Rate Limit With Istio at iFood

Transcript

1. Rate Limit With Istio at The information in this document is confidential to the person to whom it is addressed and should not be disclosed to any other person. It may not be reproduced in whole, or in part, nor may any of the information contained therein be disclosed without the prior consent of the directors of iFood. It has been used as support material for an oral presentation and, therefore, it does not represent a complete record of the topics presented in the mentioned presentation.
2. Me Jhonn Frazão - SRE at iFood - Traffic Team
3. Agenda Introduction The problem The feature Implementation Learnings
4. Introduction I didn't come to talk about: - All mesh features on iFood - The iFood's mesh journey - All the problems rate limit solved at iFood But of course we can talk later about it.
5. iFood numbers 67% Microservices with mesh 73% Traffic on mesh • Orders per month: ~100m • Requests per minute: ~28m(only mesh) • Number of microservices: ~4k • Number of k8s clusters: 28 (prod)
6. The problem
7. The problem
8. The problem
9. The problem North south traffic east west traffic
10. The problem North south traffic east west traffic
11. The problem North south traffic east west traffic
12. The problem ingress-NGINX Kong
13. The problem ingress-NGINX Kong
14. Global vs Local Global quota
15. Global vs Local Global quota "Pod" quota "Pod" quota "Pod" quota
16. Global vs Local Global quota "Pod" quota "Pod" quota "Pod" quota 300 RPS 100 RPS 100 RPS 100 RPS
17. Global vs Local Global quota "Pod" quota "Pod" quota "Pod" quota 300 RPS 100 RPS 100 RPS 100 RPS
18. Global vs Local Global quota "Pod" quota "Pod" quota "Pod" quota 300 RPS 100 RPS 100 RPS 100 RPS
19. APP Redis sidecar/gw EnvoyProxy Ratelimiter EnvoyProxy Ratelimiter Envoy Ratelimit Request Ratelimit KV Check for Source limit Limit Response Conditional Response 1 2 3 4 5 HTTP: 429/XXX Istio ratelimiter Config Map
20. APP Redis sidecar/gw EnvoyProxy Ratelimiter EnvoyProxy Ratelimiter Envoy Ratelimit Request Ratelimit KV Check for Source limit Limit Response Conditional Response 1 2 3 4 5 HTTP: 429/XXX Envoy ratelimiter Config Map
21. Ratelimit Service
22. Envoy ratelimiter
23. Envoy ratelimiter
24. Envoy ratelimiter
25. Envoy ratelimiter
26. Envoy ratelimiter
27. Envoy ratelimiter
28. Envoy ratelimiter
29. How did we do?
30. Our implementation Objectives: - Developer proximity - Predefined patterns - Fail fast
31. Our implementation Objectives: - Developer proximity - Predefined patterns - Fail fast
32. Our implementation Objectives: - Developer proximity - Predefined patterns - Fail fast
33. Our implementation Objectives: - Developer proximity - Predefined patterns - Fail fast
34. Our implementation Objectives: - Developer proximity - Predefined patterns - Fail fast
35. Our implementation Objectives: - Developer proximity - Predefined patterns - Fail fast
36. One case: Service with SLO of 250ms latency - Has a lot of different clients - Must to respond less than 250ms - Receive a lot of requests from clients - There's several dependencies bellow it
37. Learnings The global rate limit don't solve all the problems
38. Learnings Developer proximity can slow down the adoption
39. Learnings The ratelimit service needs to scale along with throughput
40. Learnings The ratelimit service per cluster
41. Learnings The redis needs to scale along with throughput
42. Learnings Turn on metrics and tracing
43. Learnings Turn on metrics and tracing
44. Learnings Have a quick exit
45. Thanks Traffic team
46. Thank You