Experience on gRPC rate limiting with Istio

Transcript

1. Experience on gRPC rate limiting with Istio
2. Hello! I am Miya Chen, a backend Enginner at AMIS Golang gRPC Kubernetes Terraform Prometheus Docker Vault
3. ▷ A kind of service mesh application ◦ Sidecar proxy ◦ Middle layer between services ◦ Decouple your application from the network ▪ Retry and timeout ▪ Monitor and tracing ▪ Service discovery ▪ Circuit breaker What is Istio?
4. https://www.nginx.com/blog/what-is-a-service-mesh/
5. Why we need Istio? ▷ gRPC is based on HTTP/2 ◦ Keep only one connection ◦ All unary and streaming requests thorugh one connection ▷ kube-proxy proxies UDP, TCP and SCTP but does not understand HTTP ▷ Can not do real load balancing and rate limiting
6. Istio Architecture ▷ Proxy (Envoy) ◦ Sidecar ◦ Extract traffic behavior as attributes ▷ Mixer ◦ Access control and usage policies ▷ Pilot ◦ Service discovery ◦ Load balancing ◦ Resiliency (retries and timeout) ▷ Others https://istio.io/docs/concepts/what-is-istio/
7. Rate Limiting ▷ What’s request we want to rate limit? ◦ By source or desination of request ▪ Ex: request from user service to notification service ◦ By request path ▪ Ex: /api/notification ▷ Request rate ◦ 100 per second
8. Istio Rate Limiting Rate limiting configuration is divided into two parts ▷ Mixer side ▷ Client side
9. Istio Rate Limiting - mixer side ▷ Quota instance ◦ Define rate limit dimensions ▪ request.headers[":path"] ▷ Quota handler ◦ Define rate limit rule for different request dimestions ▪ Set rate 100 per second for request path is /api/list ▷ Quota rule ◦ Bind instance and handler ◦ Define request matching rule
10. Istio Rate Limiting - client side ▷ Quota spec ◦ Define the quota cost and quota source ▪ 1 request charge 5 quota ▷ Quota spec binding ◦ Bind service and quota spec
12. gRPC Rate Limiting ▷ gRPC request headers are delivered as HTTP2 headers ◦ Method ➞ ":method" "POST" ◦ Scheme ➞ ":scheme" ("http" / "https") ◦ Path ➞ ":path" "/{Service-Name}/ {method name}" ◦ Content-Type ➞ "content-type" "application/grpc" ◦ … etc https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md
13. gRPC request path
14. 以 memqouta 設定為例 https://istio.io/docs/reference/config/policy-and-telemetry/adapters/memquota/ https://istio.io/docs/reference/config/policy-and-telemetry/attribute-vocabulary/
15. for all services https://istio.io/docs/reference/config/policy-and-telemetry/istio.mixer.v1.config.client/#QuotaSpec
16. Note ▷ Istio provides two types of quota handler ◦ memquota and redisquota ◦ Do Not use memquota in production ▷ Quota algorithm ◦ FIXED_WINDOW (for both) ◦ ROLLING_WINDOW (only for redisquota)
17. Thank you! miyatlchen@gmail.com https://medium.com/getamis https://github.com/MaiAmis/Careers/tree/master/AMIS