Implementing Quota as a Service @nasa9084

$ whoami • @nasa9084 • LINE corp. • Go / Kubernetes / emacs • https://blog.web-apps.tech

No content

Quota as a Service

“Quota”

Quota /kwóʊṭə/ 1. ෼୲෼ɺׂΓ౰ͯ 2. (੡଄ɾ༌ೖग़ͳͲͷ)طఆ[ׂΓ౰ͯ]਺ྔ 3. (ड͚ೖΕΔҠຽɾձһɾֶੜͳͲͷ)ఆ਺ɾఆһ —https://ejje.weblio.jp/content/quota

WHY?

Why implement “Quota as a Service”? • We are developing / managing Monitoring system • Very many requests • Easy to abuse → We need Quota/Rate Limit for our services

Don’t use Quota / RateLimit lib simply? • LINE has many services (also our team) • Need Quota / RateLimit per services • Need manage configurations for each services • Need database for each services • Not want to manage extra DBs…

Algorithms

Token Bucket • Limit the average rate of traffic • Allow some burstiness • Bucket is an abstracted container • We can implement as buffer or queue

Token Bucket Algorithm 1. Add Tokens into Bucket per 1/r seconds • Bucket can hold b Tokens 2. When n bytes packet is coming, remove n Tokens and send the packet 3. If Bucket does not have n Tokens, the packet becomes non-conformant • Drop the packet • Queue the packet until Bucket charges enough Tokens • Send with non-conformant flag

Leaky Bucket • Limit the peak rate of traffic • Not allow burstiness • Same as Generic Cell Rate Algorithm • Used for ATM Network

Leaky Bucket Algorithm • A fixed capacity bucket • If the bucket is empty, stops leaking • Packet is water • It is possible to add a specific amount of packet to the bucket • If the amount of packet would cause the bucket to exceed its capacity, then the packet is non-conformant

Fixed Window Counter • Limit requests per REAL time duration • Window is fixed • e.g. 100 requests / 10:00 - 10:59 10:00 11:00 Requests

Fixed Window Counter • Over quota in configured duration • e.g. 5 requests/hour 09:00 10:00 11:00 6 requests/hour

Sliding Window Counter • Limit requests since ${window_size} ago • Window limitation window moves as time passes

Existing Solutions

QuotaLibs

vladimir-bukhtoyarov/bucket4j • Written in Java • Based on Token Bucket algorithm • Scalable for multi-threading

tomasbasham/ratelimit • Written in python • Implemented as decorator • Not using database!

QuotaService

square/quotaservice • Written in Go • gRPC service • Based on Token Bucket algorithm • Still WIP…

lyft/ratelimit • Written in Go • As gRPC service • Assumed to use with envoy

Quota as a Service for us

Implement Quota as a Service • (Of course) Write with Go • Clean Architecture (-like) • Standard Project Layout * ᵓᴷᴷ cmd/ # main.go ᵓᴷᴷ init/ # systemd ᵋᴷᴷ internal/ ᵓᴷᴷ cmd/ ᴹ ᵓᴷᴷ httpgen/ # generate http router ᴹ ᵋᴷᴷ mockgen/ # generate mock ᵋᴷᴷ pkg/ ᵓᴷᴷ apiserver/ ᵓᴷᴷ domain ᵓᴷᴷ errors/ ᵓᴷᴷ infra/ # implementation ᵓᴷᴷ interceptor/ # gRPC middleware ᵓᴷᴷ interfaces/ # interfaces ᵓᴷᴷ middleware/ #http middleware ᵋᴷᴷ rpc/ *golang-standards/project-layout

Reduce Management Cost • Generate Codes as possible as we can • Reduce middle-wares/services managed by ourselves

Generate Codes as possible as we can • gRPC + REST • gRPC: rate limiting • REST: registration • gRPC server/client code generated from Protocol Buffers • REST server/client code generated from OpenAPI spec • Mock from interfaces

Central Dogma • Service Configuration Repository by LINE • Highly available • Version Controlled based on Git • Can watch by client • Apply config change by event base • Can mirror GitHub to Central Dogma

Reduce Services We Should Manage • Configuration Management • GitHub Pull Request for WUI + Central Dogma as Database • User Authentication / User metadata DB • LDAP + session store (Redis)

Q?