Effとリーキーバケットを使って実装するレートリミット

Scalamatsuri2023: https://scalamatsuri.org/ja/program/J1681621800

Transcript

1. & ff ͱϦʔΩʔόέοτΛ࢖࣮ͬͯ૷͢ΔϨʔτϦϛοτ
   Implement Rate Limit using Leaky Bucket with Eff ScalaMatsuri 2023 @kzmake
2. Kazuki Iwata @kzmake • SWE at Alp, inc. ࣗݾ঺հ
   $ whoami
3. Alp, inc. thealp.co.jp Scalebase • This is services for managing revenue in subscription-based businesses. • Can use integrate Scalebase into their services using our APIs. ΞϧϓͰ͸4DBMFCBTFͱ͍͏αϒεΫϦϓγϣϯϏδωεͷͨΊͷαʔϏεΛఏڙ͠ ͍ͯ·͢ɻ"1*ͷఏڙ΋։࢝͠·ͨ͠ʂ
4. ຊ୊ʹೖ͍͖ͬͯ·͢ɺϨʔτϦϛοτ
   Rate Limit
5. What is rate limit? • A technique for controlling the amount of traf fi c. • There are some algorithms: leaky bucket, token bucket, etc... ϨʔτϦϛοτ͸τϥϑΟοΫΛίϯτϩʔϧ͢ΔςΫχοΫͰ͢ɻΞϧΰϦζϜϞ σϧʹϦʔΩʔόέοτ౳͕͋Γ·͢ɻ
6. Why is rate limit used? • Performance: Prevent performance degradation from overuse or burst. • Security: Protect from malicious attacks. • Monetization: Control the amount of req/ res through monetized APIs. ϨʔτϦϛοτΛ࢖͏Ϟνϕʔγϣϯʹ͸ɺύϑΥʔϚϯεɺηΩϡϦςΟɺϚωλ Πθʔγϣϯͱ͍ͬͨจ຺͕͋Δ͔΋ɻ
7. ̏ͭͷจ຺ʹՃ͑ɺΞϓϦέʔγϣϯͷ֤ϨΠϠʔ͕Өڹ͠ɺ"1*ʹର͢Δίετͷ ؔ܎͸ෳࡶͳ΋ͷͱͳΓ͑·͢ɻ call api cost: N rate limit security performance monetization adapter usecase infrastructure domain layers contexts complexity
8. Α͋͘ΔϨʔτϦϛοτΛߟ͑ͯΈ·͠ΐ͏ɻ࢓૊Έʹґଘ͠ɺίετͷௐ੔ʹ೰· ͞ΕΔ͔΋͠Ε·ͤΜɻ call api rate limit security performance monetization adapter usecase infrastructure domain complexity rate limit 1. per client IP address 2. per user multiple executions cost: 1 cost: 1 reconcile inconsistencies .
9. ΋͠ɺϏδωε͕੒௕͠ෳࡶͳίετΛ௥Ճ͍ͨ͠ͱͳΔͱɺ5JFSͷಋೖ౳Λߟ͑ͳ ͚Ε͹͍͚ͳ͍͔΋͠Ε·ͤΜɻ call api rate limit security performance monetization adapter usecase infrastructure domain more complexity rate limit multiple executions cost: 2 cost: 1 cost: 1 apply cost: 3? or need tiers? 1. per client IP address 2. per user multiple costs reconcile inconsistencies . NEW!
10. ෳࡶ͞Λײ͡·͢Ͷ
    i want to write freely and fl exibly...
11. BUOPTPSH& ff ͱϦʔΩʔόέοτͰϨʔτϦϛοτͷෳࡶ͞Λγϯϓϧʹ͢Δͨ Ίɺࣜͱ࣮ߦʹ෼཭͠දݱͯ͠Έ·͢ɻ Simplify the complexity of rate limiting by separating it into expressions and executions using the Leaky Bucket algorithm with atnos-org/Eff.
12. ͭͷࣜͱ࣮ߦͷ̏ͭʹ෼ׂ͠ɺϨʔτϦϛοτΛ࣮ݱ͠Α͏ͱࢼΈ·ͨ͠ɻ
    call api cost rate limit request throttle execution security performance monetization adapter usecase infrastructure domain expressions ᶃ ᶄ ᶅ Run (ᶃs << ᶄ) with ᶅ to make a rate limit interpretation
13. ۩ମతʹɺϓϩάϥϜͷදݱͷͨΊಠࣗͷ5ISPUUMJOHΤϑΣΫτಋೖɺ࣮૷ʹϦʔ Ωʔόέοτͱ($3"Λద༻͠·͢ɻ 1. Original effect: Throttling using atnos-org/eff separate into expressions and executions 2. Leaky bucket use the scheduling process 3. GCRA(Generic cell rate algorithm) apply time-based leaky bucket algorithm
14. ͔͜͜ΒɺϨʔτϦϛοτΛදݱ͢ΔͨΊͷಠࣗΤϑΣΫτͱͯ͠5ISPUUMJOHΛઃ ܭ͠·͢ɻ "Throttling" Effect
15. Design "Throttling" effect ϨʔτϦϛοτΛߏ੒͢Δ̎ͭͷૢ࡞Λදݱ͢ΔͨΊɺ"%5Λఆٛ͠·͢ɻ
    Expressions: 1. Request to add N cells to the bucket 2. Throttle requests can be written as ADTs:
16. Design "Throttling" effect ·ͨɺ5ISPUUMJOHΤϑΣΫτͷ஋Λੜ੒͢Δϝιου΋४උ͓͖ͯ͠·͢ɻ
    Prepare creation with ADTs and `Eff.send`:
17. Build a program ͜ΕͰɺίετͷཁٻΛߦ͍࠷ޙʹεϩοτϦϯά͢ΔΑ͏ͳϨʔτϦϛοτΛ࣋ͭ ϓϩάϥϜΛߏஙͰ͖ͦ͏Ͱ͢Ͷɻ Can build a program as
18. දݱͷखஈΛखʹೖΕͨͷͰɺ5ISPUUMJOHΤϑΣΫτͷղऍΛ͓͜ͳ͏Πϯλʔϓ ϦλʔΛߟ͑·͢ɻ Interpretation
19. ϨʔτϦϛοτͷ༷ʑͳΞϧΰϦζϜͷத͔ΒɺϦʔΩʔόέοτͷΞϧΰϦζϜΛ ࠾༻͠·ͨ͠ɻ Leaky bucket
20. Leaky bucket ϦʔΩʔόέοτͱ͸݀ͷۭ͍ͨόέπͷϞσϧͰ͢ɻυϦοϓϓϩηεΛ࣋ͪɺҰ ఆͷ଎౓Ͱ࿙Εग़͍͖ͯ·͢ɻ • With drip process. • A constant drip out. • Over fl ow when the average water injection rate exceeds the drip out rate. drip request A: request B: request C: leaky bucket burst full?
21. Leaky bucket ࠷େ༰ྔ·Ͱ͍ͬͺ͍ʹͳΔͱड͚෇͚ͳ͍දݱɺͦΕΛϨʔτϦϛοτͱͯ͠ར༻ ͠·͢ɻ • With drip process. • A constant drip out. • Over fl ow when the average water injection rate exceeds the drip out rate. drip request A: request B: request C: leaky bucket burst full?
22. Leaky bucket Ϟσϧ؆ུԽͷͨΊɺਫྔΛηϧ΁ɺ͞Βʹ($3"Λద༻͠ɺ࣌ؒతͳϦʔΩʔό έοτ΁ͱม׵͍͖ͯ͠·͢ɻ discretization apply GCRA TAT t TATn ta
23. ($3" (FOFSJD
    DFMM
    SBUF
    BMHPSJUIN Λ؆୯ʹ͝঺հ͠·͢ɻ
    GCRA (Generic cell rate algorithm)
24. GCRA ($3"͸ཧ࿦౸ୡ࣌ࠁ 5"5 Λ༻͍ͯɺ࣍ͷεςοϓͷ5"5ΛٻΊΔΞϧΰϦζϜͰ ͢ɻ • GCRA(Generic cell rate algorithm) is a leaky bucket scheduling algorithm. • Convert bucket size and current capacity to time- based leaky bucket. Update: Reject: Params: TATn+1 = { ta + q * T (n = 0) TATn + q * T (otherwise) TATn+1 − (τ + T) ≥ ta : time of request : leak interval : time capacity of bucket ( size) ta T τ = T * : number of cells : burst capacity q τ + T
25. GCRA ֤߲͸࣌ؒతͳϦʔΩʔόέοτͷݱࡏ༰ྔ΍࠷େ༰ྔΛද͓ͯ͠ΓɺࠨͷࣜͰεέ δϡʔϧΞϧΰϦζϜΛදݱ͠·͢ɻ • GCRA(Generic cell rate algorithm) is a leaky bucket scheduling algorithm. • Convert bucket size and current capacity to time- based leaky bucket. Update: Reject: Params: TATn+1 = { ta + q * T (n = 0) TATn + q * T (otherwise) TATn+1 − (τ + T) ≥ ta : time of request : leak interval : time capacity of bucket ( size) ta T τ = T * : number of cells : burst capacity q τ + T
26. ࠷΋஫໨͢Δ΂͖఺͸ɺυϦοϓϓϩηεͷ࣮૷ͳ͠ͰϦʔΩʔόέοτΛΤϛϡ ϨʔτͰ͖Δ͜ͱͰ͢ɻ • Store only one parameter: Theoretical Arrival Time ( ). • Without dripping process. • Can be implemented using only the SETEX (= SET and EXPIRE) command provided by Redis. TATn Update: Reject: Params: TATn+1 = { ta + q * T (n = 0) TATn + q * T (otherwise) TATn+1 − (τ + T) ≥ ta : time of request : leak interval : time capacity of bucket ( size) ta T τ = T * : number of cells : burst capacity q τ + T GCRA
27. ͭ·Γɺ5"5ͷΈΛߋ৽͠ӬଓԽ͢Δ͚ͩͰදݱͰ͖ɺ3BEJTͰఏڙ͞ΕΔ4&5&9ͩ ͚Ͱ࣮૷Ͱ͖Δ΋ͷͱͳͬͯ·͢ɻ • Store only one parameter: Theoretical Arrival Time ( ). • Without dripping process. • Can be implemented using only the SETEX (= SET and EXPIRE) command provided by Redis. TATn Update: Reject: Params: TATn+1 = { ta + q * T (n = 0) TATn + q * T (otherwise) TATn+1 − (τ + T) ≥ ta : time of request : leak interval : time capacity of bucket ( size) ta T τ = T * : number of cells : burst capacity q τ + T GCRA
28. ͦΕͰ͸ΠϯλʔϓϦλʔͷ࣮૷Λߟ͍͖͑ͯ·͢ɻ
    Interpreter
29. Implement interpreter ίετͷҡ࣋ʹ4UBUFΤϑΣΫτɺ5"5ͷӬଓԽͰ,74UPSFΤϑΣΫτɺΤϥʔͷͨ Ί&JUIFSΤϑΣΫτͰղऍͯ͠Έ·͢ɻ "Throttling" effect interpret • "State" effect for maintaining the map of key/cost. • "KVStore" Effect for storing the TAT. • "Either" effect for rate limit error.
30. Implement interpreter ࠓճ͸ɺUSBOTMBUFΛ༻͍ͯ5ISPUUMJOHΤϑΣΫτΛ($3"Λద༻͠ͳ͕Β̏ͭͷ ΤϑΣΫτʹ຋༁ͯ͠Έ·͢ɻ Translate "Throttling" effect to other effects. Throttling KVStore State Either
31. "State" Effect 4UBUFΤϑΣΫτ͸ɺཁٻ͞ΕͨΩʔͱίετΛ.BQ΁Ϛʔδͯ͠ҡ࣋͢ΔͨΊʹ ࢖͍ͬͯ·͢ɻ Maintain the map of merged key/cost.
32. "State" Effect ·ͨɺϚʔδॲཧͰ͸Ճࢉͤ͞Δ͜ͱͰɺίετΛੵΈ͋͛ΔදݱΛՄೳʹ͍ͯ͠· ͢ɻ Maintain the map of merged key/cost.
33. "KVStore" Effect ,74UPSFΤϑΣΫτ͸ɺ5"5ͷߋ৽ʹͯ࢖༻͠·͢ɻLFZʹର͢Δ5"5Λऔಘɺ࣍ε ςοϓͷ5"5Λߋ৽͍͖ͯ͠·͢ɻ Store the Theoretical Arrival Time.
34. "KVStore" Effect ͋ͱ͸,74UPSFͷΠϯλʔϓϦλʔΛ3FEJTར༻ͳͲʹ͢Ε͹αʔϏεؒͰϦʔ Ωʔόέοτͷঢ়ଶΛڞ༗Ͱ͖·͢ɻ Store the Theoretical Arrival Time.
35. ࠷ޙʹϓϩάϥϜͷ࣮ߦΛΈͯΈ·͢ɻ
    Execution
36. ϓϩάϥϜͷ࠷ޙʹUISPUUMFΛ௥Ճ͠ɺΠϯλʔϓϦλʔͰ࣮ߦ͢Δ͜ͱͰϨʔτϦ ϛοτΛ࣮ݱͰ͖·ͨ͠ɻ Execute a program or performance:service/hoge
37. Ωʔʹର͢ΔͦΕͧΕͷϦʔΩʔόέοτΛΤϛϡϨʔτ͢Δ͜ͱͰෳࡶͳϨʔτϦ ϛοτ΋࣮ݱͯ͘͠Ε·͢ɻ Execute a program or security:ipAddress/192.0.2.0 or monetization:user/alice or performance:service/hoge
38. ·ͨɺA
    UISPUUMFA
    ΛऔΓআ͚͹ϓϩάϥϜͷมߋΛͤͣʹͰϨʔτϦϛοτͳ͠ ͷΞϓϦέʔγϣϯ΋࡞Ε·͢ɻ Execute a program With the rate limit for api Without the rate limit for batch, system
39. ·ͱΊ
    • It is possible to write programs fl exibly by introducing the "Throttling" effect. • By switching the implementation of the interpreter, it is possible to adopt different algorithms as well. • Removing `<< throttle[R]` makes it easy to disable rate limiting without implementing a no-op interpreter. .
40. Thank you