Rate Limiting Insights

GDG Ankara - Devfest Ankara 24'
15.12.2024
Hacettepe University / Ankara

Transcript

1. by Mert METİN
2. W h o A m I ? Senior Software Engineer 7+ years working experience Blogger, speaker MERT METİN QR to reach me
3. What is Rate Limiting? Rate Limiting Algorithims Why is it important? A g e n d a Types of Rate Limiting Where to Put Rate Limiters Returning Response
4. R e m e m b e r i n g # T w i t t e r L i m i t s
6. W h a t i s R a t e L i m i t i n g ? It is an approach to scalable and secure applications. Limiting the number of request to the application When you faced with HTTP 429 Error, there is implemented "rate limiting" approach. Picture resource: https://httpstatusdogs.com/429-too-many-requests
7. Defensive mechanism against Brute Force, DoS Attack, Web Scraping, Bots. Managing and protecting server resources - Cpu, memory, network bandwidth. Controlling traffic due to increase demands suddenly. W h y I s I t I m p o r t a n t ?
8. User Location Server Based T y p e s o f R a t e L i m i t i n g Ip Based
9. Fixed Window Sliding Window - Log Based Token Bucket Leaky Bucket S t r a t e g i e s - A l g o r i t h m s The choice of algorithm depends on the application’s specific requirements Sliding Window - Counter Based Dynamic Rate Limiter
10. T o k e n B u c k e t A token as processing request. Capacity of the bucket is defined. If the request came, token was leaving from the bucket until reaches refillment time.
11. T o k e n B u c k e t In distributed systems, we need to balance the refillment time and the request limit so that, there are no problems due to sudden resets. Solution is refill the bucket using refillment rate. Example: 5 request per 60 seconds Rate is 60/5=12 sec 10 requests in 6 seconds
12. L e a k y B u c k e t Requests are placed with FIFO (first in first out) queue structure and process regular intervals. If queue is full, there will not be accepted new request until process requests at the queue. Advantage over Token Bucket is the regular processing of requests at a fixed time interval.
13. https://www.geeksforgeeks.org/leaky-bucket-algorithm/ https://media.geeksforgeeks.org/wp-content/uploads/leakyTap-1.png L e a k y B u c k e t v s T o k e n B u c k e t
14. F i x e d W i n d o w Allows limited number of request in fixed period of time. If exceeds, rejected until replenish the window/counter. In other words "N request per period" Example: 3 requests per 60 seconds
15. Heavy requests at the boundary of two consecutive time windows can lead to increased traffic. When the maximum request limit is reached at the beginning of the time window, it can wait for a long time to wait for the next time window. It depends on length of time window. F i x e d W i n d o w Problems can be encountered with Fixed Window
16. F i x e d W i n d o w V S T o k e n B u c k e t Refillment rate can be adjusted using refillment period and request limit. Token Bucket Refillment rate is constant. Fixed Window
17. S l i d i n g W i n d o w - L o g B a s e d Incoming request and it’s timestamp store in array with request period. If a new request is received, the requests in the last time period are checked. If the request limit is not reached, push it to the array.
18. S l i d i n g W i n d o w - C o u n t e r B a s e d The logic is time window divides into time segment. Sliding window moves when each time segment passed. Example; There is 60 seconds window and it divided three segments those are 20 seconds. Request Limit is 50.
19. S l i d i n g W i n d o w - C o u n t e r B a s e d
20. L o g B a s e d v s C o u n t e r B a s e d The log-based method offers more detailed tracking and precise window control due to store timestamp. However this approach needs large data structures may be required and memory consumption is high. On the other hand; The counter-based method is more performant, but works with a slight loss of accuracy.
21. D y n a m i c R a t e L i m i t i n g Adaptive rate limiting adjusts the limit based on various factors such as client behavior, server load. In order to adjusted in real-time to optimize performance and prevent abuse
22. W h e r e t o P u t R a t e L i m i t e r s ? Client side or server side Server side is more secure and manageable than client side.
23. Is there any scalable and more reliable solutions? W h e r e t o P u t R a t e L i m i t e r s ?
24. Using RateLimit Header RateLimit-Limit: 10 RateLimit-Remaining: 1 RateLimit-Reset: 7 Returning Json object which includes meaningful message using 429 HTTP Status Code R e t u r n i n g R e s p o n s e
25. Deep dive its algorithms Where to put rate limiters Defined Rate Limiting and its importance. T o S u m U p Returning response
26. System Design Interview: An Insider’s Guide - Alex Xu - Chapter 4: Design A Rate Limiter Y o u m a y b e i n t e r e s t e d r e s o u r c e s API Rate Limiter System Design https://www.enjoyalgorithms.com/blog/design-api-rate-limiter Rate Limiting Kavramını ve Algoritmalarını Anlamak: https://mertmtn.blogspot.com/2022/09/rate-limiting-kavramn-ve-algoritmalarn.html Rate Limiting with Redis https://www.youtube.com/watch?v=YcmU24x4KyA&t=440s
27. T H A N K Y O U QR to reach me QR to slide