Slide 1

Slide 1 text

by Mert METİN

Slide 2

Slide 2 text

W h o A m I ? Senior Software Engineer 7+ years working experience Blogger, speaker MERT METİN QR to reach me

Slide 3

Slide 3 text

What is Rate Limiting? Rate Limiting Algorithims Why is it important? A g e n d a Types of Rate Limiting Where to Put Rate Limiters Returning Response

Slide 4

Slide 4 text

R e m e m b e r i n g # T w i t t e r L i m i t s

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

W h a t i s R a t e L i m i t i n g ? It is an approach to scalable and secure applications. Limiting the number of request to the application When you faced with HTTP 429 Error, there is implemented "rate limiting" approach. Picture resource: https://httpstatusdogs.com/429-too-many-requests

Slide 7

Slide 7 text

Defensive mechanism against Brute Force, DoS Attack, Web Scraping, Bots. Managing and protecting server resources - Cpu, memory, network bandwidth. Controlling traffic due to increase demands suddenly. W h y I s I t I m p o r t a n t ?

Slide 8

Slide 8 text

User Location Server Based T y p e s o f R a t e L i m i t i n g Ip Based

Slide 9

Slide 9 text

Fixed Window Sliding Window - Log Based Token Bucket Leaky Bucket S t r a t e g i e s - A l g o r i t h m s The choice of algorithm depends on the application’s specific requirements Sliding Window - Counter Based Dynamic Rate Limiter

Slide 10

Slide 10 text

T o k e n B u c k e t A token as processing request. Capacity of the bucket is defined. If the request came, token was leaving from the bucket until reaches refillment time.

Slide 11

Slide 11 text

T o k e n B u c k e t In distributed systems, we need to balance the refillment time and the request limit so that, there are no problems due to sudden resets. Solution is refill the bucket using refillment rate. Example: 5 request per 60 seconds Rate is 60/5=12 sec 10 requests in 6 seconds

Slide 12

Slide 12 text

L e a k y B u c k e t Requests are placed with FIFO (first in first out) queue structure and process regular intervals. If queue is full, there will not be accepted new request until process requests at the queue. Advantage over Token Bucket is the regular processing of requests at a fixed time interval.

Slide 13

Slide 13 text

https://www.geeksforgeeks.org/leaky-bucket-algorithm/ https://media.geeksforgeeks.org/wp-content/uploads/leakyTap-1.png L e a k y B u c k e t v s T o k e n B u c k e t

Slide 14

Slide 14 text

F i x e d W i n d o w Allows limited number of request in fixed period of time. If exceeds, rejected until replenish the window/counter. In other words "N request per period" Example: 3 requests per 60 seconds

Slide 15

Slide 15 text

Heavy requests at the boundary of two consecutive time windows can lead to increased traffic. When the maximum request limit is reached at the beginning of the time window, it can wait for a long time to wait for the next time window. It depends on length of time window. F i x e d W i n d o w Problems can be encountered with Fixed Window

Slide 16

Slide 16 text

F i x e d W i n d o w V S T o k e n B u c k e t Refillment rate can be adjusted using refillment period and request limit. Token Bucket Refillment rate is constant. Fixed Window

Slide 17

Slide 17 text

S l i d i n g W i n d o w - L o g B a s e d Incoming request and it’s timestamp store in array with request period. If a new request is received, the requests in the last time period are checked. If the request limit is not reached, push it to the array.

Slide 18

Slide 18 text

S l i d i n g W i n d o w - C o u n t e r B a s e d The logic is time window divides into time segment. Sliding window moves when each time segment passed. Example; There is 60 seconds window and it divided three segments those are 20 seconds. Request Limit is 50.

Slide 19

Slide 19 text

S l i d i n g W i n d o w - C o u n t e r B a s e d

Slide 20

Slide 20 text

L o g B a s e d v s C o u n t e r B a s e d The log-based method offers more detailed tracking and precise window control due to store timestamp. However this approach needs large data structures may be required and memory consumption is high. On the other hand; The counter-based method is more performant, but works with a slight loss of accuracy.

Slide 21

Slide 21 text

D y n a m i c R a t e L i m i t i n g Adaptive rate limiting adjusts the limit based on various factors such as client behavior, server load. In order to adjusted in real-time to optimize performance and prevent abuse

Slide 22

Slide 22 text

W h e r e t o P u t R a t e L i m i t e r s ? Client side or server side Server side is more secure and manageable than client side.

Slide 23

Slide 23 text

Is there any scalable and more reliable solutions? W h e r e t o P u t R a t e L i m i t e r s ?

Slide 24

Slide 24 text

Using RateLimit Header RateLimit-Limit: 10 RateLimit-Remaining: 1 RateLimit-Reset: 7 Returning Json object which includes meaningful message using 429 HTTP Status Code R e t u r n i n g R e s p o n s e

Slide 25

Slide 25 text

Deep dive its algorithms Where to put rate limiters Defined Rate Limiting and its importance. T o S u m U p Returning response

Slide 26

Slide 26 text

System Design Interview: An Insider’s Guide - Alex Xu - Chapter 4: Design A Rate Limiter Y o u m a y b e i n t e r e s t e d r e s o u r c e s API Rate Limiter System Design https://www.enjoyalgorithms.com/blog/design-api-rate-limiter Rate Limiting Kavramını ve Algoritmalarını Anlamak: https://mertmtn.blogspot.com/2022/09/rate-limiting-kavramn-ve-algoritmalarn.html Rate Limiting with Redis https://www.youtube.com/watch?v=YcmU24x4KyA&t=440s

Slide 27

Slide 27 text

T H A N K Y O U QR to reach me QR to slide