When Serverless Gets In the Way of Scalability

By Lily Li & Christian Zommerfelds - D2L

Presented at Functions 2018 / ServerlessDays Toronto

https://functions.events/2018/toronto/bret-mcgowen/

Transcript

1. When Serverless Gets In the Way of Scalability Lily Li & Christian Zommerfelds
4. Scalable and resilient system • Horizontally scalable • Cost efficient: $$$ • Resilient to traffic spikes and partial system failure • Low latency when system scales
5. Serverless Almost instant elasticity!
6. API Gateway Lambda DynamoDB WCU = 20
7. Naive solution 50s per success
8. WCU = 20 18 RPS
9. Rate limit 1.3s per success was 50s you save 96.6% now 1.3s
10. Unexpected traffic 20% of requests consume 10 WCU
11. Rate limit 50s per success
14. Little’s Law n = r × t Concurrency Arrival rate Latency
15. 18 rps × 555 ms = 10 n = r × t 10 max concurrency
16. Concurrency limit (n = 10) 3.8s per success was 50s you save 92.4% now 3.8s
17. What if your concurrency limit is wrong?
18. Concurrency limit (n = 50) 50 = 90 rps × 555 ms 9.5s per success was 50s you still save 81%! now 9.5s
19. Fixed concurrency limit problems System 3 NEW AUTO- SCALING
20. TCP solves a very similar problem! AIMD Performance Under Load failure detection concurrency adjustments
21. monitor max. concurrency stats limit = limit × latencyNoload/latencyActual + sqrt(limit)
22. Dynamic concurrency limit 2.9s per success
23. Dynamic concurrency limit (auto-scaling) 0.7s per success
24. Summary • Concurrency limit is better than rate limiting • Consider using dynamic concurrency limits • Use Little's Law for analysis
25. Other tips • Number of retries (Node.js SDK default of 10 can be terrible!) • Concurrency limit per invocation = no limit • Never assume, load test your system
26. Questions Lily.Li@D2L.com Christian.Zommerfelds@D2L.com