Scalability Myth Busters

Function-as-a-Service serverless offerings are advertised as the way to build event-driven applications in days or hours, and then scale them up to millions of users – all for several dollars a month.

But how does that work in today’s cloud? What do we know about the internals of FaaS implementations? Is scalability a solved problem?

Join Mikhail on a journey into the depths of how serverless scalability works, what’s common and what’s different across cloud providers, and why you should care.

Transcript

1. Serverless Scalability Myth Busters Serverless Architecture | The Hague | April 10, 2019
2. Function-as-a-Service Cloud Offerings
3. Ideal Scalability: Throughput
4. Ideal Scalability: Latency
5. Customer Value Cloud Resource Utilization
6. FIRST USERS: Scaling from 0 to 1 1 user
7. Example: Loading a map
8. Example: Loading a map
9. Example: Loading a map
10. Cold Start
11. Warm Start
12. When Does Cold Start Happen? Azure:
13. When Does Cold Start Happen? AWS:
14. When Does Cold Start Happen? GCP:
15. Cold Starts: Language Comparison
16. Dependencies Add to Cold Start
17. COLD STARTS ARE SLOW(-ER) …and depend on language and package size
18. Fighting Cold Starts: Azure [FunctionName("Warmer")] public static void WarmUp( [TimerTrigger("0 */15 * * * *")] TimerInfo timer) { // No need to do anything }
19. AWS: Single instance warming
20. AWS: Multiple instance warming
21. CONCURRENT USERS: Learning the scaling model ~100 users
22. Servers, VMs, containers, processes
23. Servers, VMs, containers, processes
24. Servers, VMs, containers, processes
25. Servers, VMs, containers, processes
26. Servers, VMs, containers, processes
27. Kubernetes usage chart
28. Kubernetes usage chart
29. FAAS RUNS ON KUBERNETES No, not for Big-3 providers
30. Functions isolation units
31. Functions isolation units
32. Functions isolation units
33. Functions isolation units
34. AWS/GCP: concurrent executions
35. AWS/GCP: pricing
36. Azure: concurrent executions
37. Azure: pricing
38. Concurrency Trade-offs
39. DEPENDENCIES: Interacting with non- serverless components ~10k users
40. Typical Application
41. Dependencies: web services
42. Dependencies: databases
43. Dependencies: web services
44. Dependencies: databases
45. MORE SCALE IS ALWAYS BETTER It’s easy to DDoS your non- serverless components
46. Limit on Trigger Level
47. AWS: Reserved Concurrency 49
48. Azure: Max Scale-out s50
49. Resource Pooling: Persist Connections const http = require('http'); const agent = new http.Agent({keepAlive: true}); exports.mycloudfunction = (req, res) => { req = http.get({ host: '...', port: 80, path: '...', agent: agent }, // ...
50. Reuse HTTP Connections in C# private static HttpClient Client = new HttpClient(); [FunctionName("MyFunc")] public static async Task Run([QueueTrigger("q")] string message) { var response = await Client.GetAsync("https://mikhail.io"); // ... the rest goes here }
51. Managing MySQL at Serverless Scale
52. SERVERLESS IS STATELESS Resources are reused between invocations, which is important for pooling and caching.
53. ETL AT SCALE: Asynchronous data processing ~1mln users
54. Async Pipelines
55. AWS Lambda Processing 100k SQS Messages
56. GCF Processing 100k Pub/Sub Messages
57. Azure Functions Processing 100k Queue Messages
58. Scaling AWS Lambda
59. Scaling Azure Function
60. CPU-intensive workload on AWS Lambda
61. CPU-intensive workload on Google Cloud Functions
62. CPU-intensive workload on Azure Functions
63. AWS Processing Speed per Reserved RAM
64. SERVERLESS IS A MAGIC BOX Learn the resource model, limits, scaling mode, RAM & CPU allocation
65. HTTP AT SCALE: Serving traffic of StackOverflow ~1mln users
66. Serving StackOverflow-like traffic
67. Requests/sec during the load test
68. 0 200 400 600 800 1000 1200 Azure Functions: Response Percentiles (ms)
69. 0 200 400 600 800 1000 1200 Azure Functions: Response Percentiles (ms)
70. 0 200 400 600 800 1000 1200 Azure Functions: Instances
71. Google Functions: Response Percentiles (ms)
72. Google Functions: Response Percentiles (ms)
73. 0 200 400 600 800 1000 1200 Google Functions: Instances
74. 0 200 400 600 800 1000 1200 AWS Lambda: Response Percentiles (ms)
75. 0 200 400 600 800 1000 1200 AWS Lambda: Response Percentiles (ms)
76. 0 200 400 600 800 1000 1200 AWS Lambda: Instances
77. SERVERLESS IS VERY SCALABLE It’s uniquely suitable for spiky load… but know the limits
78. What Problems Are There To Be Solved? 55
79. • • • • AWS Firecracker
80. • • • • Binaris
81. • • • • Cloudflare Workers
82. • • • • Follow-up reading