Performance Tales of Serverless

Function-as-a-Service serverless cloud offerings provide us with an easy way to run custom code in response to events. One promise of FaaS model is the ability to scale without limits, up or down, whenever needed.

But how does that work in practice? Can Azure Functions handle thousands of messages per second? How fast can it scale up under sudden heavy load? What kind of latency can you expect, and what factors does it depend on?

This session is a set of short tales, each one of them teaching us a lesson about practical scalability of serverless applications. I will also suggest steps to evaluate whether your application profile is suitable for serverless today.

Transcript

1. Performance Tales of Serverless CloudBrew | October 13 | 2018
3. Containers Docker AKS
4. Serverless Architecture
5. Cloud Architecture Landscape
9. Actually Serverless
10. Not Great Serverless
11. м 11 The Story …
12. 2012 2013 12
14. What did we want? 14 $
16. м 16 2017 2018
18. What do we want? 18 $
19. 19 Function-as-a-Service Cloud Offerings
20. 20 Ideal Scalability: Throughput
21. Ideal Scalability: Latency 21
22. 22 Application Overview
23. OCCASIONAL USERS: MVP just launched, we get several visitors per hour 23
24. Visitors during sample 5 hours 24
25. Dependency Call Duration (ms) 25
26. Target End-to-End Latency (ms) 26
27. AWS Lambda Latency (ms) 27
28. Google Cloud Functions Latency (ms) 28
29. Azure Functions Latency (ms) 29
30. 29 Cold Start
31. 29 Warm Start
32. Subsequent Cold Starts after quiet periods 32
33. UNDERSTAND THE RUNTIME 1. Lesson Learnt 33
34. COLD STARTS ARE SLOW(-ER) …and depend on language and package size 34 2. Lesson Learnt
35. LOAD TESTING: How do we load test? With serverless! 35
36. Load Testing Setup 36
37. Load Testing Setup 37
38. Load Testing Setup 38
39. BE CREATIVE IN USAGE SCENARIOS 39 3. Lesson Learnt
40. PROJECTED LOAD: Simulated at the target amount of users 40
41. Requests/sec during 5 hours of load test 41
42. End-to-End Latency (ms) 42
43. End-to-End Latency (ms) 43
44. SERVERLESS SCALABILITY IS LIMITED BY 44 NON-SERVERLESS DEPENDENCIES 4. Lesson Learnt
45. 45 SERVERLESS APPLICATION CAN DDOS ANY MERE MORTAL SERVICE 5. Lesson Learnt
46. ETL: Asynchronous data processing 46
47. How to improve it 47
48. Serverless ETL 48
49. GCP Functions + Pub/Sub Processing 1 Million Points 49
50. Azure Functions + Event Hub Processing 1 Million Points 50
51. AWS Lambda + Kinesis Processing 1 Million Points 51
52. AWS Processing Speed per Reserved RAM 52 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
53. SERVERLESS IS NOT JUST HTTP EVENT BASED ASYNC FTW 53 6. Lesson Learnt
54. 54 LEARN THE RESOURCE MODEL: LIMITS, SCALING MODE, RAM & CPU ALLOCATION 7. Lesson Learnt
55. TV EFFECT: We are going to be live on TV!!! 55
56. Requests/sec during the load test 56
57. # OF REQUESTS / SEC RESPONSE TIME (MS) TIME (MINUTES) Azure Functions Response Percentiles (ms) 57
58. AXIS TITLE RESPONSE TIME (MS) TIME (MINUTES) Azure Functions Response 99th Percentile (ms) 58
59. # OF REQUESTS / SEC RESPONSE TIME (MS) TIME (MINUTES) AWS Lambda Response Percentiles (ms) 59
60. Google Cloud Functions: Throttled 60
61. Google Cloud Functions: Persist Connections const http = require('http'); const agent = new http.Agent({keepAlive: true}); exports.mycloudfunction = (req, res) => { req = http.get({ host: '...', port: 80, path: '...', agent: agent }, // ... 61
62. # OF REQUESTS / SEC RESPONSE TIME (MS) TIME (MINUTES) Google Functions Response Percentiles (ms) 62
63. 63 REUSE CONNECTIONS, POOL & CACHE 8. Lesson Learnt
64. SERVERLESS IS UNIQUELY SUITABLE FOR SPIKY LOAD… 64 BUT KNOW THE LIMITS 9. Lesson Learnt
65. RUNNING IN PRODUCTION 65
66. 66 Actual Outcome • • • • 
67. DESIGN FOR SUCCESS BUT BE PREPARED TO 67 A FAILURE 10. Lesson Learnt
68. Azure Functions Internals 68
69. 69 Azure App Service
70. 70 Azure App Service
71. 71 Azure App Service
72. 72 Function App Consumption
73. Cold Starts 73
74. When do Cold Starts happen? 74
75. Cold Starts: Language Comparison 75 V1 .NET V1 Javascript V2 .NET V2 Javascript COLD START TIME (SEC)
76. Cold Starts: Language Comparison 76 V1 .NET V1 Javascript V2 .NET V2 Javascript V2 Java (Preview) COLD START TIME (SEC)
77. Cold Starts: Dependencies Add Up 77 Hello World 5 MB Zipped 35 MB Zipped COLD START TIME (SEC)
78. Fighting Cold Starts 78 [FunctionName("Warmer")] public static void WarmUp( [TimerTrigger("0 */15 * * * *")] TimerInfo timer) { // No need to do anything }
79. Resource Pooling 79
80. Don’t Do This 80 [FunctionName("MyFunc")] public static async Task Run([QueueTrigger("q")] string message) { using (var client = new HttpClient()) { var response = await client.GetAsync("https://mikhail.io"); // ... the rest goes here } }
81. Reuse Connections 81 private static HttpClient Client = new HttpClient(); [FunctionName("MyFunc")] public static async Task Run([QueueTrigger(“q")] string message) { var response = await Client.GetAsync("https://mikhail.io"); // ... the rest goes here }
82. Scalability 82
83. Queue Processor – Bcrypt Hashes 0 10 20 30 40 50 60 TIME (MINUTES)
84. HTTP Endpoint – Bcrypt Hashes 84 # OF REQUESTS / SEC RESPONSE TIME (S) TIME (MINUTES)
85. HTTP Endpoint – Scaling Out 85 # OF REQUESTS / SEC INSTANCE COUNT TIME (MINUTES)
86. Limiting Scale-out 86
87. Limit on Trigger Level
89. Singleton [Singleton] [FunctionName("MyFunc")] public static void Run( [QueueTrigger("myqueue")] string message) { // ... }
90. Is Serverless Good for My Next Project? 90
91. 91 • • • • Is Serverless Good for My Next Project?
92. Perfect Scenarios for Serverless • • • • 92
93. Visit My Blog for more