7 ways to solve Lambda cold starts

Cold starts can significantly affect your application's user experience. They occur when the AWS Lambda service needs to spin up a new execution environment before handling a new request. Initializing a new execution environment adds to the overall latency of the Lambda invocation. AWS has continuously improved the cold start performance of Lambda, including the introduction of proactive initialization. But cold starts are still a problem in many applications.

Join Yan Cui, AWS Serverless Hero, to learn more about common mistakes leading to cold-start performance issues and guide you through practical steps to enhance performance. This webinar is for developers interested in improving the efficiency and responsiveness of their serverless applications.

This webinar will teach you how to tackle cold starts by reducing their frequency and duration, including how to measure cold start durations correctly.

Transcript

2. “Cold Starts”
3. https:// fi recracker-microvm.github.io
4. https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
5. https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
6. https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
7. https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
8. API Gateway ALB Synchronous Asynchronous Lambda Request Response
9. API Gateway ALB Synchronous Asynchronous EventBridge SNS S3 Lambda Request Response Event Lambda Internal event queue
10. Synchronous Invocation under the hood
11. AWS region Invoke frontend load balancer availability zone 1 availability zone 2 availability zone 3 cold start invoke Synchronous
12. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service cold start invoke Invoke Synchronous
13. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service cold start invoke Invoke Synchronous
14. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service cold start invoke Invoke Synchronous
15. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker cold start invoke worker worker Invoke Synchronous
16. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker cold start invoke worker worker Invoke Synchronous
17. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Synchronous
18. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Synchronous
19. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Init Invoke Synchronous
20. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous
21. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker worker worker cold start invoke Invoke Invoke Synchronous
22. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous
23. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous Find space in worker fl eet
24. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous Find space in worker fl eet Create Firecracker microVM
25. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous Find space in worker fl eet Create Firecracker microVM Fetch env vars
26. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars
27. https://speedrun.nobackspacecrew.com/blog/2024/03/13/lambda-environment-variables-impact-on-coldstarts.html
28. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime
29. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module
30. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Invoke Init Invoke Synchronous Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module “InitDuration”
31. How should we measure cold start durations?
34. Trace through DynamoDB streams, SNS, SQS, Kinesis, EventBridge, etc.
35. RDS, Redis, Mongodb, Postgres, MS SQL, …
36. RDS, Redis, Mongodb, Postgres, MS SQL, …
38. BEST Serverless Observability Tool!!
39. Yan Cui AWS Serverless Hero Independent Consultant Developer Advocate @ Lumigo
40. How should we measure cold start durations?
41. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime
42. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module
43. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function
44. Find space in worker fl Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function “InitDuration”
45. Find space in worker fl Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function “InitDuration” “Duration”
46. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function User experiences all this
47. Synchronous Invocation Measurer
48. handler function User experiences all this JIT compilation
49. How to collect statistically signi fi cant sample set?
50. Synchronous Invocation Measurer Step 1 Target
51. Synchronous Invocation Measurer Step 1 Step 2 Modify any environment variable Target
52. Synchronous Invocation Measurer Step 1 Step 2 Modify any environment variable Target Step 3 = repeat step 1
56. go.aws/4d5U6at
57. When are cold starts a problem?
58. Not a problem when it doesn’t a ff ect user experience
59. (e.g. background tasks) Not a problem when it doesn’t a ff ect user experience
60. Traf fi c naturally keeps the Lambda functions “warm”
61. “Proactive Initialisation” (Lambda pre-emptively creating a new worker before it’s needed)
62. “Proactive Initialisation” (Lambda pre-emptively creating a new worker before it’s needed) Deployment Eager assignment
63. Concurrent Executions 100 Deployment
64. Concurrent Executions 100 Deployment 100 instances with new code
65. “Eager assignment”
66. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Init Invoke Synchronous
67. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Init Invoke Synchronous
68. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Init Invoke Synchronous
69. AWS region frontend load balancer availability zone 1 availability zone 2 availability zone 3 frontend invoke service counting service assignment service worker worker placement service cold start invoke worker worker Init Invoke Synchronous Invoke
70. User-facing cold starts are less frequent now
71. Lots of cold starts during a spike
72. Not a problem if it happens infrequently
73. lumigo.io
74. lumigo.io
75. Not a problem if cold starts are fast enough
76. How can we “solve” cold starts?
77. Option A. Option B. Make them less frequent Make them less impactful
78. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function
79. Lambda warmer
80. Cron job
82. Very cost e ff ective
83. Not a scalable solution
84. Not a scalable solution But not everything runs at scale ¯\_(ツ)_/¯
85. Not a scalable solution But not everything runs at scale ¯\_(ツ)_/¯ And everyone (should) cares about cost
88. Provisioned Concurrency
92. Provisioned Concurrency
93. Provisioned Concurrency On-Demand Concurrency
95. Can’t use on $LATEST alias
96. Introduces uptime cost to Lambda
97. Costly in low utilisation situations
98. Pay for uptime
99. Pay for uptime Pay for usage
100. Pay for uptime Pay for usage Lower than on- demand concurrency
101. Provisioned Concurrency comes out of the regional limit
102. DON’T use Provisioned Concurrency as default
103. use Provisioned Concurrency when: - optimization alone can’t help you reach performance target - cold starts would likely stack - tra ffi c is spiky
104. it’s about improving user experience, not eliminating cold starts
105. it’s about improving user experience, not eliminating cold starts Make cold start fast enough so it’s no problem
106. Trim your dependencies
107. Find space in worker fl Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function
108. www.youtube.com/watch?v=2EDNcPvR45w
109. www.youtube.com/watch?v=2EDNcPvR45w
110. Every 10MB costs ~100ms
111. Lazy load dependencies if they are not used in every execution
112. Bundling and mini fi cation helps
113. Bundling and mini fi cation helps Be careful of SourceMap
114. AOT compilation
115. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function
117. https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html
118. SnapStart
120. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function
121. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Take snapshot of memory and disk
122. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Take snapshot of memory and disk Cache it
123. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Take snapshot of memory and disk Cache it Deploy time
124. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Take snapshot of memory and disk Cache it Deploy time Create new environment from cache Runtime
125. ~2.8s ~1.3s with SnapStart without SnapStart
127. Can’t use on $LATEST alias
128. Java only
129. Don’t create network connections during function init
130. Don’t initialise randomness during function init
131. Don’t fetch temporary credentials during function init
132. Package Lambda as container images
133. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function
134. https://arxiv.org/pdf/2305.13162
135. Serialize container layers onto fi lesystem
136. Serialize container layers onto fi lesystem Just tarballs
137. Divide fi lesystem into 512KB chunks Serialize container layers onto fi lesystem
138. Divide fi lesystem into 512KB chunks Encrypt each chunk Serialize container layers onto fi lesystem
139. Divide fi lesystem into 512KB chunks Encrypt each chunk Cache the chunks and share across all customers Serialize container layers onto fi lesystem
140. Divide fi lesystem into 512KB chunks Encrypt each chunk Cache the chunks and share across all customers Serialize container layers onto fi lesystem Many customers have the same dependencies
141. Multi-tier caching L1: on the worker L2: distributed in AZ L3: backing store
142. Multi-tier caching L1: on the worker L2: distributed in AZ L3: backing store 67% of chunks are served from on-worker cache!
143. aaronstuyvenberg.com/posts/containers-on-lambda Node.js
144. aaronstuyvenberg.com/posts/containers-on-lambda Python
145. Application Runtime OS Virtualisation Server Storage Network Application Runtime OS Virtualisation Server Storage Network You manage Platform manages Container Image Managed Runtime
146. Prefer light-weight runtimes
147. Find space in worker fl eet Create Firecracker microVM Fetch env vars Decrypt env vars Initialise runtime Initialise function module handler function
149. https://github.com/awslabs/llrt
150. ~750ms ~50ms LLRT
151. ~750ms ~50ms LLRT No JIT compilation QuickJs + Rust
152. LLRT is not production-ready
153. 1. Lambda Warmers 2. Provisioned Concurrency 3. Trim your dependencies 4. AOT compilation 5. SnapStart 6. Package Lambda as container images 7. Prefer light-weight runtimes
154. Get Started Troubleshooting Your Serverless Applications Today - Free Lumigo.io/signup
155. Questions?