Reflections on Operating a Serverless API

This is a presentation for JavaScript Meetup Luxembourg JUNE 2018 on our experience with running APIs on Serverless Architecture (AWS Lambda). Presenting also the Serverless framework quickly

Transcript

1. Reflections on operating a Serverless API Thierry Nicola • JSLuxembourg June 2018
2. About me Name: Thierry Nicola Twitter, Github,...etc: Littleiffel Work: Doctena Luxembourg NOT Affiliated with AWS, Google or Azure! 2
3. Schedule • Concept of FaaS • FaaS at Doctena • Serverless Framework • Understanding Cold Starts • Operating Serverless API 3
4. Concept Function as a Service 4
5. Function as a Service aka • Function as a Service • Backend as a Service • Serverless • ...etc 5
6. Function as a Service Concept Deploy your code to a fully managed environment, where you are billed by execution time. The deployed code is invoked by a trigger History • 2006 - Zimki • 2008 - Google App Engine • 2014 - AWS Lambda • … Google Cloud Functions, IBM OpenWhisk, Azure Functions,.... 6
7. Function as a Service Limitations Code FaaS providers support different Architectures/Languages, you have to use one of them, Code Package Size is limted, and RAM for execution is also limited • NodeJs, Python, Java Choose carefully as great impact on “performance” (cf. later) WARNING 7
8. Function as a Service Triggers Triggers FaaS Providers offer different sort of triggers for functions • HTTP Requests, Execution Plans, Emails, DB Events.. 8
9. Function as a Service Use cases Different Use cases We have to distinguish between the following cases 1. Many Triggers and requiring Low latency (Faas As API) 2. Bursts of High Computational Tasks (Background jobs) 3. Few Triggers (Hosting personal website ) 4. ... Scenario 2 and 3 are optimal use cases for FaaS Architecture, 1 needs improvement from providers My Humble Opinion 9
10. Function as a Service Advantages Advantages (as Advertised) • Costs ◦ Pay Only for Usage • Scalability ◦ Scales for you • Productivity ◦ Focus on Code and not infrastructure 10
11. Function as a Service Disadvantages Disadvantages (as discovered) • Resource-limitations • Performance • Monitoring & Debugging • Security • Privacy • Vendor Lock 11
12. FaaS at Doctena 12
13. Doctena Service on FaaS Current State • Around 7 Services in Production on FaaS ◦ NodeJS and Java ◦ DynamoDB, MySQL, RabbitMq,... • More Services in development • Use cases ◦ API ◦ Background Jobs 13
14. Serverless Framework 14
15. Serverless Framework Serverless Framework https://serverless.com Manage Functions and Event Source within code, close to AWS CloudFormation syntax Not the only one: • AWS SAM • Google CLI • ... 15
16. Serverless Framework Advantages Advantages • Develop Offline / Run Tests • Manage Stack within Code • Integration with CI/CD Tools • Many Plugins (and simple Plugin system) • Close collaboration with Cloud Providers • Cloudformation Syntax 16
17. Demo: API API Gateway request Function A Function B Data Store 17
18. Demo & Code Walkthrough https://serverless.com/framework/docs/getting-started/ • Most simple example - https://github.com/littleiffel/jsmeetup-serverless-most-simple-example • API with ExpressJs - https://github.com/littleiffel/jsmeetup-serverless-api-demo-express 18
19. Faas Architecture Possibilities • One function per Function ◦ Clear separation of concerns • Using ExpressJS -> Create one “Function”(App) to handle all calls ◦ Creates a little overhead to re-transform event to HTTP Request 19
20. Serverless Tips • MemorySize ◦ Can be set globally and by function • Serverless-offline plugin 20
21. Understanding Cold Start 21
22. Triggering a function COLD Start Install & Deploy Function Trigger Initialization Execution Billing 22
23. Triggering a function Recycling Trigger Execution Billing Functions as long as they are used (up to a limit). If not used for a period (5 minutes) they are “terminated”, unless they are “kept warm”. 23
24. Cold Starts & Concurrency Faas Scales • A new function is spawned, when function is busy • Every new spawned function is cold-started • Heavily varying traffic Pattern (high/low) • => Many cold starts === Bad Performance === Slow response times Providers need to allow to set and control Concurrency level, or permit delay before deploying new function My Humble Opinion 24
25. Cold Starts & Concurrency Done with https://www.charlesproxy.com/ 25
26. Cold Starts & VPC Serverless with VPC (Virtual Private Cloud) • Adding Lambda to a VPC adds 10 seconds to cold start • 10 Seconds...Yes...10 Seconds • At least • Avoid to put it into a VPC • At all costs!! (if you want to stay under 10 seconds response time) 26
27. Cold Starts on API in Production 27
28. Cold Starts & Concurrency MUST READ https://hackernoon.com/im-afraid-you-re-thinking-about-aws-lambda-cold-starts-all-wrong-7d907f278a4f 28
29. Keeping Functions warm • Functions can be kept warm ◦ Up to a certain limit (max 60 minutes) • Require special function parameter or other way to trigger function without adding much load • Define a cron that triggers functions regularly ◦ You need to keep ALL parallel instances of your functions warm to avoid cold starts 29
30. Operating Serverless API 30
31. Requirements for operating an API Requirements • Availability ◦ Deploy without downtime • Scalability ◦ Handle varying Traffic patterns • Performance ◦ Response time is crucial • Productivity ◦ Develop fast (faster) • Integration 31
32. Optimizations for operating Serverless API • Programming Language ◦ Java and .Net seems most stable and fastest ◦ but... • Package Size • Memory Settings ◦ The more the better • Cold Starts ◦ Bad for compiled languages (Java,..) • Data Storage • Parallelism • Keep them Hot • VPC (AWS specific) ◦ STAY AWAY FROM VPC!!! 32
33. Function as a Service Advantages Advantages from our Experience • Costs CHECK! :-) • Scalability ◦ Functions Scale but does your DataStore? • Productivity CHECK! :-) ◦ CONDITION: Using a framework!! 33
34. Function as a Service Disadvantages Disadvantages from our Experience • Resource-limitations (No Problem) • Performance CHECK, this is a problem ◦ Cold Starts for API are a BIG problem • Monitoring & Debugging (No Problem) • No Control on ApiGateway and Parallelism ◦ WISH: Control how long requests are queue before cold-starting a new function • Vendor Lock (Optional :)) ◦ Frameworks could eventually separate Platform from Application code 34
35. Evaluating Requirements for operating Serverless API Requirements • Availability All Good ◦ Deploy without downtime Great • Scalability Problematic ◦ Does your Data Store scale? ◦ Cold-Starts problems increase with scale • Performance All Good • Productivity All Good • Integration NO VPC!!! 35
36. Function as a Service Wishlist for Providers Our Wishlist • Define Timeout for waiting before spawning new function • Control Level of Concurrency (ReservedConcurrentExecutions on AWS) • Be able to define a Lifetime of functions • Scale Data Store with Number of deployed functions 36
37. Resources Sources • How does language, memory and package size affect Serverless performance https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of- aws-lambda-a15e26d12c76 • About Cold Starts https://hackernoon.com/im-afraid-you-re-thinking-about-aws-lambda-cold-starts-all-wrong-7d9 07f278a4f • Slides by Danilo Poccia (@danilop from AWS) https://speakerdeck.com/danilop/building-serverless-apis-with-the-amazon-api-gateway-and-aw s-appsync 37