The State of Serverless Computing

Transcript

1. The State of Serverless Computing or, Fixing Dysfunction-as-a-Service Vikram Sreekanti, Chenggang Wu RISE Lab, UC Berkeley Craft Conference – Budapest, Hungary – 05/09/2019
2. 5th Graders Learning Programming San Diego, CA The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
3. UC Berkeley: CS 61A Fall 2018 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
4. Making Programmers Productive Key Question: Where will code be run? In the ☁! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
5. Background: Serverless Computing The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
6. What is serverless computing? Serverless computing is a programming abstraction that enables users to upload programs, run them at any scale, and pay only for resources used. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
7. Functions-as-a-Service • AWS Lambda, Google Cloud Functions, OpenWhisk (IBM), Azure Functions, OpenLambda, OpenFaaS, kNative… • Optimized for simplicity – register functions, enable triggers, and scale transparently The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
8. FaaS: A Simple Example Upload Cloud Storage Generate thumbnail Save thumbnail Save path Cloud Storage Cloud Database Cloud Server The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
9. Why serverless? • Today’s cloud abstractions are hard to use • Application deployment tools have improved significantly • They are oriented towards infrastructure engineers, not application developers The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
10. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Academic Interest in Serverless
11. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Academic Interest in Serverless
12. Industrial Interest in Serverless The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
13. Industrial Interest in Serverless The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
14. The State of FaaS The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
15. What is FaaS good at today? f() f() f() f() Embarrassingly parallel tasks Workflow orchestration The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
16. Wait! What about… • There are other serverless services, too! • e.g., Google Cloud Dataflow, AWS Athena, Snowflake… • Often referred to as Backend-as-a-Service (BaaS) We’re primarily interested in generality. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
17. No specialized hardware Limited execution lifetimes Limitations on FaaS Today Limited execution lifetimes No inbound network connections x IO is a bottleneck No specialized hardware The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
18. No specialized hardware Limited execution lifetimes Limitations on FaaS Today No inbound network connections x IO is a bottleneck The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
19. But that’s okay: Everything is functional! • Functional programs don’t have side effects or mutable state! • And it is called AWS Lambda The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
20. But that’s okay: Everything is functional! • Functional programs don’t have side effects or mutable state! • And it is called AWS Lambda NOT NOT The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
21. Dysfunction-as-a-Service • FaaS is not designed for functional programming because real applications share state • FaaS is poorly suited for all of these The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
22. Quantifying The Pain of FaaS How FaaS Disappoints Famous Computer Scientists The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
23. Even Functional Programming is Slow! Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 737 573 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
24. Even Functional Programming is Slow! Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 79.6 737 573 178 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
25. Even Functional Programming is Slow! Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 285 239 79.6 569 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
26. Shared Mutable State The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
27. Shared Mutable Storage The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
28. Ideal Serverless Storage Autoscaling Low Latency The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
29. Ideal Serverless Storage Autoscaling Low Latency Tradeoff! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
30. (In)Consistency Guarantees Shared Counter The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
31. (In)Consistency Guarantees Shared Counter ? The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
32. No Inbound Network Connections Easy Fault Tolerance Enables Process Migration The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
33. Indirect Communication Write Read The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
34. Indirect Communication Write Read The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
35. We can fix that! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
36. A Platform for Stateful Serverless Computing The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
37. Background: Anna • High performance across orders of magnitude in scale ü10x faster than Redis/Cassandra in a geo-distributed deployment • Autoscaling & cost-efficient ü500x faster than Amazon DynamoDB for the same cost Chenggang Wu, Jose Faleiro, Yihan Lin, and Joseph M. Hellerstein. "Anna: A KVS for Any Scale." IEEE Transactions on Knowledge and Data Engineering (2019). Chenggang Wu, Vikram Sreekanti, and Joseph M. Hellerstein. "Autoscaling Tiered Cloud Storage in Anna." Proceedings of the VLDB Endowment 12, no. 6 (2019): 624-638. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
38. Fluent: FaaS-over-Anna • Maintain disaggregation of compute & state • Make serverless a viable option for stateful applications • Use Anna for both storage and communication Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
39. Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
40. Logical disaggregation with physical colocation The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
41. Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
42. Key Idea: Caching • Enable low-latency data access by caching data close to code execution • Communication (and composition) is achieved via a fast-path on top of KVS puts and gets The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
43. Challenge: Cache Consistency • tl;dr: we can provide a variety of coordination-avoiding consistency modes (e.g., causal consistency) – which is better than S3 or DynamoDB! • This is done by encapsulating program state in lattices • Happy to discuss this in more detail later The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
44. Function Composition, Revisited Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda and Fluent. 1 10 100 1000 Fluent Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 3.59 285 239 79.6 569 6.37 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
45. Case Study: Prediction Serving The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
46. Prediction Serving • Generate predictions from pretrained machine learning models • At first blush, a great fit for serverless infrastructure Clean Input Join with Reference Data Model Replica Model Replica Model Replica Combine Results The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
47. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
48. Background: SqueezeNet • State-of-the-art image classification model (developed at Berkeley!) The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
49. Prediction Serving Median and 99th percentile latencies for SqueezeNet on Fluent and AWS SageMaker. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 0 50 100 150 200 250 300 350 400 Fluent Python AWS SageMaker Latency (ms) 90.6 153.8 282.2 122.9 156.3 361.7
50. The Future of Cloud Programming The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
51. Looking Back: Disappointed Computer Scientists • Functional programming is slow • Communication through slow storage • Poor consistency guarantees The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
52. Making FaaS Functional • Embrace state • Easy things become better • Hard things become easy A step on our road towards a programmable cloud. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
53. Our Vision • Serverless will change the way that we write software and the way that programming infrastructure works • Cloud-native programming models • Enable users to take advantage of millions of cores and petabytes of RAM The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
54. Moving Forward from FaaS Developing Autoscaling Policy Designing SLOs & SLAs The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Building Developer Tools
55. Thanks! @vsreekan= @cgwu0530 vikrams@berkeley.edu cgwu@berkeley.edu fluent-project/fluent The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary