Share
Embed
Transcript
55 slides
-
The State of Serverless Computing or, Fixing Dysfunction-as-a-Service Vikram Sreekanti,
Chenggang Wu RISE Lab, UC Berkeley Craft Conference – Budapest, Hungary – 05/09/2019 -
5th Graders Learning Programming San Diego, CA The State of
Serverless Computing Craft Conference 2019 – Budapest, Hungary -
UC Berkeley: CS 61A Fall 2018 The State of Serverless
Computing Craft Conference 2019 – Budapest, Hungary -
Making Programmers Productive Key Question: Where will code be run?
In the ☁! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Background: Serverless Computing The State of Serverless Computing Craft Conference
2019 – Budapest, Hungary -
What is serverless computing? Serverless computing is a programming abstraction
that enables users to upload programs, run them at any scale, and pay only for resources used. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Functions-as-a-Service • AWS Lambda, Google Cloud Functions, OpenWhisk (IBM), Azure
Functions, OpenLambda, OpenFaaS, kNative… • Optimized for simplicity – register functions, enable triggers, and scale transparently The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
FaaS: A Simple Example Upload Cloud Storage Generate thumbnail Save
thumbnail Save path Cloud Storage Cloud Database Cloud Server The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Why serverless? • Today’s cloud abstractions are hard to use
• Application deployment tools have improved significantly • They are oriented towards infrastructure engineers, not application developers The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
The State of Serverless Computing Craft Conference 2019 – Budapest,
Hungary Academic Interest in Serverless -
The State of Serverless Computing Craft Conference 2019 – Budapest,
Hungary Academic Interest in Serverless -
Industrial Interest in Serverless The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
Industrial Interest in Serverless The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
The State of FaaS The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
What is FaaS good at today? f() f() f() f()
Embarrassingly parallel tasks Workflow orchestration The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Wait! What about… • There are other serverless services, too!
• e.g., Google Cloud Dataflow, AWS Athena, Snowflake… • Often referred to as Backend-as-a-Service (BaaS) We’re primarily interested in generality. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
No specialized hardware Limited execution lifetimes Limitations on FaaS Today
Limited execution lifetimes No inbound network connections x IO is a bottleneck No specialized hardware The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
No specialized hardware Limited execution lifetimes Limitations on FaaS Today
No inbound network connections x IO is a bottleneck The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
But that’s okay: Everything is functional! • Functional programs don’t
have side effects or mutable state! • And it is called AWS Lambda The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
But that’s okay: Everything is functional! • Functional programs don’t
have side effects or mutable state! • And it is called AWS Lambda NOT NOT The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Dysfunction-as-a-Service • FaaS is not designed for functional programming because
real applications share state • FaaS is poorly suited for all of these The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Quantifying The Pain of FaaS How FaaS Disappoints Famous Computer
Scientists The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Even Functional Programming is Slow! Median and 99th percentile latencies
for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 737 573 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Even Functional Programming is Slow! Median and 99th percentile latencies
for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 79.6 737 573 178 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Even Functional Programming is Slow! Median and 99th percentile latencies
for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 285 239 79.6 569 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Shared Mutable State The State of Serverless Computing Craft Conference
2019 – Budapest, Hungary -
Shared Mutable Storage The State of Serverless Computing Craft Conference
2019 – Budapest, Hungary -
Ideal Serverless Storage Autoscaling Low Latency The State of Serverless
Computing Craft Conference 2019 – Budapest, Hungary -
Ideal Serverless Storage Autoscaling Low Latency Tradeoff! The State of
Serverless Computing Craft Conference 2019 – Budapest, Hungary -
(In)Consistency Guarantees Shared Counter The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
(In)Consistency Guarantees Shared Counter ? The State of Serverless Computing
Craft Conference 2019 – Budapest, Hungary -
No Inbound Network Connections Easy Fault Tolerance Enables Process Migration
The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Indirect Communication Write Read The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
Indirect Communication Write Read The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
We can fix that! The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
A Platform for Stateful Serverless Computing The State of Serverless
Computing Craft Conference 2019 – Budapest, Hungary -
Background: Anna • High performance across orders of magnitude in
scale ü10x faster than Redis/Cassandra in a geo-distributed deployment • Autoscaling & cost-efficient ü500x faster than Amazon DynamoDB for the same cost Chenggang Wu, Jose Faleiro, Yihan Lin, and Joseph M. Hellerstein. "Anna: A KVS for Any Scale." IEEE Transactions on Knowledge and Data Engineering (2019). Chenggang Wu, Vikram Sreekanti, and Joseph M. Hellerstein. "Autoscaling Tiered Cloud Storage in Anna." Proceedings of the VLDB Endowment 12, no. 6 (2019): 624-638. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Fluent: FaaS-over-Anna • Maintain disaggregation of compute & state •
Make serverless a viable option for stateful applications • Use Anna for both storage and communication Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing
Craft Conference 2019 – Budapest, Hungary -
Logical disaggregation with physical colocation The State of Serverless Computing
Craft Conference 2019 – Budapest, Hungary -
Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing
Craft Conference 2019 – Budapest, Hungary -
Key Idea: Caching • Enable low-latency data access by caching
data close to code execution • Communication (and composition) is achieved via a fast-path on top of KVS puts and gets The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Challenge: Cache Consistency • tl;dr: we can provide a variety
of coordination-avoiding consistency modes (e.g., causal consistency) – which is better than S3 or DynamoDB! • This is done by encapsulating program state in lattices • Happy to discuss this in more detail later The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Function Composition, Revisited Median and 99th percentile latencies for composing
two arithmetic functions on AWS Lambda and Fluent. 1 10 100 1000 Fluent Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 3.59 285 239 79.6 569 6.37 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Case Study: Prediction Serving The State of Serverless Computing Craft
Conference 2019 – Budapest, Hungary -
Prediction Serving • Generate predictions from pretrained machine learning models
• At first blush, a great fit for serverless infrastructure Clean Input Join with Reference Data Model Replica Model Replica Model Replica Combine Results The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Background: SqueezeNet • State-of-the-art image classification model (developed at Berkeley!)
The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Prediction Serving Median and 99th percentile latencies for SqueezeNet on
Fluent and AWS SageMaker. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 0 50 100 150 200 250 300 350 400 Fluent Python AWS SageMaker Latency (ms) 90.6 153.8 282.2 122.9 156.3 361.7 -
The Future of Cloud Programming The State of Serverless Computing
Craft Conference 2019 – Budapest, Hungary -
Looking Back: Disappointed Computer Scientists • Functional programming is slow
• Communication through slow storage • Poor consistency guarantees The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Making FaaS Functional • Embrace state • Easy things become
better • Hard things become easy A step on our road towards a programmable cloud. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Our Vision • Serverless will change the way that we
write software and the way that programming infrastructure works • Cloud-native programming models • Enable users to take advantage of millions of cores and petabytes of RAM The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary -
Moving Forward from FaaS Developing Autoscaling Policy Designing SLOs &
SLAs The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Building Developer Tools -
Thanks! @vsreekan= @cgwu0530 [email protected] [email protected] fluent-project/fluent The State of Serverless
Computing Craft Conference 2019 – Budapest, Hungary