The State of Serverless Computing or, Fixing Dysfunction-as-a-Service Vikram Sreekanti, Chenggang Wu RISE Lab, UC Berkeley Craft Conference – Budapest, Hungary – 05/09/2019 

5th Graders Learning Programming San Diego, CA The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

UC Berkeley: CS 61A Fall 2018 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Making Programmers Productive Key Question: Where will code be run? In the ☁! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Background: Serverless Computing The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

What is serverless computing? Serverless computing is a programming abstraction that enables users to upload programs, run them at any scale, and pay only for resources used. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Functions-as-a-Service • AWS Lambda, Google Cloud Functions, OpenWhisk (IBM), Azure Functions, OpenLambda, OpenFaaS, kNative… • Optimized for simplicity – register functions, enable triggers, and scale transparently The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

FaaS: A Simple Example Upload Cloud Storage Generate thumbnail Save thumbnail Save path Cloud Storage Cloud Database Cloud Server The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Why serverless? • Today’s cloud abstractions are hard to use • Application deployment tools have improved significantly • They are oriented towards infrastructure engineers, not application developers The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Academic Interest in Serverless 

The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Academic Interest in Serverless 

Industrial Interest in Serverless The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Industrial Interest in Serverless The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

The State of FaaS The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

What is FaaS good at today? f() f() f() f() Embarrassingly parallel tasks Workflow orchestration The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Wait! What about… • There are other serverless services, too! • e.g., Google Cloud Dataflow, AWS Athena, Snowflake… • Often referred to as Backend-as-a-Service (BaaS) We’re primarily interested in generality. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

No specialized hardware Limited execution lifetimes Limitations on FaaS Today Limited execution lifetimes No inbound network connections x IO is a bottleneck No specialized hardware The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

No specialized hardware Limited execution lifetimes Limitations on FaaS Today No inbound network connections x IO is a bottleneck The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

But that’s okay: Everything is functional! • Functional programs don’t have side effects or mutable state! • And it is called AWS Lambda The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

But that’s okay: Everything is functional! • Functional programs don’t have side effects or mutable state! • And it is called AWS Lambda NOT NOT The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Dysfunction-as-a-Service • FaaS is not designed for functional programming because real applications share state • FaaS is poorly suited for all of these The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Quantifying The Pain of FaaS How FaaS Disappoints Famous Computer Scientists The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Even Functional Programming is Slow! Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 737 573 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Even Functional Programming is Slow! Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 79.6 737 573 178 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Even Functional Programming is Slow! Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 285 239 79.6 569 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Shared Mutable State The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Shared Mutable Storage The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Ideal Serverless Storage Autoscaling Low Latency The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Ideal Serverless Storage Autoscaling Low Latency Tradeoff! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

(In)Consistency Guarantees Shared Counter The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

(In)Consistency Guarantees Shared Counter ? The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

No Inbound Network Connections Easy Fault Tolerance Enables Process Migration The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Indirect Communication Write Read The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Indirect Communication Write Read The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

We can fix that! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

A Platform for Stateful Serverless Computing The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Background: Anna • High performance across orders of magnitude in scale ü10x faster than Redis/Cassandra in a geo-distributed deployment • Autoscaling & cost-efficient ü500x faster than Amazon DynamoDB for the same cost Chenggang Wu, Jose Faleiro, Yihan Lin, and Joseph M. Hellerstein. "Anna: A KVS for Any Scale." IEEE Transactions on Knowledge and Data Engineering (2019). Chenggang Wu, Vikram Sreekanti, and Joseph M. Hellerstein. "Autoscaling Tiered Cloud Storage in Anna." Proceedings of the VLDB Endowment 12, no. 6 (2019): 624-638. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Fluent: FaaS-over-Anna • Maintain disaggregation of compute & state • Make serverless a viable option for stateful applications • Use Anna for both storage and communication Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Logical disaggregation with physical colocation The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Key Idea: Caching • Enable low-latency data access by caching data close to code execution • Communication (and composition) is achieved via a fast-path on top of KVS puts and gets The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Challenge: Cache Consistency • tl;dr: we can provide a variety of coordination-avoiding consistency modes (e.g., causal consistency) – which is better than S3 or DynamoDB! • This is done by encapsulating program state in lattices • Happy to discuss this in more detail later The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Function Composition, Revisited Median and 99th percentile latencies for composing two arithmetic functions on AWS Lambda and Fluent. 1 10 100 1000 Fluent Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 3.59 285 239 79.6 569 6.37 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Case Study: Prediction Serving The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Prediction Serving • Generate predictions from pretrained machine learning models • At first blush, a great fit for serverless infrastructure Clean Input Join with Reference Data Model Replica Model Replica Model Replica Combine Results The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Background: SqueezeNet • State-of-the-art image classification model (developed at Berkeley!) The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Prediction Serving Median and 99th percentile latencies for SqueezeNet on Fluent and AWS SageMaker. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 0 50 100 150 200 250 300 350 400 Fluent Python AWS SageMaker Latency (ms) 90.6 153.8 282.2 122.9 156.3 361.7 

The Future of Cloud Programming The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Looking Back: Disappointed Computer Scientists • Functional programming is slow • Communication through slow storage • Poor consistency guarantees The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Making FaaS Functional • Embrace state • Easy things become better • Hard things become easy A step on our road towards a programmable cloud. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Our Vision • Serverless will change the way that we write software and the way that programming infrastructure works • Cloud-native programming models • Enable users to take advantage of millions of cores and petabytes of RAM The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 

Moving Forward from FaaS Developing Autoscaling Policy Designing SLOs & SLAs The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Building Developer Tools 

Thanks! @vsreekan= @cgwu0530 [email protected] [email protected] fluent-project/fluent The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary