Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The State of Serverless Computing

The State of Serverless Computing

6fb14b984cce53b9229239c83ea94424?s=128

Vikram Sreekanti

May 09, 2019
Tweet

Transcript

  1. The State of Serverless Computing or, Fixing Dysfunction-as-a-Service Vikram Sreekanti,

    Chenggang Wu RISE Lab, UC Berkeley Craft Conference – Budapest, Hungary – 05/09/2019
  2. 5th Graders Learning Programming San Diego, CA The State of

    Serverless Computing Craft Conference 2019 – Budapest, Hungary
  3. UC Berkeley: CS 61A Fall 2018 The State of Serverless

    Computing Craft Conference 2019 – Budapest, Hungary
  4. Making Programmers Productive Key Question: Where will code be run?

    In the ☁! The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  5. Background: Serverless Computing The State of Serverless Computing Craft Conference

    2019 – Budapest, Hungary
  6. What is serverless computing? Serverless computing is a programming abstraction

    that enables users to upload programs, run them at any scale, and pay only for resources used. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  7. Functions-as-a-Service • AWS Lambda, Google Cloud Functions, OpenWhisk (IBM), Azure

    Functions, OpenLambda, OpenFaaS, kNative… • Optimized for simplicity – register functions, enable triggers, and scale transparently The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  8. FaaS: A Simple Example Upload Cloud Storage Generate thumbnail Save

    thumbnail Save path Cloud Storage Cloud Database Cloud Server The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  9. Why serverless? • Today’s cloud abstractions are hard to use

    • Application deployment tools have improved significantly • They are oriented towards infrastructure engineers, not application developers The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  10. The State of Serverless Computing Craft Conference 2019 – Budapest,

    Hungary Academic Interest in Serverless
  11. The State of Serverless Computing Craft Conference 2019 – Budapest,

    Hungary Academic Interest in Serverless
  12. Industrial Interest in Serverless The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  13. Industrial Interest in Serverless The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  14. The State of FaaS The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  15. What is FaaS good at today? f() f() f() f()

    Embarrassingly parallel tasks Workflow orchestration The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  16. Wait! What about… • There are other serverless services, too!

    • e.g., Google Cloud Dataflow, AWS Athena, Snowflake… • Often referred to as Backend-as-a-Service (BaaS) We’re primarily interested in generality. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  17. No specialized hardware Limited execution lifetimes Limitations on FaaS Today

    Limited execution lifetimes No inbound network connections x IO is a bottleneck No specialized hardware The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  18. No specialized hardware Limited execution lifetimes Limitations on FaaS Today

    No inbound network connections x IO is a bottleneck The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  19. But that’s okay: Everything is functional! • Functional programs don’t

    have side effects or mutable state! • And it is called AWS Lambda The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  20. But that’s okay: Everything is functional! • Functional programs don’t

    have side effects or mutable state! • And it is called AWS Lambda NOT NOT The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  21. Dysfunction-as-a-Service • FaaS is not designed for functional programming because

    real applications share state • FaaS is poorly suited for all of these The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  22. Quantifying The Pain of FaaS How FaaS Disappoints Famous Computer

    Scientists The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  23. Even Functional Programming is Slow! Median and 99th percentile latencies

    for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 737 573 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  24. Even Functional Programming is Slow! Median and 99th percentile latencies

    for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step FNs Latency (ms) 285 239 79.6 737 573 178 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  25. Even Functional Programming is Slow! Median and 99th percentile latencies

    for composing two arithmetic functions on AWS Lambda. 1 10 100 1000 Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 285 239 79.6 569 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  26. Shared Mutable State The State of Serverless Computing Craft Conference

    2019 – Budapest, Hungary
  27. Shared Mutable Storage The State of Serverless Computing Craft Conference

    2019 – Budapest, Hungary
  28. Ideal Serverless Storage Autoscaling Low Latency The State of Serverless

    Computing Craft Conference 2019 – Budapest, Hungary
  29. Ideal Serverless Storage Autoscaling Low Latency Tradeoff! The State of

    Serverless Computing Craft Conference 2019 – Budapest, Hungary
  30. (In)Consistency Guarantees Shared Counter The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  31. (In)Consistency Guarantees Shared Counter ? The State of Serverless Computing

    Craft Conference 2019 – Budapest, Hungary
  32. No Inbound Network Connections Easy Fault Tolerance Enables Process Migration

    The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  33. Indirect Communication Write Read The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  34. Indirect Communication Write Read The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  35. We can fix that! The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  36. A Platform for Stateful Serverless Computing The State of Serverless

    Computing Craft Conference 2019 – Budapest, Hungary
  37. Background: Anna • High performance across orders of magnitude in

    scale ü10x faster than Redis/Cassandra in a geo-distributed deployment • Autoscaling & cost-efficient ü500x faster than Amazon DynamoDB for the same cost Chenggang Wu, Jose Faleiro, Yihan Lin, and Joseph M. Hellerstein. "Anna: A KVS for Any Scale." IEEE Transactions on Knowledge and Data Engineering (2019). Chenggang Wu, Vikram Sreekanti, and Joseph M. Hellerstein. "Autoscaling Tiered Cloud Storage in Anna." Proceedings of the VLDB Endowment 12, no. 6 (2019): 624-638. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  38. Fluent: FaaS-over-Anna • Maintain disaggregation of compute & state •

    Make serverless a viable option for stateful applications • Use Anna for both storage and communication Anna The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  39. Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing

    Craft Conference 2019 – Budapest, Hungary
  40. Logical disaggregation with physical colocation The State of Serverless Computing

    Craft Conference 2019 – Budapest, Hungary
  41. Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing

    Craft Conference 2019 – Budapest, Hungary
  42. Key Idea: Caching • Enable low-latency data access by caching

    data close to code execution • Communication (and composition) is achieved via a fast-path on top of KVS puts and gets The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  43. Challenge: Cache Consistency • tl;dr: we can provide a variety

    of coordination-avoiding consistency modes (e.g., causal consistency) – which is better than S3 or DynamoDB! • This is done by encapsulating program state in lattices • Happy to discuss this in more detail later The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  44. Function Composition, Revisited Median and 99th percentile latencies for composing

    two arithmetic functions on AWS Lambda and Fluent. 1 10 100 1000 Fluent Lambda (S3) Lambda (Dynamo) Lambda (Direct) AWS Step Fns Latency (ms) 3.59 285 239 79.6 569 6.37 737 573 178 3346 The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  45. Case Study: Prediction Serving The State of Serverless Computing Craft

    Conference 2019 – Budapest, Hungary
  46. Prediction Serving • Generate predictions from pretrained machine learning models

    • At first blush, a great fit for serverless infrastructure Clean Input Join with Reference Data Model Replica Model Replica Model Replica Combine Results The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  47. The State of Serverless Computing Craft Conference 2019 – Budapest,

    Hungary
  48. Background: SqueezeNet • State-of-the-art image classification model (developed at Berkeley!)

    The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  49. Prediction Serving Median and 99th percentile latencies for SqueezeNet on

    Fluent and AWS SageMaker. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary 0 50 100 150 200 250 300 350 400 Fluent Python AWS SageMaker Latency (ms) 90.6 153.8 282.2 122.9 156.3 361.7
  50. The Future of Cloud Programming The State of Serverless Computing

    Craft Conference 2019 – Budapest, Hungary
  51. Looking Back: Disappointed Computer Scientists • Functional programming is slow

    • Communication through slow storage • Poor consistency guarantees The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  52. Making FaaS Functional • Embrace state • Easy things become

    better • Hard things become easy A step on our road towards a programmable cloud. The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  53. Our Vision • Serverless will change the way that we

    write software and the way that programming infrastructure works • Cloud-native programming models • Enable users to take advantage of millions of cores and petabytes of RAM The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
  54. Moving Forward from FaaS Developing Autoscaling Policy Designing SLOs &

    SLAs The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary Building Developer Tools
  55. Thanks! @vsreekan= @cgwu0530 vikrams@berkeley.edu cgwu@berkeley.edu fluent-project/fluent The State of Serverless

    Computing Craft Conference 2019 – Budapest, Hungary