Cloud Computing Made Better: An AWS View on Serverless Computing (Marvin Theimer, Amazon Web Services)

In 2019 the Berkeley RISELab published a paper giving its perspective on the emerging topic of serverless computing. Two years have now gone by and serverless computing has only grown in importance. In this talk I will discuss the predictions and conclusions made in that and other such papers from the perspective of what AWS has done in this space and how we see the future of serverless computing.

Transcript

1. Cloud Computing Made Better: An AWS View on Serverless Computing Marvin Theimer
2. The Berkeley view of serverless •“Simplified programming environment will make the cloud much easier to use” •Serverless == FaaS + BaaS •Prediction: serverless use will skyrocket •But there are still challenges: • Inadequate storage options • Inappropriate & unpredictable communication/coordination functionality • Limitations on compute capabilities • Security concerns
3. Serverless computing is popular •More than half of all new applications built at Amazon are now built with AWS Lambda •AWS customers are making trillions of invocations per month •Some AWS customers are making tens of thousands of concurrent invocations
4. Serverless == FaaS + BaaS ? What’s a function? • Lambda functions vs. Fargate containers • Step Functions, map/reduce functions, API-Gateway functions? • SQL queries? • How does Infrastructure-as-Code fit in? What’s a serverless backend service? • S3 • DynamoDB with provisioned IOPs; Kinesis with provisioned shards? • Athena vs. Redshift (Spectrum) • EC2 auto-scaling group?
5. Serverless == simple, uniform environment •Premise: the power and attraction of serverless is that it presumes a world that is: • Flat – everything is distance “1” away • Infinite – no limits • Immortal – nothing fails •Challenge: making the real world appear as if it obeyed this ideal • Oftentimes you don’t care – the world is flat enough, big enough, and robust enough for your needs • The world is getting better – the cloud is far more flat, far more elastic, and far more robust than what we used to have • BaaS can hide a lot of the complexity – but not all of it
6. What’s fundamental and what’s incidental • Fundamental: • There is no free lunch – the world is not flat, infinite, and immortal • Entitlement concept – given a particular problem and compute environment: • Cost entitlement: cost of the cheapest possible solution • Performance entitlement: performance of the most performant possible solution • Simplicity entitlement: the minimum resource management information that must be specified to implement a solution • CPS entitlement trade-off: cost, performance, simplicity – pick any two? • The real question: “who has to deal with the complexity?” • Incidental: • All the serverless challenges listed earlier
7. “Make it so” – delegating complexity •Delegation to: • Other humans – BaaS • Simple APIs hide enormous complexity • Humans handle design, implementation, and operation • Tools like compilers and query/workflow planners • High-level languages (e.g., Java, StepFunctions, SQL) translate to enormously complicated low-level programs •So, we’re done, right? Not really
8. Even simple interfaces have their limits •Garbage collection example: • It’s much easier to write a functionally correct program with GC • It’s hard to tune the program’s GC behavior if cost & performance matter •Web service example: • Svc.f(x) -> y is simple to specify for the happy case • It’s hard to deal with all the exception cases that can occur •DynamoDB example: • Functional API is simple and easy to use • Managing provisioned IOPs capacity is hard for complex workloads • On-demand IOPs management isn’t free (but may be cheaper than poor human management of IOPs)
9. The bigger challenge: playing well with others •Impedance matching – e.g., cascaded, per-workload, auto-scaling •Bottlenecks •Retries •Poison pills and dead letter queues •Delegated access •Distributed logging •BaaS 🡪 BaaS + Tools/Frameworks
10. Tools work best in well-defined environments • Compilers and query/workflow planners hide enormous complexity • They work best when targeting well-defined, constrained environments that don’t evolve too quickly • The cloud has a much bigger, less well-defined, rapidly evolving surface area • The functional interface is well-defined • The non-functional aspects of the cloud are much less well-defined • Tools do best with “constrained” complexity • The cloud has lots of “messy” complexity
11. Multi-tenancy: the opposite of a well-defined environment • Cost optimization => multi-tenancy • Multi-tenancy => unpredictable, noisy neighbor problems => CPS trade-off • The solution: • Everyone needs to tell the system “enough” about their applications & workloads • This information must propagate to all relevant parts of the system • The challenge: • How much information is “enough” and who needs to know it? • Can ML-based approaches help with messy complexity? • How much can the system observe and learn without having someone explicitly tell it about?
12. Dealing with (incidental) challenges – UCB list • Inadequate storage options • More BaaS • Inappropriate & unpredictable communication/coordination functionality • Higher level constructs/frameworks • Limitations on compute capabilities • Better underpinnings: e.g., micro-VMs, access to specialized hardware • Systemic exploitation of application-level details (e.g., micro-VM snapshots, intelligent segregation of workloads) • Security concerns • Developer-written code 🡪 operator/security engineer written, declarative config
13. An additional challenge: constraining unbounded, unintended behavior •Huge elasticity means you can consume a lot of resources very quickly •Asynchronous, ephemeral execution means you may not notice a problem till much later •Need to be able to specify appropriate budgets & guardrails
14. Putting it all together •FaaS 🡪 FaaS + managed containers + managed micro-services + … •BaaS building blocks •Higher-level constructs, frameworks, and eco-systems to combine things and enable use of planners and optimizers •How much can the system learn vs. my having to tell it?
15. Conclusions • “Serverless” computing is indeed skyrocketing in popularity • Simplifying the programing environment is indeed making cloud computing better • Production systems are getting better (simpler), but we still have a ways to go • Simplicity != serverless (Faas + Baas) • Simplicity == specifying the least amount of information to get the desired job done • Simplicity is bounded by the complexity of the most complex component and the complexity of the most complex interaction among components • Serverless computing 🡪 expressive computing ?