Serverless Architectures - Where have all the servers gone?

Serverless architectures refer to cloud applications that depend substantially on 3rd party services (Backend as a Service, BaaS) or on custom code that is run in ephemeral deployment units (Function as a Service, FaaS). By moving much behavior to the front end, such architectures reduce the need for ‚always on‘ servers. Therefore, such systems can reduce operational cost and shift operational complexity to BaaS service providers at cost of vendor dependencies and (still) immaturity of supporting services and tools.
This presentation explains the term "Serverless" and how it is changing cloud application architectures. It identifies open issues, benefits and drawbacks, as well as (in-)appropriate use cases for Serverless. It closes with a curated list of Serverless services, standalone platforms and frameworks and provides a list for further reading.

Transcript

1. Serverless Architectures Where have all the servers gone? Nane Kratzke
2. TL;DR • Serverless architectures refer to applications that depend substantially on 3rd party services (Backend as a Service, BaaS) • or on custom code that is run in ephemeral containers (Function as a Service, FaaS). • By moving much behavior to the front end, such architectures reduce the need for ‚always on‘ servers. • Therefore, such systems can reduce operational cost and shift operational complexity to BaaS service providers • at cost of vendor dependencies and (still) immaturity of supporting services and tools. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 2
3. Outline o What is Serverless? o How is Serverless changing architectures? o Benefits and drawbacks o (In-)appropriate use cases o Open issues o Serverless frameworks Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 3
4. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 4 What is Serverless?
5. What is Serverless? Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 5 The term Serverless application is used to describe (rich client like) applications that significantly depend on 3rd party (cloud) services. • These (3rd party) services are sometimes called Backend as a Service (BaaS). • A good example is the Single Sign On Service (Auth0, https://auth0.com) However, BaaS service logic can be implemented serverless as well. • This logic is run in stateless deployment units (often containers) that are event-triggered, ephemeral and fully managed by a 3rd party. • This logic is called Functions as a Service (FaaS). • The most popular service provider for FaaS is currently and probably AWS Lambda. BaaS is about using services in Serverless architectures. FaaS is about realizing services in a Serverless way.
6. Bare Metal Server The Serverless Evolution Where have all the servers gone? 6 VM Bare Metal Server Bare Metal Server A VM A B B Bare Metal Server VM VM Container Engine A B Bare Metal Server VM VM Container Engine FaaS Runtime A B ... ... A ... Virtualization Containerization Time- Sharing Dedicated Server „In recent times“ applications have been deployed to dedicated servers. In consequence, the servers were often overdimensioned and had very inefficient utilization rates. The question arised how to increase application density without touching the application design itself. Machine virtualization is mainly used to consolidate and isolate applications that otherwise would have been deployed to dedicated servers. This increases the application density on bare metal servers but the virtual machine images (deployment unit) are very large. However, the application can stay untouched. If this model is provided as a cloud service it is called Infrastructure as a Service (IaaS). To pragmatically operate more than one application per virtual machine, containerization established as a trend (Docker). A container starts faster than a virtual machine and shares the operating system with other containers, thus reducing deployment unit sizes and increasing application density per virtual machine. But the application should follow cloud architectural styles (like 12-factor-app, microservices) to fully leverage the opportunities. But a container still requests a share of CPU, memory, and storage – even if the provided service is hardly requested. It would be more ressource efficient, if services would consume ressources only if there are incoming requests. This is what FaaS runtime environments provide. So, services can timeshare a host, thus further increasing application density per host. However, this involves to follow a more rich client architecture model for the user-interface and a service- composed-of-functions approach for services. 1 2 3 4 Serverless Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems
7. Is Serverless PaaS? Are Serverless applications just another form of Platform as a Service (PaaS) like Heroku? • The key operational difference between FaaS and PaaS is scaling. • With most PaaS you still need to think about scale on a level of execution instances (dynos, machines, containers, etc.). • FaaS scaling is fine grained and completely transparent. It works on a level of individual service requests. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 7 Adrian Cockcroft VP Cloud Architecture Strategy at AWS „If your PaaS can efficiently start instances in 20ms that run for half a second, then call it Serverless.“
8. OK, what is about containers? Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 8 The argument made for PaaS holds with containers. • FaaS scaling is fine grained and completely transparent. It works on a level of individual service requests. • Most container platforms do not offer such solutions (although Kubernetes is tending towards this level providing concepts like Horizontal Pod Autoscaling). • FaaS vs. Containers? It is more a question about when to use what? Mike Roberts Serverless Expert, Founder, and Co-Author of What is Serverless?, O´Reilly „[…] FaaS is seen as a better choice for event-driven style […] and containers are seen as better choice for synchronous-request driven components […]“
9. Function as a Service (FaaS) • FaaS is about running backend code without managing own servers. • FaaS offerings do not require coding to a specific framework or library. FaaS functions can be mostly implemented as „first class“ programs in JavaScript, Python, any JVM language (Java, Clojure, Scala, …), and more languages. • Deployment is different compared with traditional systems - just upload the code to a FaaS provider and it does anything else. • Horizontal scaling is completely automatic, elastic and managed by the provider. • Functions in FaaS are triggered by event types defined by the provider, this might be file updates, scheduled events (time), messages on a message bus, or simply HTTP requests. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 9
10. Function as a Service We have to consider STATE and DURATION FaaS functions are stateless • They provide pure functional transformations of their input • or they have to use of a database, cross-application cache (e.g. Redis), or object storage (e.g. S3) to store state across requests. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 10 FaaS functions have timeouts • E.g. AWS Lambda allows no functions to execute longer than 5 minutes. • So, certain long lived tasks are not suited to FaaS functions.
11. Function as a Service We have to consider STARTUP LATENCY FaaS function respond times depend on a number of factors • and maybe anywhere between milliseconds and minutes (in very worst cases). • You should consider the overhead of starting a potential necessary runtime environment for your function code. • E.g.: JavaScript and Python are known to spun up faster than a JVM. • Latencies can get longer, especially if • a function processes events infrequently (e.g. more than 10 minutes between invocations) • or sudden spikes in traffic occur (normally 10 requests per second, but suddenly 1000 reqs/sec). • It is likely that in these cases the FaaS provider must start additional (or the first) container instances which involves longer startup times. 11 So, a low-latency trading application or a missile control system might be no appropriate use case for Serverless!!!
12. How is Serverless Changing Architectures? Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 12
13. Serverless by Example Let us start with a traditional non-serverless architecture Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 13 Client (Browser) Application Server Relational Database • Think about a traditional 3- tier client-oriented system with server-side logic. • A good example is a typical ecommerce app. • Using such an architecture the client can be relatively unintelligent. • Most of the logic will be based in the application server. So, how will Serverless change this architecture?
14. Serverless by Example Now, let us make it a Serverless architecture Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 14 Purchase Function Search Function API Gateway Authentication Service Purchase Database Product Database Native mobile app 1 3 2 4 5 The authentication logic can be replaced with a 3rd party authentication BaaS (like Auth0). The client is allowed direct access to a subset of our database. The database is fully 3rd party hosted. Server application logic now moves to the client application, making it often a native mobile app or a single-page web application. Some functionality might be kept in the „server“. It might be compute intensive or requires access to a significant amount of data like a search function. Such functionality is provided as FaaS functions that often respond to HTTP requests. Some functionality might be kept in the „server“ for security reasons or for interfacing further 3rd party BaaS. 6 An API Gateway is basically a web server that receives HTTP requests and routes them to subsequent FaaS functions or other backend services. So, service complexity is reduced at the cost of a more complex service-of-service architecture. Complexity is not reducable, it can be shifted, capsuled, managed, ... – but never reduced! If you reduce complexity, you define a different – simpler – problem!
15. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 15 Benefits and Drawbacks
16. Benefits • Reduced cost due to intensive timesharing of infrastructure and implicit sharing of the operational staff • Reduced development costs due to intensive use of BaaS (services that already exist and not have to be implemented again and again) • Easier operational management because the scaling is automatic • Reduced packaging and deployment efforts • Better time to market • Opportunities for experiments Maybe even „greener computing“? OK, no one proofed that, so far … (but there are some reasonable arguments for it) Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 16
17. Repetition of client logic • Serverless makes it easier to reuse logic on the service side. • But this involves very often similar and repetitive implementations on the client side. Inherent Drawbacks Vendor control • Vendor lock-in • Loss of server optimizations Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 17 Security concerns • Multitenancy vulnerabilities • Increased surface • Losing the protective barrier of a server-side application No in-server state for FaaS • No in-memory or in-process cacheing • External cacheing solutions like Redis or Memcached are compensating options but a order of magnitude slower.
18. Implementation Drawbacks Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 18 Do not Denial-of-Service yourself Typically your number of concurrent executions of functions is limited. This limit is applied to your account which you may use for production and testing. So testing may surprisingly affect your production deployments. Consider execution durations and latencies Remember, the executation duration of functions is limited and long lived tasks are not suited for FaaS. Startup latencies affect respond times of FaaS functions. Especially JVM-implemented functions tend to show suddenly high latencies, especially if events occur infrequently or as sudden spikes. Do not underestimate integration testing Unit testing is fairly simple. But integration testing of Serverless can be complicated. That is because the units of integration (functions) are a lot smaller and therefore Serverless architectures normally rely on integration testing a lot more than other architectural styles.
19. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 19 (In-)appropriate Use Cases
20. It is always about the use case Stupid! • As we have seen, Serverless architectures come with benefits and drawbacks. • So, some use cases are more suited for Serverless approaches than others. • The question is, which use cases look promising and which use cases do not? Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 20 Mike Roberts „There is a lot to like about Serverless architectures […], but they come with significant trade-offs. Some of these are inherent […] and can‘t be fixed. Others […] we could expect to be resolved.“
21. Remember the implementation drawbacks • Multitenancy performance A high load user or even your testing can cause other users to slow down. • Repetition of logic across client platforms This might increase development efforts for the client side. • Stateless functions Therefore FaaS seems suboptimal to realize stateful (database-like) services. • No in-process state and cacheing This might cause higher latencies. • Limitations of execution durations This might prevent long running tasks like streaming or analysis. • Startup latency This might result in long respond times especially for very infrequent requested services. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 21
22. And the ugly … ? happens whenever you try to apply Serverless to bad use cases. The good, the bad, and the ugly By use case Good use cases • User group with an uniformly distributed request volume. • Applications with a limited set of client devices. • User prefers native apps. • Stateless services. • Varying respond times can be tolerated. • Short running requests (no batch- like jobs) • No extreme low-latency or even hard real-time requirements Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 22 Bad use cases • User group with spikes in request volumes. • Necessity to support an unlimited set of client devices. • User prefers web access via browser. • Stateful services. • Respond times are low-latency and must be assured in a defined range. • Long running batch-like jobs. • Low-latency or hard real-time requirements known from low- latency trading or real-time control systems.
23. And the ugly example? Well, try to implement a FaaS video streaming function. The result will be ugly! The good, the bad, and the ugly By example Good examples • Single image processing • Videosharing • Social network sharing features • Single entity categorization using a trained neural network • Event-based processing of social media streams • (short running) database querying • Messaging Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 23 Bad examples • Batch-like image processing • Videoprocessing/-streaming • Large scale social network analysis • Neural network training • Batch-like entity categorization • Continuous observing of social media streams • Database-like storage service • Persistant message bus
24. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 24 Open Issues
25. Serverless is (still) immature Sad, but this leaves room for improvements • As we have seen, Serverless architectures have some open issues. • That is not perfect and should be considered. • However, the following points are likely to be tackled in the future. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 25 Mike Roberts „The remaining drawbacks […] are down purely to the current state of the art. With inclination […] and a heroic community these can be all wiped out.“
26. Open issues Improvements for tooling, in-process-state, platform features Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 26 Tooling There is the need for more mature and more integrated deployment, application bundling, configuration, monitoring & logging, and debugging tools. It is likely that these tools will be better integrated with future FaaS runtime environments. State To avoid in-process-state is astonishing hard to accept for a lot of developpers. This may even a show- stopper for several use cases. Better integrations with out-of-process data solutions like Redis or memcached would be helpful. Platforms Current FaaS platforms come with limitations regarding execution duration of functions, startup latency, and non- separation of execution limits. It is likely that these limitations will be attentuated in the future.
27. Open issues Finding services and proven solution patterns Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 27 Service Discovery There are no well defined or standardized solutions for service discovery of FaaS functions. This is even getting worse due to the fine granular nature of FaaS and a lack of application and versioning definition. Serverless Patterns Serverless is about to avoid ‚always-on‘ components. But ‚always-on‘ will ever be necessary. So, Serverless is typically applied as part of an hybrid architecture. How to do that is an open issue, as well as proven patterns for common use cases like media processing.
28. Open issues Operation still needs monitoring, testing, and debugging Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 28 Monitoring and debugging Serverless architectures rely on the monitoring and debugging side with whatever the vendor provides. This support is often very basic. Open and standardized APIs would be helpful to integrate more sophisticated 3rd party services. Integration testing Serverless involves to work with smaller units (functions) that are easier to unit test. However, this involves more complex integration. So, pragmatic and efficient integration testing is especially essential for microservice and Serverless approaches.
29. Serverless Frameworks
30. Serverless Services, Platforms, and Frameworks Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 30 Public FaaS Cloud Services Most public cloud service providers offer Serverless compute services, also known as function as a service (FaaS). Currently there exist no FaaS standard and Vendor Lock-In is likely. Standalone Serverless Platforms Due to missing standards public FaaS cloud services are prone to create vendor lock-in. Open Serverless platforms might be an alternative. But these platforms need infrastructures for operation. Platform Agnostic Serverless Frameworks Platform agnostic frameworks provide a provider and platform agnostic way to define and deploy Serverless code on various Serverless platforms or FaaS cloud services. • AWS Lambda https://aws.amazon.com/lambda • Google Cloud Functions https://cloud.google.com/functions • Azure Functions https://azure.microsoft.com/services /functions • OpenWhisk https://openwhisk.apache.org • Nuclio https://nuclio.io • Fn project https://fnproject.io • OpenFaaS https://www.openfaas.com • Serverless Framework https://serverless.com • Squeezer Framework https://squeezer.io/framework • SpringCloud Functions https://cloud.spring.io/spring- cloud-function This list is not complete! You will find curated and continuously updated lists here: • http://bit.ly/2rBWzXr • http://bit.ly/2FcKSbF
31. Further Reading • Serverless Architectures by Mike Roberts, 2016 https://www.martinfowler.com/articles/serverless.html; This blog post was one of the most influencing references for this presentation. • What Is Serverless? by John Chapin, Mike Roberts, O‘Reilly, 2017; You can get this ebook for free from O‘Reilly. The authors take you through the Serverless landscape: design considerations, tooling, and approaches to operational management. • Why the Future Is Serverless? by Ken Fromm, Badri Janakiraman, 2012 http://readwrite.com/2012/10/15/why-the-future-of-software- and-apps-is-serverless ; Might be one of the first articles using the term Serverless in its current meaning. • The State of The Serverless Ecosystem by Tal KimHi, 2017, https://medium.com/@talkimhi/the-state-of-the-serverless-ecosystem- c8a8b5ca56ae; This post tries to gather every company, product, tool, or framework that has something to do with the Serverless paradigm. To reach this goal seems impossible, but it is simply an awesome work. • Serverless Computing by Wikipedia, https://en.wikipedia.org/wiki/Serverless_computing, A Wikipedia link is a must. But remember: Wikipedia is a good start for a search, it should never be the end. Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 31
32. Acknowledgement Picture Reference • Cloud: Pixabay (CC0 Public Domain) • Defintion: Pixabay (CC0 Public Domain) • Serverrack: Pixabay (CC0 Public Domain) • Smileys: Pixabay (CC0 Public Domain) • Peanuts: Pixabay (CC0 Public Domain) • Question marks: Pixabay (CC0 Public Domain) • Lego bricks: Pixabay (CC0 Public Domain) • All icons: The Noun Project (CC-BY-2.0 License) Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 32 This contribution resulted as a side-effect from research that is funded by German Federal Ministry of Education and Research (Project Cloud TRANSIT, 13FH021PX4). Speaker Deck URL
33. About Prof. Dr. rer. nat. Nane Kratzke Computer Science and Business Information Systems 33 Nane Kratzke CoSA: http://cosa.fh-luebeck.de/en/contact/people/n-kratzke Blog: http://www.nkode.io Twitter: @NaneKratzke GooglePlus: +NaneKratzke LinkedIn: https://de.linkedin.com/in/nanekratzke GitHub: https://github.com/nkratzke ResearchGate: https://www.researchgate.net/profile/Nane_Kratzke Speaker Deck: https://speakerdeck.com/nkratzke