Serverless Architectures - Where have all the servers gone?

Serverless architectures refer to cloud applications that depend substantially on 3rd party services (Backend as a Service, BaaS) or on custom code that is run in ephemeral deployment units (Function as a Service, FaaS). By moving much behavior to the front end, such architectures reduce the need for ‚always on‘ servers. Therefore, such systems can reduce operational cost and shift operational complexity to BaaS service providers at cost of vendor dependencies and (still) immaturity of supporting services and tools.
This presentation explains the term "Serverless" and how it is changing cloud application architectures. It identifies open issues, benefits and drawbacks, as well as (in-)appropriate use cases for Serverless. It closes with a curated list of Serverless services, standalone platforms and frameworks and provides a list for further reading.

Transcript

1. Serverless
   Architectures
   Where have all the servers gone?
   Nane Kratzke
   

   View Slide

2. TL;DR
   • Serverless architectures refer to
   applications that depend substantially on
   3rd party services (Backend as a Service,
   BaaS)
   • or on custom code that is run in
   ephemeral containers (Function as a
   Service, FaaS).
   • By moving much behavior to the front
   end, such architectures reduce the need
   for ‚always on‘ servers.
   • Therefore, such systems can reduce
   operational cost and shift operational
   complexity to BaaS service providers
   • at cost of vendor dependencies and (still)
   immaturity of supporting services and
   tools.
   Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   2
   

   View Slide

3. Outline
   o What is Serverless?
   o How is Serverless changing
   architectures?
   o Benefits and drawbacks
   o (In-)appropriate use cases
   o Open issues
   o Serverless frameworks
   Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   3
   

   View Slide

4. Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   4
   What is Serverless?
   

   View Slide

5. What is Serverless?
   Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   5
   The term Serverless application is used to describe
   (rich client like) applications that significantly depend
   on 3rd party (cloud) services.
   • These (3rd party) services are sometimes called
   Backend as a Service (BaaS).
   • A good example is the Single Sign On Service
   (Auth0, https://auth0.com)
   However, BaaS service logic can be implemented
   serverless as well.
   • This logic is run in stateless deployment units
   (often containers) that are event-triggered,
   ephemeral and fully managed by a 3rd party.
   • This logic is called Functions as a Service (FaaS).
   • The most popular service provider for FaaS is
   currently and probably AWS Lambda.
   BaaS is about using
   services in Serverless
   architectures.
   FaaS is about realizing
   services in a Serverless
   way.
   

   View Slide

6. Bare Metal Server
   The Serverless Evolution
   Where have all the servers gone?
   6
   VM
   Bare Metal Server Bare Metal Server
   A
   VM
   A B
   B
   Bare Metal Server
   VM VM
   Container
   Engine
   A B
   Bare Metal Server
   VM VM
   Container
   Engine
   FaaS Runtime
   A
   B
   ...
   ...
   A
   ...
   Virtualization
   Containerization
   Time-
   Sharing
   Dedicated Server „In recent times“ applications have been
   deployed to dedicated servers. In consequence, the
   servers were often overdimensioned and had very
   inefficient utilization rates. The question arised
   how to increase application density without
   touching the application design itself.
   Machine virtualization is mainly used to
   consolidate and isolate applications that
   otherwise would have been deployed to
   dedicated servers. This increases the
   application density on bare metal servers but
   the virtual machine images (deployment
   unit) are very large. However, the application
   can stay untouched. If this model is provided
   as a cloud service it is called Infrastructure as
   a Service (IaaS).
   To pragmatically operate more than one
   application per virtual machine, containerization
   established as a trend (Docker). A container starts
   faster than a virtual machine and shares the
   operating system with other containers, thus
   reducing deployment unit sizes and increasing
   application density per virtual machine. But the
   application should follow cloud architectural
   styles (like 12-factor-app, microservices) to fully
   leverage the opportunities.
   But a container still requests a share of CPU, memory,
   and storage – even if the provided service is hardly
   requested. It would be more ressource efficient, if
   services would consume ressources only if there are
   incoming requests. This is what FaaS runtime
   environments provide. So, services can timeshare a
   host, thus further increasing application density per
   host. However, this involves to follow a more rich client
   architecture model for the user-interface and a service-
   composed-of-functions approach for services.
   1
   2 3 4
   Serverless
   Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   

   View Slide

7. Is Serverless PaaS?
   Are Serverless applications just
   another form of Platform as a Service
   (PaaS) like Heroku?
   • The key operational difference between
   FaaS and PaaS is scaling.
   • With most PaaS you still need to think
   about scale on a level of execution
   instances (dynos, machines, containers,
   etc.).
   • FaaS scaling is fine grained and
   completely transparent. It works on a
   level of individual service requests.
   Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   7
   Adrian
   Cockcroft
   VP Cloud Architecture
   Strategy at AWS
   „If your PaaS can
   efficiently start instances
   in 20ms that run for half
   a second, then call it
   Serverless.“
   

   View Slide

8. OK, what is about containers?
   Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   8
   The argument made for PaaS
   holds with containers.
   • FaaS scaling is fine grained and
   completely transparent. It works
   on a level of individual service
   requests.
   • Most container platforms do not
   offer such solutions (although
   Kubernetes is tending towards this
   level providing concepts like
   Horizontal Pod Autoscaling).
   • FaaS vs. Containers? It is more a
   question about when to use what?
   Mike Roberts
   Serverless Expert, Founder, and
   Co-Author of What is
   Serverless?, O´Reilly
   „[…] FaaS is seen as a better choice
   for event-driven style […]
   and containers are seen as better
   choice for synchronous-request
   driven components […]“
   

   View Slide

9. Function as a Service (FaaS)
   • FaaS is about running backend code without
   managing own servers.
   • FaaS offerings do not require coding to a specific
   framework or library. FaaS functions can be
   mostly implemented as „first class“ programs in
   JavaScript, Python, any JVM language (Java,
   Clojure, Scala, …), and more languages.
   • Deployment is different compared with traditional
   systems - just upload the code to a FaaS provider
   and it does anything else.
   • Horizontal scaling is completely automatic, elastic
   and managed by the provider.
   • Functions in FaaS are triggered by event types
   defined by the provider, this might be file
   updates, scheduled events (time), messages on a
   message bus, or simply HTTP requests.
   Prof. Dr. rer. nat. Nane Kratzke
   Computer Science and Business Information Systems
   9
   

   View Slide

10. Function as a Service
    We have to consider STATE and DURATION
    FaaS functions are stateless
    • They provide pure functional
    transformations of their input
    • or they have to use of a database,
    cross-application cache (e.g.
    Redis), or object storage (e.g. S3)
    to store state across requests.
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    10
    FaaS functions have timeouts
    • E.g. AWS Lambda allows no functions
    to execute longer than 5 minutes.
    • So, certain long lived tasks are not
    suited to FaaS functions.
    

    View Slide

11. Function as a Service
    We have to consider STARTUP LATENCY
    FaaS function respond times depend on a number of
    factors
    • and maybe anywhere between milliseconds and minutes
    (in very worst cases).
    • You should consider the overhead of starting a potential
    necessary runtime environment for your function code.
    • E.g.: JavaScript and Python are known to spun up faster
    than a JVM.
    • Latencies can get longer, especially if
    • a function processes events infrequently (e.g. more than
    10 minutes between invocations)
    • or sudden spikes in traffic occur (normally 10 requests per
    second, but suddenly 1000 reqs/sec).
    • It is likely that in these cases the FaaS provider must start
    additional (or the first) container instances which involves
    longer startup times.
    11
    So, a low-latency trading application or
    a missile control system might be no
    appropriate use case for Serverless!!!
    

    View Slide

12. How is Serverless Changing Architectures?
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    12
    

    View Slide

13. Serverless by Example
    Let us start with a traditional non-serverless architecture
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    13
    Client (Browser) Application Server Relational Database
    • Think about a traditional 3-
    tier client-oriented system
    with server-side logic.
    • A good example is a typical
    ecommerce app.
    • Using such an architecture
    the client can be relatively
    unintelligent.
    • Most of the logic will be
    based in the application
    server.
    So, how will Serverless
    change this architecture?
    

    View Slide

14. Serverless by Example
    Now, let us make it a Serverless architecture
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    14
    Purchase
    Function
    Search
    Function
    API Gateway
    Authentication Service
    Purchase Database
    Product Database
    Native
    mobile
    app
    1
    3
    2
    4
    5
    The authentication logic can be
    replaced with a 3rd party
    authentication BaaS (like Auth0).
    The client is allowed direct
    access to a subset of our
    database. The database is fully
    3rd party hosted.
    Server application
    logic now moves
    to the client
    application, making
    it often a native
    mobile app or a
    single-page web
    application.
    Some functionality might be kept in the
    „server“. It might be compute intensive
    or requires access to a significant amount
    of data like a search function.
    Such functionality is provided as FaaS
    functions that often respond to HTTP
    requests.
    Some functionality might be kept
    in the „server“ for security
    reasons or for interfacing
    further 3rd party BaaS.
    6
    An API Gateway is basically a web server that
    receives HTTP requests and routes them to
    subsequent FaaS functions or other backend services.
    So, service complexity is reduced at the cost
    of a more complex service-of-service
    architecture. Complexity is not reducable, it
    can be shifted, capsuled, managed, ... – but
    never reduced! If you reduce complexity, you
    define a different – simpler – problem!
    

    View Slide

15. Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    15
    Benefits and Drawbacks
    

    View Slide

16. Benefits
    • Reduced cost due to intensive timesharing of
    infrastructure and implicit sharing of the operational
    staff
    • Reduced development costs due to intensive use of
    BaaS (services that already exist and not have to be
    implemented again and again)
    • Easier operational management because the scaling is
    automatic
    • Reduced packaging and deployment efforts
    • Better time to market
    • Opportunities for experiments
    Maybe even „greener computing“? OK, no one proofed that,
    so far … (but there are some reasonable arguments for it)
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    16
    

    View Slide

17. Repetition of client logic
    • Serverless makes it easier to reuse
    logic on the service side.
    • But this involves very often similar
    and repetitive implementations on
    the client side.
    Inherent Drawbacks
    Vendor control
    • Vendor lock-in
    • Loss of server optimizations
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    17
    Security concerns
    • Multitenancy vulnerabilities
    • Increased surface
    • Losing the protective barrier of a
    server-side application
    No in-server state for FaaS
    • No in-memory or in-process cacheing
    • External cacheing solutions like Redis
    or Memcached are compensating
    options but a order of magnitude
    slower.
    

    View Slide

18. Implementation Drawbacks
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    18
    Do not Denial-of-Service
    yourself
    Typically your number of
    concurrent executions of
    functions is limited.
    This limit is applied to your
    account which you may use for
    production and testing.
    So testing may surprisingly affect
    your production deployments.
    Consider execution
    durations and latencies
    Remember, the executation
    duration of functions is limited
    and long lived tasks are not
    suited for FaaS.
    Startup latencies affect respond
    times of FaaS functions.
    Especially JVM-implemented
    functions tend to show suddenly
    high latencies, especially if
    events occur infrequently or as
    sudden spikes.
    Do not underestimate
    integration testing
    Unit testing is fairly simple. But
    integration testing of Serverless
    can be complicated.
    That is because the units of
    integration (functions) are a lot
    smaller and therefore Serverless
    architectures normally rely on
    integration testing a lot more
    than other architectural styles.
    

    View Slide

19. Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    19
    (In-)appropriate
    Use Cases
    

    View Slide

20. It is always about the use case
    Stupid!
    • As we have seen, Serverless architectures
    come with benefits and drawbacks.
    • So, some use cases are more suited for
    Serverless approaches than others.
    • The question is, which use cases look
    promising and which use cases do not?
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    20
    Mike Roberts
    „There is a lot to like about Serverless
    architectures […], but they come with
    significant trade-offs. Some of these are
    inherent […] and can‘t be fixed. Others […] we
    could expect to be resolved.“
    

    View Slide

21. Remember the implementation drawbacks
    • Multitenancy performance A high load user or
    even your testing can cause other users to slow down.
    • Repetition of logic across client platforms This
    might increase development efforts for the client side.
    • Stateless functions Therefore FaaS seems
    suboptimal to realize stateful (database-like) services.
    • No in-process state and cacheing This might
    cause higher latencies.
    • Limitations of execution durations This might
    prevent long running tasks like streaming or analysis.
    • Startup latency This might result in long respond
    times especially for very infrequent requested
    services.
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    21
    

    View Slide

22. And the ugly … ?
    happens whenever you
    try to apply Serverless
    to bad use cases.
    The good, the bad, and the ugly
    By use case
    Good use cases
    • User group with an uniformly
    distributed request volume.
    • Applications with a limited set of
    client devices.
    • User prefers native apps.
    • Stateless services.
    • Varying respond times can be
    tolerated.
    • Short running requests (no batch-
    like jobs)
    • No extreme low-latency or even
    hard real-time requirements
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    22
    Bad use cases
    • User group with spikes in request
    volumes.
    • Necessity to support an unlimited
    set of client devices.
    • User prefers web access via
    browser.
    • Stateful services.
    • Respond times are low-latency and
    must be assured in a defined range.
    • Long running batch-like jobs.
    • Low-latency or hard real-time
    requirements known from low-
    latency trading or real-time control
    systems.
    

    View Slide

23. And the ugly example?
    Well, try to implement
    a FaaS video streaming
    function. The result will
    be ugly!
    The good, the bad, and the ugly
    By example
    Good examples
    • Single image processing
    • Videosharing
    • Social network sharing features
    • Single entity categorization using
    a trained neural network
    • Event-based processing of social
    media streams
    • (short running) database querying
    • Messaging
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    23
    Bad examples
    • Batch-like image processing
    • Videoprocessing/-streaming
    • Large scale social network analysis
    • Neural network training
    • Batch-like entity categorization
    • Continuous observing of social
    media streams
    • Database-like storage service
    • Persistant message bus
    

    View Slide

24. Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    24
    Open Issues
    

    View Slide

25. Serverless is (still) immature
    Sad, but this leaves room for improvements
    • As we have seen, Serverless
    architectures have some open issues.
    • That is not perfect and should be
    considered.
    • However, the following points are
    likely to be tackled in the future.
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    25
    Mike Roberts
    „The remaining drawbacks […] are down
    purely to the current state of the art. With
    inclination […] and a heroic community these
    can be all wiped out.“
    

    View Slide

26. Open issues
    Improvements for tooling, in-process-state, platform features
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    26
    Tooling
    There is the need for more
    mature and more
    integrated deployment,
    application bundling,
    configuration, monitoring
    & logging, and debugging
    tools.
    It is likely that these tools
    will be better integrated
    with future FaaS runtime
    environments.
    State
    To avoid in-process-state is
    astonishing hard to accept
    for a lot of developpers.
    This may even a show-
    stopper for several use
    cases.
    Better integrations with
    out-of-process data
    solutions like Redis or
    memcached would be
    helpful.
    Platforms
    Current FaaS platforms
    come with limitations
    regarding execution
    duration of functions,
    startup latency, and non-
    separation of execution
    limits.
    It is likely that these
    limitations will be
    attentuated in the future.
    

    View Slide

27. Open issues
    Finding services and proven solution patterns
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    27
    Service Discovery
    There are no well defined or
    standardized solutions for service
    discovery of FaaS functions. This is
    even getting worse due to the fine
    granular nature of FaaS and a lack
    of application and versioning
    definition.
    Serverless Patterns
    Serverless is about to avoid
    ‚always-on‘ components. But
    ‚always-on‘ will ever be necessary.
    So, Serverless is typically applied
    as part of an hybrid architecture.
    How to do that is an open issue,
    as well as proven patterns for
    common use cases like media
    processing.
    

    View Slide

28. Open issues
    Operation still needs monitoring, testing, and debugging
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    28
    Monitoring and debugging
    Serverless architectures rely on
    the monitoring and debugging
    side with whatever the vendor
    provides. This support is often
    very basic. Open and standardized
    APIs would be helpful to integrate
    more sophisticated 3rd party
    services.
    Integration testing
    Serverless involves to work with
    smaller units (functions) that are
    easier to unit test. However, this
    involves more complex
    integration. So, pragmatic and
    efficient integration testing is
    especially essential for
    microservice and Serverless
    approaches.
    

    View Slide

29. Serverless Frameworks
    

    View Slide

30. Serverless
    Services, Platforms, and Frameworks
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    30
    Public FaaS Cloud
    Services
    Most public cloud service
    providers offer Serverless
    compute services, also known
    as function as a service (FaaS).
    Currently there exist no FaaS
    standard and Vendor Lock-In is
    likely.
    Standalone Serverless
    Platforms
    Due to missing standards
    public FaaS cloud services are
    prone to create vendor lock-in.
    Open Serverless platforms
    might be an alternative. But
    these platforms need
    infrastructures for operation.
    Platform Agnostic
    Serverless Frameworks
    Platform agnostic frameworks
    provide a provider and
    platform agnostic way to
    define and deploy Serverless
    code on various Serverless
    platforms or FaaS cloud
    services.
    • AWS Lambda
    https://aws.amazon.com/lambda
    • Google Cloud Functions
    https://cloud.google.com/functions
    • Azure Functions
    https://azure.microsoft.com/services
    /functions
    • OpenWhisk
    https://openwhisk.apache.org
    • Nuclio
    https://nuclio.io
    • Fn project
    https://fnproject.io
    • OpenFaaS
    https://www.openfaas.com
    • Serverless Framework
    https://serverless.com
    • Squeezer Framework
    https://squeezer.io/framework
    • SpringCloud Functions
    https://cloud.spring.io/spring-
    cloud-function
    This list is not complete! You will find curated
    and continuously updated lists here:
    • http://bit.ly/2rBWzXr
    • http://bit.ly/2FcKSbF
    

    View Slide

31. Further Reading
    • Serverless Architectures by Mike Roberts, 2016
    https://www.martinfowler.com/articles/serverless.html; This blog
    post was one of the most influencing references for this presentation.
    • What Is Serverless? by John Chapin, Mike Roberts, O‘Reilly, 2017;
    You can get this ebook for free from O‘Reilly. The authors take you
    through the Serverless landscape: design considerations, tooling, and
    approaches to operational management.
    • Why the Future Is Serverless? by Ken Fromm, Badri Janakiraman,
    2012 http://readwrite.com/2012/10/15/why-the-future-of-software-
    and-apps-is-serverless
    ; Might be one of the first articles using the
    term Serverless in its current meaning.
    • The State of The Serverless Ecosystem by Tal KimHi, 2017,
    https://medium.com/@talkimhi/the-state-of-the-serverless-ecosystem-
    c8a8b5ca56ae; This post tries to gather every company, product, tool, or
    framework that has something to do with the Serverless paradigm. To
    reach this goal seems impossible, but it is simply an awesome work.
    • Serverless Computing by Wikipedia,
    https://en.wikipedia.org/wiki/Serverless_computing, A Wikipedia
    link is a must. But remember: Wikipedia is a good start for a search, it
    should never be the end.
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    31
    

    View Slide

32. Acknowledgement
    Picture Reference
    • Cloud: Pixabay (CC0 Public Domain)
    • Defintion: Pixabay (CC0 Public Domain)
    • Serverrack: Pixabay (CC0 Public Domain)
    • Smileys: Pixabay (CC0 Public Domain)
    • Peanuts: Pixabay (CC0 Public Domain)
    • Question marks: Pixabay (CC0 Public Domain)
    • Lego bricks: Pixabay (CC0 Public Domain)
    • All icons: The Noun Project (CC-BY-2.0 License)
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    32
    This contribution resulted as a side-effect from research that is
    funded by German Federal Ministry of Education and Research
    (Project Cloud TRANSIT, 13FH021PX4).
    Speaker Deck URL
    

    View Slide

33. About
    Prof. Dr. rer. nat. Nane Kratzke
    Computer Science and Business Information Systems
    33
    Nane Kratzke
    CoSA: http://cosa.fh-luebeck.de/en/contact/people/n-kratzke
    Blog: http://www.nkode.io
    Twitter: @NaneKratzke
    GooglePlus: +NaneKratzke
    LinkedIn: https://de.linkedin.com/in/nanekratzke
    GitHub: https://github.com/nkratzke
    ResearchGate: https://www.researchgate.net/profile/Nane_Kratzke
    Speaker Deck: https://speakerdeck.com/nkratzke
    

    View Slide