Serverless beyond Functions

JeffConf, Milan, September 29th, 2017

One of the great advantages of serverless development is the possibility to “chain” multiple functions together, and design event-driven architectures. Applications built in this way are easier to keep under control, because our human minds are much better in looking for cause-effect relationships than understanding a complex workflow. But it is time now to go beyond that, considering serverless in its broader definition of building applications “without thinking about servers”, and use all available tools.

Transcript

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
   Danilo Poccia, Technical Evangelist
   @danilop
   Serverless beyond Functions
   Jeff
   

   View Slide

2. What is Serverless?
   

   View Slide

3. Lambda
   Function
   Do whatever you want: It’s Your Code!
   Trigger
   

   View Slide

4. Lambda
   Function
   Trigger Do whatever you want: It’s Your Code!
   S3
   Bucket
   Amazon
   API Gateway
   DynamoDB
   Table
   AWS IoT
   Rule
   Much
   More!
   Amazon
   CloudWatch
   

   View Slide

5. Lambda
   Function
   You can “chain” events to build
   Event-Driven Applications
   S3
   Bucket
   DynamoDB
   Table
   Lambda
   Function
   

   View Slide

6. Serverless Case Studies
   PhotoVogue is an online photography platform.
   Launched in 2011 and part of Vogue Italia - which
   is owned by Condé Nast Italia - it allows upcoming
   photographers to showcase their work.
   The Benefits
   Quicker provisioning, from days to hours
   90% faster
   Cut IT costs by around 30%
   Seamless scalability
   Amazon
   S3
   AWS
   Lambda
   Amazon
   API Gateway
   Amazon
   CloudFront
   “With Amazon API Gateway and AWS
   Lambda, the user experience is up to 90%
   faster. That's for both photographers
   uploading images and the editorial team
   processing them.”
   Marco Viganò
   Head of Digital Development
   

   View Slide

7. Serverless Case Studies
   F-Secure Increases Customer Insight
   and Speeds Up Activation Using AWS
   “We’ve shown the rest of the
   company that we can put our ideas
   into action faster and save 70% on
   infrastructure costs by using AWS
   compared to running hardware on-
   premises…. We’re inspiring a shift
   toward the cloud, and
   microservices in particular,
   because of these great benefits.”
   Niina Ojala
   Service Lead
   

   View Slide

8. Serverless Case Studies
   UK Driver and Vehicle Licensing Agency Supports
   Secure, Data-Driven Innovation
   “We are decomposing our applications into smaller,
   discrete components so we can choose the most
   appropriate technology”
   An API-First Approach
   “We stood up a working prototype in just a few days using
   Amazon API Gateway. The speed with which we were
   able to deliver it was unprecedented”
   Matt Lewis
   Chief Architect, Driver and Vehicle Licensing Agency
   AWS
   Lambda
   Amazon
   API Gateway
   

   View Slide

9. Serverless by Design
   

   View Slide

10. Serverless by Design
    A visual approach to serverless development.
    Event-driven.
    Think. Build. Repeat.
    

    View Slide

11. Serverless by Design
    

    View Slide

12. Serverless by Design
    

    View Slide

13. Serverless by Design
    

    View Slide

14. Serverless by Design
    

    View Slide

15. Deploying SAM templates
    aws cloudformation package
    --template-file template.yaml
    --output-template-file template-output.yaml
    --s3-bucket --s3-prefix
    aws cloudformation deploy
    --template-file ./template-output.yaml
    --stack-name
    --capabilities CAPABILITY_IAM
    

    View Slide

16. Serverless by Design
    https://sbd.danilop.net
    https://github.com/danilop/ServerlessByDesign
    

    View Slide

17. AWS SAM Local
    https://github.com/awslabs/aws-sam-local
    

    View Slide

18. “Where” is Serverless?
    

    View Slide

19. Client Back End
    AWS
    Lambda
    

    View Slide

20. CDN
    Client Back End Devices
    Sensors
    Actuators
    AWS
    Lambda
    Gateway
    

    View Slide

21. AWS Greengrass
    

    View Slide

22. Lambda Functions on a Raspberry Pi
    AWS
    Greengrass
    Lambda
    Function
    

    View Slide

23. AWS Snowball Edge
    100TB + Greengrass Core (≃ EC2 m4.4xlarge instance)
    

    View Slide

24. Snowball Edge Use Cases
    “Snowball Edge enables us to extend the
    innovative capabilities of HealthSuite, our cloud-
    enabled connected health ecosystem of devices,
    applications and digital tools supported by AWS,
    even when there is no network support.”
    Embedded Applications
    —Dale Wiggins,
    Business Leader, HealthSuite digital platform,
    Philips
    

    View Slide

25. Snowball Edge Use Cases
    “With AWS Snowball Edge, we can now collect
    100 TB of data with no intermediate steps, and
    we can also analyze the images immediately
    using the onboard compute capabilities.”
    Remote Locations for data collection and analysis
    — Bob Cowen,
    Director of Hatfield Marine Research Center,
    Oregon State University
    

    View Slide

26. AWS Lambda@Edge
    Customize content delivery while reducing load on the origin
    Events / Triggers
    Take serverless to your users
    

    View Slide

27. Lambda@Edge Use Cases
    Demo build of a
    CloudFront+S3 distribution,
    using Lambda@Edge
    to secure its HTTP headers
    

    View Slide

28. Lambda@Edge Use Cases
    These processes include applying
    transactional labels to purchases so
    Blockbuster can track customer activity, and
    providing personalized recommendations
    based on previous purchases.
    Blockbuster runs serverless compute
    processes across AWS Regions and
    Amazon CloudFront edge locations (using
    Lambda@Edge) without provisioning or
    managing servers.
    

    View Slide

29. Let’s build a Web Chat…
    

    View Slide

30. Web
    Browser
    

    View Slide

31. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    HTTPS
    Static Content
    (HTML, CSS, JS)
    

    View Slide

32. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    HTTPS
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    

    View Slide

33. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    Amazon
    API Gateway
    Lambda
    Function(s)
    DynamoDB
    Table(s)
    HTTPS
    Dynamic
    Content
    Database
    Access
    Execute
    Functions
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    

    View Slide

34. Let’s make it more interactive…
    

    View Slide

35. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    Amazon
    API Gateway
    Lambda
    Function(s)
    DynamoDB
    Table(s)
    AWS IoT
    Topic(s)
    HTTPS
    Dynamic
    Content
    Database
    Access
    MQTT over
    Secure WebSockets
    (Publish, Subscribe)
    Execute
    Functions
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    

    View Slide

36. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    Amazon
    API Gateway
    Lambda
    Function(s)
    DynamoDB
    Table(s)
    AWS IoT
    Topic(s)
    AWS IoT
    Rule(s)
    HTTPS
    Dynamic
    Content
    Database
    Access
    MQTT over
    Secure WebSockets
    (Publish, Subscribe)
    Subscribe
    Republish
    Execute
    Functions
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    

    View Slide

37. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    Amazon
    API Gateway
    Lambda
    Function(s)
    DynamoDB
    Table(s)
    AWS IoT
    Topic(s)
    AWS IoT
    Rule(s)
    HTTPS
    Dynamic
    Content
    Database
    Access
    MQTT over
    Secure WebSockets
    (Publish, Subscribe)
    Subscribe
    Republish
    Execute
    Functions
    Execute
    Functions
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    Publish
    

    View Slide

38. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    Amazon
    API Gateway
    Lambda
    Function(s)
    DynamoDB
    Table(s)
    AWS IoT
    Topic(s)
    AWS IoT
    Rule(s)
    HTTPS
    Dynamic
    Content
    Database
    Access
    MQTT over
    Secure WebSockets
    (Publish, Subscribe)
    Subscribe
    Republish
    Execute
    Functions
    Write to
    DynamoDB
    Execute
    Functions
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    Publish
    

    View Slide

39. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    Amazon
    API Gateway
    Lambda
    Function(s)
    DynamoDB
    Table(s)
    AWS IoT
    Topic(s)
    AWS IoT
    Rule(s)
    Kinesis
    Stream(s)
    HTTPS
    Dynamic
    Content
    Database
    Access
    MQTT over
    Secure WebSockets
    (Publish, Subscribe)
    Subscribe
    Republish
    Execute
    Functions
    Streaming
    Data
    Write to
    DynamoDB
    Execute
    Functions
    Execute
    Functions
    (Micro-Batches)
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    Publish
    

    View Slide

40. Web
    Browser
    CloudFront
    Edge Location
    S3
    Bucket
    Amazon
    API Gateway
    Lambda
    Function(s)
    DynamoDB
    Table(s)
    AWS IoT
    Topic(s)
    AWS IoT
    Rule(s)
    Kinesis
    Stream(s)
    HTTPS
    Dynamic
    Content
    Database
    Access
    MQTT over
    Secure WebSockets
    (Publish, Subscribe)
    Subscribe
    Republish
    Execute
    Functions
    Streaming
    Data
    Write to
    DynamoDB
    Execute
    Functions
    Execute
    Functions
    (Micro-Batches)
    Amazon
    Cognito
    Authentication
    (AWS Temporary
    Credentials)
    Static Content
    (HTML, CSS, JS)
    Publish
    

    View Slide

41. What about Topics & Rules?
    

    View Slide

42. Web
    Browser
    AWS IoT
    Topics
    chat/in/${iot:ClientId}
    chat/out
    chat/pub/${room}
    chat/stream
    $aws/events/#
    Each client has a secure “input” topic
    to receive content from the back end
    All client can publish on an “output” topic,
    they are recognizable by their clientId
    All client can subscribe and receive
    from “public” topics
    Another “output” topic
    optimized for high volumes
    AWS IoT
    Lifecycle Events
    

    View Slide

43. Web
    Browser
    AWS IoT
    Topics
    AWS IoT
    Rules
    chat/in/${iot:ClientId}
    chat/out
    chat/pub/${room}
    chat/stream
    $aws/events/#
    Each client has a secure “input” topic
    to receive content from the back end
    All client can publish on an “output” topic,
    they are recognizable by their clientId
    All client can subscribe and receive
    from “public” topics
    Another “output” topic
    optimized for high volumes
    AWS IoT
    Lifecycle Events
    Lambda function
    to process important messages,
    such as a new client connection
    Republish rule to send messages
    to the corresponding room public topic
    Lambda function to process
    high volume dreaming data
    Kinesis Stream to manage
    high volume streaming data
    Lambda function
    to process AWS IoT Lifecycle Events
    Store all messages in a database,
    to be sent to the client on connection
    

    View Slide

44. Web
    Browser
    AWS IoT
    Topics
    AWS IoT
    Rules
    chat/in/${iot:ClientId}
    chat/out
    chat/pub/${room}
    chat/stream
    $aws/events/#
    Each client has a secure “input” topic
    to receive content from the back end
    All client can publish on an “output” topic,
    they are recognizable by their clientId
    All client can subscribe and receive
    from “public” topics
    Another “output” topic
    optimized for high volumes
    AWS IoT
    Lifecycle Events
    Lambda function
    to process important messages,
    such as a new client connection
    Republish rule to send messages
    to the corresponding room public topic
    Lambda function to process
    high volume dreaming data
    Kinesis Stream to manage
    high volume streaming data
    Lambda function
    to process AWS IoT Lifecycle Events
    Store all messages in a database,
    to be sent to the client on connection
    You can use this topic
    to inject JavaScript code
    in the browser
    

    View Slide

45. So how does the flow look like?
    

    View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. View Slide

51. View Slide

52. View Slide

53. Demo
    https://chat.danilop.net/JeffConf
    

    View Slide

54. Code
    https://github.com/danilop/serverless-chat
    More Info
    http://docs.aws.amazon.com/iot/latest/developerguide/protocols.html
    http://docs.aws.amazon.com/iot/latest/developerguide/life-cycle-events.html
    http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html
    

    View Slide

55. From Event-Driven Architectures…
    

    View Slide

56. …To Functional Programming?
    

    View Slide

57. Functional Programming
    Transforming Data
    Using Expressions (Functions)
    Without Side Effects
    Immutable Data
    Pure Functions
    Higher-Order Functions
    From “Research Topics in Functional Programming” ed. D. Turner, Addison-Wesley, 1990, pp 17–42.
    1
    Why
    Functional Programming
    Matters
    John Hughes
    The University, Glasgow
    Abstract
    As software becomes more and more complex, it is more and more
    important to structure it well. Well-structured software is easy to write
    and to debug, and provides a collection of modules that can be reused
    to reduce future programming costs. In this paper we show that two fea-
    tures of functional languages in particular, higher-order functions and lazy
    evaluation, can contribute significantly to modularity. As examples, we
    manipulate lists and trees, program several numerical algorithms, and im-
    plement the alpha-beta heuristic (an algorithm from Artificial Intelligence
    used in game-playing programs). We conclude that since modularity is the
    key to successful programming, functional programming o↵ers important
    advantages for software development.
    1 Introduction
    This paper is an attempt to demonstrate to the larger community of (non-
    functional) programmers the significance of functional programming, and also
    to help functional programmers exploit its advantages to the full by making it
    clear what those advantages are.
    Functional programming is so called because its fundamental operation is
    the application of functions to arguments. A main program itself is written as
    a function that receives the program’s input as its argument and delivers the
    program’s output as its result. Typically the main function is defined in terms of
    other functions, which in turn are defined in terms of still more functions, until
    at the bottom level the functions are language primitives. All of these functions
    are much like ordinary mathematical functions, and in this paper they will be
    1
    An earlier version of this paper appeared in the The Computer Journal, 32(2):98–107,
    April 1989. Copyright belongs to The British Computer Society, who grant permission to
    copy for educational purposes only without fee provided the copies are not made for direct
    commercial advantage and this BCS copyright notice appears.
    https://www.cs.kent.ac.uk/people/staff/dat/miranda/whyfp90.pdf
    

    View Slide

58. Let’s build a “Tag Cloud” from Your Pictures
    images
    

    View Slide

59. getLabels
    groupBy
    +
    renderHtml
    images
    Amazon
    Rekognition
    browser
    Event-Driven Architecture
    Let’s build a “Tag Cloud” from Your Pictures
    

    View Slide

60. renderHtml(groupBy(getLabels([images])), htmlParams)
    Functional View
    Let’s build a “Tag Cloud” from Your Pictures
    

    View Slide

61. renderHtml(groupBy(getLabels([images])), htmlParams)
    Functional View
    Let’s build a “Tag Cloud” from Your Pictures
    

    View Slide

62. renderHtml(groupBy(getLabels([images])), htmlParams)
    Functional View
    Let’s build a “Tag Cloud” from Your Pictures
    

    View Slide

63. renderHtml(groupBy(getLabels([images])), htmlParams)
    Functional View
    Let’s build a “Tag Cloud” from Your Pictures
    

    View Slide

64. map(f, [x1, x2, …, xn]) = [f(x1), f(x2), …, f(xn)]
    Higher-Order
    Functions
    filter(f, [x1, x2, …, xn]) = [xi if f(xi) is true]
    count([a, b, a, c, a, b]) = {a: 3, b: 2, c: 1}
    

    View Slide

65. renderHtml(groupBy(getLabels([images])), htmlParams)
    ⇩
    renderHtml(count(map(getLabel, [images]))), htmlParams)
    Reusable
    More
    reusable
    

    View Slide

66. Lambda Functions + Step Functions
    Lambda
    Function
    Input
    State
    Machine
    Output
    translate
    create
    execute
    Tasks
    State passing (Immutable Data)
    Tasks are performed only if required (Lazy Evaluation)
    A Lambda Function dynamically creating a State Machine executing the logic
    

    View Slide

67. Event-Driven Architectures
    +
    Functional Programming
    +
    Standard Serverless Library
    Happy to get your feedback on this!
    

    View Slide

68. What is Serverless?
    “Build and run
    applications
    without thinking
    about servers”
    

    View Slide

69. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Danilo Poccia, Technical Evangelist
    @danilop
    Serverless beyond Functions
    Jeff
    

    View Slide