Serverless for Developers: Tips for Your Next App

Serverless for Developers: Tips for Your Next App

Serverless Meetup, Milan, November 3rd, 2016

Serverless architectures simplify operations and maintenance of production applications, abstracting from the underlining infrastructure and OS. But what about development? We are now two years in with tools such as AWS Lambda and what are the best practices to share from a development perspective? In this session you will see some of the key points that can empower a serverless developer: from event-driven design, to simplification (less is more, especially for a small team), extending the functionalities of the platform (per prefix stats for S3 buckets, geohash for DynamoDB tables, API authentication, and so on), integrating new products.

7c9b8b368924556d8642bdaed3ded1f5?s=128

Danilo Poccia

November 03, 2016
Tweet

Transcript

  1. Serverless for Developers: Tips for Your Next App Danilo Poccia,

    Technical Evangelist @danilop danilop
  2. None
  3. Credit: Gerry Cranham/Fox Photos/Getty Images http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/

  4. Credit: Gerry Cranham/Fox Photos/Getty Images http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/ 1939 London Underground

  5. AWS IoT Kinesis Stream AWS Lambda DynamoDB Table S3 Bucket

  6. Virtual Machines Containers Functions

  7. Code Microseconds to Seconds Versioning Single-Threaded Single-Task Per Memory/Second Per

    Request Free Tier The World of Lambda Functions
  8. No server is easier to manage than no server Serverless

    Computing
  9. Low Cost Security Simplicity Reliability Scalability Low Latency S3: Simple

    By Name, Simple By Nature
  10. Mobile Apps Websites Services Internet API Gateway Cache API Gateway

    CloudWatch Lambda Functions EC2 Endpoints Any Other Endpoint Amazon API Gateway
  11. Building An Infinitely Scalable Online Recording Campaign For David Guetta

    & UEFA
  12. Dev + Ops

  13. Dev + Ops + Event Driven

  14. Name Email Create New User Submit Check the “Name” syntax

    is right (only letters and spaces) Check the “Email” syntax is right (something@some.domain) Create the new user using provided Name and Email User Interface Actions
  15. Name Email Create New User Submit Check the “Name” syntax

    is right (only letters and spaces) Check the “Email” syntax is right (something@some.domain) Create the new user using provided Name and Email User Interface Actions Observers Target Action
  16. Generate thumbnails Keep another database in sync Process streaming data

    Services Functions Platform Target Action Kinesis Stream DynamoDB Table S3 Bucket
  17. area = length x width

  18. area = length x width Procedural Programming “This is a

    function!”
  19. area = length x width Procedural Programming “This is a

    function!” Reactive Programming “This is data binding!”
  20. A B Event-Driven Design For Services A triggers B or

    better B is caused by A
  21. A B C D New Services C and D are

    triggered caused by B
  22. A B C D E F ? ? What is

    causing E and F?
  23. A B C D E F New Services (E, F)

    are triggered by C
  24. A B C D E F It can be cyclic

    Think of acknowledgements
  25. A B C D E F Each service has local

    visibility
  26. A B C D E F What I need to

    know (input events) 1
  27. A B C D E F What I need to

    do (internal logic) 2
  28. A B C D E F Who I need to

    notify (output events) 3
  29. A B C D E F Who I need to

    notify (output events) What I need to know (input events) What I need to do (internal logic) 1 2 3
  30. Distributed Data Flow Vs Centralised Workflow

  31. Choreography Vs Orchestration

  32. Distributed Systems

  33. Monolith Service Service μ μ μ space (distribution, mobility) time

    (concurrency, latency) μ μ μ μ “Distributed in Time and Space” [Jonas Bonér]
  34. “…a diagram of two microservices and their shared database” Data

    should drive the boundaries
  35. A C I D tomic onsistent isolated urable

  36. A C I D tomic onsistent isolated urable A C

    I D ssociative ommutative dempotent istributed ACID 2.0
  37. A B C D E F Event-Driven Interactions replacing Distributed

    Transactions using Eventual Consistency
  38. A B C D E F Event-Driven Interactions replacing Distributed

    Transactions using Eventual Consistency Differentiate reads (queries) from writes (commands), as in the Command Query Responsibility Segregation (CQRS) pattern command query
  39. Customers Orders Warehouse Deliveries Payments Catalog Event-Driven Interactions replacing Distributed

    Transactions using Eventual Consistency
  40. Scale from Prototype to Production

  41. Responsive Resilient Elastic Message Driven The Reactive Manifesto

  42. The Reactive Manifesto Asynchronous communication is not enforced by serverless

    architectures Responsive Resilient Elastic Message Driven
  43. Serverless Architectural Patterns

  44. AWS Lambda User Client Application Back End for a Web

    or Mobile App Calling the Lambda API directly Web or Mobile Back End DynamoDB Table S3 Bucket Amazon Cognito AWS API
  45. AWS Lambda Amazon Cognito User Client Application OAuth JWT Amazon

    API Gateway Back End for a Web or Mobile App Using an API Gateway for authentication and decoupling Web or Mobile Back End Via API Gateway DynamoDB Table S3 Bucket HTTPS
  46. Using WebSockets instead of Web API to talk to the

    Back End WebSockets Back End AWS Lambda User Client Application AWS IoT DynamoDB Table S3 Bucket Amazon Cognito MQTT or Device Shadow MQTT over
 WebSockets
  47. Server ! Client { "id": "...", "html": "..." } {

    "js": "..." } Client ! Server { "cmd": "...", "data": "..." } MQTT Topics myapp/client/in/{ID} myapp/client/out/{ID} Cognito Identity ID ! {ID} 1 2 3 4 AWS Lambda User Client Application AWS IoT DynamoDB Table S3 Bucket Amazon Cognito MQTT or Device Shadow MQTT over
 WebSockets
  48. Possible optimisations depending on your data flow and your acceptable

    latency Data Source Kinesis Stream Kinesis Firehose Kinesis Analytics (augment & aggregate) AWS Lambda (batch) S3 Bucket AWS Lambda (file) Data Streams Processing Reference Data
  49. AWS Lambda (batch) Amazon Cognito User Client Application Buffering Requests

    to the Back End to reduce Lambda invocations Kinesis Stream Buffered Back End For Mobile Amazon SNS DynamoDB Table S3 Bucket
  50. AWS Lambda (batch) Amazon Cognito User Client Application Buffering Requests

    to the Back End to reduce Lambda invocations Kinesis Stream Buffered & Aggregated
 Back End For Mobile Amazon SNS DynamoDB Table S3 Bucket Kinesis Analytics (aggregate) Kinesis Stream
  51. AWS Lambda (batch) User Client Application Using WebSockets instead of

    Web API to talk to the Back End Buffering Requests to reduce Lambda invocations AWS IoT Kinesis Stream Buffered Back End For Everything MQTT or Device Shadow Amazon Cognito DynamoDB Table S3 Bucket
  52. User Client Application Using WebSockets instead of Web API to

    talk to the Back End Buffering Requests to reduce Lambda invocations Storing all customer interactions for further analysis AWS IoT Kinesis Firehose Buffered Back End Saving Interactions MQTT or Device Shadow Amazon Cognito AWS Lambda (file) S3 Bucket DynamoDB Table
  53. AWS Lambda DynamoDB Table User Client Application Amazon API Gateway

    (HTTPS) S3 Bucket (Website Hosting Routing Rules) For example, to generate image thumbnails on the first request, S3 LifeCycle Rules can delete content after N days Dynamic Cache Amazon Cognito HTTP Redirect Dynamically Build Content
  54. Content Delivery Amazon API Gateway CloudFront Distribution S3 Bucket AWS

    Lambda DynamoDB Table S3 Bucket Single DNS domain for both static and dynamic content, protected by the WAF,
 DNS health checks can react automatically to service downtime User Client Application Amazon Route 53 DNS Resolution Health Checks AWS WAF
  55. Data Analytics S3 Bucket Redshift Database DynamoDB Table #N AWS

    Lambda Kinesis Firehose Synchronize operational NoSQL databases
 with a columnar store for business intelligence and analytics Business Intelligence DynamoDB Table #1 AWS Lambda Kinesis Firehose . . . . . . . . .
  56. To receive events from everywhere on the Internet Webhooks are

    callbacks for the Web AWS Lambda Amazon API Gateway Internet Service HTTP(S) GET or POST to https://domain/SOURCE/RANDOM-HOOK Webhook Other AWS Services Other public APIs AWS KMS Secrets & Credentials
  57. To generate events for repositories that don’t support AWS Lambda

    natively Log Monitor AWS Lambda AWS Lambda AWS Lambda Amazon SNS Kinesis Stream Log Files External Application Log Monitor CloudWatch Logs or Metrics AWS Lambda
  58. S3 object auto compress S3 bucket per prefix statistics DynamoDB

    provisioned throughput dynamic scaling DynamoDB item support for geohash Relational / NoSQL data synchronisation MapReduce with Lambda functions on S3 buckets Service discovery . . . Extending Services with Your Own Functionalities
  59. Serverless === Architecture Simplification

  60. Serverless === Architecture Simplification For example, let’s build a Media

    Sharing App
  61. Client Application Get Content Index Get Content and Thumbnails Upload

    Content with Metadata Update Content Metadata Get Content Metadata Files (Multimedia) Database Build Thumbnails Update Content Index Extract and Update Metadata Functions User Resources Basic features M edia Sharing App
  62. Client Application Get Content Index Get Content and Thumbnails Upload

    Content with Metadata Update Content Metadata Get Content Metadata Files (Multimedia) Database Build Thumbnails Update Content Index Extract and Update Metadata Event: new or updated file Event: new or updated content metadata Functions User Resources Event-driven design M edia Sharing App
  63. Client Application Get Content Index Get Content and Thumbnails Upload

    Content with Metadata Update Content Metadata Get Content Metadata Files (Multimedia) Database Build Thumbnails Update Content Index Extract and Update Metadata Event: new or updated file Event: new or updated content metadata Functions User Resources Understanding the overall flow M edia Sharing App
  64. Client Application Get Content Index (S3 API) Get Content and

    Thumbnails (S3 API) Upload Content
 with Metadata (S3 API) Update Content Metadata (DynamoDB API) Get Content Metadata (DynamoDB API) S3 Bucket DynamoDB Table Build Thumbnails (Lambda function) Update Content Index (Lambda function) Extract and Update Metadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Choosing the technology M edia Sharing App
  65. Client Application Get Content Index (S3 API) Get Content and

    Thumbnails (S3 API) Upload Content
 with Metadata (S3 API) Update Content Metadata (DynamoDB API) Get Content Metadata
 (DynamoDB API) S3 Bucket DynamoDB Table Build Thumbnails (Lambda function) Update Content Index (Lambda function) Extract and Update Metadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Amazon Cognito Defining the security model M edia Sharing App
  66. Client Application Get Content Index (S3 API) Get Content and

    Thumbnails (S3 API) Upload Content
 with Metadata (S3 API) Update Content Metadata (DynamoDB API) Get Content Metadata
 (DynamoDB API) S3 Bucket DynamoDB Table Build Thumbnails (Lambda function) Update Content Index (Lambda function) Extract and Update Metadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Amazon Cognito From the features viewpoint… M edia Sharing App
  67. Client Application GET Object S3 API PUT Object S3 API

    UpdateItem DynamoDB API GetItem
 DynamoDB API S3 Bucket DynamoDB Table buildThumbnails (Lambda function) updateContentIndex (Lambda function) extractAndUpdateMetadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Amazon Cognito Amazon S3 Amazon DynamoDB AWS Lambda …To the technology viewpoint M edia Sharing App
  68. Client Application GET Object S3 API PUT Object S3 API

    UpdateItem DynamoDB API GetItem
 DynamoDB API S3 Bucket DynamoDB Table buildThumbnails (Lambda function) updateContentIndex (Lambda function) extractAndUpdateMetadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Amazon Cognito Amazon S3 Amazon DynamoDB AWS Lambda Some events are correlated M edia Sharing App
  69. Client Application GET Object S3 API PUT Object S3 API

    UpdateItem DynamoDB API GetItem
 DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata updateContentIndex (Lambda function) Event: new or updated file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB Event: new or updated content metadata Some functions can be tied together M edia Sharing App
  70. Client Application GET Object S3 API PUT Object S3 API

    UpdateItem DynamoDB API GetItem
 DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata updateContentIndex (Lambda function) Event: new or updated file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB Event: new or updated content metadata What if I want to delete content? M edia Sharing App
  71. Client Application GET Object S3 API PUT Object S3 API

    UpdateItem DynamoDB API GetItem
 DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata deleteMetadata updateContentIndex (Lambda function) Event: new, updated, or deleted file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB DELETE Object S3 API Event: new or updated content metadata REST to the rescue, the events flow is almost unchanged M edia Sharing App
  72. Client Application GET Object S3 API PUT Object S3 API

    UpdateItem DynamoDB API GetItem
 DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata deleteMetadata updateContentIndex (Lambda function) Event: new, updated, or deleted file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB DELETE Object S3 API Event: new or updated content metadata From 8 functions to 2, using 5 native API calls M edia Sharing App
  73. Serverless Architectures are not just simplifying Operations

  74. Developers can benefit from: event-driven design architectural patterns integrating new

    products architecture simplification customization of the platform Serverless Architectures are not just simplifying Operations
  75. Build Apps With Services, Not Servers

  76. Thank you @danilop danilop