Serverless for Developers: Tips for Your Next App

Slide 1

Slide 1 text

Serverless for Developers: Tips for Your Next App Danilo Poccia, Technical Evangelist @danilop danilop

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Credit: Gerry Cranham/Fox Photos/Getty Images http://www.telegraph.co.uk/travel/destinations/europe/united-kingdom/england/london/galleries/The-history-of-the-Tube-in-pictures-150-years-of-London-Underground/1939-ticket-examin/

Slide 4

Slide 4 text

Slide 5

Slide 5 text

AWS IoT Kinesis Stream AWS Lambda DynamoDB Table S3 Bucket

Slide 6

Slide 6 text

Dev + Ops

Slide 7

Slide 7 text

Dev + Ops + Event Driven

Slide 8

Slide 8 text

Name Email Create New User Submit Check the “Name” syntax is right (only letters and spaces) Check the “Email” syntax is right (something@some.domain) Create the new user using provided Name and Email User Interface Actions

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Generate thumbnails Keep another database in sync Process streaming data Services Functions Platform Target Action Kinesis Stream DynamoDB Table S3 Bucket

Slide 11

Slide 11 text

area = length x width

Slide 12

Slide 12 text

area = length x width Procedural Programming “This is a function!”

Slide 13

Slide 13 text

area = length x width Procedural Programming “This is a function!” Reactive Programming “This is data binding!”

Slide 14

Slide 14 text

A B Event-Driven Design For Services A triggers B or better B is caused by A

Slide 15

Slide 15 text

A B C D New Services C and D are triggered caused by B

Slide 16

Slide 16 text

A B C D E F ? ? What is causing E and F?

Slide 17

Slide 17 text

A B C D E F New Services (E, F) are triggered by C

Slide 18

Slide 18 text

A B C D E F It can be cyclic Think of acknowledgements

Slide 19

Slide 19 text

A B C D E F Each service has local visibility

Slide 20

Slide 20 text

A B C D E F What I need to know (input events) 1

Slide 21

Slide 21 text

A B C D E F What I need to do (internal logic) 2

Slide 22

Slide 22 text

A B C D E F Who I need to notify (output events) 3

Slide 23

Slide 23 text

A B C D E F Who I need to notify (output events) What I need to know (input events) What I need to do (internal logic) 1 2 3

Slide 24

Slide 24 text

Distributed Data Flow Vs Centralised Workflow

Slide 25

Slide 25 text

Choreography Vs Orchestration

Slide 26

Slide 26 text

Distributed Systems

Slide 27

Slide 27 text

Monolith Service Service μ μ μ space (distribution, mobility) time (concurrency, latency) μ μ μ μ “Distributed in Time and Space” [Jonas Bonér]

Slide 28

Slide 28 text

“…a diagram of two microservices and their shared database” Data should drive the boundaries

Slide 29

Slide 29 text

A C I D tomic onsistent isolated urable

Slide 30

Slide 30 text

A C I D tomic onsistent isolated urable A C I D ssociative ommutative dempotent istributed ACID 2.0

Slide 31

Slide 31 text

A B C D E F Event-Driven Interactions replacing Distributed Transactions using Eventual Consistency

Slide 32

Slide 32 text

A B C D E F Event-Driven Interactions replacing Distributed Transactions using Eventual Consistency Differentiate reads (queries) from writes (commands), as in the Command Query Responsibility Segregation (CQRS) pattern command query

Slide 33

Slide 33 text

Customers Orders Warehouse Deliveries Payments Catalog Event-Driven Interactions replacing Distributed Transactions using Eventual Consistency

Slide 34

Slide 34 text

Scale from Prototype to Production

Slide 35

Slide 35 text

Responsive Resilient Elastic Message Driven The Reactive Manifesto

Slide 36

Slide 36 text

The Reactive Manifesto Asynchronous communication is not enforced by serverless architectures Responsive Resilient Elastic Message Driven

Slide 37

Slide 37 text

Serverless Architectural Patterns

Slide 38

Slide 38 text

AWS Lambda User Client Application Back End for a Web or Mobile App Calling the Lambda API directly Web or Mobile Back End DynamoDB Table S3 Bucket Amazon Cognito AWS API

Slide 39

Slide 39 text

AWS Lambda Amazon Cognito User Client Application OAuth JWT Amazon API Gateway Back End for a Web or Mobile App Using an API Gateway for authentication and decoupling Web or Mobile Back End Via API Gateway DynamoDB Table S3 Bucket HTTPS

Slide 40

Slide 40 text

Using WebSockets instead of Web API to talk to the Back End WebSockets Back End AWS Lambda User Client Application AWS IoT DynamoDB Table S3 Bucket Amazon Cognito MQTT or Device Shadow MQTT over  WebSockets

Slide 41

Slide 41 text

Server ! Client { "id": "...", "html": "..." } { "js": "..." } Client ! Server { "cmd": "...", "data": "..." } MQTT Topics myapp/client/in/{ID} myapp/client/out/{ID} Cognito Identity ID ! {ID} 1 2 3 4 AWS Lambda User Client Application AWS IoT DynamoDB Table S3 Bucket Amazon Cognito MQTT or Device Shadow MQTT over  WebSockets

Slide 42

Slide 42 text

Possible optimisations depending on your data flow and your acceptable latency Data Source Kinesis Stream Kinesis Firehose Kinesis Analytics (augment & aggregate) AWS Lambda (batch) S3 Bucket AWS Lambda (file) Data Streams Processing Reference Data

Slide 43

Slide 43 text

AWS Lambda (batch) Amazon Cognito User Client Application Buffering Requests to the Back End to reduce Lambda invocations Kinesis Stream Buffered Back End For Mobile Amazon SNS DynamoDB Table S3 Bucket

Slide 44

Slide 44 text

AWS Lambda (batch) Amazon Cognito User Client Application Buffering Requests to the Back End to reduce Lambda invocations Kinesis Stream Buffered & Aggregated  Back End For Mobile Amazon SNS DynamoDB Table S3 Bucket Kinesis Analytics (aggregate) Kinesis Stream

Slide 45

Slide 45 text

AWS Lambda (batch) User Client Application Using WebSockets instead of Web API to talk to the Back End Buffering Requests to reduce Lambda invocations AWS IoT Kinesis Stream Buffered Back End For Everything MQTT or Device Shadow Amazon Cognito DynamoDB Table S3 Bucket

Slide 46

Slide 46 text

User Client Application Using WebSockets instead of Web API to talk to the Back End Buffering Requests to reduce Lambda invocations Storing all customer interactions for further analysis AWS IoT Kinesis Firehose Buffered Back End Saving Interactions MQTT or Device Shadow Amazon Cognito AWS Lambda (file) S3 Bucket DynamoDB Table

Slide 47

Slide 47 text

AWS Lambda DynamoDB Table User Client Application Amazon API Gateway (HTTPS) S3 Bucket (Website Hosting Routing Rules) For example, to generate image thumbnails on the first request, S3 LifeCycle Rules can delete content after N days Dynamic Cache Amazon Cognito HTTP Redirect Dynamically Build Content

Slide 48

Slide 48 text

To receive events from everywhere on the Internet Webhooks are callbacks for the Web AWS Lambda Amazon API Gateway Internet Service HTTP(S) GET or POST to https://domain/SOURCE/RANDOM-HOOK Webhook Other AWS Services Other public APIs AWS KMS Secrets & Credentials

Slide 49

Slide 49 text

To generate events for repositories that don’t support AWS Lambda natively Log Monitor AWS Lambda AWS Lambda AWS Lambda Amazon SNS Kinesis Stream Log Files External Application Log Monitor CloudWatch Logs or Metrics AWS Lambda

Slide 50

Slide 50 text

S3 object auto compress S3 bucket per prefix statistics DynamoDB provisioned throughput dynamic scaling DynamoDB item support for geohash Relational / NoSQL data synchronisation Service discovery . . . Extending Services with Your Own Functionalities

Slide 51

Slide 51 text

Serverless === Architecture Simplification

Slide 52

Slide 52 text

Serverless === Architecture Simplification For example, let’s build a Media Sharing App

Slide 53

Slide 53 text

Client Application Get Content Index Get Content and Thumbnails Upload Content with Metadata Update Content Metadata Get Content Metadata Files (Multimedia) Database Build Thumbnails Update Content Index Extract and Update Metadata Functions User Resources Basic features M edia Sharing App

Slide 54

Slide 54 text

Client Application Get Content Index Get Content and Thumbnails Upload Content with Metadata Update Content Metadata Get Content Metadata Files (Multimedia) Database Build Thumbnails Update Content Index Extract and Update Metadata Event: new or updated file Event: new or updated content metadata Functions User Resources Event-driven design M edia Sharing App

Slide 55

Slide 55 text

Client Application Get Content Index Get Content and Thumbnails Upload Content with Metadata Update Content Metadata Get Content Metadata Files (Multimedia) Database Build Thumbnails Update Content Index Extract and Update Metadata Event: new or updated file Event: new or updated content metadata Functions User Resources Understanding the overall flow M edia Sharing App

Slide 56

Slide 56 text

Client Application Get Content Index (S3 API) Get Content and Thumbnails (S3 API) Upload Content  with Metadata (S3 API) Update Content Metadata (DynamoDB API) Get Content Metadata (DynamoDB API) S3 Bucket DynamoDB Table Build Thumbnails (Lambda function) Update Content Index (Lambda function) Extract and Update Metadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Choosing the technology M edia Sharing App

Slide 57

Slide 57 text

Client Application Get Content Index (S3 API) Get Content and Thumbnails (S3 API) Upload Content  with Metadata (S3 API) Update Content Metadata (DynamoDB API) Get Content Metadata  (DynamoDB API) S3 Bucket DynamoDB Table Build Thumbnails (Lambda function) Update Content Index (Lambda function) Extract and Update Metadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Amazon Cognito Defining the security model M edia Sharing App

Slide 58

Slide 58 text

Client Application Get Content Index (S3 API) Get Content and Thumbnails (S3 API) Upload Content  with Metadata (S3 API) Update Content Metadata (DynamoDB API) Get Content Metadata  (DynamoDB API) S3 Bucket DynamoDB Table Build Thumbnails (Lambda function) Update Content Index (Lambda function) Extract and Update Metadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Amazon Cognito From the features viewpoint… M edia Sharing App

Slide 59

Slide 59 text

Client Application GET Object S3 API PUT Object S3 API UpdateItem DynamoDB API GetItem  DynamoDB API S3 Bucket DynamoDB Table buildThumbnails (Lambda function) updateContentIndex (Lambda function) extractAndUpdateMetadata (Lambda function) Event: new or updated file Event: new or updated content metadata User Amazon Cognito Amazon S3 Amazon DynamoDB AWS Lambda …To the technology viewpoint M edia Sharing App

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Client Application GET Object S3 API PUT Object S3 API UpdateItem DynamoDB API GetItem  DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata updateContentIndex (Lambda function) Event: new or updated file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB Event: new or updated content metadata Some functions can be tied together M edia Sharing App

Slide 62

Slide 62 text

Client Application GET Object S3 API PUT Object S3 API UpdateItem DynamoDB API GetItem  DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata updateContentIndex (Lambda function) Event: new or updated file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB Event: new or updated content metadata What if I want to delete content? M edia Sharing App

Slide 63

Slide 63 text

Client Application GET Object S3 API PUT Object S3 API UpdateItem DynamoDB API GetItem  DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata deleteMetadata updateContentIndex (Lambda function) Event: new, updated, or deleted file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB DELETE Object S3 API Event: new or updated content metadata REST to the rescue, the events flow is almost unchanged M edia Sharing App

Slide 64

Slide 64 text

Client Application GET Object S3 API PUT Object S3 API UpdateItem DynamoDB API GetItem  DynamoDB API S3 Bucket DynamoDB Table contentUpdated (Lambda function) buildThumbnails extractAndUpdateMetadata deleteMetadata updateContentIndex (Lambda function) Event: new, updated, or deleted file User Amazon Cognito Amazon S3 AWS Lambda Amazon DynamoDB DELETE Object S3 API Event: new or updated content metadata From 8 functions to 2, using 5 native API calls M edia Sharing App

Slide 65

Slide 65 text

Serverless Architectures are not just simplifying Operations

Slide 66

Slide 66 text

Developers can benefit from: event-driven design architectural patterns integrating new products architecture simplification customization of the platform Serverless Architectures are not just simplifying Operations