I want you to write less Python code - PyCon Sweden November 13 2020

Slide 1

Slide 1 text

Slide 2

Slide 2 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Abstract Serverless allows us to build and run applications without thinking about servers. Additional benefits are that the cloud provider can take care of tasks we usually do with code, like error handling and retries. Let's look at how serverless can reduce the Python code we write and focus on the pure business logic.

Slide 3

Slide 3 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. About me Senior Developer Advocate Background in development, operations, and management Builder of communities Father of three

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS manages Customer manages Data source integrations Physical hardware, software, networking, and facilities Provisioning Application code Container orchestration, provisioning Cluster scaling Physical hardware, host OS/kernel, networking, and facilities Application code Data source integrations Security config and updates Network config Management tasks Container orchestration control plane Physical hardware software, networking, and facilities Application code Data source integrations Work clusters Security config and updates, network config, firewall, management tasks Physical hardware software, networking, and facilities Application code Data source integrations Scaling Security config and updates Network config Management tasks Provisioning, managing scaling and patching of servers Compute and Operations AWS Lambda Serverless functions AWS Fargate Serverless containers Amazon ECS/EKS Container-management -as-a-service Amazon EC2 Infrastructure-as-a-Service Less More Opinionated

Slide 7

Slide 7 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Lambda AWS Fargate Amazon API Gateway Amazon SNS Amazon SQS AWS Step Functions COMPUTE DATA STORES INTEGRATION AWS AppSync Amazon Aurora Serverless Amazon S3 Amazon DynamoDB Amazon EventBridge

Slide 8

Slide 8 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless applications with Lambda Event source Services Changes in data state Requests to endpoints Changes in resource state Function Python Node.js Java C# Go Ruby Runtime API

Slide 9

Slide 9 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lambda can be invoked via three different methods Lambda Function Amazon SNS Amazon S3 reqs Lambda Function Asynchronous /order Amazon API Gateway Lambda Function Synchronous Amazon DynamoDB Amazon Kinesis Changes AWS Lambda Service Poll-Based

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Error handling • Two types of errors can occur • Invocation errors occur when the invocation request is rejected before your function receives it • Function errors occur when your function's code or runtime returns an error • Avoid catching errors using try/catch – if possible • Fail up the stack • Return errors to the invoking service

Slide 15

Slide 15 text

Slide 16

Slide 16 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Automatic retries • Synchronous invocation – Lambda returns failure to the invoker for retry • Asynchronous invocation – Lambda retries function errors twice • Event source mappings that read from streams – Lambda retries the entire batch of items • Bisect on function error • Maximum record age • Maximum retry attempts • Destination on failure • Event source mappings that read from queues – Lambda retries the entire batch of items • SDK retries differs between runtimes and services

Slide 17

Slide 17 text

Slide 18

Slide 18 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Building – Dead letter queues • You can add dead letter queues to • SQS • SNS • Lambda • EventBridge • SQS uses SQS as DLQ • SNS uses SQS as DLQ • Lambda uses SQS or SNS as DLQ • EventBridge uses SQS as DLQ

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Lambda Event Destinations • For asynchronous Lambda executions • Record contains details about request and response in JSON • Route execution record based on function result • Success • Failure • Use another Lambda function, SQS, SNS or EventBridge as destination • Should be favored over DLQ AWS Lambda AWS Lambda Amazon EventBridge Amazon SNS Amazon SQS

Slide 22

Slide 22 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lambda destinations example Amazon Simple Storage Service Amazon Simple Storage Service AWS Lambda Amazon SQS On failure AWS Lambda On success

Slide 23

Slide 23 text

Slide 24

Slide 24 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Standard and FIFO queues and topics • SNS and SQS supports both Standard and FIFO topics/queues. • Use FIFO when you need strict message ordering and/or only once message processing. • Message throughput is nearly unlimited for SQS Standard queues and up to 3000 transactions per second for FIFO queues with batching. • Publish API is 300 transactions per second or 10 MB per second for SNS FIFO topics and up to 30000 transactions per second (dependent of region) for Standard topics. Amazon SNS Amazon SQS

Slide 25

Slide 25 text

Slide 26

Slide 26 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Choreography Event filtering Managed & scalable SaaS integration EventBridge Events and messages enable interaction between services Orchestration Sequencing Parallel execution State management Step Functions Eventing Performance at scale Fully managed Enterprise-ready Amazon SNS Messaging Durable and scalable Fully managed Comprehensive security Amazon SQS

Slide 27

Slide 27 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Contract decoupling • Decouple through a contract (API, message) • Allow changes in implementation API API Decoupling Runtime decoupling • Decouple through asynchronous invocations • Reduce the risk of cascading failures Service A Service B Message Message Service A Service B

Slide 28

Slide 28 text

Slide 29

Slide 29 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Building – Decouple architectures AWS Step Functions Amazon EventBridge AWS Lambda Upstream Amazon Simple Queue Service AWS Lambda Amazon DynamoDB

Slide 30

Slide 30 text

Slide 31

Slide 31 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EventBridge architecture Partner event source Rules Default event bus Custom event bus SaaS event bus Amazon EventBridge

Slide 32

Slide 32 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Type of Event Buses Default Bus Events from AWS services Scheduled events Custom events Custom Buses Custom events SaaS Buses 3rd party SaaS events

Slide 33

Slide 33 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Schema registry and discovery Explicitly publish and discover Integrations for JetBrains and VS Code Language bindings for Java, Python, and TypeScript Source of truth for sharing schema Amazon EventBridge

Slide 34

Slide 34 text

Slide 35

Slide 35 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Archive and replay events with EventBridge • Create an encrypted archive of the events published to an event bus • Archive all events, or filter them using pattern matching • Store event indefinitely or set up a retention period • Replay the events stored in an archive • Events are replayed to all rules defined for the event bus or to the rules you specify • Replayed events contain an extra replay-name field • Currently, you can only replay events to the same event bus • Works with all events process by EventBridge, including events from the AWS platform, from SaaS integrations, and your own custom events • During replays, your current event throughput is unaffected

Slide 36

Slide 36 text

Slide 37

Slide 37 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. A state machine Describes a collection of computational steps split into discrete states Has one starting state and always one active state (while executing) The active state receives input, takes some action, and generates output Transitions between states are based on state outputs and rules that we define

Slide 38

Slide 38 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Step Functions Fully-managed state machines on AWS Resilient workflow automation Built-in error handling Powerful AWS service integration First-class support for integrating with your own services Auditable execution history and visual monitoring

Slide 39

Slide 39 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Summary Serverless removes the undifferentiated heavy lifting that is server operations Make use of built-in error handling and retry behavior Use dead letter queues to capture events or messages AWS Lambda destinations lets you route execution records based on result Use FIFO queues and topics for data consistency Decouple with services for events and messages Use Amazon EventBridge as your managed event bus with archiving and replay Orchestrate using AWS Step Functions