Event Driven Architectures on AWS

Slide 1

Slide 1 text

Slide 2

Slide 2 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda • Evolution of compute on AWS • What got me interested in Lambda • Building and deploying serverless apps • 19.00 – short break • Building secure, serverless architectures • How to leverage serverless for security • Where to learn more • Q&A

Slide 3

Slide 3 text

Slide 4

Slide 4 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. We are witnessing a paradigm shift Level of abstraction Focus on business logic Serverless Physical machines Virtual machines Containerization AWS Lambda AWS Fargate • Continuous scaling • Fault tolerance built in • Pay for value • Zero maintenance • Focus on business value

Slide 5

Slide 5 text

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon EC2 Instances, containers and functions Boots up in minutes Runs for days or much longer Full OS access AWS Fargate Starts in seconds Runs for minutes/hours Limited OS access AWS Lambda Starts in milliseconds Runs up to 15 minutes No OS access

Slide 6

Slide 6 text

Slide 7

Slide 7 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data collection in an EC2 based architecture (2015 and before) EC2 instance collecting data Data forwarder Search interface (Splunk) Application and OS logs Cost, usage, infrastructure and security logs

Slide 8

Slide 8 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS data collection around 2015, powered by EC2 https://github.com/marekq/list-ec2 and https://github.com/marekq/list-s3 Listing instances that were running or stopped Listing buckets and objects on S3 that had public access enabled

Slide 9

Slide 9 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data collection after discovering Lambda (~2016) Lambda Runs every hour Replaces the instance Bucket receives CSV report Replaces the forwarder Splunk on EC2 Remains unchanged, but easier to ingest historical and live data

Slide 10

Slide 10 text

Slide 11

Slide 11 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. It got even easier afterwards (2016 onwards) AWS can deliver more data directly without requiring Lambda • Data S3 inventory about stored objects can be collected automatically • Infra AWS Config captures changes in your infrastructure • Cost AWS Budgets and billing alert you about cost increases • Security GuardDuty shares various findings through events A lot of my Lambda code became obsolete over night, and this is great!

Slide 12

Slide 12 text

Slide 13

Slide 13 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. … or get notified whenever someone puts something public https://aws.amazon.com/blogs/security/how-to-use-aws-config-to-monitor-for-and-respond-to-amazon-s3- buckets-allowing-public-access/ You can monitor your environment top down and automatically remediate any issues. Blocking all public access to S3 is also a lot easier today.

Slide 14

Slide 14 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. My cost reporter Lambda has been running daily for ~4 years Source; https://github.com/marekq/s3-cost-explorer (will be updated soon, ping me fore more info!)

Slide 15

Slide 15 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Building security automation for corporate networks Lambda function pings a cloud hosted or internal system to retrieve data Amazon CloudWatch Schedule AWS Lambda Ping function Instance or VM Alarm Archive output Update database AWS Cloud Corporate data center

Slide 16

Slide 16 text

Slide 17

Slide 17 text

How to process events using Lambda Synchronous (push) Asynchronous (event) Stream-based Amazon API Gateway AWS Lambda function Amazon DynamoDB Amazon SNS /order AWS Lambda function Amazon S3 reqs Amazon Kinesis changes AWS Lambda service function

Slide 18

Slide 18 text

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved. You can combine services and build highly available architectures Sources: https://marek.rocks and https://github.com/marekq/marek.rocks

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 21 Event-driven architectures drive reliability and scalability Asynchronous Events Improve responsiveness and reduce dependencies Event Routers Abstract producers and consumers from each other Event Stores Buffer messages until services are available to process

Slide 22

Slide 22 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon EventBridge Architecture Partner event source Rules Default event bus Custom event bus SaaS event bus Amazon EventBridge

Slide 23

Slide 23 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 23 Lego uses an event-driven design Commerce platform Order & customer updates Event relay Amazon EventBridge Login Customer login Checkout Submit order Order Process order Shipping Send order to SAP Data sync Customer, VIP, wishlist sync Payment Authorize payment FIFO queue Customer login Invoke every minute Order complete Events Payment authorized Customer login Order complete Order submit

Slide 24

Slide 24 text

Slide 25

Slide 25 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 25 developers services monitor release test build delivery pipelines monitor release test build monitor release test build monitor release test build monitor release test build monitor release test build Deployment: Microservice development lifecycle

Slide 26

Slide 26 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 26 AWS CloudFormation is a greatfoundation tobuildon AWS Code Commit Execute Deploy Write templates in YAML, or JSON, or generate code via macros/transforms or higher-level languages (AWS CDK) Upload templates directly via browser, from an S3 bucket, or ideally via CI/CD pipelines to leverage testing tools Create stacks from templates using the console, CLI, or AWS SDK, or create stack sets across multiple accounts and regions Stacks, stack sets, and resources are created, updated, and managed in a uniform way

Slide 27

Slide 27 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 27 However, these solutions can accelerate your development significantly; • AWS Amplify Console and CLI The fastest way to build mobile and web applications • Serverless Application Model (SAM) CLI Build serverless apps using a YAML template with simple, clean syntax • Cloud Development Kit (CDK) Define cloud resources in your favourite programming language Serverless framework options from AWS

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 30 AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: GetProductsFunction: Type: AWS::Serverless::Function Properties: Handler: index.getProducts Runtime: nodejs10.x CodeUri: src/ Policies: - DynamoDBReadPolicy: TableName: !Ref ProductTable Events: GetResource: Type: Api Properties: Path: /products/{productId} Method: get ProductTable: Type: AWS::Serverless::SimpleTable AWS Cloud AWS SAM templates and CLI Amazon API Gateway Lambda function Table Role === To become this Allowing this

Slide 31

Slide 31 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 31 AWS Cloud Development Kit Software development framework for defining cloud infrastructure using familiar programming languages AWS CDK

Slide 32

Slide 32 text

Slide 33

Slide 33 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 33 The CDK is a great fit as you can include any AWS resource Source: https://github.com/marekq/sqs-fargate-poller

Slide 34

Slide 34 text

Slide 35

Slide 35 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 35 AWS Amplify • Check the documentation and samples on https://aws.amazon.com/amplify/ • Follow Nader Dabit (dabit3@) on Twitter to find many awesome Amplify tutorials • Sign up for Amplify Days event on 10/11 June; https://awsamplifydays.splashthat.com/ Serverless Application Repository (SAM) • Find SAM examples on GitHub; https://github.com/awslabs/serverless-application-model • Deploy full serverless apps from the Serverless App Repo, this is a great way to learn!; https://aws.amazon.com/serverless/serverlessrepo/ Cloud Development Kit (CDK) • Run a sample CDK workshop to learn the basics; https://cdkworkshop.com/ • Various GitHub repo’s contain common CDK patterns; https://twitter.com/cdkpatterns Where to learn more