Slide 1

Slide 1 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Best Practices for Orchestrating AWS Lambda Workloads Matthew Williams Evangelist @ Datadog

Slide 2

Slide 2 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 3

Slide 3 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 4

Slide 4 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Passport pics

Slide 5

Slide 5 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 6

Slide 6 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 7

Slide 7 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 8

Slide 8 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. http://sf.technovangelist.com

Slide 9

Slide 9 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Matt Williams Evangelist at Datadog @technovangelist [email protected] Organizer of DevOps Days Boston 2017 Who am I

Slide 10

Slide 10 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Who is Datadog SaaS-based Monitoring & Analytics Infrastructure, APM, Logs Open Source Agent Trillions of data points per day Booth 109 in Aria Booth 1021 in Venetian We are hiring!!

Slide 11

Slide 11 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Last year I talked about AWS Lambda…

Slide 12

Slide 12 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. @technovangelist AWS Lambda

Slide 13

Slide 13 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. …but AWS Lambda != ec2 @technovangelist

Slide 14

Slide 14 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. …but AWS Lambda != application @technovangelist

Slide 15

Slide 15 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. …applications are complicated @technovangelist

Slide 16

Slide 16 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. …lambdas should respect the single responsibility principle… @technovangelist

Slide 17

Slide 17 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 18

Slide 18 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. @technovangelist

Slide 19

Slide 19 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Other things learned over the last year Serverless is about Automation Serverless is about event-driven computing Scalability is different

Slide 20

Slide 20 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. we can create many serverless functions @technovangelist

Slide 21

Slide 21 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. we can create many serverless functions but how do we orchestrate them? @technovangelist

Slide 22

Slide 22 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Event/Message/Database as Traffic Cop Amazon Kinesis Amazon SQS Amazon DynamoDB @technovangelist

Slide 23

Slide 23 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. https://github.com/Nordstrom/hello-retail

Slide 24

Slide 24 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 25

Slide 25 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Enter AWS Step Functions @technovangelist

Slide 26

Slide 26 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. a step function is @technovangelist

Slide 27

Slide 27 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. a step function is a state machine @technovangelist

Slide 28

Slide 28 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. but what about SWF? (Amazon Simple Workflow Service) @technovangelist

Slide 29

Slide 29 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • State Machine • Transitions • Executions • States • Task • Choice • Succeed/Fail • Pass • Wait • Parallel You define the State Machine with the Amazon States Language A diagram will be generated based on that structure AWS Step Function Concepts @technovangelist

Slide 30

Slide 30 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resource * ResultPath Retry Catch TimeoutSeconds HeartbeatSeconds A task can be a Lambda or an ‘activity’ Type * Next End Comment InputPath OutputPath AWS Step Function States: Task @technovangelist

Slide 31

Slide 31 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. State Example: Task

Slide 32

Slide 32 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Choices Default Type Next End Comment InputPath OutputPath AWS Step Function States: Choice @technovangelist

Slide 33

Slide 33 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. State Example: Choice

Slide 34

Slide 34 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Cause Error Type Next End Comment InputPath OutputPath AWS Step Function States: Succeed / Fail @technovangelist

Slide 35

Slide 35 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. State Example: Succeed / Fail

Slide 36

Slide 36 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Result ResultPath Type Next End Comment InputPath OutputPath AWS Step Function States: Pass @technovangelist

Slide 37

Slide 37 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. State Example: Pass

Slide 38

Slide 38 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Seconds Timestamp SecondsPath TimestampPath Type Next End Comment InputPath OutputPath AWS Step Function States: Wait @technovangelist

Slide 39

Slide 39 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. State Example: Wait

Slide 40

Slide 40 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Branches ResultPath Retry Catch Type Next End Comment InputPath OutputPath AWS Step Function States: Parallel @technovangelist

Slide 41

Slide 41 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. State Example: Parallel

Slide 42

Slide 42 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Many ways to build Step Functions I used Serverless Framework

Slide 43

Slide 43 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Benefits of Serverless Framework Define everything in a single file: • IAM Role Statements • Lambda Function Handlers • Lambda Function Triggers • Step Function States • CloudFormation for other things

Slide 44

Slide 44 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon ECR AWS Lambda Amazon S3 Amazon DynamoDB Amazon CloudFront AWS CodeBuild AWS CodeCommit AWS CloudFormation IAM Amazon API Gateway* AWS Step Functions Amazon SNS

Slide 45

Slide 45 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. serverless.yml service: name: mattw-reinvent2017-build-website plugins: - serverless-pseudo-parameters - serverless-step-functions - serverless-webpack custom: repoName: reinvent2017-website

Slide 46

Slide 46 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. serverless.yml cont’d provider: name: aws runtime: nodejs6.10 stage: ${opt:stage, 'dev'} region: ${opt:region, 'us-east-1'} iamRoleStatements: - Effect: "Allow" Action: - "states:StartExecution" Resource: - ${self:resources.Outputs.MyStateMachine.Value} - Effect: "Allow" Action: - "codecommit:GetCommit" - "codecommit:GetDifferences" Resource: - arn:aws:codecommit:${self:provider.region}:#{AWS::AccountId}:${…custom.repoName}

Slide 47

Slide 47 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. serverless.yml cont’d functions: buildSite: handler: handler.build whatChanged: handler: handler.whatChanged startStepFunction: handler: handler.startStepFunction events: - sns: arn:aws:sns:${…provider.region}:#{AWS::AccountId}:mattw-reinvent-websitebuild environment: statemachine_arn: ${self:resources.Outputs.MyStateMachine.Value}

Slide 48

Slide 48 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. serverless.yml cont’d stepFunctions: stateMachines: FirstStateMachine: name: myStateMachine definition: StartAt: WhatChanged States: WhatChanged: Type: Task Resource: arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}\ :function:${self:service}-${self:provider.stage}-whatChanged Next: BuildDockerOrNot

Slide 49

Slide 49 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Demo @technovangelist

Slide 50

Slide 50 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CodeCommit Amazon SNS AWS Lambda

Slide 51

Slide 51 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. startStepFunction: handler: handler.startStepFunction events: - sns: arn:aws:sns…mattw-reinvent-websitebuild environment: statemachine_arn: ${self:reso…puts.MyStateMachine.Value}

Slide 52

Slide 52 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. StartStepFunction Get Step Function ARN Get SNS details Start SF Execution

Slide 53

Slide 53 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1. What changed in the repo? 2. If docker-related, build docker 1. Tell CodeBuild to rebuild docker image 2. Wait for it 3. Push to ECS repo 3. Use the docker image to build the website 4. Wait for it 5. Send Matt an SMS

Slide 54

Slide 54 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 55

Slide 55 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WhatChanged: Type: Task Resource: arn:aws:lambda:... Next: BuildDockerOrNot Retry : - ErrorEquals: - Lambda.Unknown States.ALL IntervalSeconds: 3 MaxAttempts: 4

Slide 56

Slide 56 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WhatChanged Get the commit that triggered the sf Get the previous commit Whats different If it was docker related return hasDocker = true else return hasDocker = false

Slide 57

Slide 57 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 58

Slide 58 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BuildDockerOrNot: Type: Choice Choices: - Variable: "$.hasDocker" BooleanEquals: true Next: BuildDocker - Variable: "$.hasDocker" BooleanEquals: false Next: BuildSite

Slide 59

Slide 59 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BuildDocker: Type: Task Resource: arn:aws:lambda…buildDocker Next: WaitForDockerBuild

Slide 60

Slide 60 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BuildDocker Start the CodeBuild for docker image

Slide 61

Slide 61 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. phases: install: commands: - nohup /usr/local/bin/dockerd -G dockremap --host=unix:///… - timeout -t 15 sh -c "until docker info; do echo .; sleep 1; done" pre_build: commands: - echo Logging in to Amazon ECR... - $(aws ecr get-login --no-include-email --region us-east-1) build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t mattw-stepfunction-demo:latest . - docker tag mattw-stepfunction-demo:latest ...dkr.ecr.us-east-1.amazona... post_build: commands: - echo Build completed on `date` - echo Pushing the Docker image... - docker push ...dkr.ecr.us-east-1.amazonaws.com/mattw-stepfunc...

Slide 62

Slide 62 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WaitForDockerBuild: Type: Wait Seconds: 20 Next: IsTheContainerBuilt

Slide 63

Slide 63 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IsTheContainerBuilt: Type: Task Resource: arn:aws:lambda... Next: ContainerBuiltOrNot

Slide 64

Slide 64 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 65

Slide 65 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IsTheContainerBuilt Get build info from CodeBuild Check if its done

Slide 66

Slide 66 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Slide 67

Slide 67 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ContainerBuiltOrNot: Type: Choice Choices: - Variable: "$.containerBuildDone" BooleanEquals: true Next: BuildSite - Variable: "$.containerBuildDone" BooleanEquals: false Next: WaitForDockerBuild

Slide 68

Slide 68 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. phases: build: commands: - echo "**************** In Build ******************" - aws s3 sync s3://mattw-reinvent2017-rawsitepages src/posts - export GATSBY_BUILDTIME=$(tail -1 .git/logs/HEAD | awk '{print $5}') - ln -s /backupmod/node_modules ./node_modules - gatsby build - echo "************* Build Complete ****************" post_build: commands: - echo "************** In Post Build *****************" - aws s3 sync public s3://mybucket --delete --acl public-read - echo "*********** Post Build Complete **************" # - command artifacts: files: - public/**/*

Slide 69

Slide 69 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Lambda Amazon S3 Amazon API Gateway* AWS Step Functions AWS Lambda Amazon DynamoDB AWS Lambda AWS CodeBuild

Slide 70

Slide 70 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How about monitoring? We monitor Lambda via Logs MONITORING|unix_epoch_timestamp|value|count| my.metric.name|#tag1:value,tag2 console.log(monitoringstring);

Slide 71

Slide 71 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What are the Lambda Metrics aws.lambda.duration, min, max, sum aws.lambda.errors aws.lambda.invocations aws.lambda.throttles aws.lambda.iterator_age

Slide 72

Slide 72 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What are the Step Function Metrics aws.states.execution_time (.maximum, .minimum) aws.states.executions_aborted, failed, started, succeeded, timed_out aws.states.lambda_functions_scheduled, started, succeeded, timed_out aws.states.execution_throttled aws.states.lambda_function_time (.maximum, .minimum) aws.states.lambda_function_run_time (.maximum, .minimum) aws.states.lambda_function_schedule_time (.maximum, .minimum) aws.states.lambda_functions_started

Slide 73

Slide 73 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Summary Step Functions make AWS Lambda orchestration easy Lambda and Step Functions can be an interesting part of your infrastructure Monitor your Lambda functions to ensure peak performance Source can be found at: https://github.com/DataDog/mattw-reinvent2017-demo

Slide 74

Slide 74 text

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THANK YOU! M a t t h e w W i l l i a m s - @ t e c h n o v a n g e l i s t - D a t a d o g