Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Baltimore DevOps - Orchestrating Serverless with AWS Step Functions

162d77c31766ab995b4503915a58235d?s=47 Matt Williams
September 12, 2018

Baltimore DevOps - Orchestrating Serverless with AWS Step Functions

OK, you have built your first Lambda function. It may even be useful (and not just a thumbnail generator). But an application is more than a single function; you need a lot of them. How do you get your functions working together? In this introduction Matt Williams will get you started with Step Functions and show you how easy it is to use them to orchestrate AWS Lambda.

162d77c31766ab995b4503915a58235d?s=128

Matt Williams

September 12, 2018
Tweet

Transcript

  1. Orchestrating Serverless with AWS Step Functions

  2. None
  3. matt williams - @technovangelist Baltimore DevOps

  4. None
  5. None
  6. None
  7. matt williams - @technovangelist Baltimore DevOps

  8. matt williams - @technovangelist Baltimore DevOps …but I didn’t bring

    a laptop
  9. matt williams - @technovangelist Baltimore DevOps Who am I Matt

    Williams Evangelist at Datadog mattw@datadoghq.com TW: technovangelist YT: technovangelist IG/GH/XYZ: technovangelist Organizer of: DevOps Days Boston 2017 & 2018 Serverless Days Boston 2018
  10. matt williams - @technovangelist Baltimore DevOps Who is Datadog SaaS-based

    Monitoring & Analytics Infrastructure, APM, Logs Open Source Agent Trillions of data points per day We are hiring!!
  11. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. I like to talk about Serverless…
  12. matt williams - @technovangelist Baltimore DevOps What is AWS Lambda

    Its all about the function Serverless As far as you are concerned Don’t pay for idle Scale with ease FaaS Node 8.10/6.10, Golang, C# 1/2/2.1, Python 2.7/3.6, Java8 Trigger-based
  13. matt williams - @technovangelist Baltimore DevOps Trigger Based? S3 DynamoDB

    Kinesis SNS SES Cognito CloudFormation Cloudwatch Logs & Events CodeCommit Config Alexa Lex API Gateway IoT CloudFront Lambda http://tvl.st/khvid
  14. matt williams - @technovangelist Baltimore DevOps AWS Lambda

  15. matt williams - @technovangelist Baltimore DevOps

  16. matt williams - @technovangelist Baltimore DevOps That’s pretty cool right?

    But not perfect You still have to configure everything else S3 DynamoDB API Gateway etc SAM Cloudformation Serverless
  17. matt williams - @technovangelist Baltimore DevOps AWS Lambda

  18. matt williams - @technovangelist Baltimore DevOps …but serverless !== ec2

  19. matt williams - @technovangelist Baltimore DevOps …but serverless !== applications

  20. matt williams - @technovangelist Baltimore DevOps …applications are complicated

  21. matt williams - @technovangelist Baltimore DevOps …serverless functions should respect

    the single responsibility principle…
  22. matt williams - @technovangelist Baltimore DevOps

  23. matt williams - @technovangelist Baltimore DevOps

  24. matt williams - @technovangelist Baltimore DevOps we can create many

    serverless functions
  25. matt williams - @technovangelist Baltimore DevOps we can create many

    serverless functions but how do we orchestrate them?
  26. matt williams - @technovangelist Baltimore DevOps Event/Message/Database as Traffic Cop

    Amazon Kinesis Amazon SQS Amazon DynamoDB
  27. matt williams - @technovangelist Baltimore DevOps https://github.com/Nordstrom/hello-retail

  28. matt williams - @technovangelist Baltimore DevOps Enter Step Functions

  29. matt williams - @technovangelist Baltimore DevOps Step Function Concepts State

    Machine States Task Choice Succeed/Fail Pass Wait Parallel Transitions Executions You define the State Machine with the Amazon States Language A diagram will be generated based on that structure
  30. matt williams - @technovangelist Baltimore DevOps State Example: Task

  31. matt williams - @technovangelist Baltimore DevOps State Example: Choice

  32. matt williams - @technovangelist Baltimore DevOps State Example: Succeed /

    Fail
  33. matt williams - @technovangelist Baltimore DevOps State Example: Pass

  34. matt williams - @technovangelist Baltimore DevOps State Example: Wait

  35. matt williams - @technovangelist Baltimore DevOps State Example: Parallel

  36. matt williams - @technovangelist Baltimore DevOps Many ways to build

    AWS Step Functions I used Serverless Framework (http://tvl.st/sls)
  37. matt williams - @technovangelist Baltimore DevOps Benefits of Serverless Framework

    Define everything in a single file: IAM Role Statements Function Handlers Function Triggers State Machine States AWS CloudFormation for other things
  38. matt williams - @technovangelist Baltimore DevOps Amazon ECR AWS Lambda

    Amazon S3 Amazon DynamoDB Amazon CloudFront AWS CodeBuild AWS CodeCommit AWS CloudFormation IAM Amazon API Gateway* AWS Step Functions Amazon SNS
  39. matt williams - @technovangelist Baltimore DevOps

  40. None
  41. matt williams - @technovangelist Baltimore DevOps

  42. matt williams - @technovangelist Baltimore DevOps

  43. matt williams - @technovangelist Baltimore DevOps

  44. matt williams - @technovangelist Baltimore DevOps

  45. matt williams - @technovangelist Baltimore DevOps 1.Use ML to figure

    out which slide 2.Is this slide different than the last time? 3.If so, pull slide data from Dynamo 4.If slide different then create document in s3 5.Build the site 6.Wait for it 1.Is it built 7.Done
  46. matt williams - @technovangelist Baltimore DevOps Type: Task Resource: Function

  47. matt williams - @technovangelist Baltimore DevOps

  48. None
  49. matt williams - @technovangelist Baltimore DevOps Type: Task Resource: Function

  50. matt williams - @technovangelist Baltimore DevOps

  51. matt williams - @technovangelist Baltimore DevOps

  52. matt williams - @technovangelist Baltimore DevOps Type: Task Resource: Function

  53. matt williams - @technovangelist Baltimore DevOps

  54. None
  55. matt williams - @technovangelist Baltimore DevOps How about monitoring? We

    monitor serverless via Logs MONITORING|unix_epoch_timestamp|value|count| my.metric.name|#tag1:value,tag2 console.log(monitoringstring);
  56. matt williams - @technovangelist Baltimore DevOps What are the AWS

    Lambda Metrics aws.lambda.duration, min, max, sum aws.lambda.errors aws.lambda.invocations aws.lambda.throttles aws.lambda.iterator_age
  57. matt williams - @technovangelist Baltimore DevOps What are the AWS

    Step Function Metrics aws.states.execution_time (.maximum, .minimum) aws.states.executions_aborted, failed, started, succeeded, timed_out aws.states.lambda_functions_scheduled, started, succeeded, timed_out aws.states.execution_throttled aws.states.lambda_function_time (.maximum, .minimum) aws.states.lambda_function_run_time (.maximum, .minimum) aws.states.lambda_function_schedule_time (.maximum, .minimum) aws.states.lambda_functions_started
  58. matt williams - @technovangelist Baltimore DevOps

  59. matt williams - @technovangelist Baltimore DevOps

  60. matt williams - @technovangelist Baltimore DevOps

  61. matt williams - @technovangelist Baltimore DevOps

  62. matt williams - @technovangelist Baltimore DevOps

  63. matt williams - @technovangelist Baltimore DevOps

  64. matt williams - @technovangelist Baltimore DevOps AWS Lambda and AWS

    Step Function Challenges • Logging and debugging • Don’t trust any function when it claims a step is complete • Cold start depends on your use case • Every function should do one thing…but granularity depends
  65. matt williams - @technovangelist Baltimore DevOps Matt Williams Evangelist at

    Datadog mattw@datadoghq.com all the socials: technovangelist Organizer of DevOps Days Boston 2017 & 2018