Upgrade to Pro — share decks privately, control downloads, hide ads and more …

THAT Conference - Orchestrating Serverless with Step Functions

THAT Conference - Orchestrating Serverless with Step Functions

OK, you have built your first Lambda function. It may even be useful (and not just a thumbnail generator). But an application is more than a single function; you need a lot of them. How do you get your functions working together? In this introduction I’ll get you started with Step Functions and show you how easy it is to use them to orchestrate AWS Lambda.


Matt Williams

August 07, 2018

More Decks by Matt Williams

Other Decks in Technology


  1. Orchestrating Serverless with AWS Step Functions

  2. None
  3. None
  4. None
  5. matt williams - @technovangelist

  6. None
  7. None
  8. None
  9. matt williams - @technovangelist

  10. matt williams - @technovangelist …but I didn’t bring a laptop

  11. matt williams - @technovangelist Who am I Matt Williams Evangelist

    at Datadog mattw@datadoghq.com TW: technovangelist YT: technovangelist IG/GH/XYZ: technovangelist Organizer of DevOps Days Boston 2017 & 2018
  12. matt williams - @technovangelist Who is Datadog SaaS-based Monitoring &

    Analytics Infrastructure, APM, Logs Open Source Agent Trillions of data points per day We are hiring!!
  13. © 2017, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. I like to talk about Serverless…
  14. matt williams - @technovangelist What is AWS Lambda Its all

    about the function Serverless As far as you are concerned Don’t pay for idle Scale with ease FaaS Node 8.10/6.10, Golang, C# 1/2/2.1, Python 2.7/3.6, Java8 Trigger-based
  15. matt williams - @technovangelist Trigger Based? S3 DynamoDB Kinesis SNS

    SES Cognito CloudFormation Cloudwatch Logs & Events CodeCommit Config Alexa Lex API Gateway IoT CloudFront Lambda http://tvl.st/khvid
  16. matt williams - @technovangelist AWS Lambda

  17. matt williams - @technovangelist

  18. matt williams - @technovangelist That’s pretty cool right? But not

    perfect You still have to configure everything else S3 DynamoDB API Gateway etc SAM Cloudformation Serverless
  19. matt williams - @technovangelist AWS Lambda

  20. matt williams - @technovangelist …but serverless !== ec2

  21. matt williams - @technovangelist …but serverless !== applications

  22. matt williams - @technovangelist …applications are complicated

  23. matt williams - @technovangelist …serverless functions should respect the single

    responsibility principle…
  24. matt williams - @technovangelist

  25. matt williams - @technovangelist

  26. matt williams - @technovangelist we can create many serverless functions

  27. matt williams - @technovangelist we can create many serverless functions

    but how do we orchestrate them?
  28. matt williams - @technovangelist Event/Message/Database as Traffic Cop Amazon Kinesis

    Amazon SQS Amazon DynamoDB
  29. matt williams - @technovangelist https://github.com/Nordstrom/hello-retail

  30. matt williams - @technovangelist Enter Step Functions

  31. matt williams - @technovangelist Step Function Concepts State Machine States

    Task Choice Succeed/Fail Pass Wait Parallel Transitions Executions You define the State Machine with the Amazon States Language A diagram will be generated based on that structure
  32. matt williams - @technovangelist State Example: Task

  33. matt williams - @technovangelist State Example: Choice

  34. matt williams - @technovangelist State Example: Succeed / Fail

  35. matt williams - @technovangelist State Example: Pass

  36. matt williams - @technovangelist State Example: Wait

  37. matt williams - @technovangelist State Example: Parallel

  38. matt williams - @technovangelist Many ways to build AWS Step

    Functions I used Serverless Framework (http://tvl.st/sls)
  39. matt williams - @technovangelist Benefits of Serverless Framework Define everything

    in a single file: IAM Role Statements Function Handlers Function Triggers State Machine States AWS CloudFormation for other things
  40. matt williams - @technovangelist Amazon ECR AWS Lambda Amazon S3

    Amazon DynamoDB Amazon CloudFront AWS CodeBuild AWS CodeCommit AWS CloudFormation IAM Amazon API Gateway* AWS Step Functions Amazon SNS
  41. matt williams - @technovangelist

  42. None
  43. matt williams - @technovangelist

  44. matt williams - @technovangelist

  45. matt williams - @technovangelist

  46. matt williams - @technovangelist

  47. matt williams - @technovangelist 1.Use ML to figure out which

    slide 2.Is this slide different than the last time? 3.If so, pull slide data from Dynamo 4.If slide different then create document in s3 5.Build the site 6.Wait for it 1.Is it built 7.Done
  48. matt williams - @technovangelist Type: Task Resource: Function

  49. matt williams - @technovangelist

  50. None
  51. matt williams - @technovangelist Type: Task Resource: Function

  52. matt williams - @technovangelist

  53. matt williams - @technovangelist

  54. matt williams - @technovangelist Type: Task Resource: Function

  55. matt williams - @technovangelist

  56. None
  57. matt williams - @technovangelist How about monitoring? We monitor serverless

    via Logs MONITORING|unix_epoch_timestamp|value|count| my.metric.name|#tag1:value,tag2 console.log(monitoringstring);
  58. matt williams - @technovangelist What are the AWS Lambda Metrics

    aws.lambda.duration, min, max, sum aws.lambda.errors aws.lambda.invocations aws.lambda.throttles aws.lambda.iterator_age
  59. matt williams - @technovangelist What are the AWS Step Function

    Metrics aws.states.execution_time (.maximum, .minimum) aws.states.executions_aborted, failed, started, succeeded, timed_out aws.states.lambda_functions_scheduled, started, succeeded, timed_out aws.states.execution_throttled aws.states.lambda_function_time (.maximum, .minimum) aws.states.lambda_function_run_time (.maximum, .minimum) aws.states.lambda_function_schedule_time (.maximum, .minimum) aws.states.lambda_functions_started
  60. matt williams - @technovangelist

  61. matt williams - @technovangelist

  62. matt williams - @technovangelist

  63. matt williams - @technovangelist

  64. matt williams - @technovangelist

  65. matt williams - @technovangelist

  66. matt williams - @technovangelist AWS Lambda and AWS Step Function

    Challenges • Logging and debugging • Don’t trust any function when it claims a step is complete • Cold start depends on your use case • Every function should do one thing…but granularity depends
  67. matt williams - @technovangelist Matt Williams Evangelist at Datadog mattw@datadoghq.com

    all the socials: technovangelist Organizer of DevOps Days Boston 2017 & 2018
  68. matt williams - @technovangelist