THAT Conference - Orchestrating Serverless with Step Functions

OK, you have built your first Lambda function. It may even be useful (and not just a thumbnail generator). But an application is more than a single function; you need a lot of them. How do you get your functions working together? In this introduction I’ll get you started with Step Functions and show you how easy it is to use them to orchestrate AWS Lambda.

Transcript

1. Orchestrating Serverless with AWS Step Functions
5. matt williams - @technovangelist
9. matt williams - @technovangelist
10. matt williams - @technovangelist …but I didn’t bring a laptop
11. matt williams - @technovangelist Who am I Matt Williams Evangelist at Datadog mattw@datadoghq.com TW: technovangelist YT: technovangelist IG/GH/XYZ: technovangelist Organizer of DevOps Days Boston 2017 & 2018
12. matt williams - @technovangelist Who is Datadog SaaS-based Monitoring & Analytics Infrastructure, APM, Logs Open Source Agent Trillions of data points per day We are hiring!!
13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. I like to talk about Serverless…
14. matt williams - @technovangelist What is AWS Lambda Its all about the function Serverless As far as you are concerned Don’t pay for idle Scale with ease FaaS Node 8.10/6.10, Golang, C# 1/2/2.1, Python 2.7/3.6, Java8 Trigger-based
15. matt williams - @technovangelist Trigger Based? S3 DynamoDB Kinesis SNS SES Cognito CloudFormation Cloudwatch Logs & Events CodeCommit Config Alexa Lex API Gateway IoT CloudFront Lambda http://tvl.st/khvid
16. matt williams - @technovangelist AWS Lambda
17. matt williams - @technovangelist
18. matt williams - @technovangelist That’s pretty cool right? But not perfect You still have to configure everything else S3 DynamoDB API Gateway etc SAM Cloudformation Serverless
19. matt williams - @technovangelist AWS Lambda
20. matt williams - @technovangelist …but serverless !== ec2
21. matt williams - @technovangelist …but serverless !== applications
22. matt williams - @technovangelist …applications are complicated
23. matt williams - @technovangelist …serverless functions should respect the single responsibility principle…
24. matt williams - @technovangelist
25. matt williams - @technovangelist
26. matt williams - @technovangelist we can create many serverless functions
27. matt williams - @technovangelist we can create many serverless functions but how do we orchestrate them?
28. matt williams - @technovangelist Event/Message/Database as Traffic Cop Amazon Kinesis Amazon SQS Amazon DynamoDB
29. matt williams - @technovangelist https://github.com/Nordstrom/hello-retail
30. matt williams - @technovangelist Enter Step Functions
31. matt williams - @technovangelist Step Function Concepts State Machine States Task Choice Succeed/Fail Pass Wait Parallel Transitions Executions You define the State Machine with the Amazon States Language A diagram will be generated based on that structure
32. matt williams - @technovangelist State Example: Task
33. matt williams - @technovangelist State Example: Choice
34. matt williams - @technovangelist State Example: Succeed / Fail
35. matt williams - @technovangelist State Example: Pass
36. matt williams - @technovangelist State Example: Wait
37. matt williams - @technovangelist State Example: Parallel
38. matt williams - @technovangelist Many ways to build AWS Step Functions I used Serverless Framework (http://tvl.st/sls)
39. matt williams - @technovangelist Benefits of Serverless Framework Define everything in a single file: IAM Role Statements Function Handlers Function Triggers State Machine States AWS CloudFormation for other things
40. matt williams - @technovangelist Amazon ECR AWS Lambda Amazon S3 Amazon DynamoDB Amazon CloudFront AWS CodeBuild AWS CodeCommit AWS CloudFormation IAM Amazon API Gateway* AWS Step Functions Amazon SNS
41. matt williams - @technovangelist
43. matt williams - @technovangelist
44. matt williams - @technovangelist
45. matt williams - @technovangelist
46. matt williams - @technovangelist
47. matt williams - @technovangelist 1.Use ML to figure out which slide 2.Is this slide different than the last time? 3.If so, pull slide data from Dynamo 4.If slide different then create document in s3 5.Build the site 6.Wait for it 1.Is it built 7.Done
48. matt williams - @technovangelist Type: Task Resource: Function
49. matt williams - @technovangelist
51. matt williams - @technovangelist Type: Task Resource: Function
52. matt williams - @technovangelist
53. matt williams - @technovangelist
54. matt williams - @technovangelist Type: Task Resource: Function
55. matt williams - @technovangelist
57. matt williams - @technovangelist How about monitoring? We monitor serverless via Logs MONITORING|unix_epoch_timestamp|value|count| my.metric.name|#tag1:value,tag2 console.log(monitoringstring);
58. matt williams - @technovangelist What are the AWS Lambda Metrics aws.lambda.duration, min, max, sum aws.lambda.errors aws.lambda.invocations aws.lambda.throttles aws.lambda.iterator_age
59. matt williams - @technovangelist What are the AWS Step Function Metrics aws.states.execution_time (.maximum, .minimum) aws.states.executions_aborted, failed, started, succeeded, timed_out aws.states.lambda_functions_scheduled, started, succeeded, timed_out aws.states.execution_throttled aws.states.lambda_function_time (.maximum, .minimum) aws.states.lambda_function_run_time (.maximum, .minimum) aws.states.lambda_function_schedule_time (.maximum, .minimum) aws.states.lambda_functions_started
60. matt williams - @technovangelist
61. matt williams - @technovangelist
62. matt williams - @technovangelist
63. matt williams - @technovangelist
64. matt williams - @technovangelist
65. matt williams - @technovangelist
66. matt williams - @technovangelist AWS Lambda and AWS Step Function Challenges • Logging and debugging • Don’t trust any function when it claims a step is complete • Cold start depends on your use case • Every function should do one thing…but granularity depends
67. matt williams - @technovangelist Matt Williams Evangelist at Datadog mattw@datadoghq.com all the socials: technovangelist Organizer of DevOps Days Boston 2017 & 2018
68. matt williams - @technovangelist