© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Danilo Poccia Principal Evangelist, Serverless @danilop AWS AppSync & Step Functions From frontend to backend, there and back again

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark https://aws.amazon.com/events/summits/amsterdam/

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is GraphQL? GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data Traditional data-fetching GraphQL /posts /postInfo /postJustTitle /postsByAuthor /postNameStartsWithX /commentsOnPost

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. How does GraphQL work? { "id": "1", "name": "Get Milk", "priority": "1" }, { "id": "2", "name": "Go to gym", "priority": "5" },… type Query { getTodos: [Todo] } type Todo { id: ID! name: String description: String priority: Int duedate: String } query { getTodos { id name priority } } Model data with application schema Client requests what it needs Only that data is returned

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Is REST dead then? • When data drives UI • Structured data • Query-driven • Client-driven development Use GraphQL • When you leverage HTTP • Caching, Content types • Hypermedia (HATEOAS) • For resources (e.g., Amazon S3) Use REST Pick the appropriate protocol for your use case

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Schema type Event { id: ID! name: String where: String when: String description: String comments: [Comment] } type Comment { commentId: String! eventId: ID! content: String! createdAt: String! }

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Schema Mutation type Mutation { createEvent( name: String!, when: String!, where: String!, description: String! ): Event deleteEvent(id: ID!): Event commentOnEvent( eventId: ID!, content: String!, createdAt: String! ): Comment }

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Schema Mutation Query type Query { getEvent(id: ID!): Event listEvents( filter: TableEventFilterInput, limit: Int = 10, nextToken: String ): EventConnection }

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Schema Mutation Query Subscription type Subscription { subscribeToEventComments(eventId: String!): Comment @aws_subscribe(mutations: ["commentOnEvent"]) }

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Schema Mutation Query Subscription Realtime? YES Batching? YES Pagination? YES Relations? YES Aggregations? YES Search? YES Offline? YES

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Introducing AWS AppSync AWS AppSync is a managed service for application data using GraphQL with real-time capabilities and an offline programming model Real-time collaboration Offline programming model with sync Flexible database options Fine-grained access control

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AppSync DynamoDB Table Lambda Function GraphQL Schema Upload Schema GraphQL Query Mutation Subscription Real-time Offline Users AppSync API Data Sources Resolvers Pipelines / Functions Velocity Templates (VTL) Auth Cognito User Pool AWS IAM API Key OpenID Connect Elasticsearch Service HTTP Endpoint Relational Database

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AppSync DynamoDB Table Lambda Function GraphQL Schema Upload Schema GraphQL Query Mutation Subscription Real-time Offline Users AppSync API Data Sources Auth Cognito User Pool AWS IAM API Key OpenID Connect Create DynamoDB Table Create Resource Elasticsearch Service Resolvers Pipelines / Functions Velocity Templates (VTL) HTTP Endpoint Relational Database

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AppSync DynamoDB Table Lambda Function GraphQL Schema Upload Schema GraphQL Query Mutation Subscription Real-time Offline Users AppSync API Data Sources Auth Cognito User Pool AWS IAM API Key OpenID Connect Autogenerate Schema & Resolvers Im port a D ynam oD B Table Elasticsearch Service Resolvers Pipelines / Functions Velocity Templates (VTL) HTTP Endpoint Relational Database

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AppSync DynamoDB Table Lambda Function GraphQL Schema Upload Schema GraphQL Query Mutation Subscription Real-time Offline Users AppSync API Data Sources Auth Cognito User Pool AWS IAM API Key OpenID Connect Other Databases O ther D atabases Elasticsearch Service Resolvers Pipelines / Functions Velocity Templates (VTL) HTTP Endpoint Relational Database

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AppSync DynamoDB Table Lambda Function GraphQL Schema Upload Schema GraphQL Query Mutation Subscription Real-time Offline Users AppSync API Data Sources Auth Cognito User Pool AWS IAM API Key OpenID Connect Legacy Applications Legacy Application Elasticsearch Service Resolvers Pipelines / Functions Velocity Templates (VTL) HTTP Endpoint Relational Database

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AppSync DynamoDB Table Lambda Function Elasticsearch Service GraphQL Schema Upload Schema GraphQL Query Mutation Subscription Real-time Offline Users AppSync API Data Sources Auth Cognito User Pool AWS IAM API Key OpenID Connect Com plex Q ueries DynamoDB to Elasticsearch Sync Function Resolvers Pipelines / Functions Velocity Templates (VTL) HTTP Endpoint Relational Database

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. M usixm atch AW S Sum m it M ilan 2018

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Amplify Amplify your apps. Build on a flexible, scalable, and reliable serverless backend. Easy-to-Use Library Choose your cloud services and easily connect them to your app with just a few lines of code. Powerful Toolchain Effortlessly create and maintain sophisticated serverless backends for your apps. Beautiful UI Components Accelerate app development by leveraging our beautiful out-of-the box UI components. O pen Source

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AW S Am plify CLI

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AW S Am plify CLI Effortlessly configure backends Hosting for static assets and websites Easily create APIs and generate client code $ cd $ amplify init $ amplify add $ amplify push $ amplify add hosting $ amplify publish $ amplify codegen add $ amplify codegen generate

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AW S Am plify Console

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AW S Am plify Console

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Sample Schema type Note { id: ID! title: String! content: String! } type Query { getNote(id: ID!): Note listNotes: [Note] } type Mutation { createNote(title: String!, content: String!): Note updateNote(id: ID!, title: String!, content: String!): Note deleteNote(id: ID!): Note } schema { query: Query mutation: Mutation }

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Transformers type Note @model { id: ID! title: String! content: String! } AW S Am plify

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL Transformers type Note @model @auth(rules: [ { allow: owner } ]) @searchable @versioned { id: ID! title: String! content: String! } AW S Am plify

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS AppSync & Amplify Demo

λ λ λ DBMS λ λ λ λ λ λ λ λ λ Queue Modern serverless app

Modern serverless app

"I want to sequence functions" "I want to select functions based on data" "I want to retry functions" "I want try/catch/finally" Functions into apps "I have code that runs for hours" "I want to run functions in parallel"

Coordination must-haves • Scales out • Doesn’t lose state • Deals with errors/timeouts • Easy to build & operate • Auditable

Application Lifecycle in AWS Step Functions Visualize in the Console Define in JSON Monitor Executions

Define in JSON and Then Visualize in the Console { "Comment": "Hello World Example", "StartAt" : "HelloWorld", "States" : { "HelloWorld" : { "Type" : "Task", "Resource" : "${lambdaArn}", "End" : true } } }

Execute One or One Million Start End HelloWorld Start End HelloWorld Start End HelloWorld Start End HelloWorld Start End HelloWorld Start End HelloWorld Start End HelloWorld Start End HelloWorld

Maximum Execution Time AWS Lambda Functions 15 minutes AWS Step Functions State Machines 1 year

Monitor Executions from the Console

Seven State Types Task A single unit of work Choice Adds branching logic Parallel Fork and join the data across tasks Wait Delay for a specified time Fail Stops an execution and marks it as a failure Succeed Stops an execution successfully Pass Passes its input to its output

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ExtractImageMetadata ImageTypeCheck TransformMetadata Rekognition Thumbnail StoreMetadata Start End Synchronous task Task: a single unit of work Synchronous: Lambda { "ExtractImageMetadata": { "Type": "Task", "Resource": "arn:aws:lambda:mars-ter …", … } }

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Step Functions Tasks – Service Integrations AWS Lambda invoke a Lambda function AWS Batch submit a Batch job and wait for it to complete Amazon DynamoDB get, put, update or delete an item Amazon ECS/Fargate run an ECS task and waits for it to complete Amazon SNS publish a message to a SNS topic Amazon SQS send a SQS message AWS Glue start a Glue job Amazon SageMaker create a training or transform job

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ExtractImageMetadata TransformMetadata Rekognition Thumbnail StoreMetadata Start End Task: a single unit of work Asynchronous: "Activity Task", Polled by workers { "ExtractImageMetadata": { "Type": "Task", "Resource": "arn:aws:states:mars-ter …", … } } Asynchronous task

API actions on activity task Register Activity Task - Returns ARN Poll For task (by ARN) Report Success Report Failure Report Heartbeat

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. O pen Source A pache License https://states-language.net/spec.html

Build Visual Workflows from State Types Task Choice Fail Parallel

Image Recognition and Processing Backend Task Choice Fail Parallel https://github.com/awslabs/lambda-refarch-imagerecognition

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Step Functions Demo

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark AWS Step Functions Case Studies

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Vending Pass problem 1. Thirsty consumer presents card, can buy a drink with Vending Pass or debit 2. But mobile wallet display can get out of sync under certain conditions 3.

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Vending Pass problem • Just wait for backend to catch up • Create a two-step state machine: • Wait (90 seconds) • Update mobile wallet • Cheap and simple!

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Transcode 350 clips/day into 14 formats, fast • It’s all done with FFmpeg. The processing time is just about 100% of the video length • Aargh! Video processing problem

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Derive keyframe locations within the source • Split the source at the keyframes • Process segments (typically 0.5 sec per) in parallel • Concatenate segments • Elapsed time: ~20 min down to ~2 minutes Video processing solution

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Details How to get from Lambda to Amazon S3: ffmpeg -f concat –safe 0 -I filelist.ffcat –c copy pipe:0 | aws s3 cp – s3://output-bucket/output-file.mp4

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Details

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let’s peek at the code! "Init": { "Type": "Pass", "ResultPath": "$.operation", "Result": "prepare-split", "Next": "PrepareSplit" }, "PrepareSplit": { "Type": "Task", "Resource": "arn:…:ffmpeg-18HFP9FXFL6P", "Next": "SplitPrepared" }, "SplitPrepared": { "Type": "Pass", "ResultPath": "$.operation", "Result": "perform-split", "Next": "PerformSplit" }, "PerformSplit": { "Type": "Task", "Resource": "arn:…:ffmpeg-18HFP9FXFL6P", "Next": "SplitPerformed" }, "SplitPerformed": { "Type": "Pass", "ResultPath": "$.operation", "Result": "poll-results", "Next": "PreparePoll" }, "PreparePoll": { "Type": "Pass", "ResultPath": "$.pollstatus", "Result": "Submitted", "Next": "WaitForResults" } Hmm…the same function, throughout

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let’s peek at the code! "WaitForResults": { "Type": "Choice", "Choices": [ { "Variable": "$.pollstatus", "StringEquals": "Complete", "Next": "SegmentsCompleteState" }, { "Variable": "$.pollstatus", "StringEquals": "Progressing", "Next": "loop_wait_using_seconds" }, { "Variable": "$.pollstatus", "StringEquals": "Submitted", "Next": "loop_wait_using_seconds" } ] } "loop_wait_using_seconds": { "Type": "Wait", "Seconds": 3, "Next": "loop" }, "loop": { "Type": "Task", "Resource": "arn:…:ffmpeg-18HFP9FXFL6P", "Next": "WaitForResults" }, "SegmentsCompleteState": { "Type": "Pass", "ResultPath": "$.operation", "Result": "concat", "Next": "CompleteState" }, "CompleteState": { "Type": "Task", "Resource": " arn:…:ffmpeg-18HFP9FXFL6P", "End": true } Counts the segments, sets $.pollstatus Stitches segments together

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Takeaways Use Amazon S3 data events to trigger parallel Lambda processing: win Use Amazon S3 URLs to stream video to Lambda: win Scaling to 1,000 Lambdas, rather than 1,000 EC2 instances: win Process video segments in parallel: win

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. More state machines Image Recognition and Processing Backend EBS Snapshot Management https://aws.amazon.com/step-functions/getting-started

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What are you building? #stepfunctions https://console.aws.amazon.com/states/

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Thank you! @danilop