Slide 1

Slide 1 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. B E R L I N | 1 5 + 1 6 M A Y 2 0 2 4

Slide 2

Slide 2 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Elevating workflows with AWS Step Functions and generative AI Dennis Kieselhorst S V S 2 0 1 (he/ him) Principal Solutions Architect AWS Diana Cheng (she/ her) Senior Solutions Architect AWS

Slide 3

Slide 3 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda Explore the problem landscape Build a generative AI application Leverage direct integrations with Amazon Bedrock Leverage HTTPS endpoints in AWS Step Functions Get started today

Slide 4

Slide 4 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our goal today is to answer how to . . . Easily integrate generative AI into your applications with less code Accelerate building generative AI applications with new features from AWS Step Functions Build resilient workflows from the start

Slide 5

Slide 5 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Generative AI is the answer. What was the question? Accelerate drug discovery Tailor investment strategies Enhance medical images Create product listings

Slide 6

Slide 6 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sequencing is hard Human in the loop Function Chaining 1 2 3 1 2 3 Coordination by database 3 2 1 Parallel Processing Retries Error handling

Slide 7

Slide 7 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s build a generative AI application What we’re building • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video

Slide 8

Slide 8 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. What’s involved with building an application Store data Covert video to text Use FM to generate title Get feedback Decision logic Use FM to generate avatar Send notification

Slide 9

Slide 9 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions S E R V E R L E S S V I S U A L W O R K F L O W S E R V I C E • Pay per use • Scales automatically • Fully managed • Drag and drop or ASL • Built-in error handling • Integrates with over 220 AWS services

Slide 10

Slide 10 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Directly compose applications from and

Slide 11

Slide 11 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. CALLBACK (.waitForTaskToken) Pause until a task token is returned then progress JOB-RUN (.sync) Wait for the request to complete then progress REQUEST-RESPONSE Wait for an HTTP response then progress Control service integrations with call patterns

Slide 12

Slide 12 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Real world: Live streaming and on-demand video S E R V E R L E S S V I D E O I S A L I V E V I D E O S T R E A M I N G A P P L I C A T I O N , B U I L T W I T H S E R V E R L E S S A R C H I T E C T U R E Live broadcasts from AWS experts Introduction to Building with AWS Step Functions Hello everyone and thanks for watching, please give me an emoji if you can hear me loud and clear. I want to tell you all about my talks coming up later this year at AWS re:Invent. Watch and interact live Playback on-demand

Slide 13

Slide 13 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Machine learning and generative AI Integrate directly with Amazon Transcribe API to convert video to text https://s12d.com/video

Slide 14

Slide 14 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why direct integration is powerful app.js const AWS = require('aws-sdk'); const docClient = new AWS.DynamoDB.DocumentClient(); async function queryItems(){ try { const data = await docClient.getItem(params).promise() return data } catch (err) { return err } } var params = { "TableName": "reinvent2023!", "Key": { "PK": {"S": "Wardrobe"}, "SK": {"S": ”shoes"} } } exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } } A Lambda function that queries Amazon DynamoDB has multiple lines of code AWS Lambda Amazon DynamoDB

Slide 15

Slide 15 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why direct integration is powerful GetItem DLQ Catch error Retry Even a single task “workflow” adds value with built-in error handling, catch, retry, observability, reduction of custom code, and centralized logging of each workload app.js const AWS = require('aws-sdk'); const docClient = new AWS.DynamoDB.DocumentClient(); async function queryItems(){ try { const data = await docClient.getItem(params).promise() return data } catch (err) { return err } } var params = { "TableName": "reinvent2023", "Key": { "PK": {"S": "Wardrobe"}, "SK": {"S": ”shoes"} } } exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } }

Slide 16

Slide 16 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why direct integration is powerful Drill down to trace execution path of every workload request

Slide 17

Slide 17 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video

Slide 18

Slide 18 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. API Layer Amazon Bedrock Prompt / text embedding Fine-tune Foundation Models Amazon Bedrock How do I access foundation models?

Slide 19

Slide 19 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } } Integrating with Amazon Bedrock app.js const AWS = require('aws-sdk'); prompt_data = get_data_from_S3(bucket, key) request = json.dumps({ 'prompt': f'Human:{prompt_data}\n\nAssistant:', 'max_tokens_to_sample': 1028, 'temperature': 1, 'top_k': 250, 'top_p': 0.999, 'stop_sequences': ['\n\nHuman:'] }) client = boto3.client('bedrock-runtime') response = bedrock_client.invoke_model( modelId=event["ModelId"], body=json.dumps(event["Body"]), ) body = json.loads(response["body"].read().decode("utf-8")) response["body"] = body

Slide 20

Slide 20 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimized integration for Amazon Bedrock

Slide 21

Slide 21 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Invoke model Create model customization job .sync Prompt: “Write me a title...” S3 bucket Two optimized integrations to simplify building and scaling serverless generative AI applications

Slide 22

Slide 22 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 1: Generate title and description

Slide 23

Slide 23 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Select model Dynamic input Amazon States Language (ASL) for Amazon Bedrock Invoke model Inference parameters Use Amazon S3 as input or output

Slide 24

Slide 24 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video

Slide 25

Slide 25 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS account Outside AWS How do I access foundation models outside AWS?

Slide 26

Slide 26 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Accessing a public API Client Server API Secrets retrieval OAuth token handling Graceful retry I/O handling

Slide 27

Slide 27 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Accessing a public API Amazon S3 bucket Public API Amazon DynamoDB AWS Secrets Manager 1. Fetch credentials 2. Manage token/rate control 3. Retrieve request data 1 2 3

Slide 28

Slide 28 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Public HTTPS API integration

Slide 29

Slide 29 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Easily integrate with public HTTPS APIs AWS Step Functions workflow Hugging Face

Slide 30

Slide 30 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Easily integrate with public HTTPS APIs Test state Test step independently to validate business logic Authorization OAuth, Basic, and API key Transform data Specify URL-encoding for request body Handle errors Retry and catch using http status code

Slide 31

Slide 31 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Authenticate using Amazon EventBridge ConnectionArn Optional Query Parameters, Headers, and Request Body Amazon States Language (ASL) for HTTPS endpoint HTTP invoke Endpoint

Slide 32

Slide 32 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 1: Generate multiple titles

Slide 33

Slide 33 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. What if something goes wrong… Issue arises Recover quickly Fix the issue

Slide 34

Slide 34 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you handle hard failures? Transcribe Bedrock Hugging Face Transform Transform 2 Human Review Transcribe Bedrock Hugging Face Bedrock Hugging Face Transform Transform 2 Transform 2 Transform

Slide 35

Slide 35 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Redrive Executions

Slide 36

Slide 36 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Recover from failure faster with Redrive Human Review Transcribe Bedrock Hugging Face Transform 2 Transform Human Review Transcribe Bedrock Hugging Face Transform 2 Transform Transform Transform 2 Transform Transform 2

Slide 37

Slide 37 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Execution event history for easy observability E A S I L Y T R A C K E V E R Y S T E P O F Y O U R W O R K F L O W E X E C U T I O N New ExecutionRedriven event Filter down to what you need CloudWatch logs to dive deeper New Redrive count

Slide 38

Slide 38 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video

Slide 39

Slide 39 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Wait for callback—human/external process Callback pattern Call an external resource with a token. Pause workflow for a callback event with the token. Wait for as long as you need—minutes, days, weeks, or months: • Human activity • Third-party API • Legacy application Start End Callback Task Next Task

Slide 40

Slide 40 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 2: Human feedback loop • Feedback loop • Async channels— email, WebSocket • API to send response back • Decisions using choice state • Looping to regenerate

Slide 41

Slide 41 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video

Slide 42

Slide 42 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 3: Generate avatar

Slide 43

Slide 43 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Prompt chaining You are an award-winning fiction writer, and you are writing a new story about {story_description}. Before writing the story, describe five characters that will be in the story. Your response should be formatted as a JSON array, with each element in the array containing a "name" key for the character's name and a "description" key with the character's description. An example of a valid response is below, inside… Amazon Bedrock Anthropic’s Claude Prompt chaining • Connect multiple prompts to generate complex content • Feed response from one model to the next Use cases • Writing blogs and articles • Response validation • Conversational LLM

Slide 44

Slide 44 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions workflow Amazon SQS Amazon API Gateway AWS Lambda Architecture 1 2 1. Generate title, description... 2. Invoke workflow

Slide 45

Slide 45 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions workflow Architecture AWS IoT Core 3 3. Send the title, description to user 4. Send the chosen title and description along with token 4 Amazon API Gateway

Slide 46

Slide 46 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions workflow Architecture 5 5. Send S3 pre-signed URL of avatar to the user AWS IoT Core

Slide 47

Slide 47 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Slide 48

Slide 48 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. We set out to answer . . . Easily integrate generative AI into your applications with less code Accelerate building generative AI applications with new features from AWS Step Functions Build resilient workflows from the start AWS Step Functions, visual authoring experience, native integrations, and human in the loop Optimized integrations for Amazon Bedrock, AWS Step Functions HTTPS APIs, and TestState API Error handling, Redrive, and observability

Slide 49

Slide 49 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Resources https://s12d.com/svs201

Slide 50

Slide 50 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Continue your AWS serverless learning Learn at your own pace Increase your knowledge Earn an AWS Serverless badge Expand your serverless skills with our learning plans on AWS Skill Builder Use our Ramp-Up Guides to build your serverless knowledge Demonstrate your knowledge by achieving digital badges https://s12d.com/serverless-learning

Slide 51

Slide 51 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! Please complete the session survey in the mobile app Dennis Kieselhorst Diana Cheng dkieselh@amazon.de linkedin.com/in/kieselhorst dchengab@amazon.de @daianacheng