Elevating workflows with AWS Step Functions and generative AI

Generative AI is changing the way businesses operate—providing opportunities to accelerate innovation. Large language models (LLMs) provide organizations with unique building blocks that would otherwise require extensive investment to build generative AI applications. In this session, discover how new capabilities on AWS Step Functions can make it easier to compose these building blocks into production-grade applications. See a demo of how you can use Step Functions for orchestrating generative AI applications with Amazon Bedrock foundation models. Explore patterns such as prompt chaining that can be tailored to your own specific use case, and enrich your workflows with capabilities from the many AWS services using natural language.

Transcript

1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. B E R L I N | 1 5 + 1 6 M A Y 2 0 2 4
2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Elevating workflows with AWS Step Functions and generative AI Dennis Kieselhorst S V S 2 0 1 (he/ him) Principal Solutions Architect AWS Diana Cheng (she/ her) Senior Solutions Architect AWS
3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda Explore the problem landscape Build a generative AI application Leverage direct integrations with Amazon Bedrock Leverage HTTPS endpoints in AWS Step Functions Get started today
4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Our goal today is to answer how to . . . Easily integrate generative AI into your applications with less code Accelerate building generative AI applications with new features from AWS Step Functions Build resilient workflows from the start
5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Generative AI is the answer. What was the question? Accelerate drug discovery Tailor investment strategies Enhance medical images Create product listings
6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sequencing is hard Human in the loop Function Chaining 1 2 3 1 2 3 Coordination by database 3 2 1 Parallel Processing Retries Error handling
7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Let’s build a generative AI application What we’re building • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. What’s involved with building an application Store data Covert video to text Use FM to generate title Get feedback Decision logic Use FM to generate avatar Send notification
9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions S E R V E R L E S S V I S U A L W O R K F L O W S E R V I C E • Pay per use • Scales automatically • Fully managed • Drag and drop or ASL • Built-in error handling • Integrates with over 220 AWS services
10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Directly compose applications from and
11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. CALLBACK (.waitForTaskToken) Pause until a task token is returned then progress JOB-RUN (.sync) Wait for the request to complete then progress REQUEST-RESPONSE Wait for an HTTP response then progress Control service integrations with call patterns
12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Real world: Live streaming and on-demand video S E R V E R L E S S V I D E O I S A L I V E V I D E O S T R E A M I N G A P P L I C A T I O N , B U I L T W I T H S E R V E R L E S S A R C H I T E C T U R E Live broadcasts from AWS experts Introduction to Building with AWS Step Functions Hello everyone and thanks for watching, please give me an emoji if you can hear me loud and clear. I want to tell you all about my talks coming up later this year at AWS re:Invent. Watch and interact live Playback on-demand
13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Machine learning and generative AI Integrate directly with Amazon Transcribe API to convert video to text https://s12d.com/video
14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why direct integration is powerful app.js const AWS = require('aws-sdk'); const docClient = new AWS.DynamoDB.DocumentClient(); async function queryItems(){ try { const data = await docClient.getItem(params).promise() return data } catch (err) { return err } } var params = { "TableName": "reinvent2023!", "Key": { "PK": {"S": "Wardrobe"}, "SK": {"S": ”shoes"} } } exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } } A Lambda function that queries Amazon DynamoDB has multiple lines of code AWS Lambda Amazon DynamoDB
15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why direct integration is powerful GetItem DLQ Catch error Retry Even a single task “workflow” adds value with built-in error handling, catch, retry, observability, reduction of custom code, and centralized logging of each workload app.js const AWS = require('aws-sdk'); const docClient = new AWS.DynamoDB.DocumentClient(); async function queryItems(){ try { const data = await docClient.getItem(params).promise() return data } catch (err) { return err } } var params = { "TableName": "reinvent2023", "Key": { "PK": {"S": "Wardrobe"}, "SK": {"S": ”shoes"} } } exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } }
16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why direct integration is powerful Drill down to trace execution path of every workload request
17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. API Layer Amazon Bedrock Prompt / text embedding Fine-tune Foundation Models Amazon Bedrock How do I access foundation models?
19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } } Integrating with Amazon Bedrock app.js const AWS = require('aws-sdk'); prompt_data = get_data_from_S3(bucket, key) request = json.dumps({ 'prompt': f'Human:{prompt_data}\n\nAssistant:', 'max_tokens_to_sample': 1028, 'temperature': 1, 'top_k': 250, 'top_p': 0.999, 'stop_sequences': ['\n\nHuman:'] }) client = boto3.client('bedrock-runtime') response = bedrock_client.invoke_model( modelId=event["ModelId"], body=json.dumps(event["Body"]), ) body = json.loads(response["body"].read().decode("utf-8")) response["body"] = body
20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimized integration for Amazon Bedrock
21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Invoke model Create model customization job .sync Prompt: “Write me a title...” S3 bucket Two optimized integrations to simplify building and scaling serverless generative AI applications
22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 1: Generate title and description
23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Select model Dynamic input Amazon States Language (ASL) for Amazon Bedrock Invoke model Inference parameters Use Amazon S3 as input or output
24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS account Outside AWS How do I access foundation models outside AWS?
26. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Accessing a public API Client Server API Secrets retrieval OAuth token handling Graceful retry I/O handling
27. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Accessing a public API Amazon S3 bucket Public API Amazon DynamoDB AWS Secrets Manager 1. Fetch credentials 2. Manage token/rate control 3. Retrieve request data 1 2 3
28. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Public HTTPS API integration
29. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Easily integrate with public HTTPS APIs AWS Step Functions workflow Hugging Face
30. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Easily integrate with public HTTPS APIs Test state Test step independently to validate business logic Authorization OAuth, Basic, and API key Transform data Specify URL-encoding for request body Handle errors Retry and catch using http status code
31. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Authenticate using Amazon EventBridge ConnectionArn Optional Query Parameters, Headers, and Request Body Amazon States Language (ASL) for HTTPS endpoint HTTP invoke Endpoint
32. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 1: Generate multiple titles
33. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. What if something goes wrong… Issue arises Recover quickly Fix the issue
34. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do you handle hard failures? Transcribe Bedrock Hugging Face Transform Transform 2 Human Review Transcribe Bedrock Hugging Face Bedrock Hugging Face Transform Transform 2 Transform 2 Transform
35. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Redrive Executions
36. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Recover from failure faster with Redrive Human Review Transcribe Bedrock Hugging Face Transform 2 Transform Human Review Transcribe Bedrock Hugging Face Transform 2 Transform Transform Transform 2 Transform Transform 2
37. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Execution event history for easy observability E A S I L Y T R A C K E V E R Y S T E P O F Y O U R W O R K F L O W E X E C U T I O N New ExecutionRedriven event Filter down to what you need CloudWatch logs to dive deeper New Redrive count
38. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
39. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Wait for callback—human/external process Callback pattern Call an external resource with a token. Pause workflow for a callback event with the token. Wait for as long as you need—minutes, days, weeks, or months: • Human activity • Third-party API • Legacy application Start End Callback Task Next Task
40. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 2: Human feedback loop • Feedback loop • Async channels— email, WebSocket • API to send response back • Decisions using choice state • Looping to regenerate
41. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
42. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Requirement 3: Generate avatar
43. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Prompt chaining You are an award-winning fiction writer, and you are writing a new story about {story_description}. Before writing the story, describe five characters that will be in the story. Your response should be formatted as a JSON array, with each element in the array containing a "name" key for the character's name and a "description" key with the character's description. An example of a valid response is below, inside… Amazon Bedrock Anthropic’s Claude Prompt chaining • Connect multiple prompts to generate complex content • Feed response from one model to the next Use cases • Writing blogs and articles • Response validation • Conversational LLM
44. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions workflow Amazon SQS Amazon API Gateway AWS Lambda Architecture 1 2 1. Generate title, description... 2. Invoke workflow
45. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions workflow Architecture AWS IoT Core 3 3. Send the title, description to user 4. Send the chosen title and description along with token 4 Amazon API Gateway
46. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Step Functions workflow Architecture 5 5. Send S3 pre-signed URL of avatar to the user AWS IoT Core
47. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved.
48. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. We set out to answer . . . Easily integrate generative AI into your applications with less code Accelerate building generative AI applications with new features from AWS Step Functions Build resilient workflows from the start AWS Step Functions, visual authoring experience, native integrations, and human in the loop Optimized integrations for Amazon Bedrock, AWS Step Functions HTTPS APIs, and TestState API Error handling, Redrive, and observability
49. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Resources https://s12d.com/svs201
50. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Continue your AWS serverless learning Learn at your own pace Increase your knowledge Earn an AWS Serverless badge Expand your serverless skills with our learning plans on AWS Skill Builder Use our Ramp-Up Guides to build your serverless knowledge Demonstrate your knowledge by achieving digital badges https://s12d.com/serverless-learning
51. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! Please complete the session survey in the mobile app Dennis Kieselhorst Diana Cheng dkieselh@amazon.de linkedin.com/in/kieselhorst dchengab@amazon.de @daianacheng