Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elevating workflows with AWS Step Functions and generative AI

Elevating workflows with AWS Step Functions and generative AI

Generative AI is changing the way businesses operate—providing opportunities to accelerate innovation. Large language models (LLMs) provide organizations with unique building blocks that would otherwise require extensive investment to build generative AI applications. In this session, discover how new capabilities on AWS Step Functions can make it easier to compose these building blocks into production-grade applications. See a demo of how you can use Step Functions for orchestrating generative AI applications with Amazon Bedrock foundation models. Explore patterns such as prompt chaining that can be tailored to your own specific use case, and enrich your workflows with capabilities from the many AWS services using natural language.

Dennis Kieselhorst

May 15, 2024
Tweet

More Decks by Dennis Kieselhorst

Other Decks in Technology

Transcript

  1. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. B E R L I N | 1 5 + 1 6 M A Y 2 0 2 4
  2. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Elevating workflows with AWS Step Functions and generative AI Dennis Kieselhorst S V S 2 0 1 (he/ him) Principal Solutions Architect AWS Diana Cheng (she/ her) Senior Solutions Architect AWS
  3. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Agenda Explore the problem landscape Build a generative AI application Leverage direct integrations with Amazon Bedrock Leverage HTTPS endpoints in AWS Step Functions Get started today
  4. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Our goal today is to answer how to . . . Easily integrate generative AI into your applications with less code Accelerate building generative AI applications with new features from AWS Step Functions Build resilient workflows from the start
  5. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Generative AI is the answer. What was the question? Accelerate drug discovery Tailor investment strategies Enhance medical images Create product listings
  6. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Sequencing is hard Human in the loop Function Chaining 1 2 3 1 2 3 Coordination by database 3 2 1 Parallel Processing Retries Error handling
  7. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Let’s build a generative AI application What we’re building • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
  8. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. What’s involved with building an application Store data Covert video to text Use FM to generate title Get feedback Decision logic Use FM to generate avatar Send notification
  9. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Step Functions S E R V E R L E S S V I S U A L W O R K F L O W S E R V I C E • Pay per use • Scales automatically • Fully managed • Drag and drop or ASL • Built-in error handling • Integrates with over 220 AWS services
  10. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Directly compose applications from and
  11. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. CALLBACK (.waitForTaskToken) Pause until a task token is returned then progress JOB-RUN (.sync) Wait for the request to complete then progress REQUEST-RESPONSE Wait for an HTTP response then progress Control service integrations with call patterns
  12. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Real world: Live streaming and on-demand video S E R V E R L E S S V I D E O I S A L I V E V I D E O S T R E A M I N G A P P L I C A T I O N , B U I L T W I T H S E R V E R L E S S A R C H I T E C T U R E Live broadcasts from AWS experts Introduction to Building with AWS Step Functions Hello everyone and thanks for watching, please give me an emoji if you can hear me loud and clear. I want to tell you all about my talks coming up later this year at AWS re:Invent. Watch and interact live Playback on-demand
  13. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Machine learning and generative AI Integrate directly with Amazon Transcribe API to convert video to text https://s12d.com/video
  14. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Why direct integration is powerful app.js const AWS = require('aws-sdk'); const docClient = new AWS.DynamoDB.DocumentClient(); async function queryItems(){ try { const data = await docClient.getItem(params).promise() return data } catch (err) { return err } } var params = { "TableName": "reinvent2023!", "Key": { "PK": {"S": "Wardrobe"}, "SK": {"S": ”shoes"} } } exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } } A Lambda function that queries Amazon DynamoDB has multiple lines of code AWS Lambda Amazon DynamoDB
  15. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Why direct integration is powerful GetItem DLQ Catch error Retry Even a single task “workflow” adds value with built-in error handling, catch, retry, observability, reduction of custom code, and centralized logging of each workload app.js const AWS = require('aws-sdk'); const docClient = new AWS.DynamoDB.DocumentClient(); async function queryItems(){ try { const data = await docClient.getItem(params).promise() return data } catch (err) { return err } } var params = { "TableName": "reinvent2023", "Key": { "PK": {"S": "Wardrobe"}, "SK": {"S": ”shoes"} } } exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } }
  16. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Why direct integration is powerful Drill down to trace execution path of every workload request
  17. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
  18. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. API Layer Amazon Bedrock Prompt / text embedding Fine-tune Foundation Models Amazon Bedrock How do I access foundation models?
  19. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. exports.handler = async (event, context) => { try { const data = await queryItems() return { body: JSON.stringify(data) } } catch (err) { return { error: err } } } Integrating with Amazon Bedrock app.js const AWS = require('aws-sdk'); prompt_data = get_data_from_S3(bucket, key) request = json.dumps({ 'prompt': f'Human:{prompt_data}\n\nAssistant:', 'max_tokens_to_sample': 1028, 'temperature': 1, 'top_k': 250, 'top_p': 0.999, 'stop_sequences': ['\n\nHuman:'] }) client = boto3.client('bedrock-runtime') response = bedrock_client.invoke_model( modelId=event["ModelId"], body=json.dumps(event["Body"]), ) body = json.loads(response["body"].read().decode("utf-8")) response["body"] = body
  20. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Optimized integration for Amazon Bedrock
  21. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Invoke model Create model customization job .sync Prompt: “Write me a title...” S3 bucket Two optimized integrations to simplify building and scaling serverless generative AI applications
  22. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Requirement 1: Generate title and description
  23. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Select model Dynamic input Amazon States Language (ASL) for Amazon Bedrock Invoke model Inference parameters Use Amazon S3 as input or output
  24. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
  25. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS account Outside AWS How do I access foundation models outside AWS?
  26. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Accessing a public API Client Server API Secrets retrieval OAuth token handling Graceful retry I/O handling
  27. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Accessing a public API Amazon S3 bucket Public API Amazon DynamoDB AWS Secrets Manager 1. Fetch credentials 2. Manage token/rate control 3. Retrieve request data 1 2 3
  28. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Public HTTPS API integration
  29. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Easily integrate with public HTTPS APIs AWS Step Functions workflow Hugging Face
  30. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Easily integrate with public HTTPS APIs Test state Test step independently to validate business logic Authorization OAuth, Basic, and API key Transform data Specify URL-encoding for request body Handle errors Retry and catch using http status code
  31. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Authenticate using Amazon EventBridge ConnectionArn Optional Query Parameters, Headers, and Request Body Amazon States Language (ASL) for HTTPS endpoint HTTP invoke Endpoint
  32. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Requirement 1: Generate multiple titles
  33. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. What if something goes wrong… Issue arises Recover quickly Fix the issue
  34. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do you handle hard failures? Transcribe Bedrock Hugging Face Transform Transform 2 Human Review Transcribe Bedrock Hugging Face Bedrock Hugging Face Transform Transform 2 Transform 2 Transform
  35. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Redrive Executions
  36. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Recover from failure faster with Redrive Human Review Transcribe Bedrock Hugging Face Transform 2 Transform Human Review Transcribe Bedrock Hugging Face Transform 2 Transform Transform Transform 2 Transform Transform 2
  37. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Execution event history for easy observability E A S I L Y T R A C K E V E R Y S T E P O F Y O U R W O R K F L O W E X E C U T I O N New ExecutionRedriven event Filter down to what you need CloudWatch logs to dive deeper New Redrive count
  38. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
  39. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Wait for callback—human/external process Callback pattern Call an external resource with a token. Pause workflow for a callback event with the token. Wait for as long as you need—minutes, days, weeks, or months: • Human activity • Third-party API • Legacy application Start End Callback Task Next Task
  40. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Requirement 2: Human feedback loop • Feedback loop • Async channels— email, WebSocket • API to send response back • Decisions using choice state • Looping to regenerate
  41. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Building a generative AI application • Create multiple titles and descriptions for video • Ask human to provide feedback • Create an avatar for the video
  42. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Requirement 3: Generate avatar
  43. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Prompt chaining You are an award-winning fiction writer, and you are writing a new story about {story_description}. Before writing the story, describe five characters that will be in the story. Your response should be formatted as a JSON array, with each element in the array containing a "name" key for the character's name and a "description" key with the character's description. An example of a valid response is below, inside… Amazon Bedrock Anthropic’s Claude Prompt chaining • Connect multiple prompts to generate complex content • Feed response from one model to the next Use cases • Writing blogs and articles • Response validation • Conversational LLM
  44. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Step Functions workflow Amazon SQS Amazon API Gateway AWS Lambda Architecture 1 2 1. Generate title, description... 2. Invoke workflow
  45. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Step Functions workflow Architecture AWS IoT Core 3 3. Send the title, description to user 4. Send the chosen title and description along with token 4 Amazon API Gateway
  46. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Step Functions workflow Architecture 5 5. Send S3 pre-signed URL of avatar to the user AWS IoT Core
  47. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. We set out to answer . . . Easily integrate generative AI into your applications with less code Accelerate building generative AI applications with new features from AWS Step Functions Build resilient workflows from the start AWS Step Functions, visual authoring experience, native integrations, and human in the loop Optimized integrations for Amazon Bedrock, AWS Step Functions HTTPS APIs, and TestState API Error handling, Redrive, and observability
  48. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Resources https://s12d.com/svs201
  49. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Continue your AWS serverless learning Learn at your own pace Increase your knowledge Earn an AWS Serverless badge Expand your serverless skills with our learning plans on AWS Skill Builder Use our Ramp-Up Guides to build your serverless knowledge Demonstrate your knowledge by achieving digital badges https://s12d.com/serverless-learning
  50. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Thank you! Please complete the session survey in the mobile app Dennis Kieselhorst Diana Cheng [email protected] linkedin.com/in/kieselhorst [email protected] @daianacheng