Deploying a production-level RAG

Transcript

2. Workshop: Developing a production- level RAG workflow Bethany Jepchumba AI Cloud Advocate, Microsoft @bethanyjep
3. Agenda  Overview of LLMOps  Overview of Prompt flow  Build a production level RAG solution
4. Demo Contoso Chat (Agent) deployed endpoint for RAG solution Contoso Outdoors (Web) client integration with deployed endpoint 1. Basic Chat 2. Contextualized to customer 3. Grounded in product data 4. Content safety by default 1️⃣ 2️⃣ 3️⃣ 4️⃣ What | Contoso Chat – Customer Support Agent
5. LLM Lifecycle in the real world Saf e Rollout/Staging Managing SEND FEEDBACK PREPARE FOR APP DEPLOYMENT ADVANCE PROJECT Find LLMs Try prompts Hypo thesis BUSINESS NEED Deploy LLM App/UI Quo ta and cost management REVERT PROJECT Prompt Engine ering or Fine-tuning Retrieval Augmente d Generation Evalua tion Exceptio n Handling C ontent Filtering M onitoring Operationalizing Building/ augmenting Ideating/ exploring
6. Identify business use case Run flow against sample data Evaluate prompt flow Satisfied? Run flow against larger dataset Evaluate prompt flow Satisfied? Deploy endpoint No No Yes Yes Modify flow (prompts and tools, etc.) Integrate into application Develop flow based on prompt to extend the capability Connect to your data Build your basic prompt flow Add monitoring and alerts 1. Ideating/exploring 2. Building/augmenting 3. Operationalizing LLM Lifecycle Considerations
7. What is Prompt Flow? Development tool to streamline the entire development cycle of LLM applications. Simplifies the process of prototyping, experimenting, iterating, deploying and monitoring your AI applications.
8. Retrieval Augmented Generation (RAG) User question Retrieve related data Retriever over Knowledge Base Augment the prompt Generate response Large Language Model Send Results Workflow
9. Azure Open AI Question embedding Chat Flow Request Orchestration Azure AI Search Product Document Vector Search Azure Cosmos DB Customer Database lookup Azure Open AI Prompt to GPT 35 Turbo Answer Question How | Use Prompt Flow with RAG Architecture
10. Azure AI Resource Azure AI Services Azure AI Project Azure AI Search Azure CosmosDB Manage Ops including billing, permissions, policies, compute, service access Built-in capabilities you can activate. Use default Open AI and Content Safety services Build Workspace to organize work & save state. Use Prompt Flow, Filters & Deployments Vector Search required for RAG. Add indexes for your product data for efficient query Managed NoSQL database for app data at scale. Use it for customer id and order history Where | Build on Azure. With Azure AI Platform
11. Evaluation metrics Do the model's generated answers align with information from the input source Groundedness Is the model's generated response relevant for given question Relevance Does the language model produce output that resembles human- like language Coherence
12. Join the Azure AI Community on Discord Find workshop resources and chat with presenters in the #ms-ai-tour forums: aka.ms/aitour/contoso-chat/discord
13. aka.ms/aitour/contoso-chat Walkthrough in the GitHub repository
14. Thank you!