Mastering LLM-Based AI Applications: A Deep Dive into Prompt Flow Henk Boelman Senior Cloud Advocate @ Microsoft

@hboelman Github.com/hnky https://www.henkboelman.com Henk Boelman Senior Cloud Advocate @ Microsoft

Bring your own data

Using your data Challenge I want to reason over my very long documents, but the token limits are not sufficient Solution Build a vector database Retrieve relevant data and construct prompt at run-time

Retrievers: Externalizing Knowledge “Find the most relevant snippets in a large data collection, using unstructured input as query” App UX Orchestrator Azure OpenAI Azure Cognitive Search Data Sources (files, databases, etc.) Query → Knowledge Prompt + Knowledge → Response

Retrieving Using Semantic Similarity Vector representations (or embeddings)  Learned such that “close” vectors represent items with similar meaning  May encode words, sentences, images, audio, etc.  Some map multiple media types into the same space  Azure OpenAI embeddings API, OSS embeddings (e.g., SBERT, CLIP)

Vector-based Retrieval Search Query I need warm waterproof shoes to go on a hike. [0.023883354, 0.021508986, 0.044205155, 0.019588541, 0.031198505, …] Create Embedding Search Service Vector Search Search Index [0.031198505, 0.094205155, 0.044205155, 0.019588541, 0.019588541, 0.019588541 …] Create Embeddings 1 - TrailWalker hiking Shoes 2 - TrekHiker Walking Booths Search Result

System prompt Meta Prompt ## Task You are an AI agent for the Contoso Trek outdoor products retailer. As the agent, you answer questions briefly, succinctly, and in a personable manner using markdown and even add some personal flair with appropriate emojis. ## Documents The following documentation should be used in the response. The response should specifically include the product id. TrailWalker hiking Shoes The Adventurer Pro Backpack is designed to provide comfort, durability, and ample storage space for your outdoor adventures. Familiarize yourself with the key features of the backpack […] TrekHiker Walking Booths The Adventurer Pro Backpack is designed to provide comfort, durability, and ample storage space for your outdoor adventures. Familiarize yourself with the key features of the backpack […] I need warm waterproof shoes to go on a hike. Prompt Sure, I'd be happy to help! 😊 Based on the available documentation, I can recommend two choices from the Contoso Trek catalog. 1.Product: TrailWalker Hiking Shoes ID: 36244753 Brand: TrekReady […] Prompt Response

Azure Machine Learning Prompt flow Benefits • Create AI workflows that consume various language models and data sources using the frameworks and APIs of your choice • The prompt flow can be executed locally or in the cloud. • One platform to quickly iterate through build, tune, & evaluate for your GenAI workflow • Evaluate the quality of AI workflows with pre-built and custom metrics • Easy historical tracking and team collaboration • Easy deployment and monitoring

Retrieval Augmented Generation User Question Query My Data Retriever over Knowledge Base Add Results to Prompt Query Model Large Language Model Send Results Workflow

Demo RAG Application