Generative AI Infodays Bonn 27.5.-29.5.24 Extended RAG – mit LangChains Multi- Retriever-Ansatz zu Fragen mit AI die beste Antwortquelle auswählen Marco Frodl [email protected] Principal Consultant for Generative AI @marcofrodl

Why is it important? Generative AI Extended RAG - AI-based Retriever Selection Generative AI AI understands and generates natural language AI can access knowledge from the training phase

Generative AI Extended RAG - AI-based Retriever Selection What is RAG? https://aws.amazon.com/what-is/retrieval-augmented-generation/ RAG = Ingestion + Retrieval

Generative AI Extended RAG - AI-based Retriever Selection Demo: RAG

Generative AI Extended RAG - AI-based Retriever Selection

Generative AI Extended RAG - AI-based Retriever Selection About Me Marco Frodl Principal Consultant for Generative AI Thinktecture AG X: @marcofrodl E-Mail: [email protected] https://www.thinktecture.com/thinktects/marco-frodl/

Ingestion Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell Splitted (smaller) parts Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Document Metadata: Reference to original document

Our sample content Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Generative AI Extended RAG - AI-based Retriever Selection Demo: Qdrant Vector-DB

Similarity search in a Vector DB Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Ingestion++ HyQE: Hypothetical Question Embedding Generative AI Extended RAG - AI-based Retriever Selection Simple Advanced RAG in a nutshell LLM, e.g. GPT-3.5-turbo Transformed document Write 3 questions, which are answered by the following document. Chunk of Document Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Metadata: content of original chunk

Ask me anything Generative AI Extended RAG - AI-based Retriever Selection Simple RAG Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector Workflow Terms - Retriever - Chain Elements Embedding- Model Vector- DB Python LLM LangChain

Generative AI Extended RAG - AI-based Retriever Selection Demo: Simple RAG

Similarity search in a Vector DB – Limits with K Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Similarity search in a Vector DB – Threshold Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Generative AI Extended RAG - AI-based Retriever Selection How to Debug/Trace Generative AI-Apps?

Generative AI Extended RAG - AI-based Retriever Selection Demo: Debugging

Generative AI Extended RAG - AI-based Retriever Selection GenAI Tracing

Just one Vector DB/Retriever? • Multiple Generative AI-Apps • Scaling and Hosting • Query Parameter per Retriever • Prompts per Retriever • Fast Updates & Re-Indexing • Access Rights • Custom Retriever Generative AI Extended RAG - AI-based Retriever Selection What’s wrong with Simple RAG? ✅ ✅ ✅ On-Premise AI-Apps Cloud Docs Public Tickets Features Website Sales Docs Internal Tickets

Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or

Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Retriever Selection LLM Vector DB A Vector DB B or

Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector

Generative AI Extended RAG - AI-based Retriever Selection Demo: Dynamic Retriever Selection with AI

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Thank you! Any questions? Marco Frodl @marcofrodl Principal Consultant for Generative AI