Extended RAG – mit Langchains Multi-Retriever-Ansatz zu Fragen mit AI die beste Antwortquelle auswählen

Retrieval Augmented Generation (RAG) verwendet Daten aus Retrievern wie Vektor-Datenbanken, damit die relevanten Informationen gefunden werden und Benutzerfragen beantwortet werden können.

Wenn mehrere Retriever (z.B. Support-Tickets, Produkte, Kundenliste) verwendet werden sollen, kann die Auswahl der optimalen Datenquelle zur gestellten Frage eine Herausforderung sein. Marco Frodl wird das MultiRouteChain-Paradigma aus dem LangChain-Framework vorstellen, mit dem die AI dynamisch die Auswahl der besten Antwortquelle übernimmt. Die Live-Coding-Demonstration wird zeigen, wie MultiRouteChain die Leistung von RAG für das Beantworten von Benutzeranfragen verbessert.

Transcript

1. Generative AI Infodays Bonn 27.5.-29.5.24 Extended RAG – mit LangChains Multi- Retriever-Ansatz zu Fragen mit AI die beste Antwortquelle auswählen Marco Frodl marco.frodl@thinktecture.com Principal Consultant for Generative AI @marcofrodl
2. Why is it important? Generative AI Extended RAG - AI-based Retriever Selection Generative AI AI understands and generates natural language AI can access knowledge from the training phase
3. Generative AI Extended RAG - AI-based Retriever Selection What is RAG? https://aws.amazon.com/what-is/retrieval-augmented-generation/ RAG = Ingestion + Retrieval
4. Generative AI Extended RAG - AI-based Retriever Selection Demo: RAG
5. Generative AI Extended RAG - AI-based Retriever Selection
6. Generative AI Extended RAG - AI-based Retriever Selection About Me Marco Frodl Principal Consultant for Generative AI Thinktecture AG X: @marcofrodl E-Mail: marco.frodl@thinktecture.com https://www.thinktecture.com/thinktects/marco-frodl/
7. Ingestion Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell Splitted (smaller) parts Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Document Metadata: Reference to original document
8. Our sample content Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell
9. Generative AI Extended RAG - AI-based Retriever Selection Demo: Qdrant Vector-DB
10. Similarity search in a Vector DB Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell
11. Ingestion++ HyQE: Hypothetical Question Embedding Generative AI Extended RAG - AI-based Retriever Selection Simple Advanced RAG in a nutshell LLM, e.g. GPT-3.5-turbo Transformed document Write 3 questions, which are answered by the following document. Chunk of Document Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Metadata: content of original chunk
12. Ask me anything Generative AI Extended RAG - AI-based Retriever Selection Simple RAG Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector Workflow Terms - Retriever - Chain Elements Embedding- Model Vector- DB Python LLM LangChain
13. Generative AI Extended RAG - AI-based Retriever Selection Demo: Simple RAG
14. Similarity search in a Vector DB – Limits with K Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell
15. Similarity search in a Vector DB – Threshold Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell
16. Generative AI Extended RAG - AI-based Retriever Selection How to Debug/Trace Generative AI-Apps?
17. Generative AI Extended RAG - AI-based Retriever Selection Demo: Debugging
18. Generative AI Extended RAG - AI-based Retriever Selection GenAI Tracing
19. Just one Vector DB/Retriever? • Multiple Generative AI-Apps • Scaling and Hosting • Query Parameter per Retriever • Prompts per Retriever • Fast Updates & Re-Indexing • Access Rights • Custom Retriever Generative AI Extended RAG - AI-based Retriever Selection What’s wrong with Simple RAG? ✅ ✅ ✅ On-Premise AI-Apps Cloud Docs Public Tickets Features Website Sales Docs Internal Tickets
20. Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or
21. Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Retriever Selection LLM Vector DB A Vector DB B or
22. Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector
23. Generative AI Extended RAG - AI-based Retriever Selection Demo: Dynamic Retriever Selection with AI
24. Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG
25. Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG
26. Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG
27. Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG
28. Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG
29. Thank you! Any questions? Marco Frodl @marcofrodl Principal Consultant for Generative AI