Advanced RAG – Dynamically Selecting the best Retrievers for Queries with AI

Retrieval Augmented Generation (RAG) uses data from retrieval systems such as Vector DBs to find the relevant information to answer a user query. When multiple retrieval systems are to be used, selecting the optimal retrieval system for a query can be a challenge. Marco Frodl will present the MultiRouteChain paradigm from the LangChain framework, which enables AI-based dynamic retrieval system selection based on semantic matching of the user query and the retrieval system focus. The live coding will illustrate how MultiRouteChain improves the performance of RAG for answering user queries by selecting the most relevant QA chain for retrieval for each query.

The talk for the AI-Meetup Frankfurt is very hands-on with lots of live demos and coding in Python and LangChain framework to show the difference between a simple RAG approach and this more real-world approach.

Transcript

1. Generative AI Advanced RAG AI-based Retriever Selection Marco Frodl marco.frodl@thinktecture.com Principal Consultant for Generative AI @marcofrodl
2. Generative AI Advanced RAG - AI-based Retriever Selection About Me Marco Frodl Principal Consultant for Generative AI Thinktecture AG @marcofrodl marco.frodl@thinktecture.com https://www.thinktecture.com/thinktects/marco-frodl/
3. Why is it important? Generative AI Advanced RAG - AI-based Retriever Selection Generative AI AI understands and generates natural language AI can access knowledge from the training phase
4. Generative AI Advanced RAG - AI-based Retriever Selection What is RAG? https://aws.amazon.com/what-is/retrieval-augmented-generation/ RAG = Ingestion + Retrieval
5. Generative AI Advanced RAG - AI-based Retriever Selection Demo: RAG
6. Generative AI Advanced RAG - AI-based Retriever Selection
7. Our sample content Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell
8. Ingestion Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell Splitted (smaller) parts Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Document Metadata: Reference to original document
9. Generative AI Advanced RAG - AI-based Retriever Selection Demo: Qdrant Vector-DB
10. Similarity search in a Vector DB Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell
11. Ingestion++ HyQE: Hypothetical Question Embedding Generative AI Advanced RAG - AI-based Retriever Selection Simple Advanced RAG in a nutshell LLM, e.g. GPT-3.5-turbo Transformed document Write 3 questions, which are answered by the following document. Chunk of Document Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Metadata: content of original chunk
12. Ask me anything Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector Workflow Terms - Retriever - Chain Elements Embedding- Model Vector- DB Python LLM LangChain
13. Generative AI Advanced RAG - AI-based Retriever Selection Demo: Simple RAG
14. Similarity search in a Vector DB – Limits with K Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell
15. Similarity search in a Vector DB – Threshold Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell
16. Generative AI Advanced RAG - AI-based Retriever Selection How to Debug/Trace Generative AI-Apps?
17. Generative AI Advanced RAG - AI-based Retriever Selection Demo: Debugging
18. Generative AI Advanced RAG - AI-based Retriever Selection GenAI Observability
19. Just one Vector DB/Retriever? • Multiple Generative AI-Apps • Scaling and Hosting • Query Parameter per Retriever • Prompts per Retriever • Fast Updates & Re-Indexing • Access Rights • Custom Retriever Generative AI Advanced RAG - AI-based Retriever Selection What’s wrong with Simple RAG? ✅ ✅ ✅ On-Premise AI-Apps Cloud Docs Public Tickets Features Website Sales Docs Internal Tickets
20. Best source determination before the search Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or
21. Best source determination before the search Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG Retriever Selection LLM Vector DB A Vector DB B or
22. Best source determination before the search Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector
23. Generative AI Advanced RAG - AI-based Retriever Selection Demo: Dynamic Retriever Selection with AI
24. Dynamic Retriever Selection with AI Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG
25. Dynamic Retriever Selection with AI Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG
26. Dynamic Retriever Selection with AI Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG
27. Dynamic Retriever Selection with AI Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG
28. Dynamic Retriever Selection with AI Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG
29. Thank you! Any questions? Marco Frodl @marcofrodl Principal Consultant for Generative AI