Advanced RAG – Dynamically Selecting the best Retrievers for Queries with AI

Slide 1

Slide 1 text

Generative AI Advanced RAG AI-based Retriever Selection Marco Frodl [email protected] Principal Consultant for Generative AI @marcofrodl

Slide 2

Slide 2 text

Generative AI Advanced RAG - AI-based Retriever Selection About Me Marco Frodl Principal Consultant for Generative AI Thinktecture AG @marcofrodl [email protected] https://www.thinktecture.com/thinktects/marco-frodl/

Slide 3

Slide 3 text

Why is it important? Generative AI Advanced RAG - AI-based Retriever Selection Generative AI AI understands and generates natural language AI can access knowledge from the training phase

Slide 4

Slide 4 text

Generative AI Advanced RAG - AI-based Retriever Selection What is RAG? https://aws.amazon.com/what-is/retrieval-augmented-generation/ RAG = Ingestion + Retrieval

Slide 5

Slide 5 text

Generative AI Advanced RAG - AI-based Retriever Selection Demo: RAG

Slide 6

Slide 6 text

Generative AI Advanced RAG - AI-based Retriever Selection

Slide 7

Slide 7 text

Our sample content Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 8

Slide 8 text

Ingestion Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell Splitted (smaller) parts Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Document Metadata: Reference to original document

Slide 9

Slide 9 text

Generative AI Advanced RAG - AI-based Retriever Selection Demo: Qdrant Vector-DB

Slide 10

Slide 10 text

Similarity search in a Vector DB Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 11

Slide 11 text

Ingestion++ HyQE: Hypothetical Question Embedding Generative AI Advanced RAG - AI-based Retriever Selection Simple Advanced RAG in a nutshell LLM, e.g. GPT-3.5-turbo Transformed document Write 3 questions, which are answered by the following document. Chunk of Document Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Metadata: content of original chunk

Slide 12

Slide 12 text

Ask me anything Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector Workflow Terms - Retriever - Chain Elements Embedding- Model Vector- DB Python LLM LangChain

Slide 13

Slide 13 text

Generative AI Advanced RAG - AI-based Retriever Selection Demo: Simple RAG

Slide 14

Slide 14 text

Similarity search in a Vector DB – Limits with K Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 15

Slide 15 text

Similarity search in a Vector DB – Threshold Generative AI Advanced RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 16

Slide 16 text

Generative AI Advanced RAG - AI-based Retriever Selection How to Debug/Trace Generative AI-Apps?

Slide 17

Slide 17 text

Generative AI Advanced RAG - AI-based Retriever Selection Demo: Debugging

Slide 18

Slide 18 text

Generative AI Advanced RAG - AI-based Retriever Selection GenAI Observability

Slide 19

Slide 19 text

Just one Vector DB/Retriever? • Multiple Generative AI-Apps • Scaling and Hosting • Query Parameter per Retriever • Prompts per Retriever • Fast Updates & Re-Indexing • Access Rights • Custom Retriever Generative AI Advanced RAG - AI-based Retriever Selection What’s wrong with Simple RAG? ✅ ✅ ✅ On-Premise AI-Apps Cloud Docs Public Tickets Features Website Sales Docs Internal Tickets

Slide 20

Slide 20 text

Best source determination before the search Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or

Slide 21

Slide 21 text

Best source determination before the search Generative AI Advanced RAG - AI-based Retriever Selection Advanced RAG Retriever Selection LLM Vector DB A Vector DB B or

Slide 22

Slide 22 text

Slide 23