Slide 1

Slide 1 text

Generative AI Infodays Bonn 27.5.-29.5.24 Extended RAG – mit LangChains Multi- Retriever-Ansatz zu Fragen mit AI die beste Antwortquelle auswählen Marco Frodl [email protected] Principal Consultant for Generative AI @marcofrodl

Slide 2

Slide 2 text

Why is it important? Generative AI Extended RAG - AI-based Retriever Selection Generative AI AI understands and generates natural language AI can access knowledge from the training phase

Slide 3

Slide 3 text

Generative AI Extended RAG - AI-based Retriever Selection What is RAG? https://aws.amazon.com/what-is/retrieval-augmented-generation/ RAG = Ingestion + Retrieval

Slide 4

Slide 4 text

Generative AI Extended RAG - AI-based Retriever Selection Demo: RAG

Slide 5

Slide 5 text

Generative AI Extended RAG - AI-based Retriever Selection

Slide 6

Slide 6 text

Generative AI Extended RAG - AI-based Retriever Selection About Me Marco Frodl Principal Consultant for Generative AI Thinktecture AG X: @marcofrodl E-Mail: [email protected] https://www.thinktecture.com/thinktects/marco-frodl/

Slide 7

Slide 7 text

Ingestion Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell Splitted (smaller) parts Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Document Metadata: Reference to original document

Slide 8

Slide 8 text

Our sample content Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 9

Slide 9 text

Generative AI Extended RAG - AI-based Retriever Selection Demo: Qdrant Vector-DB

Slide 10

Slide 10 text

Similarity search in a Vector DB Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 11

Slide 11 text

Ingestion++ HyQE: Hypothetical Question Embedding Generative AI Extended RAG - AI-based Retriever Selection Simple Advanced RAG in a nutshell LLM, e.g. GPT-3.5-turbo Transformed document Write 3 questions, which are answered by the following document. Chunk of Document Embedding- Model Embedding 𝑎 𝑏 𝑐 … Vector- Database Metadata: content of original chunk

Slide 12

Slide 12 text

Ask me anything Generative AI Extended RAG - AI-based Retriever Selection Simple RAG Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector Workflow Terms - Retriever - Chain Elements Embedding- Model Vector- DB Python LLM LangChain

Slide 13

Slide 13 text

Generative AI Extended RAG - AI-based Retriever Selection Demo: Simple RAG

Slide 14

Slide 14 text

Similarity search in a Vector DB – Limits with K Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 15

Slide 15 text

Similarity search in a Vector DB – Threshold Generative AI Extended RAG - AI-based Retriever Selection Simple RAG in a nutshell

Slide 16

Slide 16 text

Generative AI Extended RAG - AI-based Retriever Selection How to Debug/Trace Generative AI-Apps?

Slide 17

Slide 17 text

Generative AI Extended RAG - AI-based Retriever Selection Demo: Debugging

Slide 18

Slide 18 text

Generative AI Extended RAG - AI-based Retriever Selection GenAI Tracing

Slide 19

Slide 19 text

Just one Vector DB/Retriever? • Multiple Generative AI-Apps • Scaling and Hosting • Query Parameter per Retriever • Prompts per Retriever • Fast Updates & Re-Indexing • Access Rights • Custom Retriever Generative AI Extended RAG - AI-based Retriever Selection What’s wrong with Simple RAG? ✅ ✅ ✅ On-Premise AI-Apps Cloud Docs Public Tickets Features Website Sales Docs Internal Tickets

Slide 20

Slide 20 text

Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or

Slide 21

Slide 21 text

Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Retriever Selection LLM Vector DB A Vector DB B or

Slide 22

Slide 22 text

Best source determination before the search Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG Question Retriever Selection 0-N Search Results Question Answer LLM Embedding Model Vector DB A Question as Vector Vector DB B LLM Prepare Search or Question Prepare Search Search Results Question Answer LLM Vector DB Embedding Model Question as Vector

Slide 23

Slide 23 text

Generative AI Extended RAG - AI-based Retriever Selection Demo: Dynamic Retriever Selection with AI

Slide 24

Slide 24 text

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Slide 25

Slide 25 text

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Slide 26

Slide 26 text

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Slide 27

Slide 27 text

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Slide 28

Slide 28 text

Dynamic Retriever Selection with AI Generative AI Extended RAG - AI-based Retriever Selection Advanced RAG

Slide 29

Slide 29 text

Thank you! Any questions? Marco Frodl @marcofrodl Principal Consultant for Generative AI