Oct 2024 - Without good retrieval, RAG is just AG - All Guesswork

Information retrieval-based techniques to get best out of you RAG !

Transcript

1. The ‘R’ in RAG is Retrieval Atita Arora
2. Without good retrieval, RAG is just AG - All Guesswork. Atita Arora
3. - Building world class Search - Implementation & Evaluation of LLM Applications About me Started 2008 - Computer Applications - Strategic Business Management Opensource Loves to travel, eat, cook Mom of 2 boys
4. To begin with : Information Retrieval is not a new problem ;)
5. The world of RAG is ever evolving !
6. Ref : https://arxiv.org/html/2312.10997v5
7. ++++ Many more Query Enrichment Techniques
8. Ref : https://arxiv.org/html/2312.10997v5
9. Improving Retrieval in RAG - Source Selection: Identifying the best data source for retrieval. - Granularity: Determining the level of detail to retrieve. - Pre-processing: Preparing data for efficient retrieval. - Embedding Model Choice: Selecting the appropriate embedding model for accurate relevance matching.
10. Types of Retrieval in RAG Targeted results Break down & answer Dynamic control
11. It depends … • Domain • Use-case ◦ Information need ◦ Expected Query complexity ◦ Desired Response complexity • Data type ◦ Federated ◦ Structured ◦ Unstructured ◦ Multilingual ◦ Multimodal
12. Why is it vital to establish Information need ? • Helps in understanding attributes of relevance • It establishes how well do we interpret the query against the results • Assist and support Precision and recall metrics by helping to identify knowledge gaps • Drives everything from data preprocessing techniques to query expansion , routing , relaxation or enrichment
13. When selling a solution, you don’t just sell the product - but a productized process to understand the users definition of success, and customize your product to deliver on it. You need to be a trusted partner, advisor, and consultant. You don’t just load your data and leave. Ref : https://softwaredoug.com/blog/2024/10/13/the-hidden-dangers-search-solution
14. Derivatives of Retrieval Strategy - Lexical search (BM25) is ancient (org. 1970s) and still is a solid baseline for IR systems , keep it in your back pocket !!! - Excels in handling extensive texts and domain-specific jargon. - Leverage Metadata filtering / NER to limit irrelevant results - Semantic search via embeddings is powerful BUT - Hundreds of tokens compressed into a single vector - Information loss is inevitable Makes sense to keep in mind when experimenting with chunk sizes - Nuances of Multilingual and Multimodal content
15. Zero-shot NER as Metadata filter
16. Adaptive Retrieval (https://arxiv.org/pdf/2403.14403) Moving towards GAR (Generative AI Augmented Retrieval) Techniques
17. Adaptive Retrieval (https://arxiv.org/pdf/2403.14403) Further Experimentation / Improvements - Embedding Models - Chunking strategy - NER / Multimodal websearch - Classifiers / Rerankers - Prompt Techniques - CoT , ReAct
18. Retrieval Evaluation Subjectivity Domain representation Context Query Understanding Metrics Suitability Multimodality Build an Evaluation dataset of Ground Truth Mean Average Precision DCG / NDCG F1 Score Semantic Similarity
19. Techniques to improve Retrieval in Qdrant - Leverage different type of vectors to build a suitable retrieval strategy - Leverage Payload index to filter relevant results - Leverage Retrieval Quality Experimentation to determine ideal HNSW params - Leverage Quantization techniques to optimize storage and search performance - Leverage Hybrid and Multistage queries to combine multiple queries and results for efficient and faster searching. - Leverage hot and cold indexes to manage latency - Leverage search quality tool to assess search quality against exact search
20. Qdrant Prefetch (All in one) https://github.com /qdrant/qdrant-rag -eval/blob/master/ workshop-rag-eva l-oxford-llm2024/n otebook/advance d_hybrid_search_ qdrant_rag.ipynb
21. 1. Retrieval Source: - RAG relies on external knowledge to enhance LLMs. - Types of retrieval sources: Unstructured Data, Semi-structured data, Structured data, and LLMs-generated Content. - Growing trend in recent researches towards utilizing content generated by LLMs themselves for retrieval and enhancement purposes. 2. Data Structure: - Importance of text as the mainstream source of retrieval. - Expansion of retrieval sources to include semi-structured data (PDF) and structured data (Knowledge Graph, KG) for enhancement. - Common unstructured data sources such as Wikipedia Dump, cross-lingual text, and domain-specific data. 3. Retrieval Granularity: - Importance of retrieval granularity in influencing the relevance and burden of retrieval for dense retrievers. - Examples of retrieval granularity: Token, Phrase, Sentence, Proposition, Chunks, Document for text; Entity, Triplet, sub-Graph for Knowledge Graph. 4. Indexing Optimization: - Chunking Strategy for document segmentation. - Metadata attachments and their role in limiting retrieval scope. - The application of structural indexing for efficient data retrieval. 5. Query Optimization: - Importance of expanding and transforming queries for enhanced context and relevance.
22. R2AG : Incorporating Retrieval Information into Retrieval Augmented Generation • Challenges faced by LLMs in utilizing retrieval information and handling retrieved documents effectively. • Proposed R2AG framework to bridge the semantic gap between retrievers and LLMs, enabling deeper comprehension of retrieved documents and more accurate responses. • Description of the R2-Former as a trainable module that bridges between retrievers and LLMs through a Transformer-based model. • Importance of retrieval-aware prompting in the generation process for LLMs to effectively utilize the retrieval information. Example : Here are examples of retrieval-aware prompting based on the concepts discussed in the given article: 1. Include Extracted Information: "Given the retrieved documents discussing the benefits of renewable energy sources such as solar and wind power, describe the environmental advantages of transitioning to renewable energy in your response." 2. Contextual Reference: "After reviewing the historical context of World War II from the retrieved sources, explain the impact of key battles on the outcome of the war in Europe." 3. Incorporate Specific Details: "Considering the statistical data obtained from recent economic reports regarding market trends, analyze the implications of rising interest rates on consumer spending habits." 4. Addressing Specific Findings: "Based on the information extracted from medical studies on cardiovascular health, outline the recommended lifestyle changes for reducing the risk of heart disease." 5. Utilizing Retrieved Facts: "Using the country-specific economic data retrieved from financial reports, compare and contrast the GDP growth rates of emerging markets in Asia."
23. Thank you !! A free forever 1GB cluster included for trying out. No credit card required. Feel free to reach us at : info@qdrant.com