From Hybrid Retrieval to RAG With Haystack and OpenSearch

Slide deck of the presentation from David Tippett form OpenSearch and Bilge Yucel from deepset at Open NLP Meetup https://www.meetup.com/open-nlp-meetup/events/296555345/

In their talk, David and Bilge explore the benefits of hybrid search, including hybrid search for retrieval augmented generative pipelines (RAG). They explain how they have made use of hybrid search technique on Haystack website itself to create a search application for tutorials by leveraging custom Haystack pipelines and OpenSearch. Additionally, they showcase building flexible search and LLM applications with open source tools and models.

Recoding of the session: https://youtu.be/Gn5SV8q887s?si=qVUIUCoA6w7M8URB

Transcript

2. From Hybrid Retrieval to RAG With Haystack and OpenSearch Open NLP Meetup Bilge Yücel & David Tippett October 19th, 2023
3. David Tippett Senior Developer Advocate AWS • Twitter: @dtaivpp • Linkedin: David Tippett • GitHub: @dtaivpp Bilge Yücel Developer Advocate deepset • Twitter: @bilgeycl • Linkedin: Bilge Yucel • GitHub: @bilgeyucel
4. Agenda 1 Search Applications and Retrieval 2 Keyword-based vs Semantic Search 3 Hybrid Retrieval with Haystack and OpenSearch 4 Hybrid Retrieval to RAG 5 Q&A
5. Search and Retrieval
6. Documents Documents Search and Retrieval
7. Documents Documents Search Applications Find the right document - Document Search
8. Documents Documents Search Applications Find the right document - Document Search Answer questions - Extractive QA - Generative QA
9. Document Retrieval
10. Documents Document Retrieval
11. Documents Document Retrieval 1. Inference takes time. Relevant search results reduces the amount of content a model has to look through 2. Relevant search results provides LLMs with the relevant context with which to generate an answer
12. What is Haystack? • Fully open-source framework built in Python for custom LLM applications • Provides tools that developers need to build state-of-the-art NLP systems • Building blocks: Pipelines & Components
13. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
14. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
15. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
16. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
17. So what is OpenSearch? data cluster manager coordinating data dashboards data OpenSearch Cluster
18. OpenSearch’s Community 10,000+ Stars 320M+ Downloads 22,700+ Pull Requests 500+ Contributors
19. Documents What kind of retrieval should you use? 🦄 What kind of queries do you expect? 🦄 What kind of data do you have?
20. Documents Keyword Search vs Semantic Search: Wild West Documents Source: https://opensearch.org/blog/semantic-search-solutions/
21. Keyword-based Search vs Semantic Retrieval • Keyword-based Retrieval ◦ Sparse Vectors ◦ TF-IDF, BM25 ◦ “Exact” match ◦ Fast, lightweight, effective • Semantic Retrieval ◦ Dense Embeddings ◦ Trained models: OpenAI, Cohere, Sentence Transformers… ◦ Captures semantic similarity
22. Examples: Haystack Tutorials
23. Keyword-based vs Semantic Keyword-based Semantic
24. Keyword-based vs Semantic Keyword-based Semantic
25. What if I have a domain specific data and want better performance? 🤔 Train your own embedding model ⭐ Hybrid Retrieval ⭐
26. What is Hybrid Retrieval? • It’s the combination of sparse and dense retrievers: Hybrid Retrieval Keyword-based Search Semantic Search
27. Hybrid Retrieval on Haystack Website
28. Indexing Pipeline
29. Indexing Pipeline
30. Indexing Pipeline
31. Hybrid Retrieval Pipeline
32. Hybrid Retrieval Pipeline
33. Hybrid Retrieval Pipeline
34. Hybrid Retrieval Pipeline: query clauses Using OpenSearch, we can set custom fields for the BM25Retriever: • fields • fuzziness • operator
35. Recap 1 Search Applications and Retrieval ✅ 2 Keyword-based vs Semantic Search ✅ 3 Hybrid Retrieval with Haystack and OpenSearch ✅ 4 Hybrid Retrieval to RAG 5 Q&A
36. What is Retrieval Augmented Generation (RAG)?
38. RAG Pipeline with Hybrid Retrieval
39. RAG Pipeline with Hybrid Retrieval Hybrid retrieval
40. RAG Pipeline with Hybrid Retrieval Hybrid retrieval
41. RAG Pipeline with Hybrid Retrieval • Connection to LLMs • Receives input • Sends a prompt • Returns a response
42. RAG Pipeline with Hybrid Retrieval
43. RAG Pipeline with Hybrid Retrieval
45. OpenSearch Roadmap 2.9 Released: July 24, 2023 - Pre-Filtering in FAISS - ML Connectors - Search Pipelines - Byte Sized Vectors 2.10 Released: Sept 25, 2023 - Conversational Memory - RAG Pipelines - Hybrid Search 2.11 Released: Oct 18, 2023 - Multi-model Search - Neural Search Default Models - Model Statistics API - Sparse Retrieval
46. 46 Resources and where to find us Follow 👇 Haystack on Twitter Discover 👇 What’s coming in Haystack 2.0 Join 👇 Haystack Community on Discord
47. 47 Bilge Yücel Bilge Yücel
48. RAG Pipeline with Hybrid Retrieval
49. RAG Pipeline with Hybrid Retrieval • SentenceTransformersRanker • DiversityRanker • LostInTheMiddleRanker • RecentnessRanker • CohereRanker
50. DiversityRanker Hybrid Retrieval
51. DiversityRanker Context Window Hybrid Retrieval
52. DiversityRanker Diversity Ranker Context Window Context Window Hybrid Retrieval
53. DiversityRanker Diversity Ranker Context Window Context Window Retrieval