Document Ranking: Taking RAG Pipelines in Haystack to the Next Level

In this lightning talk at the AI Conference, I spoke about ranking techniques for retrieval-augmented generation pipelines with Haystack.
1. What is retrieval-augmentation and why is it an effective technique?
2. Ranking of retrieved documents
3. Diversity ranking
4. Lost in the middle ranking

Transcript

1. Document Ranking: Taking RAG Pipelines in Haystack to the Next Level AI Conference San Francisco Tuana Çelik September 26, 2023
3. 3 What is Retrieval Augmentation? • LLMs do not know the answer to everything • We can help them by giving them the relevant context • Once retrieved, we augment the prompt (instruction) with the relevant context
4. 4 Instruction Context What is Retrieval Augmentation?
5. Question ReRanker Retriever Relevant Documents
6. ReRanking: Diversity Similarity Search
7. ReRanking: Diversity Similarity Search Context Window
8. Diversity Ranker Similarity Search Context Window Context Window ReRanking: Diversity
9. • “Lost in the Middle: How Language Models Use Long Contexts”, Liu et al (2023) [1] • [1 2 3 4 5 6 7 8 9 10] -> [1 3 5 7 9 10 8 6 4 2] • Place best paragraphs at the beginning and end of the context window • Proven to help in answer extraction, might help in generative AI [1] https://arxiv.org/abs/2307.03172 ReRanking: Lost in the Middle
10. 10 What is Haystack? • Fully open-source framework built in Python to designed to build LLM applications • Core NLP tasks covered • Production focused
12. 12 Resources and where to find us • We have an open Discord server • We have regular meetups! Go check out the Open NLP Meetup • GitHub: Haystack is open source ⭐ • Twitter @deepset_ai To read 👇 Introducing DiversityRanker and LostInTheMiddleRanker Discover 👇 What’s coming in Haystack 2.0