Agent-Powered Retrieval with Haystack and Qdrant

Presented at Qdrant Vector Space Day, September 2025

Abstract: Are agents the future of retrieval or an overcomplication? This session compares traditional retrieval pipelines with agent-powered approaches built using Haystack and Qdrant. We’ll construct a traceable, debuggable agent that plans tool usage, selects retrieval strategies, and explains its steps. Then we’ll evaluate latency, accuracy, and robustness against a strong non-agent baseline on a movie dataset stored in Qdrant. You’ll learn to read agent traces and decide when agents add value complex multi-step tasks, ambiguous queries, or dynamic toolsets and when a simpler pipeline is preferable.

Transcript

1. Agent-Powered Retrieval with Haystack and Qdrant Promise and Pitfalls Bilge Yücel, Developer Relations Engineer @ deepset
2. Hello 👋 Bilge Yücel • Developer Relations Engineer at deepset 🥑 • B.Sc. Computer Science • M.Sc. Artificial Intelligence • Learning & teaching how to build with AI in/bilge-yucel @bilgeyucl
3. Who is deepset? Company Solving Custom AI challenges since 2018. HQ in Berlin and NYC. Backed by: Leading open source framework & commercial platforms for custom enterprise-grade AI Products Used by 70 Thought leaders
4. Scan the QR Code to receive materials and ask your questions!
5. Are agents the future of search, or do classic retrievers still rule?
6. Traditional Retrieval • Retrieve relevant information based on query • Different algorithms: keyword-based, semantic, hybrid, …
7. Agent-Powered Retrieval • Use agent architecture for retrieval • Retrieval as a tool → function calling
8. Movie Recommendation Assistant with Qdrant and Haystack 🍿
9. 🍿 Tech Stack • 🎬 Pablinho/movies-dataset → 10k movies with title, rating, description,... • Haystack → AI Orchestration Framework (⭐22.8k, open source, python) • Qdrant → Vector Database (free cloud cluster) • FastEmbed (Qdrant/minicoil-v1) → Sparse Neural Embedding • OpenAI (gpt-4o-mini) → Cheap, powerful
10. Load Data and Create Embeddings
11. Upload Data to Qdrant Collection
12. Retrieval Pipeline
13. Retrieval Pipeline
14. Retrieval Pipeline
15. Retrieval Pipeline + Filtering
16. Agent-Based Retrieval
17. 🛠 Define the Tool • Name → retrieval_tool • Description → "Use this tool to get movies fitting to the user criteria" • Parameter → JSON schema • Function
18. JSON Schema • Define each parameter, query and metadata_filters • Be precise with the metadata_filter by providing field names and options in detail • 💡Alternative: Treat field names as parameters
19. Agent • Bring them all together, create a Tool • Pass that to your Agent alongside prompt and the LLM (OpenAIChatGenerator)
20. Test the Agent
21. Observability? Covered 👍 • Get insights on the tool names and tool calls • Connect it to LLM observability tools
22. Advantages of Agent-Powered Retrieval
23. 1⃣ Query Rewrite • Query: I love Titanic. Which movie should I watch next?
24. 2⃣ Metadata Extraction • Query: Find me a highly-rated action movie about car racing
25. 3⃣ Error/Retry mechanism • Query: Can you get me a turkish movie rated above 8 that I can watch with my kids? Preferably an animation • Answer: It seems the search for Turkish animated movies rated above 8 was quite narrow, as I couldn't find any that met all your criteria….
26. Reality Check
27. Comparison Traditional Retrieval Agent-based Retrieval • Works with complex, ambiguous queries • Error recovery & Retry • Filter generation • Slower 4 secs) • Less cheap :) • Fails with complex queries • No error recovery • Manual filtering • Faster 0.5 secs) • Cheap
28. 🤝 Middle Ground • Filter generation (more performant than pure retrieval) ✅ • Cheaper (than agent-based) ✅ • Faster (than agent-based) ✅ • Not every LLM supports ❌ • No query enhancements ❌ • No error handling ❌ Structured Output Generation + Retrieval
29. Thank you Weʼre hiring! deepset.ai/careers deepset.ai | haystack.deepset.ai in/bilge-yucel @bilgeyucl Receive materials and ask questions