Improve LLM-based Applications with Fallback Mechanisms

While RAG addresses the common LLM pitfalls, challenges like handling out-of-domain queries still persist. Learn the significance of fallback mechanisms to tackle these issues gracefully, incorporating strategies like web searches and alternative data sources to improve the user experience of your system. In this session, we’ll discover various fallback techniques and practical implementation using Haystack, empowering you to develop resilient LLM-based systems for diverse scenarios without human intervention.

Talk details: https://pretalx.com/pyconde-pydata-2024/talk/QCNXLW/

Transcript

1. Improve LLM-based Applications with Fallback Mechanisms Bilge Yücel PyCon DE & PyData Berlin 2024
2. About me •🥑 Developer Relations Engineer at deepset •🏗 Open source LLM Framework: Haystack •📍 Istanbul, Turkey •💃 Latin music/dances Bilge Yücel @bilgeycl Bilge Yücel
3. Agenda Introduction to Haystack Retrieval Augmented Generation Fallback Mechanisms Q&A
4. Large Language Models (LLMs)
5. Retrieval Augmented Generation
6. Retrieval Augmented Generation
7. Retrieval Augmented Generation
9. • Haystack is an open source framework for building production-ready LLM applications
10. • Haystack is an open source framework for building production-ready LLM applications • It lets you quickly try out the latest AI models while being flexible and easy to use
11. • Haystack is an open source framework for building production-ready LLM applications • It lets you quickly try out the latest AI models while being flexible and easy to use • Building blocks: Components & Pipelines
12. Haystack
13. Haystack
14. RAG PIPELINE
15. RAG PIPELINE
16. RAG PIPELINE
17. RAG PIPELINE
18. RAG PIPELINE
19. RAG PIPELINE
21. RAG Pipeline When is Taylor Swift’s next concert?
22. RAG Pipeline When is Taylor Swift’s next concert? The documents do not provide any information about Taylor Swift's upcoming concert.
23. Fallback Mechanisms
24. Fallback to Web
25. Fallback to Web - Prompt Template
26. Fallback to Web - Conditional Router
27. Fallback to Web
28. Fallback to Web When is Taylor Swift’s next concert?
29. Fallback to Web When is Taylor Swift’s next concert? Based on the documents retrieved, Taylor Swift's upcoming concert is on May 9-12 in Paris at the Paris La Defense arena.
30. Fallback to Other Sources
31. Fallback to Web Search 󰟴 Tutorial Building Fallbacks to Websearch with Conditional Routing
32. Meetup: All Things LLM 📅 April 25, 6PM 📍Mindspace Krausenstraße lu.ma/berlin-genai-meetup
33. • Events lu.ma/haystack • X (Twitter) @Haystack_AI • YouTube @Haystack_AI @bilgeycl Bilge Yücel 🔗Haystack GitHub 🔗Haystack Website 🔗Discord community
34. Problem - Wrong Format
35. Solution - Wrong Format
36. Custom Component
37. OutputValidator
38. Loop-Based Auto-Correction
39. Loop-Based Auto-Correction 󰟴 Tutorial Generating Structured Output with Loop-Based Auto-Correction