Improve LLM-based
Applications with
Fallback Mechanisms
Bilge Yücel
PyCon DE & PyData Berlin 2024
Slide 2
Slide 2 text
About me
•🥑 Developer Relations Engineer at deepset
•🏗 Open source LLM Framework: Haystack
•📍 Istanbul, Turkey
•💃 Latin music/dances
Bilge Yücel
@bilgeycl Bilge Yücel
Slide 3
Slide 3 text
Agenda
Introduction to Haystack
Retrieval Augmented Generation
Fallback Mechanisms
Q&A
Slide 4
Slide 4 text
Large Language Models (LLMs)
Slide 5
Slide 5 text
Retrieval Augmented
Generation
Slide 6
Slide 6 text
Retrieval Augmented Generation
Slide 7
Slide 7 text
Retrieval Augmented Generation
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
● Haystack is an open source framework for building
production-ready LLM applications
Slide 10
Slide 10 text
● Haystack is an open source framework for building
production-ready LLM applications
● It lets you quickly try out the latest AI models while being
flexible and easy to use
Slide 11
Slide 11 text
● Haystack is an open source framework for building
production-ready LLM applications
● It lets you quickly try out the latest AI models while being
flexible and easy to use
● Building blocks: Components & Pipelines
Slide 12
Slide 12 text
Haystack
Slide 13
Slide 13 text
Haystack
Slide 14
Slide 14 text
RAG PIPELINE
Slide 15
Slide 15 text
RAG PIPELINE
Slide 16
Slide 16 text
RAG PIPELINE
Slide 17
Slide 17 text
RAG PIPELINE
Slide 18
Slide 18 text
RAG PIPELINE
Slide 19
Slide 19 text
RAG PIPELINE
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
RAG Pipeline
When is Taylor Swift’s
next concert?
Slide 22
Slide 22 text
RAG Pipeline
When is Taylor Swift’s
next concert?
The documents do not
provide any
information about
Taylor Swift's
upcoming concert.
Slide 23
Slide 23 text
Fallback
Mechanisms
Slide 24
Slide 24 text
Fallback to Web
Slide 25
Slide 25 text
Fallback to Web - Prompt Template
Slide 26
Slide 26 text
Fallback to Web - Conditional Router
Slide 27
Slide 27 text
Fallback to Web
Slide 28
Slide 28 text
Fallback to Web
When is Taylor Swift’s
next concert?
Slide 29
Slide 29 text
Fallback to Web
When is Taylor Swift’s
next concert? Based on the documents
retrieved, Taylor Swift's
upcoming concert is on
May 9-12 in Paris at the
Paris La Defense arena.
Slide 30
Slide 30 text
Fallback to Other Sources
Slide 31
Slide 31 text
Fallback to Web Search
Tutorial
Building Fallbacks to Websearch
with Conditional Routing
Slide 32
Slide 32 text
Meetup: All Things LLM
📅 April 25, 6PM
📍Mindspace
Krausenstraße
lu.ma/berlin-genai-meetup
Slide 33
Slide 33 text
● Events lu.ma/haystack
● X (Twitter) @Haystack_AI
● YouTube @Haystack_AI
@bilgeycl
Bilge Yücel
🔗Haystack GitHub 🔗Haystack Website 🔗Discord community
Slide 34
Slide 34 text
Problem - Wrong Format
Slide 35
Slide 35 text
Solution - Wrong Format
Slide 36
Slide 36 text
Custom Component
Slide 37
Slide 37 text
OutputValidator
Slide 38
Slide 38 text
Loop-Based Auto-Correction
Slide 39
Slide 39 text
Loop-Based Auto-Correction
Tutorial
Generating Structured
Output with Loop-Based
Auto-Correction