Most AI Agents Are Broken. Let’s Fix That

PyCon Italia 2026 Talk

---
Let’s face it: most AI agents are glorified demos. They look flashy, but they’re brittle, hard to debug, and rarely make it into real products. Why? Because wiring an LLM to a few tools is easy. Engineering a robust, testable, and scalable system is hard.

This talk is for practitioners, data scientists, AI engineers, and developers who want to stop tinkering and start shipping. We’ll take a candid look at the common reasons agent systems fail and introduce practical patterns to fix them using Haystack, an open-source framework purpose-built for production-grade LLM pipelines.

You’ll learn how to design agents that are:

- Modular, so they’re easy to extend and evolve
- Observable, so you can trace failures and understand behavior
- Maintainable, so they don’t become one-off science projects

Whether you’re just starting to explore agents or trying to tame an unruly prototype, you’ll leave with a clear, actionable blueprint to build something that’s not just smart, but also reliable.

Transcript

1. Most AI Agents Are Broken. Letʼs Fix That Bilge Yücel, Sr. DevRel Engineer @ deepset
2. Hello 👋 Bilge Yücel • Sr. DevRel Engineer at deepset • B.Sc. Computer Science • M.Sc. Artificial Intelligence • Learning & teaching how to build with AI in/bilge-yucel @bilgeyucl
3. Who is deepset? Company Solving Custom AI challenges since 2018. Offices in Berlin, Munich, Hamburg, London and Barcelona. Backed by: Leading open source framework & commercial platforms for custom enterprise-grade AI Products Used by 70 Thought leaders
4. Where are you on your agent journey?
5. What is an AI agent? An AI agent is an LM-based system that autonomously pursues a goal by interacting with its environment using tools. Human LLM Call Environment Action Feedback Stop
6. Why Agents Fail? 🤔
7. “Agent involves a lot more work than expected and 90% of it is pure engineering. Nothing really to do with the LLM, but it is how to blend LLM into an agentic workflow that makes senseˮ - a Haystack community member
8. Agent Harness Agent = Model + Harness Harness Engineering: How to Build Reliable AI Agents by Engineering the System, Not the Model
9. Failure Reason 1 Too Much Trust, Too Little Design 🛠 Too Many Tools Complexity grows exponentially with every additional tool. 📉 No Fallback Plans No contingency when the primary agentic path, tools fails. 🔍 No Observability Without tracing, debugging failures becomes nearly impossible. 🛡 No Guardrails Missing safety and validation checks for agent inputs/outputs.
10. Failure Reason 2 Wrong Use Case • You have complex, multi-step problems requiring diverse actions • Tasks involve multiple tools / sources • Goals are clear but the optimal path to achieve them isn't predetermined
11. Failure Reason 2 Wrong Use Case • You have complex, multi-step problems requiring diverse actions • Tasks involve multiple tools / sources • Goals are clear but the optimal path to achieve them isn't predetermined • When interactions can follow predictable patterns (e.g. Q&A • Tasks can be decomposed into clear steps • Stability, robustness and efficiency are prioritized over automation potential
12. Failure Reason 2 Wrong Use Case • You have complex, multi-step problems requiring diverse actions • Tasks involve multiple tools / sources • Goals are clear but the optimal path to achieve them isn't predetermined • When interactions can follow predictable patterns (e.g. Q&A • Tasks can be decomposed into clear steps • Stability, robustness and efficiency are prioritized over automation potential Agents for Reasoning, Pipelines for Defined Flow
13. How to design reliable agents?
14. • Open-source AI orchestration framework by deepset • Backbone of the Haystack Enterprise Platform • Agents, RAG & Context Engineering with visibility, control and modularity • Building blocks: Components & Pipelines & Agents Component Component Component Component Pipeline pip install haystack-ai
15. Haystack Agents User Request Generated Answer Agent LM (e.g. OpenAI, Anthropic, Google, Open Models) System Prompt Tools  Python Functions  External APIs  Haystack Pipelines Components Agents  MCP Servers
16. Letʼs Build!
17. Itinerary Agent A multi-agent travel itinerary planning system that to create comprehensive travel plans with accommodation optimization and detailed daily itineraries. Orchestration Haystack Agent Tools Various MCP Servers • Perplexity • Google Maps • Optimal Route Model Provider OpenAI (gpt-4.1 Deployment Hayhooks REST API layer User Interface Open WebUI Guardrails, streaming, & observability
18. https://itinerary-agent.deepset.ai
19. Itinerary Agent - Agent Architecture
20. Observability - Langfuse
21. Itinerary Agent - LLM
22. Itinerary Agent - Guardrails
23. Itinerary Agent - Tools
24. Itinerary Agent - Tools Specify the tools
25. Itinerary Agent - Tools BM25 search for progressive discovery Specify the tools
26. Itinerary Agent - Sub Agents
27. Itinerary Agent - Sub Agents Agents as tools through ComponentTool Only last_message is returned
28. Itinerary Agent
29. Itinerary Agent Error Handling If a tool call fails, the error is sent back to the LLM by default. Resilience Ensure uptime with FallbackChatGenerator for model connections. Termination Stops on direct text output or reaching max_agent_step limit.
30. Soon: Haystack 3.0 ☀  Simplified UX Improving the user experience of the Agent component for better accessibility.  Enhanced Harness Expanding harness capabilities to provide more robust agents.  Advanced Streaming Improved streaming capabilities for real-time data processing.
31. Practical Tips For Building Agents Optimal Autonomy Maintain control over your system by finding the right level of autonomy. Multi-Agent Systems Split tasks between specialized agents like planners, coders, and fixers. 'Fat' Tools Encapsulate logic, retries, and fallbacks directly within the tool logic. Guardrails & Observability Validate all inputs/outputs and log every step with comprehensive tracing. Human-in-the-Loop Ensure agents are overseen and guided by people for optimal reliability. Minimize Tools Avoid unnecessary complexity in context; every additional tool increases the risk of failure.
32. Unbreakable AI Agents with Haystack  Human-in-the-Loop View Tutorial  Mem0 Memory Store View Cookbook  AI Guardrails & Safety Explore Content Moderation  Multi-Agent Systems System Design Guide  Context Engineering for Agentic Systems Deep Dive into Agent Logic
33. Thank you! 🤖 Demo: itinerary-agent.deepset.ai 🌐 Demo GitHub: github.com/deepset-ai/itinerary-agent STAY CONNECTED 🌟 Haystack GitHub: github.com/deepset-ai/haystack 🤝 Ambassadors: haystack.deepset.ai/ambassadors in/bilge-yucel @bilgeyucl Get the presentation