Building and Deploying Agentic Workflows with Haystack

Deeplearning.AI AI Dev 25
https://ai-dev.deeplearning.ai/

Building an AI agent sounds exciting until you start engineering one. It’s not just about the LLM, but about designing a workflow where the LLM fits into a composable, scalable, and deployable system. That’s where Haystack shines, providing the building blocks to create, refine, and deploy agentic pipelines into production. In this talk, we will explore how to construct an agentic pipeline using tool-calling support, leveraging Haystack’s modular architecture to keep workflows flexible and adaptable. Then, we’ll take it beyond the prototype phase by deploying the pipeline and demonstrating how deepset Studio helps visualize, test, and refine the system before it goes live. This session will focus on practical strategies for building agentic applications that go beyond PoCs to become fully operational and production-ready. Whether you are experimenting with agents or looking to deploy them at scale, you’ll walk away with the tools to build, iterate, and ship with confidence.

Transcript

1. Building and Deploying Agentic Workflows with Haystack
2. Bilge Yucel Hello 👋 • DevRel Engineer at deepset • Open source LLM orchestration framework Haystack • Istanbul, Turkey bilgeyucel in/bilge-yucel @bilgeyucl
3. Agenda • Introduction to Haystack • Building an Agent • Deploying an Agent • Q&A
4. AI Agents Overview of a LLM-powered autonomous agent system Source: https://lilianweng.github.io/posts/20230623-agent/
5. • Open-source LLM orchestration framework by deepset • Provides the tools that Python developers need to build real world, agentic AI systems • Building blocks: Components & Pipelines
6. Directed (cyclic) Graphs → Flexible, modular, customizable
7. Custom Components
8. Building an Agent with Haystack
9. “Agent involves a lot more work than expected and 90% of it is pure engineering. Nothing really to do with LLM, but it is how to blend LLM into an agentic workflow that makes senseˮ - a community member
10. Github Issue Resolver Agent • Takes a Github issue URL • Comments on the issue with instructions on how to solve it • Navigates through the repo and understand code
11. Github Issue Resolver Agent
12. Deploying an Agent with Haystack
13. Hayhooks • Tool to deploy and serve Haystack pipelines as REST APIs • Wrap Pipelines with custom logic and expose them via HTTP endpoints • OpenAI-compatible chat completion endpoints→ Open WebUI
14. Development Environment for Haystack • Drag, drop, and construct Haystack pipelines • Ready-made pipelines • Bring your own files or connect to your database • Deploy on Studio or export pipelines • Test, debug, and share your prototype • Free and open to everyone deepset Studio
15. Build a Pipeline Deploy to Development Environment
16. Build a Pipeline Playground
17. Build a Pipeline Export Pipeline
18. Thank You! Any Questions? bilgeyucel in/bilge-yucel @bilgeyucl github.com/deepset-ai/github-agent haystack.deepset.ai