Haystack 2.0: The story of a rewrite

To rewrite or not to rewrite: it's a major question.

Releasing new software versions with breaking changes can be disruptive to a community, but sometimes they are necessary in the long run to move forward.

Haystack is a free open source Python LLM framework. It was launched in 2020, before LLMs were cool. In 2023 we decided to undergo a major re-architecture, culminating in the GA release of Haystack 2.0. It wasn't an easy decision. By involving the open source community and some big companies in our design process early on, we are confident we built a more usable, flexible foundation for years to come.

In this talk I'll tell you the story of this rewrite. The decisions we made to bring the project forward with the right level of flexibility / composability in the rapidly changing LLM landscape. I won't only show you the new features 2.0 provides, but give you a peek into our future roadmap. You'll walk away with a better understanding of how modern LLM frameworks can help you solve problems for yourself and your users, as well as an enriched understanding of how to think for the long-term when building for an open source community.

You’ll see how the strength of Haystack modularity and ease of use makes it stand out from other libraries. Demos will make it much clear and give you some great ideas on how to integrate Haystack in your projects.

Transcript

1. Haystack 2.0 The story of a rewrite
2. Senior Software Engineer Silvano Cerza
3. Senior Software Engineer Silvano Cerza
4. Senior Software Engineer Silvano Cerza
5. Senior Software Engineer Silvano Cerza
6. Senior Software Engineer Silvano Cerza
7. Senior Software Engineer Silvano Cerza
8. Agenda 1. A brief intro of Haystack 2. Why a rewrite? 3. What to do 4. Haystack in detail 5. Q/A
9. What is Haystack? ??? !!!
10. Jul 2019 Nov 2019 Sep 2021 Nov 2022 A timeline
11. To rewrite or not to rewrite Tackle technical debt Business business Split the team on different projects Paradigm change
12. To rewrite or not to rewrite ⏳⏳ ⏳⏳ ⏳⏳ 💰💰 💰💰 💰💰
13. So we made the decision
15. # TODO 1. Focus on the pain points 2. Listen your community 3. Consider new use cases 4. Set a deadline 5. Decide the future of your past 6. Remember it’s not just code
16. What we’ve done
17. The Pipeline ??? !!!
18. The Pipeline ? ? ! ! !
19. Components ????? ? ?
20. Components
21. Dependencies Haystack Extras Dependencies
22. Dependencies Haystack Core Dependencies Lazy Imports Haystack Integrations
23. pip install haystack-ai
24. Improve LLM-based Applications with Fallback Mechanisms Room A1 Wednesday 11:40 Bilge Yücel DevRel Engineer
25. Thursday 18:00 Mindspace Krausenstraße lu.ma/berlin-genai-meetup
26. discord.gg/haystack lu.ma/berlin-genai-meetup haystack.deepset.ai @silvanocerza@mastodon.world linkedin.com/in/silvanocerza