Supercharge Conversational AI with RAG, Agents, and Memory

The slide deck of a webinar hosted by Pinecone: Supercharge Conversational AI with RAG, Agents, and Memory

Recording: https://www.youtube.com/watch?v=TYuoQjWur8w&t=1s

In this workshop, we will delve into RAG, agents, and memory — how to integrate them all using Pinecone and Haystack to create high performance, up-to-date, conversational AI.

One of the greatest challenges of developing with LLMs is their lack of up-to-date knowledge of the world or niche that they operate in. Retrieval Augmented Generation (RAG) allows us to tackle this issue by giving LLMs direct access to external information.

By pairing this with AI Agents, we can build systems with incredible accuracy and flexibility. The superpower of LLMs is their ability to converse in natural language, so to make the most of RAG and Agents we incorporate short-term memory, a fundamental component of conversational systems such as chatbots and personal assistants.

Transcript

1. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Supercharge Conversational AI with RAG, Agents, and Memory
2. Bilge Yücel Developer Advocate, Deepset James Briggs Staff Developer Advocate, Pinecone 2
3. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023)
4. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Not only a problem with up to date info 🔒 Internal data 󰡸 Niche domain 🥚 Freshness 🔮 Hallucinations
5. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) The Problem
6. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) The Solutions 🏋 Fine-tuning 📚 Retrieval Augmented Generation (RAG)
7. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Fine-tuning ✅ Chatbot behavior ✅ Kind of accurate knowledge of data that doesn’t update often and is not too sensitive to errors ❌ Trust in results (as we cannot provide citations) ❌ Data that updates quickly (like company stock catalogue)
8. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) RAG ✅ Fast changing knowledge that must be managed ✅ Adding citations to improve user trust ✅ Learning and storing facts about users ❌ Chatbot behavior
9. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Need all of these things? Use both!
10. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) What is RAG?
11. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) RAG swaps this:
12. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) For this…
13. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) How do we Feed External Knowledge to the LLM?
14. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Could we stuff everything into the context window? (No) Arxiv Paper: N. Liu, et. al., Lost in the Middle: How Language Models Use Long Contexts (2023), Stanford University Our Article: A. Catav, Less is More: Why Use Retrieval Instead of Larger Context Windows (2023), Pinecone
15. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Arxiv Paper: N. Liu, et. al., Lost in the Middle: How Language Models Use Long Contexts (2023), Stanford University Our Article: A. Catav, Less is More: Why Use Retrieval Instead of Larger Context Windows (2023), Pinecone
16. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Is There a Better Way? We need to selectively feed highly relevant info into our context LLMs work in natural language — so ideally our search should too Using a vector DB, we can retrieve relevant docs with “natural language” — we can do a semantic search
17. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) 🏦 Where is the Bank of England? 🌱 Where is the grassy bank? 🛩 How does a plane bank? 🐝 “the bees decided to have a mutiny against their queen” 🐝 “flying stinging insects rebelled in opposition to the matriarch” Semantic Meaning
18. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Semantic Search
19. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Retrieval Augmented Generation (RAG) ✅ Retrieval of highly relevant docs using natural language search ✅ Scalable to billions of records ✅ Data management like a traditional DB 🚀 No context stuffing required!
20. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Retrieval Augmented Generation
21. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Vector DB Given a ‘query’ vector, Pinecone returns the most similar ‘context’ vectors
22. Webinar: Supercharge Conversational AI with RAG, Agents, and Memory (November 2023) Pinecone • Purpose-built, cloud-native vector database for SotA vector search • Scales to 10B+ records • Integrations with ML frameworks and LLMs like Haystack! • Provides completely managed infrastructure with automatic scaling, load balancing, and fault tolerance
23. What is Haystack? • Fully open-source framework built in Python for custom LLM applications • Provides tools that developers need to build state-of-the-art NLP systems • Building blocks: Pipelines & Components
24. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
25. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
26. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
27. What is Haystack? • Fully open-source framework built in Python for custom LLM applications
28. Indexing Pipeline
29. Indexing Pipeline
30. Indexing Pipeline
31. RAG Pipeline
32. RAG Pipeline • Embeds query • Retrieves relevant documents
33. RAG Pipeline • Connection to LLMs • Receives input • Sends a prompt • Returns a response
34. RAG Pipeline - Prompt
35. RAG Pipeline - Prompt
36. RAG Pipeline
37. Haystack Agent • Uses LLMs • Understands and processes the information retrieved from external sources
38. Haystack Agent • Uses LLMs • Understands and processes the information retrieved from external sources
39. • An Agent is in essence, a PromptNode that connects to an LLM and has a very clever initial prompt • An Agent has access to a variety of Tools • Tools may be: Haystack Pipelines, data connectors, the web… • Each Tool may be useful to achieve one specific task, or simply, it may provide the Agent access to a certain knowledge base • An Agent can create an action plan to resolve complex queries, and do so by invoking and orchestrating the Tools that it has at its disposal • Memory: whole conversation or summary Features of an Agent
40. Agent
41. Agent Prompt
42. Agent Prompt
43. Agent Prompt
44. Agent Prompt
45. Agent Tool - RAG Pipeline
46. Agent Prompt Node
47. Agent Prompt Node
48. Agent Reasoning
49. Agent -> Conversational AI Memory
50. Conversational Agent Prompt
51. Memory
52. Conversational Agent
53. Conversational Agent
54. DEMO
55. Q&A Follow 👇 Haystack on Twitter: Haystack_AI What’s coming in Haystack 2.0 Haystack Community on Discord Pinecone Community Pinecone Twitter