A Hands-on AI Agent Guide with Gemma 💎, Ollama 🦙 and more!

DevFest 2024 Istanbul

✨ Are you curious about building an AI agent from the ground up using open-source tools? ✨

Join this session, where we’ll create an intelligent AI agent with Gemma, Ollama, and other open source tools. We’ll start from scratch, exploring key components such as vector databases, RAG, prompt engineering, and evaluation to build a fully functional agent.

Whether you're new to AI development or looking to expand your toolkit, this session will provide practical insights, code examples, and everything you need to start building AI-driven applications.

Transcript

1. A HANDS-ON AI AGENT GUIDE with Gemma, Ollama and more!
2. 🥑 Developer Relations Engineer at deepset 󰎲 🏗 Open source LLM Framework: Haystack 🎓 Sabanci University B.Sc. 🧠 KU Leuven M.Sc. 📍 Istanbul, Turkey ABOUT ME @BILGEYCL /IN/BILGE-YUCEL BILGEYUCEL Bilge Yucel
3. AGENDA • AI AGENTS • BUILDING TOOLS • 󰟲 BUILDING TIME • Q&A
4. AI AGENTS An agent in an LLM-powered system is an autonomous entity that uses a language model as its core to plan, reflect, store memory, and leverage external tools for executing complex tasks and interacting with external data sources efficiently
5. AI AGENTS Overview of a LLM-powered autonomous agent system. Source: https://lilianweng.github.io/posts/2023-06-23-agent/
6. • Can you mail me the latest Hacker News articles? ◦ Reason + Act = ReAct ◦ Tool calls, function calling ▪ find latest Hacker News articles ▪ generate an email ▪ send it AI AGENTS EXAMPLES
7. • Generate JSON from {this} text ◦ Self-reflection: Generates a JSON, verifies it with the second iteration AI AGENTS EXAMPLES
8. • Who is Taylor Swift? + When was she born? ◦ Memory: “When was Taylor Swift born?” AI AGENTS EXAMPLES
9. BUILDING TOOLS • Gemma 2 • Ollama • Haystack
10. GEMMA 2 • An open language model by Google • 2b, 9b, 27b • 8192 tokens • wide range of tasks: question answering, commonsense reasoning, mathematics, science, and coding • Gemma License • More info in Gemma 2: Improving Open Language Models at a Practical Size
11. OLLAMA OLLAMA • OSS to run models locally on macOS, Linux, and Windows • Models come with Ollama • Mostly 4-bit quantization • GPU -> CPU • Mostly for PoC
12. OLLAMA OLLAMA
13. OLLAMA How to use OLLAMA • Install Ollama to your computer • >> ollama pull <model_name> • >> ollama serve
14. OLLAMA HAYSTACK • Haystack is an open source Python framework for building production-ready LLM applications • RAG, Semantic search, Agent… • Prototyping, evaluation, deployment, monitoring… • Building blocks: Components & Pipelines
15. OLLAMA Web-Enhanced Self-Reflecting Agent • Retrieval Augmented Generation (RAG) • If not enough context, goes to web to enrich the context 🌐 • If enough context, finishes ✅
16. OLLAMA Web-Enhanced Self-Reflecting Agent
17. OLLAMA Web-Enhanced Self-Reflecting Agent
18. OLLAMA Web-Enhanced Self-Reflecting Agent
19. OLLAMA PROMPT <start_of_turn>user {% if web_documents %} You were asked to answer the following query given the documents but the context was not enough. Here is the user question: {{ query }} Context: {% for document in documents %} {{document.content}} {% endfor %} {% for document in web_documents %} URL: {{document.meta.link}} TEXT: {{document.content}} — {% endfor %} Answer the question based on the given context. If you have enough context to answer this question, return your answer with the used links. If you don't have enough context to answer, say 'N0_ANSWER'. {% else %} Answer the following query based on the documents retrieved from Haystack's documentation. Documents: {% for document in documents %} {{document.content}} {% endfor %} Query: {{query}} If you have enough context to answer this question, just return your answer If you don't have enough context to answer, say 'N0_ANSWER'. {% endif %} <end_of_turn> <start_of_turn>model
20. OLLAMA PROMPT <start_of_turn>user {% if web_documents %} You were asked to answer the following query given the documents but the context was not enough. Here is the user question: {{ query }} Context: {% for document in documents %} {{document.content}} {% endfor %} {% for document in web_documents %} URL: {{document.meta.link}} TEXT: {{document.content}} — {% endfor %} Answer the question based on the given context. If you have enough context to answer this question, return your answer with the used links. If you don't have enough context to answer, say 'N0_ANSWER'. {% else %} Answer the following query based on the documents retrieved from Haystack's documentation. Documents: {% for document in documents %} {{document.content}} {% endfor %} Query: {{query}} If you have enough context to answer this question, just return your answer If you don't have enough context to answer, say 'N0_ANSWER'. {% endif %} <end_of_turn> <start_of_turn>model
21. OLLAMA PIPELINE
22. OLLAMA LET’S RUN
23. OLLAMA LET’S RUN
24. OLLAMA Q&A @BILGEYCL /IN/BILGE-YUCEL BILGEYUCEL Web-Enhanced Self-Reflecting Agent