[Rocky Mountain Ruby 2024] - Building AI Agents in Ruby

The Coatue AI report is putting AI models at the centerpiece of all modern tech stacks going forward that Application Devs will be using to build on top of. It would not be controversial to say that the Ruby ecosystem lacks in its support and adoption of AI, ML and DS libraries. If we’d like to stay relevant in the future, we need to start building the foundations now. We’ll look at what Generative AI is, what kind of applications developers in other communities are building and how Ruby can be used to build similar applications today. We’ll cover Retrieval Augmented Generation (RAG), vector embeddings and semantic search, prompt engineering, and what the state of art (SOTA) in evaluating LLM output looks like today. We will also cover AI Agents, semi-autonomous general purpose LLM-backed applications, and what they’re capable of today. We'll make a case why Ruby is a great language to build these applications because of its strengths and its incredible ecosystem. After the slides, we'll build an AI Agent in 15 min.

Transcript

1. Building AI Agents in Ruby Rocky Mountain Ruby 2024 Tuesday, October 8th, 2024 by Andrei Bondarev
2. Work Source Labs LLC Patterns AI
3. My Impact
4. GenAI Impact Before: 1 month Label data 3 months Train custom model 3 months Deploy (optimize) After: Few days Prompt engineering Few weeks Basic RAG (if needed) Few days Deploy
5. Common ML tasks Data Structuring Summarization Classification Language Translation Content Generation Named Entity Recognition
6. Capabilities, an API call away Adoption Cost
7. AI in every stack
8. AI Agents in every Enterprise
9. (Re-)Rise of AI Agents 1950s 1970s — 1980s 1990s — 2000s Intelligent Machines Expert Systems 2010s Software Agents 2020s Chatbots LLMs as Agents
10. The Vision
11. AI Agent ƻ Definition: An autonomous software system capable of perceiving its environment, making decisions, and taking actions to achieve specific goals. ♻ Environment awareness 2 Decision-making Ƣ Action-taking
12. Agent vs Assistant Conversational Assistant Conversational system that continuously takes directions from a human Autonomous Agent Autonomous system that independently executes a task (like a background job)
13. Use-cases Automating business processes Mundane low-IQ tasks Personal assistant (co- pilot) Time-consuming tasks Tasks in a consulting business: Creating invoices from timesheets Categorizing business expenses Writing project proposals (incl. service offering, meeting notes) Writing job descriptions. Writing JIRA tickets.
14. AI Agent components: Reasoning & Planning (LLMs) Tasks/Goals/Objectives/Workflows Triggers Memory Tools / Functions Evals Observability Analytics Fine-tuning
15. Reasoning & Planning Cornerstone for problem-solving, decision-making and critical analysis. Primary forms of reasoning Deductive — drawing a specific conclusion from general facts. Inductive — making a broad generalization from specific observations Abductive — finding the simplest explanation for an observation Plan formulation Decomposing a top-level task into numerous sub- tasks. Plan reflection Leveraging feedback mechanism to reflect upon a plan and evaluate its merits.
16. Chain-of-Thought (CoT) Paper: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2022) Forcing the AI to explain it's reasoning. Without Chain-of-Thought prompting With Chain-of-Thought prompting
17. Business logic Tasks/Goals/Objectives/Workflows/"Standard Operating Procedures" Standard Operating Procedures in e-commerce. New Order Return Order
18. Triggers State change Schedule ⏰ Event-driven "6 Manual ▶
19. Memory Saving the context, execution progress, tool calling to memory
20. Retrieval Augmented Generation (RAG)
21. Tool/Function Calling Structured Outputs Response adhere to a predefined JSON schema External Tools Intent detection
22. Tool Calling Use tools to do the following: Get data from external sources (APIs) Get real-time data Take actions Execute deterministic tasks1 Without Tools Using the Tool (Code Interpreter)
23. Tool Calling Function definition User's message Function invocation
24. AI Agent diagram LLMs AI Agent Tools Triggers ⏰ Instructions Memory User Store/Retriever Take Actions Reason/Plan Business logic Converse
25. langchainrb Ruby framework for building LLM-powered applications
26. Demo
27. Nerds & Threads Selling comfortable nerdy t-shirts for software engineers that work from home AI Agent ú Customer Management ✉ Email Service  Payment Gateway Service Order Management Inventory Management Shipping Service
28. Business logic (in code) The Ruby on Rails promise: "Developers focus on writing business logic and not the 'plumbing'" Old World (before AI) Business logic in models and service objects New World (after AI) Business logic in prompts
29. Code
30. Tool Definitions
31. Demo
32. Text-to-SQL
33. Why would you use this? Changing requirements on the fly Intelligence in your process Tackling complex workflows
34. Evaluations Benchmarks Comparing to a large dataset of question-answer pairs. "LLM as a Judge" Asking LLM whether the answer fits a list of criteria.
35. Benchmarks huggingface gretelai/gsm8k-synthetic-diverse-405b · Datasets at Hugging Face We ʼ re on a journey to advance and democratize artificial intelligence through open source and open science .
36. Agent Reliability Responsibilities # of Tasks Decision Tree SIMPLER COMPLEX INCREASES Reliability DECREASES RELIABLE UNREALIABLE
37. System reliability Modern software fails because: AI systems fail because: Dependencies Inaccurate or incomplete data / Bias in data Doesn't scale Compute limits Cloud outages Cloud outages Cyber attacks Adversarial attacks Insufficient testing (bugs) Black box behavior Unclear liability & accountability Engineering problems that will be solved.
38. Why Ruby?
39. Recap
40. Thank you! ɉ @rushing_andrei @andreibondarev in/andreibondarev andrei@sourcelabs.io Discord