Building AI Agents in Ruby Rocky Mountain Ruby 2024 Tuesday, October 8th, 2024 by Andrei Bondarev

Work Source Labs LLC Patterns AI

My Impact

GenAI Impact Before: 1 month Label data 3 months Train custom model 3 months Deploy (optimize) After: Few days Prompt engineering Few weeks Basic RAG (if needed) Few days Deploy

Common ML tasks Data Structuring Summarization Classification Language Translation Content Generation Named Entity Recognition

Capabilities, an API call away Adoption Cost

AI in every stack

AI Agents in every Enterprise

(Re-)Rise of AI Agents 1950s 1970s — 1980s 1990s — 2000s Intelligent Machines Expert Systems 2010s Software Agents 2020s Chatbots LLMs as Agents

The Vision

AI Agent ƻ Definition: An autonomous software system capable of perceiving its environment, making decisions, and taking actions to achieve specific goals. ♻ Environment awareness 2 Decision-making Ƣ Action-taking

Agent vs Assistant Conversational Assistant Conversational system that continuously takes directions from a human Autonomous Agent Autonomous system that independently executes a task (like a background job)

Use-cases Automating business processes Mundane low-IQ tasks Personal assistant (co- pilot) Time-consuming tasks Tasks in a consulting business: Creating invoices from timesheets Categorizing business expenses Writing project proposals (incl. service offering, meeting notes) Writing job descriptions. Writing JIRA tickets.

AI Agent components: Reasoning & Planning (LLMs) Tasks/Goals/Objectives/Workflows Triggers Memory Tools / Functions Evals Observability Analytics Fine-tuning

Reasoning & Planning Cornerstone for problem-solving, decision-making and critical analysis. Primary forms of reasoning Deductive — drawing a specific conclusion from general facts. Inductive — making a broad generalization from specific observations Abductive — finding the simplest explanation for an observation Plan formulation Decomposing a top-level task into numerous sub- tasks. Plan reflection Leveraging feedback mechanism to reflect upon a plan and evaluate its merits.

Chain-of-Thought (CoT) Paper: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (2022) Forcing the AI to explain it's reasoning. Without Chain-of-Thought prompting With Chain-of-Thought prompting

Business logic Tasks/Goals/Objectives/Workflows/"Standard Operating Procedures" Standard Operating Procedures in e-commerce. New Order Return Order

Triggers State change Schedule ⏰ Event-driven "6 Manual ▶

Memory Saving the context, execution progress, tool calling to memory

Retrieval Augmented Generation (RAG)

Tool/Function Calling Structured Outputs Response adhere to a predefined JSON schema External Tools Intent detection

Tool Calling Use tools to do the following: Get data from external sources (APIs) Get real-time data Take actions Execute deterministic tasks1 Without Tools Using the Tool (Code Interpreter)

Tool Calling Function definition User's message Function invocation

AI Agent diagram LLMs AI Agent Tools Triggers ⏰ Instructions Memory User Store/Retriever Take Actions Reason/Plan Business logic Converse

langchainrb Ruby framework for building LLM-powered applications

Demo

Nerds & Threads Selling comfortable nerdy t-shirts for software engineers that work from home AI Agent ú Customer Management ✉ Email Service Payment Gateway Service Order Management Inventory Management Shipping Service

Business logic (in code) The Ruby on Rails promise: "Developers focus on writing business logic and not the 'plumbing'" Old World (before AI) Business logic in models and service objects New World (after AI) Business logic in prompts

Code

Tool Definitions

Demo

Text-to-SQL

Why would you use this? Changing requirements on the fly Intelligence in your process Tackling complex workflows

Evaluations Benchmarks Comparing to a large dataset of question-answer pairs. "LLM as a Judge" Asking LLM whether the answer fits a list of criteria.

Benchmarks huggingface gretelai/gsm8k-synthetic-diverse-405b · Datasets at Hugging Face We ʼ re on a journey to advance and democratize artificial intelligence through open source and open science .

Agent Reliability Responsibilities # of Tasks Decision Tree SIMPLER COMPLEX INCREASES Reliability DECREASES RELIABLE UNREALIABLE

System reliability Modern software fails because: AI systems fail because: Dependencies Inaccurate or incomplete data / Bias in data Doesn't scale Compute limits Cloud outages Cloud outages Cyber attacks Adversarial attacks Insufficient testing (bugs) Black box behavior Unclear liability & accountability Engineering problems that will be solved.

Why Ruby?

Recap

Thank you! ɉ @rushing_andrei @andreibondarev in/andreibondarev [email protected] Discord