[Wroclove.rb 2024] Building LLM-powered applications in Ruby

Transcript

1. Building LLM-powered applications in Ruby Wroclove.rb 2024 April 14, 2024 by Andrei Bondarev
2. My Impact
3. What is Generative AI? Large Language Models (LLMs) Deep Learning Artificial Neural Networks (models) with general-purpose language understanding and generation. Exploded in popularity after the Attention Is All You Need (2017) research paper that introduced the Transformers architecture.
4. The GenAI Impact Supervised learning 1 month Get labeled data 3 months Train AI model on data 3 months Deploy (run) model Prompt-based AI Few days Specify prompt Few days Deploy model
5. LLMs excel at Structuring Data Summarizing Data Classifying Data Translating Languages Generating Content Answering Questions
6. LLM in every stack
7. AI-centric tech stack
8. Decision tree traversal
9. Business logic (in code) The Ruby on Rails promise: "Developers focus on writing business logic and not reinventing engineering solutions" Old World (before AI) Business logic in models and service objects New World (after AI) Business logic in prompts
10. Business logic (in prompts)
11. AI Agents ƻ Autonomous (semi-autonomous) general purpose LLM-powered programs Can use Tools (APIs, other systems) via 'Function Calling' Work best with powerful LLMs Can be used to automate workflows/business processes and execute multi-step tasks
12. Agent Reliability Responsibilities # of Tasks Decision Tree SIMPLER COMPLEX INCREASES Reliability DECREASES RELIABLE UNREALIABLE
13. Reliability x Focus Reliable Unreliable General Focused AGI Today Pre-Transformers Today Today
14. Slow adoption 1 Fast changing 2 IP / Copyright issues 3 Lack of tooling 4 Risky THE CITY - NYC News NYC AI Chatbot Touted by Adams Tells Businesses to Break the Law The Microsoft-powered bot says bosses can take workerʼs tips and that landlords can discriminate based on source of income. That's not right. aibusiness.com Air Canada Held Responsible for Chatbotʼs Hallucinations Air Canada's chatbot gave a traveler wrong airfare information. The traveler sued when the airline refused to give a refund. GM Authority GM Dealer Chat Bot Agrees To Sell 2024 Chevy Tahoe For $1 One customer recently managed to trick a dealer chat bot to agree to sell them a new 2024 Chevy Tahoe for just $1.
15. Prompt Engineering Alchemy Ŀ Randomness ¯\_(ツ)_/¯ [Large Language Models as Optimizers](https://arxiv.org/pdf/2309.03409.pdf)
16. Jailbreaking The process of getting a GenAI model to do or say unintended things through prompting Many-shot jailbreaking — including large amounts of text in a specific configuration that forces LLMs to produce potentially harmful responses, despite their being trained not to do so.
17. LLM Deficiencies Hallucinations Model generating incorrect or non-sensical text Outdated Data Example: GPT-4 was trained on data up to April 2023 Relevant Knowledge is Not Used
18. Retrieval Augmented Generation (RAG)
19. Detailed explanation 1 Generate vector embeddings from user's question. 2 Retrieve relevant documents by running similarity search in a vector database. 3 Construct the RAG prompt to send to the LLM. 4 Get the response back from the LLM in natural language
20. Vector embeddings An approach to cluster data in an N-dimensional space, organized by its meaning. LLMs encode meaning behind texts in the embedding space or "latent space". OpenAI's text-embedding-ada-002 model uses 1536 dimensions.
21. Vector or "Latent" space
22. Semantic/Vector search
23. RAG prompt
24. Naive RAG
25. Evals Human evals ( / ) GPT-4 RAGAS metric (0.0 - 1.0 score) Faithfulness Ensuring retrieved context can act as a justification for the generated answer Context Relevance Context is focused, with little to no irrelevant information Answer Relevance The answer addresses the actual question
26. Advanced RAG Strategy
27. Interfaces ❌ Open chat ✅ Intelligent quick actions
28. langchainrb ⭐ Most popular Ruby framework for building LLM-powered applications Use-cases: LLMs Agents Retrieval-Augmented Generation
29. Langchain.rb Approach Vendor-agnostic Support most popular providers Best-practices Staying on top of emerging research Batteries-included Ease of use
30. Langchain.rb Approach llm = Langchain::LLM::OpenAI.new # Cohere, Gemini, Mistral, and more llm.chat() llm.embed() Common interface for LLMs: vector_db = Langchain::LLM::Weaviate.new # Chroma, Qdrant, and more vector_db.add_texts() vector_db.similarity_search() vector_db.ask() # Naive RAG Common interface for vectorsearch DBs:
31. Building an Assistant
32. Why (not) Ruby? Pragmatism Focus on business outcomes OOP principles "There's a gem for that"
33. My Most Popular Tweet Twitter Andrei Bondarev on Twitter / X 1. Ruby Developer is sad a library exists in Python but not in Ruby.2. Ruby Developer copy- pastes said library' s files one by one into ChatGPT.3. ChatGPT converts Python code to Ruby code.4. Ruby Developer fixes a few undefined errors and imports the correct…
34. Learnings from OSS Be responsive Be friendly , Be helpful Ƽ
35. Langchain.rb Contributors
36. Langchain.rb Discord
37. "We do not have it all figured out yet…" "But we're optimistic!" —AI Engineering Community
38. Thank you! Questions? @rushing_andrei @andreibondarev in/andreibondarev andrei@sourcelabs.io