Ruby On RAG - Building AI Use Cases for Fun and Profit

This talk will cover what RAG is, how it works, and why we should be building RAG applications in Ruby and Rails. I'll share some code examples of what a toy RAG pipeline looks like in native Ruby and do a live demo of a simple RAG application. I'll also share a perspective as to why Ruby and Rails are great tools for building LLM applications and that the future languages for building such applications are whatever languages are most natural to you.

Transcript

1. Ruby On RAG Building AI Use Cases for Fun and Profit 1
2. I am Landon Gray Founder & AI Engineer @ Identus Consulting Hello! 2
3. Overview ◉ Fun ◦ What is RAG ◦ What Problem does it solve ◦ How it works? ▪ Indexing ▪ Retrieval & Generation ◉ Profit - Practical ◦ Demo 3
4. What is Rag? 4
5. “ Retrieval Augmented Generation (RAG) is way to augment the LLM knowledge with additional information. 5
6. Typical LLM Query-Response Cycle 6
7. Hallucinations (making stuff up) 7
8. Non Relevant Data 8
9. 9 How do we deal with this?
10. Try Adding Documents Limited Context Window!
11. Token Token can be thought of as a piece of a word. Context Window & Tokens Context Window The maximum number of tokens that can be used in a single request, inclusive of both input and output tokens. 11 https://platform.openai.com/docs/models https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them
12. Rule of Thumb 1 token~= 4 chars in English 100 tokens ~= 75 words Context Window & Tokens 12 https://platform.openai.com/docs/models https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them Example 6 tokens = “Ruby is a fine programming language” GPT4-o Context Window = 128,000 tokens ~= 96,000 words
13. 96,000 Words Visualized: To Kill a Mockingbird Approx. 100,000 words 13
14. 96,000 Words Visualized: Ender’s Game Approx. 100,000 words 14
15. 96,000 Words Visualized: 1984 Approx. 88,000 words 15
16. 16 How we calculate the token size of various documents?
17. OpenAI - Tokenizer OpenAI has a tool to help us understand how many tokens a piece of text might contain. 17
18. tiktoken_ruby tiktoken_ruby is a fast BPE (Byte Pair Encoding) tokenizer for use with OpenAI's models. 18
19. Limited Context Window
20. Indexing
21. Chunking 21
22. Chunking (Text Splitting)
23. Pass chunks to the LLM Which Chunks?
24. 24 We’ll get back to that…
25. Embeddings 25
26. Embedding = Array of Floats 26
27. Embedding = Vector = Array of Floats 27
28. Chunks to Array of Floats (Embedding) Chunks Embedding
29. “ Vector embeddings are a way to convert words and sentences and other data into numbers that capture their meaning and relationships. 29
30. Effecient Semantic Search 30
31. 31 Cake Water Milk Cookie Relationships?
32. 32 Cake Water Milk Cookie Drink Dessert
33. 33 Cake Water Milk Cookie Drink Dessert Water Milk Cake Cookie
34. 34 Drink Dessert Water Milk Cake Cookie Meaning Maintained!
35. Store 35
36. Store our Embeddings
37. Quick Recap - Indexing Chunking Embedding Store (Vector Database)
38. 38 Time to get back to that thing…
39. Pass chunks to the LLM Which Chunks?
40. Pass chunks to the LLM
41. Similarity Search 41
42. Observations 42
45. Graph our data 45
47. 47 Vector Store (Vector Space)
53. Similarity Search 53
55. Find 3 chunks similar to Star Wars
58. Find 3 chunks similar to Star Wars
59. Find 3 chunks similar to Star Wars
60. Find 3 chunks similar to Star Wars
61. Retrieval & Generation
62. Retrieval & Generation
63. Quick Recap: Similarity Search
64. Generate a prompt Query: Search Results:
65. Clarification: Query vs Prompt Prompt Final text input ingest by the LLM. Which often contains the Query Query Input text generated by some human or system.
66. Generate a prompt Query: Search Results: Prompt:
67. Pass to LLM to get Response
68. Quick Recap - Search & Generation
69. That’s It! 69
71. Demo 71
72. Thank You 72