Building LLM Powered Features

Links from the talk

https://lmstudio.ai/
https://www.youtube.com/watch?v=2eWuYf-aZE4
https://www.youtube.com/watch?v=kCc8FmEb1nY
https://github.com/karpathy/nanochat
https://www.manning.com/books/build-a-large-language-model-from-scratch
https://airtable.com/app8FCbqCU88hfOmw/shroEheCLsxBqz5eF/tbl5tknm2fgOUCVpA/viwBx10aPYYmALBUL

Transcript

1. 
 Building LLM Powered Features ! Radoslav Stankov
2. Radoslav Stankov @rstankov rstankov.com
3. https://tips.rstankov.com
5. I'm not going talk about...
6. 1.AGI I'm not going talk about...
7. 1.AGI 2.AI taking developers jobs I'm not going talk about...
8. 1.AGI 2.AI taking developers jobs 3.Vibe coding I'm not going talk about...
9. 1.AGI 2.AI taking developers jobs 3.Vibe coding 4.How fast AI is moving I'm not going talk about...
10. 1.AGI 2.AI taking developers jobs 3.Vibe coding 4.How fast AI is moving 5.Agents / MCP I'm not going talk about...
12. LLM-based features will become stable in every application, just like databases. Knowing how to work with LLMs will be as essential a skill for a developer. My thoughts
13. LLMs are already incredible, and there is years of work to be done to fully productize the capabilities that exist today.
14. 
 Building LLM Powered Features ! Radoslav Stankov
25. .gguf !
30. GPU Model Network API
31. await fetch("https://api.openai.com/v1/responses", { method: "POST", headers: { "Content-Type": "application/json", "Authorization": "Bearer YOUR_OPENAI_API_KEY" }, body: JSON.stringify({ model: "gpt-5.1", input: [ { role: "system", content: "You are a helpful assistant." }, { role: "user", content: "Explain what LLM is" } ], temperature: 0.7 }) }); Chat mode
32. await fetch("https://api.openai.com/v1/responses", { method: "POST", headers: { "Content-Type": "application/json", "Authorization": "Bearer YOUR_OPENAI_API_KEY" }, body: JSON.stringify({ model: "gpt-5.1", instructions: "Explain concepts clearly and concisely.", input: "What is an LLM?", temperature: 0.7 }) }); Instructions + Input Mode
33. Instructions (system prompt)
34. Input (user prompt)
35. Instructions + Input Output LLM
36. lets zoom in...
37. Instructions + Input Output LLM
38. Instructions + Input Output
39. Instructions + Input Output
40. Instructions + Input Output
41. Instructions + Input Tokenizer Output
42. Instructions + Input Tokenizer All tokens so far (context) Output
43. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) Output
44. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Output
45. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Next-token probabilities Output
46. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Next-token probabilities Sampling (temperature, top-k, etc.) Output
47. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Next-token probabilities Sampling (temperature, top-k, etc.) Select one token (next token) Output
48. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Next-token probabilities Sampling (temperature, top-k, etc.) Select one token (next token) EOS token? Output
49. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Next-token probabilities Sampling (temperature, top-k, etc.) Select one token (next token) EOS token? append token to context false Output
50. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Next-token probabilities Sampling (temperature, top-k, etc.) Select one token (next token) EOS token? Detokenizer true append token to context false Output
51. Instructions + Input Tokenizer All tokens so far (context) Embeddings (vectors) LLM 
 (transformer) Next-token probabilities Sampling (temperature, top-k, etc.) Select one token (next token) EOS token? Detokenizer true append token to context false Output
54. https://www.youtube.com/watch?v=2eWuYf-aZE4
55. https://www.youtube.com/watch?v=kCc8FmEb1nY
56. The best ChatGPT that $100 can buy. https://github.com/karpathy/nanochat
57. https://www.manning.com/books/build-a-large-language-model-from-scratch
59. Instructions + Input Output LLM
60. SQL Output Database
62. 
 Building LLM Powered Features ! Radoslav Stankov
65. ! Retrieval-Augmented Generation (RAG) Injecting relevant data from external sources like databases or files into the LLM’s context so it can produce more accurate outputs.
66. ! Retrieval-Augmented Generation (RAG) 1/ user query 2/ retrive data 3/ receive data 4/ query + data 5/ response 6/ answer
67. ! Retrieval-Augmented Generation (RAG) 1/ user query 2/ retrive data 3/ receive data 4/ query + data 5/ response 6/ answer
68. ! Retrieval-Augmented Generation (RAG) 1/ user query 2/ retrive data 3/ receive data 4/ query + data 5/ response 6/ answer
69. ! Retrieval-Augmented Generation (RAG) 1/ user query 2/ retrive data 3/ receive data 4/ query + data 5/ response 6/ answer
70. ! Retrieval-Augmented Generation (RAG) 1/ user query 2/ retrive data 3/ receive data 4/ query + data 5/ response 6/ answer
71. ! Retrieval-Augmented Generation (RAG) 1/ user query 2/ retrive data 3/ receive data 4/ query + data 5/ response 6/ answer
72. ! Retrieval-Augmented Generation (RAG) 1/ user query 2/ retrive data 3/ receive data 4/ query + data 5/ response 6/ answer
73. ! Levels of LLM integrations ! Single features ! Workflows ! Agents
74. ! Single features Single LLM call and result
75. ! Workflows Multistep LLMs calls orchestrated by code
76. ! Agents LLMs calling tools in a loop to achieve a goal
80. const PROMPT = ` You are search assistant for {form_description} form. Convert the user's natural language query into a JSON object using only the fields defined in the HTML form. {form} # Context The current date is "{current_date}" The user name is "{user_name}" The user ID is "{user_id}" # Instructions - ... `; export default { async transactionsSearchParams({ query, formCode, currentUser }) { const prompt = PROMPT .replace('{form_description}', 'financial transactions',) .replace('{form}', formCode) .replace('{current_date}', new Date().toISOString().slice(0, 10)) .replace('{user_id}', String(currentUser.id)) .replace('{user_name}', currentUser.name); return await OpenAiApi.response({ label: `TransactionsSearch: ${query}`, userId: currentUser.id, instructions: prompt, input: query, }); }, };
81. formCode = ` <form> <select name="user_id"> <option value="">Kасиер</option> <option value="43917">Демо Потребител</option> </select> <select name="source"> <option value="">Източник</option> <option value="payment">Плащане</option> <option value="epay">Платено през ePay</option> <option value="easy_pay">Платено през EasyPay</option> <option value="icard">Платено през iCard</option> <option value="bank_transfer">Платено с банков трансфер</option> </select> <label>Равно <input type="date" name="date[eq]" /></label> <label>От <input type="date" name="date[gteq]" /></label> <label>До <input type="date" name="date[lteq]" /></label> <select name="kind"> <option value="">Тип</option> <option value="income">Приход</option> <option value="expense">Разход</option> </select> <input type="search" name="query" placeholder="Заглавие" /> <label>Равно <input type="text" name="amount[eq]" /></label> <label>От <input type="text" name="amount[gteq]" /></label> <label>До <input type="text" name="amount[lteq]" /></label> <label>Равно <input type="date" name="created_at[eq]" /></label> <label>От <input type="date" name="created_at[gteq]" /></label> <label>До<input type="date" name="created_at[lteq]" /></label> </form> `; const params = await transactionsSearchParams({ currentUser, formCode, query }); redirectTo(paths.transactionsSearch(params));
87. ! Evals LLM evals are automated tests that measure how reliably a model behaves across real scenarios so you can ship LLM features with confidence.
88. 1. Generate a lot of outputs with different inputs. Analyze them. 2. Build a couple of tests with something like the VCR gem 3. Record LLM interactions and review manually with UI 4. Adjust prompt and go through steps 1/2 5. Process depends on specific features. ...this takes time, tries, and tokens ($$$) ! My current process
89. 1. Generate a lot of outputs with different inputs. Analyze them. 2. Build a couple of tests with something like the VCR gem 3. Record LLM interactions and review manually with UI 4. Adjust prompt and go through steps 1/2 5. Process depends on specific features. ...this takes time, tries, and tokens ($$$) ! My current process
91. Recap
92. 0/ ! How LLMs work 1/ ! RAG 2/ ! Levels of LLM integrations - single feature, workflow, agent 3/ ! Context 4/ ! Evals ! Recap
94. Learning resources
95. https://bit.ly/4cznIfA
96. https://rstankov.com/appearances
97. https://rstankov.com/appearances