AI Agents: How Do They Work and How to Build Them @ Shift 2025

Shift 2025, Zadar, Croatia

Did you know you already have all the skills you need to build an AI agent? You only need some JavaScript knowledge and an API key (optional but highly recommended). AI agents have been a hot topic recently. Everybody talks about amazing, intelligent AI agents that often sound like popular science-fiction movies. Let’s talk about AI agents in plain and simple language and learn how to build them using JavaScript and your favorite LLM.

Transcript

1. Have you heard about AI agents?
2. Yeah, the intelligent agents who will take your job in a few years!
3. I don't want to scare you, but…
4. "By2034,AI"agents"will: •Replace70%ofofficework(McKinsey) •Add$7trilliontotheglobaleconomy(Goldman) Mostjobswillbecomeobsolete. Here'swhatyouneedtoknow(&howyoushouldprepare)"!
5. "By2034,AI"agents"will: •Replace70%ofofficework(McKinsey) •Add$7trilliontotheglobaleconomy(Goldman) Mostjobswillbecomeobsolete. Here'swhatyouneedtoknow(&howyoushouldprepare)"! Somebodyontwitter (+McKinsey&Goldman)
8. So, welcome to my woodworking course!
9. AI AGENTS
10. AI AGENTS with some JavaScript
11. AI AGENTS with some JavaScript How work and how to build them
12. AI AGENTS with some JavaScript How work and how to build them
13. When people talk about AI agents, we often imagine:
14. But in reality, AI agents are more like:
15. Don't get me wrong. These agents can be extremely helpful!
16. How LLMs work But before we continue, let's see
28. Source: https://platform.openai.com/tokenizer
29. Prompts: - Prompts are instructions - You te! an LLM what you want, and it tries to reply based its training and your instructions - more clear instructions = better reply - LLM always answer, but not always based on truth
30. Prompts are just instructions
31. So, how do they work?
32. You give your instructions
34. You get some unexpected wisdom or hallucination
35. But how do LLMs know how to reply?
37. Tokens and vectors
38. @slobodan_
39. Some parts of your instructions might be more important than other or you might want to make them repeatable
40. System prompts
41. Slobodan Stojanovic cofounder/CTO @ Vacation Tracker @slobodan_
42. Before we talk about AI agents, let me tell you a story from 6 months ago…
43. Meet my "friend" Claude. I ask it many weird things all the time Way more weird than this one, trust me
44. This specific question is interesting because Claude could not answer it* *It can today, I know! Don’t kill my vibe just yet…
46. Some other LLMs can answer this question! But I like Claude. Can I help it to answer?
50. By the way, we just created an AI agent!
51. I know ChatGPT can search the internet. Claude 4 can, too. But these are also agents. These agents are now built into the ChatGPT and Claude products.
52. An agent searched the web and provided missing info
53. How AI agents work
54. LLMs can do and are good at this:
55. LLMs will always reply. Some replies might not be meaningful (hallucinations).
56. Agents are LLMs + something that provides missing information & capabilities
58. A "While" loop
59. An AI Agent is like a "while" loop that keeps asking available tools to provide additional information or capability until it has all it needs to complete the task or answer the question
60. The tool can be anything that provides the missing information or capabilities
61. I am the "tool" here!
62. But, while loops can be expensive!
63. Not because of the code complexity of a while loop, but because you invoke an LLM at least once in each iteration!
65. What does it mean to be "expensive" in this context? It depends on your use case! But be careful.
66. How to be careful: - Define spending limits - Make sure you do not iterate indefinitely (i.e., stop after N retries) - Use cheaper models for simple evaluations - Add monitoring and alarms
67. Where do we write these while loops?
68. Anywhere you want!
69. If you really really want, you can be that "while" loop
70. I am the "while loop" here!
71. But you can write your "while loop" anywhere you need it. This while loop can be in an app, in a terminal, on a server, in a browser, etc.
72. Just be careful not to expose your LLM secret keys and limit their usage because, remember, these while loops can be expensive.
73. How to write a while loop? - Define a system prompt with clear explanation of a! the tools you want to support (i.e., when and how to invoke them) - Ask an LLM to reply in the strict JSON format - Make sure you parse and validate reply correctly - Handle errors and have a limit on the number of iterations
74. LLMs are good at talking to humans, but these replies are not easy to parse in the code
76. You can ask an LLM to reply in the JSON format
77. User: // Some long instructions But always reply with valid JSON and nothing else! Here's your JSON: ```json { "some": "JSON", Assistant:
78. I SAID JSON ONLY!!! Works, sometimes
79. But there's something else you can do!
80. Write the beginning of the reply in the API request!
81. User: // Your instructions Answer with valid JSON and nothing else. { " Assistant: System: // Your system prompt
82. // Your instructions Answer with valid JSON and nothing else. { " // Your system prompt some": "valid", "JSON": true } User: Assistant: System:
83. It's good to understand how these "while loops" work. But, you don't really need to write your own while loop!
84. Popular AI Agents tools & frameworks: - LlamaIndex - LangChain - AutoGen - Amazon Bedrock Agents - Many other alternatives…
85. Also, now all major LLMs have built-in agent capabilities (or simply, they can use tools)
86. https://platform.openai.com/docs/guides/function-calling
87. https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview
88. Built-in tools pros: - Less errors (no n"d to force an LLM to return JSON) - No third-party tools - We! defined format
89. Built-in tools cons: - A bit harder to switch models (you n"d to write a sma! adapter/wrapper)
90. Let's build an AI agent!
93. We could build this in many different ways in production. For example, it can look similar to the following diagram
95. In production you need to think about: - Web application firewa! (WAF) with rate limiting - Error handling - Rate limits (for your app + LLMs + other services) - Monitoring - Conversation storage (i.e., DynamoDB) - And many other things
96. I'll show the most important parts of the code, only
97. Define available tools
99. Then you can invoke the LLM with the list of tools
101. The response
103. Then you send your API request, wait for the reply and do the following:
106. Agent response
108. That's it!
109. The complete code example is more complicated. Check the full example here: https://slobodan.me/posts/ai-agents/
110. A few other things to check
111. https://docs.anthropic.com/en/docs/agents-and-tools/mcp
112. https://slobodan.me/posts/5-prompt-engineering-tips-for-developers/
113. https://slobodan.me/posts/ai-agents/
114. A quick summary
115. @slobodan_ • An AI Agent is like "while" loops with tools • You have all the skills you need to build tools • AI agents aren't scary, and they can be useful • Go, build agents, and have fun
116. https://slobodan.me @slobodan_