How I created a serverless AI Agent that helped more than 1k artists and music producers

In my spare time I love playing guitar. I use a music production software called Ableton Live to make beats and add effects to my guitar. This software is very complex and have thousands of features. Sometimes I had to navigate for hours on tutorials, videos and documentations to be able to make a simple beat or the desired effects on my guitar. Ableton is also very visual, it have complex elements with lots of tabs, buttons and knobs. Without being able to see what you have to interact with, it is very hard to navigate the UI, so putting the manual on a CustomGPT or a NotebookLM wouldn't work because these tools are textual only. This is why I decided to create an AI Agent using Typescript, Next.js, Vercel, AI SDK and Neon Serverless Postgres that was able to understand the manual and all the images and answer questions about this software. I decided also to make it Open Source and now it has helped more than 1k music producers and artists.

Transcript

1. HOW I CREATED A SERVERLESS AI AGENT THAT HELPED MORE THAN 1k ARTISTS AND MUSIC PRODUCERS RAFAEL SARACENI 21.10.25 SERVERLESSDAYS MILANO
2. HOW MANY OF YOU HAVE EVER FELT OVERWHELMED TRYING TO LEARN A NEW, COMPLEX SOFTWARE WITHOUT VISUAL GUIDES?
3. HI, I AM RAFA 3 Software Engineer specialised in AI Powered B2B SaaS More than 10 years of experience building digital products. Italian 󰏢 and Brazilian 󰎙 Currently based in Turin, Italy. Latin America 󰎙 󰐏 󰎆 Europe 󰎾 Australia 󰎉
4. 4 THE PROBLEM
5. 5 THE PROBLEM
6. 6 WHY DON’T YOU JUST DOWNLOAD THE PDF MANUAL AND FEED IT TO NOTEBOOKLM OR A CUSTOM GPT?
8. 8 THE SOLUTION
9. 9 APP ARCHITECTURE OVERVIEW React + Next.js + AI SDK GPT-4o mini Hosting + DB
10. 10 RAG PIPELINE OVERVIEW Web Scraping Embeddings Vector Store Image Description
11. 11 GETTING ALL THE URLS
12. 12 SCRAPING URLS List of URLs For each URL Resource Markdown Content Embeddings Markdown Content Relational DB Vector Store
13. 13 TIP: SAVING MARKDOWN SCRAPPED CONTENT TO DB • No need to scrape data again if I need to regenerate embeddings. • Easier to access scraped data for debugging • If the content of the page scrapped fits the context window it can be used instead of embeddings for better results
14. 14 CHUNKING STRATEGY
15. 15 EXTRACTING IMAGES AND GENERATING DESCRIPTIONS Resource Markdown Content For each URL Media • resource_id • url • description Extract Media URLs using Regex
17. 17 This image shows the Ableton Live interface, specifically a sample clip view. The clip is named "AS_132_A_ML27_5," and the waveform is displayed in the center. Above the waveform, there is a timeline with beat markers from 1 to 2.4. Below the waveform, there are parameters for start and end points, loop, position, length, signature, groove, and BPM, alongside warp mode controls and pitch adjustments.
18. 18 DB SCHEMA Resource Raw markdown content of a scraped url • Id • Content • Description • Title • Url • Source Embedding Embedding of a chunk of the content from a resource • Id • ResourceId • Content • Embedding • ChunkSize • Overlap • ChunkPosition Media The description of an media url found on the resource content • Id • ResourceId • Url • MimeType (PNG, JPG) • Description
21. 21 PROMPT ENGINEERING + TOOLS
22. 22 HOW IT WORKS Agent 🤖 Decide which tools to call to get the content to answer the question via RAG User 󰞦 asks a question Tools ⚙ Agent receives markdown content from the RAG tools Agent 🤖 Check for any urls on the markdown content and call the image description tool passing the urls Tools ⚙ Agent receives the description of each images found on the markdown content Agent 🤖 Compose an answer containing the relevant images User 󰞦 receives a complete markdown formatted answer containing relevant UI images
23. 23 DEPLOYING
24. 24 NEON DB
25. 25 TIME TO SHARE! 🚀
26. 26 TIME TO SHARE! 🚀
27. 27 ALMOST 2K VISITORS ON THE FIRST WEEK OF LAUNCH 🚀
28. 28 ALMOST 2.5K QUESTIONS ANSWERED IN MORE THAN 20 LANGUAGES 🚀
29. 29 MY FIRST DONATION FROM AN USER 🚀
30. 30 WRAPPING UP • Build something that solves a problem of you • See if there are people out there who have the same problem and talk to them • Share with people who really care about this problem on the right place • Use tools you are comfortable with • Hater gonna hate
31. THANK YOU 31 RAFAEL SARACENI 21.10.25 SERVERLESSDAYS MILANO Let’s connect! Professional site https://saraceni.me Rafael Saraceni www.linkedin.com/in/rafaelsaraceni saraceni-br bsky.app/profile/saraceni-br.bsky.social