Building AI Applications with Java, LLMs, and Spring AI

This presentation will guide you through building Java applications with AI capabilities, leveraging Generative AI and Large Language Models (LLMs) using Spring AI. You will learn how to:

- Integrate chat models into your applications, including prompt design and managing structured outputs.
- Implement Retrieval-Augmented Generation (RAG) workflows to combine LLMs with your custom data, covering both basic and advanced use cases.
- Enable agentic behavior by integrating your APIs with LLMs through tool calling.
- Explore approaches for evaluating and monitoring LLM-based applications.
- Enhance the developer experience with tools such as Testcontainers and Ollama.

Transcript

1. Thomas Vitale JUG November 2025 Building AI Applications With Java, LLMs, and @thomasvitale.com
2. Software Engineer Thomas Vitale @thomasvitale.com
3. Machine Learning Subset of Arti fi cial Intelligence Platform/Infrastructure Platform Engineers HTTP API Application Developer Model Training Model Inference ML Engineers Data Preparation Data Scientists @thomasvitale.com
4. Java for AI-Infused Applications Integrations with Model Inference Platform/Infrastructure Platform Engineers Model Training Model Inference ML Engineers Data Preparation Data Scientists Application Developers Application @thomasvitale.com
5. spring.io/projects/spring-ai @thomasvitale.com
6. spring.io/projects/spring-ai @thomasvitale.com
7. ollama.com @thomasvitale.com
8. ollama.com @thomasvitale.com
9. Application Inference Service Consum es LLM s Architecture Database Reads/writes data Observability Platform Exports telem etry Spring Boot Application Vaadin Spring AI Arconia @thomasvitale.com
10. Arconia Dev Services and OpenTelemetry arconia dev gradle bootRun mvn spring-boot:run @thomasvitale.com arconia.io
11. Chat Inference Service Request Response Single Interactions Answer Application @thomasvitale.com Question
12. Multimodality Inference Service Request Response Modalities and Structured Output Question Answer Application Format Instructions Output Converter @thomasvitale.com
13. Chat Memory Inference Service Request Response Question Answer Application Multiple Interactions @thomasvitale.com Augment with Memory Memory Read Update Memory Write
14. Prompt Stuffing Inference Service Request Response Augmenting Prompts with Context Answer Application @thomasvitale.com Question Context
15. Retrieval Augmented Generation Inference Service Request Response Question Answer Application Augment with Context Web Search Engines Search Engine HTTP API @thomasvitale.com Context
16. Retrieval Augmented Generation Inference Service Request Response Question Answer Application Augment with Context Vector Stores Vector Store Semantic Search @thomasvitale.com
17. Ingestion Pipeline @thomasvitale.com Data Preparation Document Reader Reads from Writes To Document Transformer Document Writer
18. Retrieval Augmented Generation Inference Service Request Response Question Answer Application Augment with Context Prompt Augmentation with Retrieved Context Source Query @thomasvitale.com
19. Tools Inference Service Request Tool Calling Question Response Answer Application API Tool Call Tool Execution Tool Call Request Tool Call Response @thomasvitale.com
20. Agents @thomasvitale.com Orchestration Query Response LLM uses uses Tool 1 uses Tool2 Agent
21. API MCP Server MCP Inference Service Request Tools Question Response Answer Application Tool Call MCP Client Tool Call Request Tool Call Response @thomasvitale.com
22. Building AI Applications https://github.com/ThomasVitale/modular-rag https://github.com/ThomasVitale/llm-apps-java-spring-ai Thomas Vitale @thomasvitale.com thomasvitale.com With Java, LLMs, and