AI isn’t a Python-only diet:​ Agentic AI with Spring joins the menu

Large Language Models (LLMs) are transforming how we build intelligent applications, but bringing them into enterprise systems, and evolving them into Agentic AI systems, can be challenging. In this talk, we’ll explore how Spring AI simplifies LLM integration, making it easy to connect to different providers with minimal effort and swap models without rewriting business logic. You’ll learn how to build intelligent Agents that can reason, act, and interact using tools and workflows. We’ll also cover Retrieval-Augmented Generation (RAG) to enrich model responses and reduce hallucinations by grounding outputs in your own data, and how the Model Context Protocol (MCP) enables seamless interoperability between AI services, and existing systems. Through practical examples, you’ll see how to design and implement production-ready, context-aware, agent-driven applications using Spring AI.

Transcript

1. Raffaele Auriemma AI isn’t a Python-only diet: Agentic AI with Spring joins the menu
2. Better ask ERNI AI means coding in Python… or does it? 2
3. Better ask ERNI 3 What’s the first step?
4. Better ask ERNI 4 Need to swap the model...
5. Better ask ERNI 5 Why not use Spring? Dependency injection Declarative configuration Composable components Observability built in
6. Better ask ERNI Software Architect with a soft spot for Java. 6 Hi, I'm Raffaele! Principal Software Architect @ https://www.linkedin.com/in/raffaeleauriemma/
7. Better ask ERNI Built for 7 What is Spring AI? Declarative API for: • Chat & Embeddings • Prompt Templates • Vector Search • Function Calling (Tools, MCP)
8. Better ask ERNI Chat 8
9. Better ask ERNI 9 Configuration spring: ai: openai: api-key: ${OPENAI_API_KEY} <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-openai</artifactId> </dependency> pom.xml: application.yml:
10. Better ask ERNI 10 Chat Client API @RestController public class ChatAiController { private final ChatClient chatClient; public ChatAiController(ChatClient.Builder chatClientBuilder) { this.chatClient = chatClientBuilder.build(); } @GetMapping("/chat") public String chat(String userMessage) { return chatClient.prompt() .user(userMessage) .call() .content(); } }
11. Better ask ERNI 11 Change model spring: ai: vertex: ai: gemini: project-id: ${GEMINI_PROJECT_ID} location: europe-west1 <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-vertex-ai-gemini</artifactId> </dependency> pom.xml: application.yml:
12. Better ask ERNI Memory 12
13. Better ask ERNI 13 Memory public MemoryChatService(ChatClient.Builder chatClientBuilder, JdbcChatMemoryRepository jdbcChatMemoryRepository) { ChatMemory chatMemory = MessageWindowChatMemory.builder() .chatMemoryRepository(jdbcChatMemoryRepository) .maxMessages(15) .build(); this.chatClient = chatClientBuilder .defaultAdvisors( MessageChatMemoryAdvisor.builder(chatMemory).build() ) .build(); }
14. Better ask ERNI 14 Chat Id public String chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, chatId)) .call() .content(); }
15. Better ask ERNI Streaming Response 15
16. Better ask ERNI 16
17. Better ask ERNI 17 Configuration <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> </dependency> pom.xml: <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> or
18. Better ask ERNI 18 Chat Client public String chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, chatId)) .call() .content(); } public Flux<String> chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, chatId)) .stream() .content(); }
19. Better ask ERNI 19 Chat API @GetMapping public String chat(String userMessage) { return chatService.chat(chatId, userMessage) } @GetMapping(produces = MediaType.TEXT_EVENT_STREAM_VALUE) public Flux<String> chat(String userMessage) { return chatService.chat(chatId, userMessage) }
20. Better ask ERNI AI and Cybersecurity 20
21. Better ask ERNI 21
22. Better ask ERNI 22 Personas public class ChatService { private static final String SYSTEM = """ You are a customer support agent for SpringAI Hotel. ... """; this.chatClient = chatClientBuilder .defaultSystem(SYSTEM) .build(); public String chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .call() .content(); }
23. Better ask ERNI 23 https://genai.owasp.org/llm-top-10/
24. Better ask ERNI 24 AI Guardrails Guardrails LLM Checked Input Question Checked Output Answer User
25. Better ask ERNI 25 Advisors You are an EU safety guard, responsible for GDPR compliance. Your task it to ensure the user question doesn't include any sensitive data, such as credit card numbers, home addresses, social security numbers, or any other information that qualifies as personally identifiable information (PII). If you detect any of such sensitive data in the user question, you must disallow it. If not, you can allow it. https://github.com/spring-projects/spring-ai/issues/1081
26. Better ask ERNI 26 Advisors public class SensitiveDataInputAdvisor implements CallAdvisor { public ChatClientResponse advisorCall(ChatClientRequest chatClientRequest, CallAdvisorChain advisorChain) { InputAdvisorResult inputAdvisorResult = chatClient.prompt().system(SYSTEM) .user(chatClientRequest.prompt().getUserMessage().getText()).call() .entity(InputAdvisorResult.class); if (VALIDATION_RESULT) { return advisorChain.nextCall(chatClientRequest); } else { return ChatClientResponse.builder() .chatResponse(ChatResponse.builder() .generations(List.of(new Generation( new AssistantMessage("Invalid input detected!")))) .build()) .context(chatClientRequest.context()).build(); } } }
27. Better ask ERNI 27 Advisors this.chatClient = chatClientBuilder.clone() .defaultAdvisors(SensitiveDataInputGuardrailAdvisor.builder() .chatClientBuilder(chatClientBuilder.clone()) .build()) .build();
28. Better ask ERNI RAG Retrieval-Augmented Generation 28
29. Better ask ERNI 29 RAG Embedding Search LLM Request Question Answer User Vector Store
30. Better ask ERNI 30 RAG public VectorStore ragVectorStore(VectorStore vectorStore, JdbcTemplate jdbcTemplate) { List<Document> documents = getDocsFromPdf(); TextSplitter textSplitter = new TokenTextSplitter(); List<Document> splitDocuments = textSplitter.apply(documents); vectorStore.add(splitDocuments); return vectorStore; }
31. Better ask ERNI 31 Built in ETL private List<Document> getDocsFromPdf() { PagePdfDocumentReader pdfReader = new PagePdfDocumentReader(pdfResource, PdfDocumentReaderConfig.builder() .withPageTopMargin(0) .withPageExtractedTextFormatter( ExtractedTextFormatter.builder() .withNumberOfTopTextLinesToDelete(0) .build()) .withPagesPerDocument(1) .build()); return pdfReader.read(); }
32. Better ask ERNI 32 RAG Advisor public ChatService(ChatClient.Builder chatClientBuilder, ChatMemory chatMemory, VectorStore vectorStore) { this.chatClient = chatClientBuilder .defaultSystem(SYSTEM) .defaultAdvisors( MessageChatMemoryAdvisor.builder(chatMemory).build(), QuestionAnswerAdvisor.builder(vectorStore).build() ) .build(); }
33. Better ask ERNI AI Agents 33
34. Better ask ERNI 34 AI Agents AI workflows are structured, rule-based automation processes for repetitive tasks, offering predictability but lacking flexibility. AI agents are autonomous systems that make independent decisions, adapt to new information, and dynamically choose tools and strategies to achieve a goal, offering greater autonomy but potentially less control.
35. Better ask ERNI 35 MCP (Model Context Protocols) MCPs are like a universal API gateway for AI models. MCPs act as the “REST standard” for LLMs, letting the model: • Discover what tools exist (like service discovery) • Understand how to call them (like OpenAPI specs) • Execute the right operations (like API requests) • And use their responses to enrich its reasoning (like microservice orchestration)
36. Better ask ERNI 36 AI Agents LLM Question Answer User API
37. Better ask ERNI 37 https://github.com/modelcontextprotocol/servers
38. Better ask ERNI 38 Tools @Tool(description = "Get booking details") public BookingDetails getBookingDetails(String bookingNumber, String firstName, String lastName) { try { return bookingService.getBookingDetails(bookingNumber, firstName, lastName); } catch (Exception _) { return new BookingDetails(bookingNumber, firstName, lastName); } } @Tool(description = "Cancel booking") public void cancelBooking(String bookingNumber, String firstName, String lastName) { bookingService.cancelBooking(bookingNumber, firstName, lastName); }
39. Better ask ERNI 39 Tools public ChatService(ChatClient.Builder chatClientBuilder, ChatMemory chatMemory, VectorStore vectorStore, BookingTools bookingTools) { this.chatClient = chatClientBuilder .defaultSystem(SYSTEM) .defaultAdvisors( MessageChatMemoryAdvisor.builder(chatMemory).build(), QuestionAnswerAdvisor.builder(vectorStore).build() ) .defaultTools(bookingTools) .build(); }
40. Better ask ERNI Spring AI Hotel Booking DEMO 40 https://github.com/ERNI-Academy/spring-ai-hotel-booking
41. Better ask ERNI Observability 41
42. Better ask ERNI 42
43. Better ask ERNI Real Use Case 43
44. Better ask ERNI 44
45. Better ask ERNI 45 Embedding Search AI Agent Request Question Answer User Vector Store API RAG
46. Better ask ERNI AI isn’t magic, but it’s an incredible tool. Explore it, build with it, and enjoy the journey. 46
47. Better ask ERNI Q & A 47 https://www.linkedin.com/in/raffaeleauriemma/ https://github.com/ERNI-Academy/spring-ai-hotel-booking
48. Swiss Software Engineering Better ask ERNI
49. Better ask ERNI Backup 49
50. Better ask ERNI 50
51. Better ask ERNI 51 https://docs.spring.io/spring-ai/reference/api/tools.html
52. Better ask ERNI 52 https://docs.spring.io/spring-ai/reference/api/tools.html class DateTimeTools { @Tool(description = "Set a user alarm for the given time") void setAlarm(@ToolParam(description = "Time in ISO-8601 format") String time) { LocalDateTime alarmTime = LocalDateTime.parse(time, DateTimeFormatter.ISO_DATE_TIME); System.out.println("Alarm set for " + alarmTime); } }
53. Better ask ERNI 53 https://docs.spring.io/spring-ai/reference/api/tools.html class CustomerTools { @Tool(description = "Update customer information") void updateCustomerInfo(Long id, String name, @ToolParam(required = false) String email) { System.out.println("Updated info for customer with id: " + id); } }