AI isn’t a Python-only diet: Spring AI joins the menu ​

Large Language Models (LLMs) are transforming how we build intelligent applications, but bringing them into enterprise systems can be challenging. In this talk, we’ll explore how Spring AI simplifies LLM integration, making it easy to connect to different providers with minimal effort and swap models without rewriting business logic. You’ll also learn how to use Retrieval-Augmented Generation (RAG) to enrich model responses and reduce hallucinations by grounding model outputs in your own data, and how the Model Context Protocol (MCP) enables seamless interoperability between AI services and existing systems. Through practical examples, you’ll see how to build production-ready, context-aware applications with Spring AI.

Transcript

1. Raffaele Auriemma AI isn’t a Python-only diet: Spring AI joins the menu
2. Better ask ERNI AI means coding in Python… or does it? 2
3. Better ask ERNI 3 What’s the first step?
4. Better ask ERNI 4 Need to swap the model...
5. Better ask ERNI 5 Why not use Spring? Dependency injection Declarative configuration Composable components Observability built in
6. Better ask ERNI Software Architect with a soft spot for Java. 6 Hi, I'm Raffaele! Principal Software Architect @ https://www.linkedin.com/in/raffaeleauriemma/
7. Better ask ERNI Built for 7 What is Spring AI? Declarative API for: • Chat & Embeddings • Prompt Templates • Vector Search • Function Calling (Tools, MCP)
8. Better ask ERNI Chat 8
9. Better ask ERNI 9 Configuration spring: ai: openai: api-key: ${OPENAI_API_KEY} <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-openai</artifactId> </dependency> pom.xml: application.yml:
10. Better ask ERNI 10 Chat Client API @RestController public class ChatAiController { private final ChatClient chatClient; public ChatAiController(ChatClient.Builder chatClientBuilder) { this.chatClient = chatClientBuilder.build(); } @GetMapping("/chat") public String chat(String userMessage) { return chatClient.prompt() .user(userMessage) .call() .content(); } }
11. Better ask ERNI 11 Change model spring: ai: vertex: ai: gemini: project-id: ${GEMINI_PROJECT_ID} location: europe-west1 <dependency> <groupId>org.springframework.ai</groupId> <artifactId>spring-ai-starter-model-vertex-ai-gemini</artifactId> </dependency> pom.xml: application.yml:
12. Better ask ERNI Memory 12
13. Better ask ERNI 13 Memory public MemoryChatService(ChatClient.Builder chatClientBuilder, JdbcChatMemoryRepository jdbcChatMemoryRepository) { ChatMemory chatMemory = MessageWindowChatMemory.builder() .chatMemoryRepository(jdbcChatMemoryRepository) .maxMessages(15) .build(); this.chatClient = chatClientBuilder .defaultAdvisors( MessageChatMemoryAdvisor.builder(chatMemory).build() ) .build(); }
14. Better ask ERNI 14 Chat Id public String chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, chatId)) .call() .content(); }
15. Better ask ERNI Streaming Response 15
16. Better ask ERNI 16 Configuration <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> </dependency> pom.xml: <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> or
17. Better ask ERNI 17 Chat Client public String chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, chatId)) .call() .content(); } public Flux<String> chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, chatId)) .stream() .content(); }
18. Better ask ERNI 18 Chat API @GetMapping public String chat(String userMessage) { return chatService.chat(chatId, userMessage) } @GetMapping(produces = MediaType.TEXT_EVENT_STREAM_VALUE) public Flux<String> chat(String userMessage) { return chatService.chat(chatId, userMessage) }
19. Better ask ERNI AI and Cybersecurity 19
20. Better ask ERNI 20
21. Better ask ERNI 21 Personas public class ChatService { private static final String SYSTEM = """ You are a customer support agent for SpringAI Hotel. ... """; this.chatClient = chatClientBuilder .defaultSystem(SYSTEM) .build(); public String chat(String chatId, String userMessage) { return chatClient.prompt() .user(userMessage) .call() .content(); }
22. Better ask ERNI 22 https://genai.owasp.org/llm-top-10/
23. Better ask ERNI 23 AI Guardrails Guardrails LLM Checked Input Question Checked Output Answer User
24. Better ask ERNI 24 Advisors You are an EU safety guard, responsible for GDPR compliance. Your task it to ensure the user question doesn't include any sensitive data, such as credit card numbers, home addresses, social security numbers, or any other information that qualifies as personally identifiable information (PII). If you detect any of such sensitive data in the user question, you must disallow it. If not, you can allow it. https://github.com/spring-projects/spring-ai/issues/1081
25. Better ask ERNI 25 Advisors public class SensitiveDataInputAdvisor implements CallAdvisor { public ChatClientResponse advisorCall(ChatClientRequest chatClientRequest, CallAdvisorChain advisorChain) { InputAdvisorResult inputAdvisorResult = chatClient.prompt().system(SYSTEM) .user(chatClientRequest.prompt().getUserMessage().getText()).call() .entity(InputAdvisorResult.class); if (VALIDATION_RESULT) { return advisorChain.nextCall(chatClientRequest); } else { return ChatClientResponse.builder() .chatResponse(ChatResponse.builder() .generations(List.of(new Generation( new AssistantMessage("Invalid input detected!")))) .build()) .context(chatClientRequest.context()).build(); } } }
26. Better ask ERNI 26 Advisors this.chatClient = chatClientBuilder.clone() .defaultAdvisors(SensitiveDataInputGuardrailAdvisor.builder() .chatClientBuilder(chatClientBuilder.clone()) .build()) .build();
27. Better ask ERNI RAG Retrieval-Augmented Generation 27
28. Better ask ERNI 28 RAG Embedding Search LLM Request Question Answer User Vector Store
29. Better ask ERNI 29 RAG public VectorStore ragVectorStore(VectorStore vectorStore, JdbcTemplate jdbcTemplate) { List<Document> documents = getDocsFromPdf(); TextSplitter textSplitter = new TokenTextSplitter(); List<Document> splitDocuments = textSplitter.apply(documents); vectorStore.add(splitDocuments); return vectorStore; }
30. Better ask ERNI 30 Built in ETL private List<Document> getDocsFromPdf() { PagePdfDocumentReader pdfReader = new PagePdfDocumentReader(pdfResource, PdfDocumentReaderConfig.builder() .withPageTopMargin(0) .withPageExtractedTextFormatter( ExtractedTextFormatter.builder() .withNumberOfTopTextLinesToDelete(0) .build()) .withPagesPerDocument(1) .build()); return pdfReader.read(); }
31. Better ask ERNI 31 RAG Advisor public ChatService(ChatClient.Builder chatClientBuilder, ChatMemory chatMemory, VectorStore vectorStore) { this.chatClient = chatClientBuilder .defaultSystem(SYSTEM) .defaultAdvisors( MessageChatMemoryAdvisor.builder(chatMemory).build(), QuestionAnswerAdvisor.builder(vectorStore).build() ) .build(); }
32. Better ask ERNI AI Agents 32
33. Better ask ERNI 33 AI Agents AI workflows are structured, rule-based automation processes for repetitive tasks, offering predictability but lacking flexibility. AI agents are autonomous systems that make independent decisions, adapt to new information, and dynamically choose tools and strategies to achieve a goal, offering greater autonomy but potentially less control.
34. Better ask ERNI 34 MCP (Model Context Protocols) MCPs are like a universal API gateway for AI models. MCPs act as the “REST standard” for LLMs, letting the model: • Discover what tools exist (like service discovery) • Understand how to call them (like OpenAPI specs) • Execute the right operations (like API requests) • And use their responses to enrich its reasoning (like microservice orchestration)
35. Better ask ERNI 35 AI Agents LLM Question Answer User API
36. Better ask ERNI 36 https://github.com/modelcontextprotocol/servers
37. Better ask ERNI 37 Tools @Tool(description = "Get booking details") public BookingDetails getBookingDetails(String bookingNumber, String firstName, String lastName) { try { return bookingService.getBookingDetails(bookingNumber, firstName, lastName); } catch (Exception _) { return new BookingDetails(bookingNumber, firstName, lastName); } } @Tool(description = "Cancel booking") public void cancelBooking(String bookingNumber, String firstName, String lastName) { bookingService.cancelBooking(bookingNumber, firstName, lastName); }
38. Better ask ERNI 38 Tools public ChatService(ChatClient.Builder chatClientBuilder, ChatMemory chatMemory, VectorStore vectorStore, BookingTools bookingTools) { this.chatClient = chatClientBuilder .defaultSystem(SYSTEM) .defaultAdvisors( MessageChatMemoryAdvisor.builder(chatMemory).build(), QuestionAnswerAdvisor.builder(vectorStore).build() ) .defaultTools(bookingTools) .build(); }
39. Better ask ERNI Spring AI Hotel Booking DEMO 39 https://github.com/ERNI-Academy/spring-ai-hotel-booking
40. Better ask ERNI Observability 40
41. Better ask ERNI 41
42. Better ask ERNI AI isn’t magic, but it’s an incredible tool. Explore it, build with it, and enjoy the journey. 42
43. Better ask ERNI Q & A 43
44. Swiss Software Engineering Better ask ERNI
45. Better ask ERNI Backup 45
46. Better ask ERNI 46
47. Better ask ERNI 47 https://docs.spring.io/spring-ai/reference/api/tools.html
48. Better ask ERNI 48 https://docs.spring.io/spring-ai/reference/api/tools.html class DateTimeTools { @Tool(description = "Set a user alarm for the given time") void setAlarm(@ToolParam(description = "Time in ISO-8601 format") String time) { LocalDateTime alarmTime = LocalDateTime.parse(time, DateTimeFormatter.ISO_DATE_TIME); System.out.println("Alarm set for " + alarmTime); } }
49. Better ask ERNI 49 https://docs.spring.io/spring-ai/reference/api/tools.html class CustomerTools { @Tool(description = "Update customer information") void updateCustomerInfo(Long id, String name, @ToolParam(required = false) String email) { System.out.println("Updated info for customer with id: " + id); } }