Pimp Your LLM! – Model Context Protocol with Spring AI

This is the slide deck for my session "Pimp Your LLM! – Model Context Protocol with Spring AI"

Transcript

1. Pimp Your LLM! Model Context Protocol with Spring AI Kai Tödter
2. Who am I? ▪ Distinguished Key Expert at Siemens Smart Infrastructure ▪ Open-Source Lover ▪ VR Enthusiast and Gamer ▪ E-mail: kai@toedter.com ▪ Mastodon: https://mastodon.social/@kaitoedter ▪ GitHub: github.com/toedter 11/3/2025 2 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License.
3. Show Hands! 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 3
4. What is Model Context Protocol (MCP)? ▪ Definition: An open standard for how AI models connect with tools, data, and environments ▪ Purpose: Lets models take action, access info, and go beyond just generating text ▪ Universality: Compatible across different AI systems and platforms ▪ Analogy: Like USB-C for AI ▪ one universal plug for many intelligent agents 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 4
5. Benefits of MCP ▪ Interoperability: Works across different tools, languages, and providers ▪ Modularity: Swap or upgrade tools without breaking things ▪ Scalability: Supports complex workflows across many systems ▪ Open Standard: No vendor lock-in ▪ encourages collaboration and innovation 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 5 •Open Standard: No vendor lock-in—encourages collaboration and innovation Let me know if you'd like this styled for a pitch deck, technical audience, or something else
6. MCP Flows 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 6 MCP Protocol (stdio) MCP Protocol (stdio) MCP Protocol (Streamable HTTP, SSE) Local Data Source Your Computer Internet Remote Service A Remote Service B Web APIs Web APIs MCP Server MCP Server MCP Server Host with MCP Client (IDE, Claude Desktop, CLI, Java Application)
7. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 7 Claude Desktop Demo
8. Movie Quote Demo 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 8
9. Movie Rank Demo 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 9
10. Weather Forecast Demo 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 10
11. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 11 MCP Server with Spring AI
12. Requirements ▪ Java 17 ▪ Spring AI 1.1.0-SNAPSHOT ▪ Spring Boot 3.5.x ▪ GitHub: https://github.com/toedter/spring-ai-mcp-demos 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 12
13. Providing the Tools 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 13 @Bean public ToolCallbackProvider tools( QuoteService quoteService, MovieService movieService, WeatherService weatherService) { return MethodToolCallbackProvider .builder() .toolObjects(quoteService, movieService, weatherService) .build(); }
14. Quote Service 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 14 @Service public class QuoteService { @Tool(description = "Get Kai's favorite movie quote") public String getKaisFavoriteMovieQuote() { return "With great power comes great responsibility."; } }
15. Movie Service 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 15 @Tool(description = """ Get the top-ranked movies from IMDb. The API supports pagination using the pageNumber and pageSize parameters. The pageNumber starts at 0. The default pageSize is 10, with a maximum of 250.""") public String getTopRankedMovies(int pageNumber, int pageSize) { return restClient.get() .uri("/movies?page[number]={pageNumber}&page[size]={pageSize}", pageNumber, pageSize) .retrieve() .body(String.class); }
16. Weather Service 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 16 @Tool(description = """ Get the current weather forecast for a specific latitude and longitude. Units can be 'metric' or 'imperial’; the default is 'metric'. """) public String getWeatherForecastByLocation( double latitude, double longitude, String units) { return restClient.get() .uri("/current?latitude={latitude}" + "&longitude={longitude}&units={units}", latitude, longitude, units) .retrieve() .body(String.class); }
17. Config for STDIO
18. application.yml for STDIO 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 18 spring: ai: mcp: server: name: demo-mcp-server version: 0.0.1 enabled: true stdio: true main: banner-mode: off web-application-type: none logging.pattern.console:
19. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 19 Security
20. OAuth2 Authorization Server ▪ We need an authorization server ▪ To enable OAuth2 based authorization ▪ To validate the access tokens (JWT) ▪ We use Spring Authorization Server for the demo ▪ See https://spring.io/projects/spring-authorization-server 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 20
21. Authorization Server Config spring security: oauth2: authorizationserver: client: mcp-client: registration: client-id: "mcp-auth-client" client-secret: "{noop}secret" client-authentication-methods: - "client_secret_post" authorization-grant-types: - "client_credentials" scopes: - "openid" - "profile" - "mcp.tools" require-authorization-consent: true 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 21
22. CURL call to get Access Token curl -X POST http://localhost:9000/oauth2/token \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "client_id=mcp-auth-client" \ -d "client_secret=secret" \ -d "grant_type=client_credentials" \ -d "scope=mcp.tools" 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 22
23. The Decoded Payload of the JWT { "sub": "mcp-auth-client", "aud": "mcp-audience", "nbf": 1757256088, "scope": [ "mcp.tools" ], "iss": "http://localhost:9000", "exp": 1757256388, "iat": 1757256088, "jti": "61f3b3d4-b6a1-4b35-8c48-f81f4d3b6622" } 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 23
24. Application yaml for Security server.port: 8082 spring: security: oauth2: resourceserver: jwt: jwk-set-uri: http://localhost:9000/oauth2/jwks issuer-uri: http://localhost:9000 audiences: mcp-audience 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 24
25. Application yaml for Streamable HTTP spring: ai: mcp: server: name: demo-mcp-server version: 0.0.1 protocol: STREAMABLE enabled: true streamable-http: mcp-endpoint: /api/mcp keep-alive-interval: 30s 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 25
26. Run the Streamable HTTP Demo ▪ Start Authorization Server ▪ Get Access Token ▪ Start secure MCP Server ▪ Call: npx @modelcontextprotocol/inspector ▪ URL: http://localhost:8082/api/mcp ▪ Configure Authorization with Bearer Token ▪ Set Client ID to mcp-auth-client and Scope to mcp.tools ▪ Connect 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 26
27. @modelcontextprotocol/inspector 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 27
28. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 28 MCP Streamable HTTP Demo
29. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 29 MCP Client with Spring AI
30. Chat Controller @RestController public class ChatController { private final ChatClient chatClient; public ChatController(ChatClient.Builder chatClientBuilder, ToolCallbackProvider tools) { this.chatClient = chatClientBuilder .defaultSystem("You are a friendly assistant.") .defaultToolCallbacks(tools) .build(); } 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 30
31. Chat Controller @GetMapping("/chat") public String chat(@RequestParam String message) { return this.chatClient .prompt() .user(message) .call() .content(); } 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 31
32. MCP Client Config spring: ai: mcp: client: name: demo-mcp-client version: 0.0.1 type: SYNC stdio: servers-configuration: classpath:mcp_servers_config.json 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 32
33. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 33 MCP Client Demo
34. MCP Client in Web Browser 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 34
35. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 35 MCP FAQ
36. MCP FAQ ▪ Does it scale? ▪ How many MCP servers can be connected to one MCP client? ▪ How to deal with many federated MCP servers? ▪ Is the Authorization state of the art? ▪ Is SSE a good choice? ▪ According to recent updates in the MCP spec (as of March 2025), SSE is being deprecated in favor of Streamable HTTP 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 36
37. 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 37 Discussion
38. Links ▪ Spring AI MCP Demos: https://github.com/toedter/spring-ai-mcp-demos ▪ Spring AI: https://spring.io/projects/spring-ai ▪ MCP: https://modelcontextprotocol.io/ 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 38
39. License ▪ This work is licensed under a Creative Commons Attribution 4.0 International License. ▪ See http://creativecommons.org/licenses/by/4.0/ 11/3/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 39