Pimp Your LLM! – Model Context Protocol with Spring AI

This is the slide deck for my session "Pimp Your LLM! – Model Context Protocol with Spring AI"

Transcript

1. Pimp Your LLM! Model Context Protocol with Spring AI Kai Tödter
2. Who am I? ▪ Distinguished Key Expert at Siemens Smart Infrastructure ▪ Open-Source Lover ▪ VR Enthusiast and Gamer ▪ E-mail: kai@toedter.com ▪ Mastodon: https://mastodon.social/@kaitoedter ▪ GitHub: github.com/toedter 10/1/2025 2 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License.
3. Show Hands! 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 3
4. What is Model Context Protocol (MCP)? ▪ Definition: MCP is an open protocol that defines how AI models interact with tools, data, and environments ▪ Purpose: Enables models to perform actions, retrieve information, or manipulate content beyond simple text generation ▪ Universality: Works across different AI systems, models, and platforms ▪ Analogy: Think of MCP like the “USB-C” of AI, like one universal plug that fits many intelligent agents. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 4
5. Benefits of MCP ▪ Interoperability: Connects models and tools regardless of provider or language ▪ Modularity: Add, remove, or upgrade tools easily without breaking systems ▪ Scalability: Facilitates complex workflows across many tools and services ▪ Open Standard: No lock-in to proprietary formats—fosters collaboration and innovation 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 5
6. MCP Flows 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 6 MCP Protocol (stdio) MCP Protocol (stdio) MCP Protocol (Streamable HTTP, SSE) Local Data Source Your Computer Internet Remote Service A Remote Service B Web APIs Web APIs MCP Server MCP Server MCP Server Host with MCP Client (IDE, Claude Desktop, CLI, Java Application)
7. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 7 Claude Desktop Demo
8. Movie Quote Demo 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 8
9. Movie Rank Demo 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 9
10. Weather Forecast Demo 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 10
11. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 11 MCP Server with Spring AI
12. Requirements ▪ Java 17 ▪ Spring AI 1.1.0-SNAPSHOT ▪ Spring Boot 3.5.x ▪ GitHub: https://github.com/toedter/spring-ai-mcp-demos 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 12
13. Providing the Tools 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 13 @Bean public ToolCallbackProvider tools( QuoteService quoteService, MovieService movieService, WeatherService weatherService) { return MethodToolCallbackProvider .builder() .toolObjects(quoteService, movieService, weatherService) .build(); }
14. Quote Service 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 14 @Service public class QuoteService { @Tool(description = "Get Kai's favorite movie quote") public String getKaisFavoriteMovieQuote() { return "With great power comes great responsibility."; } }
15. Movie Service 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 15 @Tool(description = """ Get the top-ranked movies from IMDb. The API supports pagination using the pageNumber and pageSize parameters. The pageNumber starts at 0. The default pageSize is 10, with a maximum of 250.""") public String getTopRankedMovies(int pageNumber, int pageSize) { return restClient.get() .uri("/movies?page[number]={pageNumber}&page[size]={pageSize}", pageNumber, pageSize) .retrieve() .body(String.class); }
16. Weather Service 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 16 @Tool(description = """ Get the current weather forecast for a specific latitude and longitude. Units can be 'metric' or 'imperial’; the default is 'metric'. """) public String getWeatherForecastByLocation( double latitude, double longitude, String units) { return restClient.get() .uri("/current?latitude={latitude}" + "&longitude={longitude}&units={units}", latitude, longitude, units) .retrieve() .body(String.class); }
17. Config for STDIO
18. application.yml for STDIO 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 18 spring: ai: mcp: server: name: demo-mcp-server version: 0.0.1 enabled: true stdio: true main: banner-mode: off web-application-type: none logging.pattern.console:
19. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 19 Security
20. OAuth2 Authorization Server ▪ We need an authorization server ▪ To enable OAuth2 based authorization ▪ To validate the access tokens (JWT) ▪ We use Spring Authorization Server for the demo ▪ See https://spring.io/projects/spring-authorization-server 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 20
21. Authorization Server Config spring security: oauth2: authorizationserver: client: mcp-client: registration: client-id: "mcp-auth-client" client-secret: "{noop}secret" client-authentication-methods: - "client_secret_post" authorization-grant-types: - "client_credentials" scopes: - "openid" - "profile" - "mcp.tools" require-authorization-consent: true 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 21
22. CURL call to get Access Token curl -X POST http://localhost:9000/oauth2/token \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "client_id=mcp-auth-client" \ -d "client_secret=secret" \ -d "grant_type=client_credentials" \ -d "scope=mcp.tools" 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 22
23. The Decoded Payload of the JWT { "sub": "mcp-auth-client", "aud": "mcp-audience", "nbf": 1757256088, "scope": [ "mcp.tools" ], "iss": "http://localhost:9000", "exp": 1757256388, "iat": 1757256088, "jti": "61f3b3d4-b6a1-4b35-8c48-f81f4d3b6622" } 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 23
24. Application yaml for Security server.port: 8082 spring: security: oauth2: resourceserver: jwt: jwk-set-uri: http://localhost:9000/oauth2/jwks issuer-uri: http://localhost:9000 audiences: mcp-audience 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 24
25. Application yaml for Streamable HTTP spring: ai: mcp: server: name: demo-mcp-server version: 0.0.1 protocol: STREAMABLE enabled: true streamable-http: mcp-endpoint: /api/mcp keep-alive-interval: 30s 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 25
26. Run the Streamable HTTP Demo ▪ Start Authorization Server ▪ Get Access Token ▪ Start secure MCP Server ▪ Call: npx @modelcontextprotocol/inspector ▪ URL: http://localhost:8082/api/mcp ▪ Configure Authorization with Bearer Token ▪ Connect 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 26
27. @modelcontextprotocol/inspector 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 27
28. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 28 MCP Streamable HTTP Demo
29. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 29 MCP Client with Spring AI
30. Chat Controller @RestController public class ChatController { private final ChatClient chatClient; public ChatController(ChatClient.Builder chatClientBuilder, ToolCallbackProvider tools) { this.chatClient = chatClientBuilder .defaultSystem("You are a friendly assistant.") .defaultToolCallbacks(tools) .build(); } 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 30
31. Chat Controller @GetMapping("/chat") public String chat(@RequestParam String prompt) { return this.chatClient .prompt() .user(prompt) .call() .content(); } 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 31
32. MCP Client Config spring.application.name: spring-ai-mcp-client-stdio server.port: 8083 logging.level.org.springframework.ai: DEBUG spring.ai.mcp.client: transport: stdio model: ollama/qwen3:8b type: SYNC toolcallback: enabled: true stdio.servers-configuration: classpath:/mcp_servers_config.json 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 32
33. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 33 MCP Client Demo
34. MCP Client in Web Browser 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 34
35. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 35 MCP FAQ
36. MCP FAQ ▪ Does it scale? ▪ How many MCP servers can be connected to one MCP client? ▪ How to deal with many federated MCP servers? ▪ Is the Authorization state of the art? ▪ Is SSE a good choice? ▪ According to recent updates in the MCP spec (as of March 2025), SSE is being deprecated in favor of Streamable HTTP 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 36
37. 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 37 Discussion
38. Links ▪ Spring AI MCP Demos: https://github.com/toedter/spring-ai-mcp-demos ▪ Spring AI: https://spring.io/projects/spring-ai ▪ MCP: https://modelcontextprotocol.io/ 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 38
39. License ▪ This work is licensed under a Creative Commons Attribution 4.0 International License. ▪ See http://creativecommons.org/licenses/by/4.0/ 10/1/2025 © Kai Tödter, Licensed under a Creative Commons Attribution 4.0 International License. 39