Optimize your architecture for the next wave of AI-infused applications

About this Session:
While LLMs like ChatGPT dominate the conversation, the real challenge lies in the anatomy of AI-infused applications. This session explores why traditional software architecture often fails when deploying AI into production and provides a roadmap for building scalable, resilient AI ecosystems.

Key takeaways:

• The Anatomy of AI Apps: Understanding the unique challenges of AI-infused application architecture compared to traditional SDLC.

• Agentic AI Integration: How to leverage autonomous agents to distribute intelligence throughout your apps without disrupting legacy software investments.

• Model Context Protocol (MCP): A deep dive into an enterprise use case using AI agents with MCP to revolutionize customer service and satisfaction.

• Future-Proof Logic: Strategies for evolving application logic even as underlying LLMs change over time.

------------------------------------

Conference Details:
Conference: apidays Paris 2025 | part of FOST 2025
Theme: The APIs meet AI conference: Innovation, Security, Sovereignty, Sustainability
Date: 9 - 11 December 2025 • CNIT Forest, Paris

--------------------------

Resources from apidays:
Join our upcoming conferences: https://www.apidays.global/
Read the latest API news: https://www.apiscene.io
Explore the API Landscape: https://apilandscape.apiscene.io/

Transcript

2. A journey to… 2 Optimize your architecture for the next wave of AI-infused applications
3. 3 Attribution: ICC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1340533 By Charles Darwin in Transmutation of Species 1837 - HTML / JPG, Public Domain, https://commons.wikimedia.org/w/index.php?curid=2648102 Perpetual Change Common Descent Multiplication of Species Gradualism Natural Selection
4. 4.
5. 5 AI - the latest Wave in Information Technology Evolution
6. 6 AI - the latest Wave in Information Technology Evolution Average M new users / year 100 25 20 8
7. 7 Tech waves are hard
8. 75% 25% 92% 1% 30% Source: BCG Radar 2025 McKinsey 2025 AI report Gartner prediction 2024 Challenges in the transition to the AI Wave 8. 92% firms plan to invest more in AI 1% believe investments at maturity McKinsey AI report 2025 Survey of 3800+ employees and execs Gen AI projects will be abandoned after proof of concept by end of 2025 Gartner, Rita Sallam, 2024 Prediction at Gartner Analytics and Data Summit 75% rank AI top 3 priority 25% see significant value from AI BCG Radar 2025 Survey of 1800+ CxOs
9. Platform Engineering / Platform as Product Anatomy of the Modern Application Platform 9. (Micro)services Containers Domain modeling APIs, events + integration Architecture and design practices Value delivery on open hybrid cloud Secure, repeatable development process Applications Public | Private | Hybrid
10. Evolution of Distributed Architecture 10 Monolithic N-Tier Microservices
11. AI adds distributed intelligence with Agentic architecture 11 Monolithic N-Tier Microservices Agentic
12. AI is another leap forward in app architecture 12. Scalability Composability Client autonomy
13. Encoding of logic for software is done with programming languages 13
14. New programming languages have emerged for each new technology domain 14 Enterprise software Java C# (.NET) Web browsers Javascript Typescript Mobile Swift Kotlin AI/ML Python
15. 15 Enterprise software Java C# (.NET) Web browsers Javascript Typescript Mobile Swift Kotlin AI/ML Python
16. Source: BBC: Decades-long mystery of ginger cats revealed MuyInterestante, Christian Perez: Descubren el secreto genético que revela por qué los gatos naranjas son naranjas Let’s not forget the cats! 16.
17. Source: Cleveland Clinic - nervous system Connectivity between apps and services is like a nervous system 17 Neuron types for message transmission: ▸Motor - brain/spinal cord ⇔ muscles ▸Sensory - senses ⇒ brain ▸Interneurons - motor ⇔ sensory neurons
18. New connectivity interfaces to support new communications challenges 18 Interface Protocol/Format Communication Pattern Typical Use Case Strengths Weaknesses RPC Binary/Custom Synchronous (req- resp) Distributed systems, microservices Transparent remote calls, language agnostic Tightly coupled, less flexible SOA/SOAP XML over HTTP/Other Synchronous (req- resp) Enterprise integration Strong standards, security, extensibility Verbose, complex, slower REST API HTTP (JSON/XML) Synchronous (req- resp) Web/mobile APIs, microservices Lightweight, easy, flexible, scalable Weaker contracts, less built-in security Event Streaming Text/Binary Asynchronous, real- time Real-time data, notifications Real-time, decoupled, scalable Complexity, error handling required MCP JSON-RPC (A)synchronous LLM/AI tool integration Standard, simplified access to tools and resources Niche, less general- purpose
19. Defacto standard for interfacing LLMs Model Context Protocol (MCP) for Agentic AI 19. MCP as the interface for AI architecture, to simplify agent orchestration: ▸ Layered on existing interfaces ▸ Complement other interface types ▸ Coexist into the future - not an either/or
20. Customer support 20 Finance Support Website
21. Customer support with Agentic AI 21 Moderation Structured Output
22. Customer support with Agentic AI 22 Finance Support Website Moderation Structured Output Routing Processing
23. Customer support with Agentic AI 23 Moderation Structured Output Routing Processing Finance Support Website Exception
24. Customer support with Agentic AI 24 Moderation Structured Output Routing Processing Finance Support Website Exception
25. Customer service app with Agentic AI 25 Moderation Structured Output Routing Processing Finance Support Website Exception Moderation Svc Intake Moderated Input Messages
26. Customer service app with Agentic AI 26 Moderation Structured Output Routing Processing Finance Support Website Exception Structure Svc Structured
27. Customer service app with Agentic AI 27 Moderation Structured Output Routing Processing Finance Support Website Exception Router Customer Data Svc Router Svc RDBMS
28. Customer service app with Agentic AI 28 Moderation Structured Output Routing Processing Finance Support Website Exception Router Customer Data Svc Router Svc RDBMS Moderation Svc Intake Moderated Input Messages Structure Svc Structured
29. Customer service app with Agentic AI 29 WALKTHROUGH
30. 30 User request Moderated Structure Svc Structured Hello, I can't find the invoice for my most recent purchase, please send my invoice history. Sincerely, Thomas thomashardy@example.com
31. 31 Structured Svc (Quarkus agent) Moderated Structure Svc Structured @RegisterAiService() @ApplicationScoped public interface InformationExtractor { @SystemMessage(""" Extract the customer support email information. Do not make up information. If a piece of information is not present, leave the field in the response blank """) StructuredInformation extractInformationFrom(String text); }
32. 32 Components of prompt System prompt Enriched input Moderated Structure Svc Structured Extract the customer support email information. Do not make up information. If a piece of information is not present, leave the field in the response blank You must answer strictly in the following JSON format: {\n\"reason\": (type: string),\n\"sentiment\": (type: string),\n\"customerId\": (type: string),\n\"companyName\": (type: string),\n\"customerName\": (type: string),\n\"country\": (type: string),\n\"emailAddress\": (type: string),\n\"phone\": (type: string),\n\"productName\": (type: string),\n\"escalate\": (type: boolean)\n}
33. 33 API Request from AGENT to LLM Moderated Structure Svc Structured User request Hello, I can't find the invoice for my most recent purchase, please send my invoice history. Sincerely, Thomas thomashardy@example.com { "model": "gpt-4o-mini", "messages": [ { "role": "system", "content": "Extract the customer support email information. Do not make up information. If a piece of information is not present, leave the field in the response blank" }, { "role": "user", "content": "Hello, I can't find the invoice for my most recent purchase, please send my invoice history. \n\nSincerely, \nThomas \nthomashardy@example.com \nYou must answer strictly in the following JSON format: {\n\"reason\": (type: string),\n\"sentiment\": (type: string),\n\"customerId\": (type: string),\n\"companyName\": (type: string),\n\"customerName\": (type: string),\n\"country\": (type: string),\n\"emailAddress\": (type: string),\n\"phone\": (type: string),\n\"productName\": (type: string),\n\"escalate\": (type: boolean)\n}" } ], "temperature": 0.0, "top_p": 1.0, "presence_penalty": 0.0, "frequency_penalty": 0.0 } System prompt User prompt Enriched input
34. 34 "role": "assistant", "content": "{\n \"reason\": \"Request for invoice history\",\n \"sentiment\": \"Neutral\",\n \"customerId\": \"\",\n \"companyName\": \"\",\n \"customerName\": \"Thomas\",\n \"country\": \"\",\n \"emailAddress\": \"thomashardy@example.com\",\n \"phone\": \"\",\n \"productName\": \"\",\n \"escalate\": false\n}", "refusal": null, "annotations": [] }, Gets added to the message for the next request Moderated Structure Svc Structured JSON response from LLM
35. 35 Finance Svc Finance API Finance
36. MCP 36 1. Agentic App registers MCP server and gets tool metadata 2. Agentic App calls LLM with user query. The payload contains the tool metadata. The LLM responds with a tool call. 3. The Agentic App calls the MCP Server with the tool name and parameters 4. The MCP Server invokes the tool and returns the result to the Agentic App. 5. The Agentic App calls the LLM with the chat history and the tool call result. Agentic App MCP Server Tool Impl. 1. 3. 2. 5. 4.
37. 37 Finance Svc Finance API MCP Server Finance • Gets tool definitions from MCP Server • Adds them to the prompt • Builds the API Request to the LLM with the list of tools + previous message Quarkus agent
38. 38 Finance Svc Finance API MCP Server Finance • Gets tool definitions from MCP Server • Adds them to the prompt • Builds the API Request to the LLM with the list of tools + previous message Quarkus agent { "model" : "gpt-4o-mini", "messages" : [ { "role" : "system", "content" : "You are an expert tool calling, review the user message and call the best matching tool" }, { "role" : "user", "content" : "This is the user request. It contains the customerId and the user request. If you called a tool, add the function name to the response.\nRequest: {\n \"customer_id\" : \"AROUT\",\n \"request\" : \"Hello, I can't find the invoice for my most recent purchase, please send my invoice history. \\n\\nSincerely, \\nThomas \\nthomashardy@example.com\"\n}\n\nYou must answer strictly in the following JSON format: {\n\"tools\": (type: array of string),\n\"records\": (type: array of org.parasol.customerservice.finance.ai.model.FinanceRecord: {\n\"id\": (type: integer),\n\"customer\": (type: string),\n\"orderDate\": (type: date string (2023-12-31)),\n\"shippedDate\": (type: date string (2023-12- 31)),\n\"status\": (type: string),\n\"value\": (type: java.math.BigDecimal)\n})\n}" } ], "temperature" : 0.0, "top_p" : 1.0, "presence_penalty" : 0.0, "frequency_penalty" : 0.0, "tools" : [ { "type" : "function", "function" : { "name" : "findMissingReceipt", "description" : "Start the process to find a lost receipt for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getInvoiceHistory", "description" : "Retrieve a customer's invoice history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getOrderHistory", "description" : "Retrieve a customer's order history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "startDuplicateChargeDispute", "description" : "Start the process to dispute a duplicate charge for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } } ] } System prompt System prompt You are an expert tool calling, review the user message and call the best matching tool Enriched input User prompt
39. 39 Finance Svc Finance API MCP Server Finance • Gets tool definitions from MCP Server • Adds them to the prompt • Builds the API Request to the LLM with the list of tools + previous message Quarkus agent { "model" : "gpt-4o-mini", "messages" : [ { "role" : "system", "content" : "You are an expert tool calling, review the user message and call the best matching tool" }, { "role" : "user", "content" : "This is the user request. It contains the customerId and the user request. If you called a tool, add the function name to the response.\nRequest: {\n \"customer_id\" : \"AROUT\",\n \"request\" : \"Hello, I can't find the invoice for my most recent purchase, please send my invoice history. \\n\\nSincerely, \\nThomas \\nthomashardy@example.com\"\n}\n\nYou must answer strictly in the following JSON format: {\n\"tools\": (type: array of string),\n\"records\": (type: array of org.parasol.customerservice.finance.ai.model.FinanceRecord: {\n\"id\": (type: integer),\n\"customer\": (type: string),\n\"orderDate\": (type: date string (2023-12-31)),\n\"shippedDate\": (type: date string (2023-12- 31)),\n\"status\": (type: string),\n\"value\": (type: java.math.BigDecimal)\n})\n}" } ], "temperature" : 0.0, "top_p" : 1.0, "presence_penalty" : 0.0, "frequency_penalty" : 0.0, "tools" : [ { "type" : "function", "function" : { "name" : "findMissingReceipt", "description" : "Start the process to find a lost receipt for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getInvoiceHistory", "description" : "Retrieve a customer's invoice history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getOrderHistory", "description" : "Retrieve a customer's order history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "startDuplicateChargeDispute", "description" : "Start the process to dispute a duplicate charge for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } } ] } System prompt User prompt This is the user request. It contains the customerId and the user request. If you called a tool, add the function name to the response.\nRequest: {\n \"customer_id\" : \"AROUT\",\n \"request\" : \"Hello, I can't find the invoice for my most recent purchase, please send my invoice history. \\n\\nSincerely, \\nThomas \\nthomashardy@example.com\"\n}\n\nYou must answer strictly in the following JSON format: {\n\"tools\": (type: array of string),\n\"records\": (type: array of org.parasol.customerservice.finance.ai.model.FinanceR ecord: {\n\"id\": (type: integer),\n\"customer\": (type: string),\n\"orderDate\": (type: date string (2023-12-31)),\n\"shippedDate\": (type: date string (2023-12-31)),\n\"status\": (type: string),\n\"value\": (type: java.math.BigDecimal)\n})\n} Enriched input User prompt
40. 40 Finance Svc Finance API MCP Server Finance • Gets tool definitions from MCP Server • Adds them to the prompt • Builds the API Request to the LLM with the list of tools + previous message Quarkus agent { "model" : "gpt-4o-mini", "messages" : [ { "role" : "system", "content" : "You are an expert tool calling, review the user message and call the best matching tool" }, { "role" : "user", "content" : "This is the user request. It contains the customerId and the user request. If you called a tool, add the function name to the response.\nRequest: {\n \"customer_id\" : \"AROUT\",\n \"request\" : \"Hello, I can't find the invoice for my most recent purchase, please send my invoice history. \\n\\nSincerely, \\nThomas \\nthomashardy@example.com\"\n}\n\nYou must answer strictly in the following JSON format: {\n\"tools\": (type: array of string),\n\"records\": (type: array of org.parasol.customerservice.finance.ai.model.FinanceRecord: {\n\"id\": (type: integer),\n\"customer\": (type: string),\n\"orderDate\": (type: date string (2023-12-31)),\n\"shippedDate\": (type: date string (2023-12- 31)),\n\"status\": (type: string),\n\"value\": (type: java.math.BigDecimal)\n})\n}" } ], "temperature" : 0.0, "top_p" : 1.0, "presence_penalty" : 0.0, "frequency_penalty" : 0.0, "tools" : [ { "type" : "function", "function" : { "name" : "findMissingReceipt", "description" : "Start the process to find a lost receipt for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getInvoiceHistory", "description" : "Retrieve a customer's invoice history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getOrderHistory", "description" : "Retrieve a customer's order history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "startDuplicateChargeDispute", "description" : "Start the process to dispute a duplicate charge for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } } ] } System prompt Enriched input "type" : "function", "function" : { "name" : "getInvoiceHistory", "description" : "Retrieve a customer's invoice history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" Enriched input User prompt
41. 41 Finance Svc Finance API MCP Server Finance { "model" : "gpt-4o-mini", "messages" : [ { "role" : "system", "content" : "You are an expert tool calling, review the user message and call the best matching tool" }, { "role" : "user", "content" : "This is the user request. It contains the customerId and the user request. If you called a tool, add the function name to the response.\nRequest: {\n \"customer_id\" : \"AROUT\",\n \"request\" : \"Hello, I can't find the invoice for my most recent purchase, please send my invoice history. \\n\\nSincerely, \\nThomas \\nthomashardy@example.com\"\n}\n\nYou must answer strictly in the following JSON format: {\n\"tools\": (type: array of string),\n\"records\": (type: array of org.parasol.customerservice.finance.ai.model.FinanceRecord: {\n\"id\": (type: integer),\n\"customer\": (type: string),\n\"orderDate\": (type: date string (2023-12-31)),\n\"shippedDate\": (type: date string (2023-12- 31)),\n\"status\": (type: string),\n\"value\": (type: java.math.BigDecimal)\n})\n}" } ], "temperature" : 0.0, "top_p" : 1.0, "presence_penalty" : 0.0, "frequency_penalty" : 0.0, "tools" : [ { "type" : "function", "function" : { "name" : "findMissingReceipt", "description" : "Start the process to find a lost receipt for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getInvoiceHistory", "description" : "Retrieve a customer's invoice history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "getOrderHistory", "description" : "Retrieve a customer's order history based on their customer id.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } }, { "type" : "function", "function" : { "name" : "startDuplicateChargeDispute", "description" : "Start the process to dispute a duplicate charge for a customer.", "parameters" : { "type" : "object", "properties" : { "customerId" : { "type" : "string", "description" : "the customer id" } }, "required" : [ "customerId" ] } } } ] } System prompt Enriched input User prompt • Gets tool definitions from MCP Server • Adds them to the prompt • Builds the API Request to the LLM with the list of tools + previous message Quarkus agent • Decides which tool to use "findMissingReceipt” "getInvoiceHistory" "getOrderHistory" "startDuplicateChargeDispute" LLM
42. 42 Finance Svc Finance API MCP Server Finance Quarkus agent • Gets tool definitions from MCP Server • Adds them to the prompt • Builds the API Request to the LLM with the list of tools + previous message • Agent calls MCP server { "jsonrpc": "2.0", "id": 2, "method": "tools/call", "params": { "name": "getInvoiceHistory", "arguments": { "customerId": "AROUT" } } } API call to the MCP server
43. Customer service app with Agentic AI 43 Moderation Structured Output Routing Processing Finance Support Website Exception Router Customer Data Svc Router Svc RDBMS Moderation Svc Intake Moderated Input Messages Structure Svc Structured
44. Key Takeaways 44 ▸ Improve success rate by taking an evolutionary approach ▸ Use Python where appropriate but continue with Java ▸ World of AI is world of APIs, but adapt to new interfaces like MCP ▸ Agentic approaches are key to infuse AI into your application architecture ▸ Rethink your application infusing AI into the design