Solving Context Bloat: Semantic Tool Routing in Multi-Server MCP Environments

The MCP ecosystem grows more valuable as tool catalogs expand — but that growth creates fragility if context management isn't solved at the architectural level. This session advances a community-owned pattern that any MCP implementer can adopt, regardless of framework or vendor. By sharing reference architectures and open-source implementation guidance, it helps the broader agentic community build systems that scale sustainably as MCP server proliferation continues.

Transcript

1. AI CONNECTIVITY Beating Context Bloat Hugo Guerrero DEVELOPER ADVOCATE KONG MAR 2026 MCP Patterns
2. AI CONNECTIVITY 2 2 Standardized tool access The MCP Promise Integration to enterprise systems
3. AI CONNECTIVITY 3 The MCP Promise vs. Reality Promise Standardized agentic access to all enterprise tools THE CONNECTIVITY EXPLOSION Every new tool added is another paragraph in the system prompt A 'System Prompt' that looks like a legal contract Reality Result
4. AI CONNECTIVITY 4 4 THE HIDDEN SCALING PROBLEM Context Bloat ▪ Too many tools ▪ Large prompts ▪ Slower reasoning ▪ Higher hallucination risk
5. AI CONNECTIVITY 5 Idealistic MCP Architecture Tool Tool MCP Server MCP Clients / AI Agents
6. AI CONNECTIVITY © Kong Inc. 6 Let’s break some things DEMO
7. AI CONNECTIVITY 7 Idealistic MCP Architecture Tool Tool MCP Server MCP Clients / AI Agents
8. AI CONNECTIVITY 8 Idealistic Reality-checked MCP Architecture Tool Tool Tool Tool Tool Tool Tool MCP Server MCP Server MCP Server MCP Clients / AI Agents 100x 10x
9. AI CONNECTIVITY ▪ Latency Tax: LLMs take longer to 'reason' over massive prompts. ▪ Fiscal Tax: You are paying for those thousands of "Tool Definition" tokens every single turn. ▪ Intelligence Tax: "Lost in the Middle" syndrome, where the LLM misses the right tool because of the noise. 9 3 Tax Hits on AI Performance WHY "JUST IN CASE" FAILS
10. AI CONNECTIVITY CONFIDENTIAL | NOT TO BE SHARED EXTERNALLY © Kong Inc. Agents should receive tools just in time, not just in case
11. AI CONNECTIVITY 11 MCP Missing Component Tool Tool Tool Tool Tool Tool Tool MCP Server MCP Server MCP Server MCP Clients / AI Agents HELP!!
12. AI CONNECTIVITY 12 MCP Gateway Architecture Tool Tool Tool Tool Tool Tool Tool MCP Server MCP Server MCP Server MCP Clients / AI Agents AI (MCP) Gateway ▪ Au/Az ▪ ACLs ▪ o11y
13. AI CONNECTIVITY © Kong Inc. 13 Let’s break some things DEMO
14. AI CONNECTIVITY 14 MCP Gateway Architecture Tool Tool Tool Tool Tool Tool Tool MCP Server MCP Server MCP Server MCP Clients / AI Agents AI (MCP) Gateway ▪ Au/Az ▪ ACLs ▪ o11y
15. AI CONNECTIVITY 15 MCP Gateway Architecture Tool Tool Tool Tool Tool Tool Tool MCP Server MCP Server MCP Server MCP Clients / AI Agents AI (MCP) Gateway MCP Registry ▪ Service Discovery ▪ Endpoint Resolution ▪ Run Time Discovery & Resolution
16. AI CONNECTIVITY 16 MCP Gateway Architecture Tool Tool Tool Tool Tool Tool Tool MCP Server MCP Server MCP Server MCP Clients / AI Agents AI (MCP) Gateway MCP Registry AI (LLM) Gateway LLMs LLMs LLMs
17. AI CONNECTIVITY ▪ Agent handles Prompt ▪ Agent searches registry for servers ▪ MCP gateway filters tools ▪ Injects relevant authorized tools 17 MCP Tool Selection using the Registry and Gateway patterns HOW IT WORKS 17
18. AI CONNECTIVITY © Kong Inc. 18 Let’s break some things DEMO
19. AI CONNECTIVITY 19 What we just saw 1 2 3 We were able to seamlessly manage context scope DEMO We apply identity-based security (Alice vs. Bob) Dynamically added tools without rebuilding
20. AI CONNECTIVITY ▪ MCP will dramatically increase tool counts ▪ Prompt injection does not scale ▪ Semantic selection makes agent infrastructure viable at enterprise level 20 The future of agent systems is not bigger prompts; it is smarter context TAKEAWAYS
21. AI CONNECTIVITY