Where Your Agent Leaks Tokens, The MCP Token Tax

Your agent’s actual request is about 1.6% of the tokens it sends. The rest is tool schemas, history, and raw results re-read on every turn. In this presentation, I walk through four token leaks in MCP-based agents (catalog, payloads, cache misses, blind spots), demo the same prompt in three tool modes with live token counts, and share what to measure first.

Transcript

1. AAIF COMMUNITY BOSTON The MCP Token Tax Watch an agent’s context bloat. Then put it on a diet. Hugo Guerrero Senior AI Architect & Evangelist, Solo.io Governance Board & TSC Member, AsyncAPI Initiative
2. THE HOOK Same request. Very different bill NO TOOLS 52 3 MCP SERVERS → input tokens on the first model call · 58 TOOLS 16,482 input tokens on the first model call “Tell me a Chuck Norris joke.” One tool call is enough. The agent paid for 58 tools. Demo repo, claude-haiku-5-5, no caching, median of 3 runs. Yours will differ. @hguerreroo | theagenticarch.com 2
3. THE PLAN Build it, break it, fix it four ways 01 02 03 04 05 06 Build Bloat Just in time Compress Code Flows An agent with a jokes MCP Add GitHub and Jira Find servers in agentregistry Headroom in front of the model One endpoint, tool modes Tools as Arazzo workflows (if time) ▶ demo 1 ▶ demo 1 ▶ demo 3 ▶ demo 4 ▶ demo 5 ▶ demo 2 One agent, one task, one meter. The meter reads tokens, calls, latency and cost on every run. @hguerreroo | theagenticarch.com 3
4. WHY IT MATTERS Tool definitions are a tax on every call COST LATENCY ACCURACY You pay for tools the task never used The model reads it all before it answers The right tool is buried Tool definitions are input tokens on every model call. Prompt caching lowers the price per token. It does not make the prompt smaller. A longer prompt delays the first token, on every turn of the agent loop. With many tools to choose from, models choose worse. We did not measure this today. The numbers are in the hidden backup slides at the end of the deck. @hguerreroo | theagenticarch.com 4
5. DEMO 1 · BUILD AND BLOAT ▶ DEMO 1 · 3 MIN Add MCP servers, watch the meter WHAT THE MODEL CARRIES ON ITS FIRST CALL no tools + jokes @hguerreroo | theagenticarch.com + GitHub • The request never changes. • Each server adds its whole tool catalog. • The joke needed one tool. The agent carried all of them. • The meter prints tokens, calls and latency on every run. + Jira 5
6. THE LEAK MAP Anatomy of one model call LEAK 1 · tool definitions LEAK 2 standard Tool definitions · 39% · tool results LEAK 3 prefix · history, request, cache Tool results · 60% Your request: 0.2% search ↑ The catalog is gone. The payload stays. codeSearch Both shrink, if the script filters. Relative sizes of the input of the last model call, release brief task. Exact token counts are in the hidden backup. @hguerreroo | theagenticarch.com 6
7. BEFORE THE FIRST TOOL Choose the harness first: it is a fixed tax WHAT IT MEANS • A fixed cost, paid on every call, before your first tool. • A full-featured harness can cost more than a whole set of MCP servers. • Small windows and slow local models feel it most. • Lean has a price: Pi has no sandbox or permission prompts by default. • A gateway cannot shrink the harness. It controls what the harness pulls in over MCP. Relative sizes, one sample each, not a benchmark. Figures are in the hidden backup. @hguerreroo | theagenticarch.com 7
8. FIX 1 · JUST IN TIME ▶ DEMO 2 · 2 MIN Don’t load tools. Look them up Agent Two small tools: find_mcp_servers connect_mcp_server agentregistry → A catalog of servers. Name and description of each. No schemas. agentgateway → One route per server. Identity and policy live here. MCP servers → jokes · github · jira 1 find_mcp_servers(“joke”) The registry returns a few one-line entries, ranked by keyword. 2 connect_mcp_server(“jokes”) The agent connects to the URL the registry holds, through the gateway. 3 Next turn Only that server’s tools are in the context. @hguerreroo | theagenticarch.com 8
9. FIX 1 · RESULTS It pays when the task touches few servers A joke needs one server A GitHub question needs one server A release brief needs three servers BIG SAVING Only the jokes server is loaded. The other two never enter the context. SMALL SAVING One big release payload dominates the call, and the registry does not touch payloads. NO SAVING The agent ends up loading everything, after extra lookups. The registry decides at the granularity of a server. For a finer unit, see tool modes. @hguerreroo | theagenticarch.com 9
10. FIX 2 · COMPRESSION ▶ DEMO 3 · 2 MIN · OPTIONAL Squeeze what still comes back Agent → Headroom :8787 → agentgateway → Claude ACTS ON DATA DEPENDS ON THE PAYLOAD LOSSY HAS A COST Not on the catalog Big and repetitive wins It can drop items It compresses JSON, logs, code and text before they reach the model. A local proxy, so nothing leaves your machine. Large JSON lists compress well. Short results barely change, which is what we saw here. In an earlier test it cut a list of 30 items to about half. Keep identifiers and complete lists out of lossy mode. @hguerreroo | theagenticarch.com 10
11. FIX 3 · TOOL MODES One endpoint, four ways to present the tools standard search code codeSearch THE MODEL SEES THE MODEL SEES THE MODEL SEES THE MODEL SEES Every tool, full schema get_tool + invoke_tool run_code, with the API in its description get_tool + run_code LARGE CATALOG TINY CATALOG LARGE CATALOG TINY CATALOG USE IT FOR USE IT FOR USE IT FOR USE IT FOR Small catalogs Many tools, a few per task Chains of calls Many tools and chains All four are routes on one gateway over the same servers. Only toolMode differs. Tool modes are an Enterprise agentgateway feature. @hguerreroo | theagenticarch.com 11
12. FIX 3 · HOW IT WORKS The model writes the plan. The gateway runs it RUN_CODE · WRITTEN BY THE MODEL const [rel, issues, joke] = await Promise.all([ github_get_latest_release({owner: "agentgateway", repo: "agentgateway"}), jira_searchIssues({jql: "project = AGW AND statusCategory != Done", maxResults: 1}), jokes_getJoke({}) ]); ({ release: rel && {tag: rel.tag_name, name: rel.name, published: rel.published_at}, issues: issues && {total: issues.total ?? issues.totalCount, keys: issues.issues?.map(i=>i.key)}, joke }) get_tool The model asks for the signatures it needs, nothing else. run_code One script calls the tools, in parallel where it can. result Intermediate data stays in the sandbox. Only the last value returns. @hguerreroo | theagenticarch.com 12
13. DEMO 4 · TOOL MODES ▶ DEMO 4 · 4 MIN · KEEP Same task, four tool modes search The catalog is gone. The payload stays. code Fewer calls. The catalog stays. codeSearch Both shrink, if the script filters. More calls. Fewer tokens. Results vary run to run. Relative input tokens for one task. Exact numbers are in the hidden backup. @hguerreroo | theagenticarch.com 13
14. THE TRADE-OFFS What the savings cost More calls Fewer tokens come from more, smaller round trips. Each one adds latency. A sandbox to trust Time and CPU limits on every script, and authorization on every tool it calls. Ask about egress and audit, as for any model-generated code. @hguerreroo | theagenticarch.com Code that must be right The model writes the script. If it does not filter, the raw payload comes back into the context. Evaluate answers, not only tokens. Lossy shortcuts Compression can drop items from a list. Keep identifiers and complete lists out of lossy mode. 14
15. FIX 4 · TOOLS AS FLOWS ▶ DEMO 5 · 2 MIN · OPTIONAL Document the flow once. Expose it as one tool ARAZZO · release-brief.arazzo.yaml workflowId: releaseBrief summary: Release brief for a repo and its Jira project inputs: { owner, repo, jql } steps: - stepId: latestRelease operationId: $sourceDescriptions.github.getLatestRelease - stepId: openIssues operationId: $sourceDescriptions.jira.searchIssues parameters: [{ name: jql, in: query, value: $inputs.jql }] - stepId: joke operationId: $sourceDescriptions.jokes.getJoke outputs: release: $steps.latestRelease.outputs.tag notDoneIssues: $steps.openIssues.outputs.total joke: $steps.joke.outputs.text @hguerreroo | theagenticarch.com WHAT THE AGENT SEES One tool. One call. The order of the calls is written in the workflow, not worked out by the model each time. CAVEATS The flow is fixed: the agent loses its freedom to improvise. Arazzo is young, not designed for LLMs, and the runner used here is alpha. 15
16. THE SCORECARD Which lever for which leak LEVER SHRINKS HOW MUCH WATCH OUT FOR Registry, just in time Catalog, per server Large when few servers are needed Retrieval quality, extra calls, cache busting Search mode Catalog, per tool Medium: the payload stays Extra round trip per schema Code mode Calls and results Medium: saves by fewer calls Description still carries the API codeSearch Catalog and calls Large, if the script filters Model-written code, sandbox limits Compression Payloads, history Small on small results Lossy mode can drop items Flows (Arazzo) Calls and catalog Large when the flow fits Fixed flow, young tooling The levers act on different leaks, so they do not simply add up. Measure your own agents. Figures from the demo are in the hidden backup. @hguerreroo | theagenticarch.com 16
17. TAKEAWAYS What to do on Monday 1 Measure Tokens, calls and latency per task. A meter is 100 lines; this repo has one. 2 Choose and scope Pick the harness on purpose. Give each agent only the servers its task needs, with identity and policy in the gateway. 3 Disclose or code Search or codeSearch for big catalogs. Code mode when tasks chain calls. 4 Compress carefully Only where results are big and loss is acceptable. Check the answers. 5 Cap and govern Token budgets, a cost catalog, and authorization on every tool, in every mode. @hguerreroo | theagenticarch.com 17
18. THANK YOU LINKEDIN CONNECT X / TWITTER @hguerreroo BLUESKY hguerreroo.bsky.social Not bigger prompts. Smarter context. YOUTUBE theagenticarch LINKEDIN RUN THE DEMO YOURSELF github.com/hguerrero/ mcp-context-bloat-demo Hugo Guerrero linkedin.com/in/hugoguerrero @hguerreroo | theagenticarch.com hugoguerrero WEB theagenticarch.com