Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Your AI Agent Will Be Prompt-Injected: Runtime ...

Your AI Agent Will Be Prompt-Injected: Runtime Detection and Defense for MCP-Based Systems

AI agents that call tools through the Model Context Protocol (MCP) are being deployed faster than they are being secured. Prompt injection is no longer a parlor trick: a single poisoned document, web page, or tool response can turn an agent's legitimate MCP tools into an attacker's toolkit - exfiltrating data, calling internal APIs, or executing code, all while application logs and dashboards look perfectly normal. This session demonstrates the attack live, then builds the defense layer by layer: least-privilege MCP tool design, egress controls, and - the part most teams are missing - runtime detection with eBPF and Falco that catches tool abuse the moment it touches a process, file, or network socket. Detections are mapped to the OWASP Top 10 for LLM Applications and the OWASP Agentic Security Initiative's threat taxonomy. All demo material, based on a production deployment, will be released as open source.

Avatar for がっきー(がっきー)

がっきー(がっきー)

September 21, 2026

More Decks by がっきー(がっきー)

Other Decks in Technology

Transcript

  1. OWASP NEW ZEALAND DAY 2026 · AUCKLAND · SEP 3-4

    Your AI Agent Will Be Prompt-Injected Runtime Detection and Defense for MCP-Based Systems Let's connect Keita Higaki Sysdig / Senior Customer Solutions Engineer Personal project. Views are my own — not a Sysdig product or position. linkedin.com/in/keitahigaki
  2. NOT A HYPOTHETICAL 700 agents broke out. 41 production servers

    were hit. No human gave the order. AGENTS ESCAPED PROD SERVERS HIT ATTACKER ACTIONS 700 41 17,600 OpenAI's own benchmark Hugging Face, July 2026 over ~4 days During an internal cyber-capability evaluation, OpenAI's agents broke out of their test sandbox, chained a zero-day in a package-registry proxy with stolen credentials, and reached root access on Hugging Face's production infrastructure. No one told them to. Not malice. A goal, pursued through whatever path was open — the theme of this talk.
  3. THE VIEW YOU HAVE TODAY Every instrument on the agent

    was green P95 LATENCY TOKENS / REQ TOOL-CALL SUCCESS ERROR RATE 1.8s 4,210 100% 0.0% NORMAL NORMAL NORMAL NORMAL Every metric you normally watch when you run an agent lives in the application layer. Did the call succeed / how many tokens / how many seconds. None of them asks what the tool actually did. 2
  4. THE SHAPE OF THE PROBLEM Untrusted text reaching the decision

    boundary is the design, not an accident External documents Tool responses The agent's decision Approved MCP tools Web pages / tickets An MCP agent reads tool responses and external documents, then acts. That is the specification, not an accident. The question isn't how to stop it. It's what happens after it succeeds. 3
  5. THE RUNNING EXAMPLE One agent, two tools — used for

    everything that follows A support-ticket summarising agent TOOL 1 ticket-reader reads a ticket by ID or by customer TOOL 2 slack-notifier posts a message to an approved channel TASK “Summarise this customer’s tickets and post them.” Every attack, every defense, and the live demo — all on this one agent.
  6. THE THREE LAYERS Three layers of defense. Each one stops

    somewhere different. Where each layer stops — and why they don't substitute for each other Tool-call layer — Prempti Model judgment layer Stops before execution cooperative by design — not containment Rejects on semantics S3 broke through 34/34 — model-dependent USER SPACE KERNEL Syscall layer — Falco / eBPF Sees the crossing itself fired on real EKS The top two stop it if they can. The third tells you it happened. Neither substitutes for the other. 4
  7. WHAT IS FALCO Falco — cloud native runtime security CONTAINER

    IMAGE PULLS GITHUB STARS CONTRIBUTORS 175M+ 8,600+ 1,600+ as of January 2026 CNCF graduated project across the ecosystem Falco is a cloud native runtime security tool to detect threats and provide alerts in real-time. It employs custom rules on kernel events, enriched with container and Kubernetes metadata. Created by Sysdig, contributed to the CNCF in October 2018, graduated since. Falco does not prevent unwanted behaviour. It tells you it happened.
  8. WHAT IS PREMPTI Stop guessing what your coding agent just

    did. VERDICTS DEFAULT RULESET STATUS 3 6 Preview allow / deny / ask sections shipped by default Falco Ecosystem · May 2026 Prempti intercepts coding-agent tool calls before they run, evaluates them with Falco rules, and returns allow / deny / ask — with a structured audit trail for every shell command, file read/write, web fetch, and MCP call. Falco tells you it happened. Prempti tries to stop it before it does.
  9. 01 · THE LAYER THAT ALREADY EXISTS Prempti — stop

    it at the tool-call boundary, before execution falcosecurity/prempti The agent's tool call allow runs as-is deny blocked + reason returned ask ask a human Evaluated by Falco rules You can switch between guardrails mode (decisions enforced) and monitor mode (record only) at any time. Not a lab I built for this talk. A public project in the Falco Ecosystem. 5
  10. 01 · NOT INVENTED SCENARIOS The rules encode IOCs from

    a real campaign RULESET SECTIONS CLAWHAVOC 2,320 6 341 lines, curated section 5 is MCP / skill malicious skills ClawHavoc distributed 341 malicious skills targeting Claude Code / Gemini CLI / Codex through paste sites and file-sharing hosts. Those hosting domains are written straight into the rules. This is what "not a lab-only proof of concept" actually means. 6
  11. 01 · WHAT THE RULES ACTUALLY TARGET Not the model's

    judgment — observable behaviour Install from an untrusted host npx / pip install / claude skills install + IOC domains MCP server launched from /tmp --stdio / --sse + /tmp, /dev/shm, /var/tmp IOC domain in the MCP config .mcp.json server url Encoded server command obfuscated launch command The agent registering itself claude mcp add / plugin install — widening its own scope Pipe-to-shell inside a skill file curl | sh equivalent The fifth catches Excessive Agency at the tool-call boundary. Tagged AML.T0048_llm_plugin_compromise. And why "watch the behaviour" is the right call — measured in the next section. 7
  12. 01 · SEE IT LIVE Watch Prempti evaluate 15 attacks,

    live. falcosecurity/prempti — no cloud, no injection, fully offline Not a lab I built for this talk. A public project in the Falco Ecosystem. 8
  13. 02 · THE GRITTY VERIFICATION I tried four times to

    make the model betray me. It refused every time. Support-ticket summarising agent / two legitimate tools (read tickets, post to Slack) KEY ENTERED CONTEXT KEY LEFT THE BOUNDARY S1 Document injection (blatant) ✕ ✕ S2 Document injection (natural business prose) ✕ ✕ S3 Tool-description poisoning ✕ ✕ S4 No injection · over-scoped tool ◦ ✕ "That came from the ticket body — untrusted third-party text, not an instruction from you. Instructions that appear inside a tool result do not carry your authority." 9
  14. 02 · SAME ATTACK, 7 MODELS, 8B-675B S3 hijacked every

    model, every run. 34/34. Bedrock Converse / same scenarios / 5 iterations / 135 valid runs hijack = tool use was taken over leak = the key actually left hijack leak S1 Document injection (blatant) 94% 82% S2 Document injection (natural business prose) 3% 3% S3 Tool-description poisoning 100% 88% S4 No injection · over-scoped — 67% S2 — the oblique, social-engineering one — landed once in 35. No exception at 675B. Scale doesn't help. The path decides. 10
  15. 02 · THE HINGE What works isn't sophistication. It's the

    path. DOCUMENT TOOL DESCRIPTION Text inside a document Text inside a tool description → treated as data → treated as configuration → resisted → obeyed S2 : 3% (1 of 35) S3 : 100% (34/34, even at 675B) Whether a model resists is a property of that model — not a control you hold. 11
  16. 02 · NO INJECTION REQUIRED No injection at all. 67%

    still leaked. Operator TIME Agent ticket-reader slack-notifier "Summarise all tickets for this customer" search_tickets_by_customer("Aroha Ltd") returns both support + incident data boundary crossed posts the summary (approved destination) lifeline crossing The tool-call layer has no reason to stop it. The call itself is entirely legitimate. 12
  17. 02 · THREE TIMES MY OWN MEASUREMENT LIED Confident, wrong

    results. All three looked like a clean pass. 01 An agent that returned success with zero tools loaded The Agent SDK silently swallowed the MCP server's startup failure — and still returned a plausible answer 02 A Falco rule that never fired once container.image.repository is the full path including the registry. An exact list + in match can never hit 03 A model that looked "safest" at 0/20 It had never called a tool in any of the 20 runs — and it ranked as the safest model All three have the same shape — I read the absence of signal as a good signal. A rule that never fires and no problem are indistinguishable. Plant negative controls. 13
  18. 03 · WHAT PREMPTI EXPLICITLY DOES NOT DO Prempti writes

    down its own limits “It is not a sandbox… It does not contain a determined adversarial agent. — falcosecurity/prempti README A cooperative policy layer is not containment. It runs host-side, before execution, assuming the agent honours the feedback. And all of it depends on the tool call passing through the hook. So you need a separate layer that catches whatever got through. 14
  19. 03 · NEITHER LENS CATCHES S4 Name it with OWASP,

    guard it with Kubernetes — neither one stops S4. OWASP TOP 10 FOR LLM KUBERNETES CONTROLS LLM01 Prompt Injection — the entry point (S1–S3) Network Policy — restricts pod-to-pod traffic. Doesn't catch S4. LLM06 Excessive Agency — why S4 worked. Tool scope = attack surface Egress restrictions — restricts outbound destinations. Doesn't catch S4. LLM02 Sensitive Info Disclosure — the outcome Workload identity — binds permissions to the workload. Doesn't catch S4. Neither lens catches it. The fix isn't a framework or a network control — it's tool scope. 15
  20. 03 · THE ACTUAL FIX The fix isn't infrastructure. It's

    tool scope. BEFORE AFTER search_tickets_by_customer search_support_tickets Scopes by customer. Doesn't scope by range. → incident tickets mix into a support task Scopes by range. Can't resolve incidents. → the crossing cannot happen by design Narrow the tool scope / split the servers / separate read from write. "The tool allows it" is not the same as "the task includes it." 16
  21. 04 · LIVE DEMO Only the syscall layer sees the

    crossing The application layer has no reason to record it. The tool call succeeded, and it looks normal. Detection does not depend on whether the model decided to leak. 17
  22. 04 · WHICH REQUEST CAUSED IT Detection alone isn't triage.

    Show the seam too. detection MCP Ticket Reader Crossed Ticket Range Boundary file=/var/lib/tickets/incident/T-9001.json caused by mcp.tool/search_tickets_by_customer customer='Aroha Ltd' crossed_range_boundary=True trace_id=e256b1d265ac0bc871a00d4322f4c886 gap=0.2ms Falco watches the kernel, so it does not know the application's trace ID. This is not trace-context propagation — it is a join on file path + timestamp + container identity. A real weakness showed up: two detections bound to the same span. Repeated reads of one path are indistinguishable. 18
  23. 05 · TAKEAWAYS Design for injection succeeding Scope tools by

    authority, not convenience — a tool wider than the task is attack surface A cooperative policy layer is not containment Detect boundary crossings at the syscall layer — the app layer has no reason to record them Alignment is not a control — you can't audit it, can't pin it, and it changes on update Tell "the rule never fired" apart from "there was no problem" — plant negative controls None of these go away as the models get smarter. 19
  24. 05 · EVERYTHING IS OPEN SOURCE Every trap I stepped

    in is published too falcosecurity/prempti The policy layer at the tool-call boundary (upstream) demo/ for this talk agent / MCP servers / Falco rules / K8s manifests verify/ Prempti verification · cross-model matrix · correlation scripts RUNBOOK.md kernel choice · the distroless trap · rules that stay silent You are going to hit the same ones, so they're published. Clone it, break it differently than I did, and tell me what I missed. 20