Zero Trust Agents: Secure AO-Workloads with OPA and Kubernetes #CLC26

AI agents are evolving from passive chatbots into autonomous actors that can independently alter system states. This ‘capacity to act’ carries risks such as tool misuse and PII leakage.

In this talk, we present a cloud-native reference architecture for Kubernetes that secures agent-based workloads.

We demonstrate how the Open Policy Agent (OPA) enforces fine-grained ‘policy-as-code’ validation before every tool invocation via callback hooks and other appropriate mechanisms. We also use CNCF tools for real-time sanitisation of sensitive data. And we outline a ‘Zero Trust Agent’ architecture that combines LLM flexibility with strict compliance guarantees.

1. Zero Trust Agents Sichere KI-Workloads mit OPA und Kubernetes Mario-Leander Reimer mlr@qaware.de AI generated
2. Mario-Leander Reimer Managing Director | CTO @LeanderReimer #cloudnativenerd #qaware #gernperDude 2
3. Legacy De-Risking Agentic Automation Make legacy systems and infrastructure gradually more understandable, reduce risks and modernise them sensibly. Boost productivity in your core business with AI agents – ensuring high compliance and deep integration. • Due Diligence: System and environment analysis with a concrete action plan to facilitate clear decision-making. • AI Portfolio Management: Identifying high-value use cases and linking them to data, compliance and architecture. • System transparency (Helix): Code and runtime analysis using experts and AI agents to accelerate re-engineering. • Team-as-a-Service: Measurable traction through 1 joint team, a 6-month duration and 1 business KPI. • Fixed-price guarantee: Incremental replacement or long-term reliability, with QAware retaining full responsibility for delivery. • A fully independent platform: Open-source agentic layer in the customer’s cloud, with no vendor lock-in. 3
4. Better Assistance and Process Automation with Agentic AI
5. The problem: each domain builds agents on its own. … stacks and tools are constantly changing, hardly any governance. Agents are emerging everywhere. FRONT END Chat · Language · Companion · Workflow Engine Chat, workflow and custom agents are developed in parallel by business departments and IT — with each team using its own models, rules and logging systems. No one has a clear overview. AGENTS UNDER WRITING SALES INPUT MANAGEMENT Risk Agent Consultancy Assistant Classification Agent DAMAGE PROCESSING CUSTOMERS SERVICE Regulatory Agent Dialogue Agent Costs spiralling out of control The LLM overhead per agent, process and team cannot be measured. Each agent has its own guardrails + other domains Safety rules are implemented on a team-by-team basis — or not at all. Are they effective? If there are errors: no traces. Agent actions are not logged or measured centrally. INFRASTRUCTURE DORA · AI Act · GDPR · NIS2 Core systems · Data platform · Services Audit-ready answers are missing when auditors ask questions. The proliferation of unmanaged agents poses a significant risk for companies! If an agent gives the wrong answer tomorrow — who will notice? Who can stop him?
6. Why Guardrails and Policies? Real-World AI Failures. When traditional code fails, it throws an error. When LLMs fail, they speak with total authority. Financial & Legal Liability Regulatory & Safety Catastrophes Air Canada • Legally Binding Speech NYC "MyCity" Chatbot • Illegal Business Advice • Chatbot hallucinated a retroactive bereavement fare policy. • Advised business owners to steal employee tips & break labor laws. • Court Ruling: Tribunal rejected defense; ruled LLM output is legally binding company speech. • RAG Failure: Naive retrieval over 2,000+ pages without factual guardrails. • Takeaway: Chatbots cannot be isolated from enterprise liability. Chevy Dealership • Prompt Injection & Jailbreak Google Bard Launch • $100B Hallucination • Bot claimed JWST took the first exoplanet photos in a public promo video. • Financial Fallout: Alphabet stock dropped ~8-9% in a single day (~$100B market cap). • Takeaway: Lack of automated factual-verification gates on key outputs. • Prompted to agree to sell a $50k 2024 Tahoe for $1.00 USD ("no takesies backsies"). • Takeaway: Excessive agency & zero instruction sanitization. NEDA "Tessa" Bot • Harmful Health Guidance • Recommended calorie restriction to eating disorder patients; forced shutdown. The Core Difference in AI Quality Assurance "When traditional software fails, it throws a 500 Internal Server Error or a NullPointerException. When an LLM fails, it speaks to your customers with absolute authority, makes legally binding promises, tells people to break the law, or knocks billions off your stock price. Testing GenAI isn't an optional polish phase — it is the difference between production reliability and enterprise liability."
8. From uncontrolled proliferation of agents to an Agentic Control Plane with the Agentic Layer FRONT END Chat · Language · Companion · Workflow Engine Agentic Layer. An open-source stack, which makes the control plane a concrete, production-ready solution. AGENTIC LAYER UNDER WRITING SALES INPUT MANAGEMENT DAMAGE PROCESSING CUSTOMERS SERVICE Risk Agent Consultancy Assistant Classification Agent Regulatory Agent Dialogue Agent ACCESS Vendor-neutral Switch between LLM providers or cloud services – without having to modify an agent. CONTROL GUARDRAILS Modular Each component can be used individually — there’s no need to choose one way or the other. Agent Gateway Tool Gateway AI Gateway Guardrails TRANSPARENCY GOVERNANCE OPERATIONS QUALITY Observability Compliance & Audit Runtime Testbench Security & Guardrails Policies are centralised and interchangeable — not hard-coded into every agent. Testability No agent without a passed exam — even after every update, not just at go-live. PLATFORM & INFRASTRUCTURE Kubernetes · Core systems · Data platform · Services Open Source The code and roadmap are in your hands — not the provider’s. The control plane is the blueprint. The agentic layer is the implementation.
9. An AI agent is an autonomous software system that not only answers questions, but also pursues goals independently, makes decisions and carries out actions in order to solve a complex task. It identifies the users objective, works out the best way to achieve it, uses tools independently and, ideally, only gets back to the user once the job is done.
10. The 5 key characteristics of an AI agent Autonomy: It operates largely without human intervention to achieve its objective. It decides for itself how to achieve that objective. Planning & Reasoning: He can break down complex tasks into logical steps, set priorities and adapt strategies if things go wrong. Perception: It ‘sees’ its environment – it can capture and analyse text, data streams, files and directories, or sensory inputs. Tool use: He can actively interact with software. He uses APIs, writes and executes code, uses web browsers and searches databases. Memory: It has short-term memory (so as not to forget the current task) and often long-term memory (to learn from experience and past feedback).
11. AI agents can be built in accordance with the principles of a microservices architecture. AI Agent LLM, LAM, SLM, domain-specific foundation models Prompts, flow control, sub agents GenAI usage, memory logic Tool Server OpenAI Tools (MCP) Integration logic MCP AI Agent Memory … ❏ ❏ ❏ Clear responsibilities Domain specific Highly Distributed … A2A … ?
12. The 5 Zones of Agentic Trust Analyzing trust levels, threat vectors, and feedback loops in AI agents Persists malicious data Modifies goal Zone 1: Input Spreads to neighbouring agents Selects and calls tools Zone 2: Planning Zone 3: Execution Zone 4: Memory Zone 5: A2A Trust: None Trust: None Trust: Task Trust: Uncertain Trust: Peer Agent Every input channel is potentially dangerous, even tool descriptions or other documents. Data and instructions share the same context within the reasoning engine. Every 'tool call' exercises specific privileges on behalf of the agent or the end user. Write operations from past sessions persist as future execution context. Cascading agent systems inherit the trust of the weakest peer input. Results feedback Poisons future sessions Compromised output Vier grüne Häkchen, trotzdem gehackt: Threat Modeling für agentenbasierte KI, Christian Schneider, devSec() 2026
13. OWASP Top 10 for Large Language Model Applications Critical security risks covering the full generative AI and LLM application lifecycle INPUT & DATA SECURITY EXECUTION & AGENCY SUPPLY CHAIN & GOVERNANCE LLM01 Prompt Injection: Attackers manipulate inputs to override behavior or execute unauthorized actions. LLM05 Improper Output Handling: Unvalidated model outputs passed directly to downstream systems. LLM03 Supply Chain Vulnerabilities: Compromised third-party models, packages, plugins, or datasets. LLM02 Sensitive Data Disclosure: Unintended exposure of confidential data, secrets, or PII in outputs. LLM06 Excessive Agency: Granting LLMs unnecessary permissions, tool capabilities, or privileges. LLM09 Misinformation: Hallucinated or inaccurate or compromised outputs relied upon for critical workflows. LLM04 Data & Model Poisoning: Tampering with training data, RAG bases, or embeddings. LLM08 Vector & Embedding Risks: Vulnerabilities in retrieval layers, databases, and vector stores. LLM10 Unbounded Consumption: Resource management failures leading to excessive API costs or DoS. LLM07 System Prompt Leakage: Exposing internal system instructions and operating controls. https://genai.owasp.org/llm-top-10/
14. Open Policy Agent: Policy-as-Code for Cloud Native A CNCF Graduated project providing unified, context-aware policy enforcement. CORE CONCEPT KEY CAPABILITIES Unified Engine: General-purpose policy engine that decouples policy decision-making from application logic. Decoupled Lifecycle: Update authorization policies in real time without redeploying underlying code. Rego Language: High-level, declarative language designed specifically for expressing policy rules over complex JSON/data structures. Context-Aware (ABAC): Evaluates user attributes, agent roles, environment state, and time simultaneously. Deployment Flexibility: Runs as a sidecar, standalone daemon, or embedded library. Fine-Grained Controls: Deep inspection of structured inputs (JSON/REST/gRPC) to validate parameters. CNCF Status: Graduated open-source project, establishing industry standards for compliance. https://www.openpolicyagent.org/ Deterministic Safety: Imposes rigid compliance guarantees on probabilistic actors. EXAMPLE USE CASES STACK Kubernetes Admission: Enforce container security standards and resource limits via Gatekeeper. API & Service Mesh: Offload authorization logic from Envoy proxies and microservices. Agentic AI Guardrails: Intercept autonomous tool calls and validate arguments against policy. CI/CD & IaC: Audit Terraform configurations and deployment manifests before release.
15. Rego: The Policy Language for OPA Declarative fine-grained authorization for AI Agents & Tool Execution package crm.claims_agent import rego.v1 # Allow only ClaimsAgent calls to the CRM tools needed Declarative & High-Level Rego focuses on what queries should return, not how to compute them. It seamlessly handles complex nested JSON structures. allowed_tools := { "crm.get_customer", "crm.search_claims", "crm.add_claim_note", } default allow := false allow if { } Decoupled Policy Logic Moves authorization out of hardcoded agent logic into reviewable, independently testable policy files. Deterministic Guardrails Enforces strict allow/deny decisions over tool names, arguments, user roles, and execution history. input.agent.role == "ClaimsAgent" Hot-Reloadable & Audit-Ready input.tool in allowed_tools Policies update without redeploying code, enabling centralized governance and full compliance tracking.
16. Attribute-Based Access Control (ABAC) Context-aware evaluation of user attributes, agent roles, environment state, and time simultaneously. package authz.abac User & Agent Attributes import rego.v1 Evaluates identity metadata like department, assigned roles, and explicit authority levels (e.g., approval limits). # Dynamic ABAC Policy Rules default allow := false allow if { # User & Agent Role Attributes Environment & Temporal State Incorporates real-time contextual factors such as time of day, network origin, and system operational state. input.user.department == "Finance" input.agent.role == "ClaimsAgent" Fine-Grained Controls # Environment: Business Hours Validates request parameters against specific resource attributes dynamically rather than relying on static permissions. input.env.hour >= 8 input.env.hour <= 18 # Context State Check } input.action == "approve_payout" Deterministic Enforcement input.resource.amount <= input.user.max_approval Provides explicit allow/deny guarantees over complex autonomous AI agent tool calls and workflows.
17. Executing Rego Policies: Local Hooks & Proxies Flexible integration options for agent callbacks, local hooks, and sidecar proxies. WASM IN-MEMORY CLIENT SDK REST RUST BINARY ENVOY PROXY JS / Python / Java Wasm OPA Client SDK wasmtime CLI / Library opa-envoy Sidecar Ecosystem: Node.js, Python, Java Ecosystem: HTTP / REST Service Ecosystem: Binary / Rust Ecosystem: Kubernetes / Cloud Native Best For: High-throughput agent loops, in-process callbacks, and WebAssembly execution. Best For: Calling remote or local OPA daemon instances over standard REST APIs. Best For: CI/CD pipelines, CLI tools, or lightweight container hooks. Best For: Network-level API gateway enforcement and agent traffic control. Key Benefit: Zero network latency. Policies evaluate instantly inside host memory. Key Benefit: Centralized policy engine management with universal language support. Hot reload of policies supported. Key Benefit: Zero external runtime dependencies with ultra-fast cold start times in single-digit ms. Key Benefit: Offloads authorization entirely from application code via gRPC External Authorization filter.
18. High-Level Architecture Overview End-to-end integration of gateways, runtimes, and cross-cutting policies and guardrails Cognigy / Parloa / Voice CROSS-CUTTING Frontend · API Consumers TRANSPARENCY Chat · Workflow Engine · Companions Observability Logs · Metrics Tracing Agent Gateway GUARDRAILS Authentication · Rate limiting · Routing Guardrails Guidelines · Content filters, Policies Agent runtime operator GOVERNANCE Agent A Agent B Agent C ↕ A2A ↕ A2A ↕ A2A Compliance and Audit Immutable logs QUALITY AI Gateway Tool Gateway Costs · Failover · Load Balancing MCP Standard · Actions Testbench Analysis · Regression SECURITY Large language models Tools & APIs OpenAI · Anthropic · Our own models Legacy systems · Data platform Authz IAM, Identity, Certs Agent Gateway: Unified entry point (A2A, OpenAI API, MCP, AG UI) AI Gateway: LLM abstraction – swap models without changing the code Tool Gateway: MCP proxy for controlled tool access Agent Runtime: Kubernetes operator for the agent lifecycle Guardrails: Enforcement of access policies (local or remote) at all three gateways Testbench: Testing agents prior to rollout using Testkube and RAGAS, PromptFoo, et.al Observability: via OpenTelemetry – Logs, Metrics and Traces
19. Zero Trust AI Agents: Beyond Policy Enforcement Securing identity, privileges, runtime environments, and verification across autonomous workloads 1. IDENTITY & AUTH 2. LEAST PRIVILEGE 3. RUNTIME ISOLATION Workload Identity (SPIFFE/SPIRE) Issue cryptographic, short-lived mTLS X.509 SVIDs to agent pods; eliminate static, long-lived API keys. Strict Mutual TLS (Istio/Linkerd) Enforce mTLS for all agent-to-service & agent-to-LLM comms with automatic cert rotation. Distroless & Read-Only Build images with gcr.io/distroless (no shells, package managers) & enforce readOnly root filesystem. Behavior Monitoring (Tetragon/Falco) Use eBPF at kernel level to detect anomalous process spawns or unauthorized network connections. eBPF Egress Filtering (Cilium) Restrict outbound traffic at Layer 7 strictly to authorized external LLM endpoints and tool APIs. Sandboxed Code (gVisor/Kata) Isolate non-deterministic outputs (LLM code interpreters) in kernel-level sandboxes/microVMs. Just-In-Time Secrets (Vault) Inject ephemeral, dynamic credentials into agent context only for tool execution duration. Kernel Security Profiles Enforce strict Seccomp & AppArmor profiles to block unexpected syscalls if hijacked. Immutable Decision Logging Cryptographically log LLM prompts, agent tool calls, and OPA policy decisions into a tamper-proof audit trail for EU AI Act compliance. OAuth 2.0 Token Exchange (RFC 8693): Exchange incoming user JWTs for downscoped, short-lived tokens restricted to the specific tool/service needed. 4. VERIFICATION
20. agentic-layer/ lreimer/opa-ai-agent-guardrails
21. Let´s connect! Mario-Leander Reimer Managing Director mario-leander.reimer@qaware.de QAware GmbH | Aschauer Straße 30 | 81549 München | GF: Dr. Josef Adersberger, Michael Stehnken, Michael Rohleder, Mario-Leander Reimer| Niederlassungen in München, Mainz, Rosenheim, Darmstadt | +49 89 232315-0 | info@qaware.de | www.qaware.de