Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Keine Panik vor Agenten! Per Anhalter durch die...

Keine Panik vor Agenten! Per Anhalter durch die neue Galaxis der AI Gateways #KI Navigator

„Keine Panik!“ – Dieser Rat von Douglas Adams ist heute wichtiger denn je. Während unsere API-Welt bisher streng deterministisch war, betreten wir nun eine Galaxis, die probabilistisch, kontexthungrig und agentengetrieben ist. Dein alter „Traffic Cop“ (das klassische API Gateway) ist mit dieser neuen Art der Kommunikation schlicht überfordert.

In dieser Session nehmen wir dich mit auf eine Reise durch die Galaxis der AI Gateways. Wir diskutieren, warum wir für die Ära der generativen KI mehr als nur andere Infrastruktur Komponenten brauchen, sondern radikal neue Protokolle und Cross-Cutting Concerns wichtig sind: während MCP (Model Context Protocol) den universellen Zugriff auf Daten regelt, ermöglichen Standards wie A2A und UCP die autonome Kollaboration zwischen Agenten, und A2UI sorgt dafür, dass auch die Benutzeroberfläche dynamisch generiert werden kann.

Doch wie lässt sich dieser „Zoo“ aus Protokollen sicher orchestrieren? Hier kommt das AI Gateway ins Spiel. In packenden Live-Demos demonstrieren wir dir, wie wir mit modernen Gateways mittels Semantic Caching Latenzen eliminieren, Kosten durch Token-Rate-Limiting senken und kritische Unternehmensdaten durch PII-Redaction schützen. Wir präsentieren dir Architekturen, mit denen wir Legacy-APIs sofort „agent-ready“ machen, ohne eine einzige Zeile Backend-Code anzupassen.

Komm mit uns auf die Reise und mach deine APIs bereit für die KI-Revolution – und vergiss dein Handtuch nicht!

M.-Leander Reimer PRO

November 18, 2026

Transcript

25 slides
  1. Legacy De-Risking Agentic Automation Make legacy systems and infrastructure gradually

    more understandable, reduce risks and modernise them sensibly. Boost productivity in your core business with AI agents – ensuring high compliance and deep integration. • Due Diligence: System and environment analysis with a concrete action plan to facilitate clear decision-making. • AI Portfolio Management: Identifying high-value use cases and linking them to data, compliance and architecture. • System transparency (Helix): Code and runtime analysis using experts and AI agents to accelerate re-engineering. • Team-as-a-Service: Measurable traction through 1 joint team, a 6-month duration and 1 business KPI. • Fixed-price guarantee: Incremental replacement or long-term reliability, with QAware retaining full responsibility for delivery. • A fully independent platform: Open-source agentic layer in the customer’s cloud, with no vendor lock-in. 3
  2. The problem: each domain builds agents on its own. …

    stack and tools constantly changing, hardly any governance. Agents are emerging everywhere. FRONT END Chat · Language · Companion · Workflow Engine Chat, workflow and custom agents are developed in parallel by business departments and IT — with each team using its own models, rules and logging systems. No one has a clear overview. AGENTS UNDER WRITING SALES INPUT MANAGEMENT Risk Agent Consultancy Assistant Classification Agent DAMAGE PROCESSING CUSTOMERS SERVICE Regulatory Agent Dialogue Agent Costs spiralling out of control The LLM overhead per agent, process and team cannot be measured. Each agent has its own guardrails + other domains Safety rules are implemented on a team-by-team basis — or not at all. Are they effective? If there are errors: no traces. Agent actions are not logged or measured centrally. INFRASTRUCTURE DORA · AI Act · BaFin · GDPR Core systems · Data platform · Services Audit-ready answers are missing when auditors ask questions. The proliferation of unmanaged agents poses a significant risk for companies! If an agent gives the wrong answer tomorrow — who will notice? Who can stop him?
  3. An AI agent is an autonomous software system that not

    only answers questions, but also pursues goals independently, makes decisions and carries out actions in order to solve a complex task. It identifies the users objective, works out the best way to achieve it, uses tools independently and, ideally, only gets back to the user once the job is done.
  4. AI agents can be built in accordance with the principles

    of a microservices architecture. Microservice Principles AI Agent Clear Responsibilities Prompts, flow control, sub agents Single focused domain scope per agent service. Domain Specific Tailored models, memory & business tools. LLM, LAM, SLM, domain-specific foundation models GenAI usage, memory logic OpenAI Tool Server Tools (MCP) Manageable Size Integration logic MCP Modular code, small & maintainable prompts. AI Agent ... Highly Distributed Decoupled inter-agent communication (A2A). ... Memory A2A ... ?
  5. Quick History of Inter Process Communication (IPC) RPC Oct 1983

    Birrel und Nielson HTTP/1.0 Mai 1996 RFC 1945 XML-RPC 1998 DCOM 18.09.1996 Win95 CORBA 3.0 July 2002 HTTP/1.1 Juni 1999 RFC 2616 GraphQL Sep 2015 RESTful Applications 2014 (?) gRPC 1.0 Aug 2016 2025 RPC 14.01.1976 RFC 707 CORBA 2.0 August 1996 CORBA 1.0 Oct 1991 CORBA 2.3 Juni 1999 Java RMI Feb 1997 JDK 1.1 REST 2000 by Roy T. Fielding JSON-RPC v1 2005 HTTP/2.0 v2 2010 Mai 2015 RFC 7540 SOAP 1.2 2003 A2A MCP
  6. MCP is the USB-C port for AI applications. MCP Host

    (AI Application) MCP Client A Dedicated Connection MCP Client B Dedicated Connection MCP Server A MCP Server B Local (e.g. CLI, Disk) Remote (e.g. CRM) modelcontextprotocol/modelcontextprotocol Data Layer • Lifecycle Management: Manages connection setup, negotiation, and teardown. • Server Features: Provides tools, resources, and prompt templates to clients. • Client Features: Allows server to trigger client actions (e.g. user input prompts). • Utility Features: Supports real-time notifications and progress tracking. Transport Layer • Stdio transport: Uses standard I/O for fast, direct local inter-process communication. • Streamable HTTP transport: Uses HTTP POST with Server-Sent-Events (SSE) for streaming.
  7. Options for OpenAPI-to-MCP conversion Framework-based (e.g. FastMCP): A framework for

    the rapid transformation of OpenAPI specifications into executable MCP servers with minimal coding effort. Configuration-based (e.g. openapi-to-mcpserver): Generates YAML/JSON configuration files for MCP servers directly from OpenAPI specifications. Ideal for declarative interface definitions. Template overrides allow flexible injection of authentication headers (e.g. OAuth, API keys) and route mapping, as well as composition. Generator-based (e.g. API to MCP Converter VS Code extension): A development tool that scans local repositories for API definitions and generates MCP wrapper code. System integration via MCP presents a number of challenges that must be taken into account!
  8. Without A2A, integrating different agents presents further challenges. Standardized Communication:

    JSON-RPC 2.0 over HTTP(S). Agent Discovery: "Agent Cards" describe capabilities and connection details. Flexible Interaction: Supports synchronous request/response, streaming (SSE) and asynchronous push notifications. Rich Data Exchange: Processes text, files and structured JSON data. Enterprise-Ready: Takes security, authentication and auditability into account.
  9. Proper management of Agent, LLM and MCP APIs right from

    the start is essential to prevent the Big Ball of AI Mud . https://thenewstack.io/history-service-mesh/
  10. The Triple Gateway Architecture API Consumer / Client Layer 

      Agent Gateway Tool Gateway AI Gateway Controls access to AI Agents Controls access to MCP Server Controls access to LLM Backends Backend Infrastructure & Services
  11. Option A - API Management Solutions ▪ Fully-fledged API management

    solutions: Kong, Tyk, Mulesoft, 3scale, Gravitee.io, Apigee, … ▪ Make sense for enterprise use cases. Often expensive. Some offer stripped-down open-source versions. ▪ Offer additional features such as API catalogues, developer portals, API key management, payment options or integration with EAM solutions ▪ Usually deployed and operated centrally for all agents and applications within the organisation ▪ The 80/20 rule applies (as with any COTS product).
  12. Option B - Build Your Own Gateway ▪ There are

    numerous frameworks to choose from here: PortKey, LiteLLM, Bifrost, MFflow, Spring AI, Quarkus, … ▪ This option offers maximum flexibility – anything you can program is possible! ▪ However, the team has to develop, maintain and operate the gateway. That involves a lot of work and carries risks!
  13. Option C - AI-Native & Cloud-native Gateways ▪ A wide

    range of options: Traefik AI, Envoy AI, KrakenD AI, AISIX, Bifrost, PortKey AI, kgateway, Higress, agentgateway, LiteLLM, ... ▪ Open source with an additional enterprise offering ▪ Lightweight and often easy to set up and use ▪ The supported functions differ, in some cases significantly ▪ However, they can often be extended using custom plugins ▪ Offer deep integration with cloud-native platforms (K8s) and cloud providers
  14. High-Level Architecture Overview End-to-end integration of gateways, runtimes, and cross-cutting

    policies and guardrails CROSS-CUTTING Frontend · API Consumers TRANSPARENCY Chat · Workflow Engine · Companions Observability Logs · Metrics Tracing Agent Gateway GUARDRAILS Authentication · Rate limiting · Routing Guardrails Guidelines · Content filters, Policies Agent runtime operator GOVERNANCE Agent A Agent B Agent C ↕ A2A ↕ A2A ↕ A2A Compliance and Audit Immutable logs QUALITY AI Gateway Tool Gateway Costs · Failover · Load Balancing MCP Standard · Actions Testbench Analysis · Regression SECURITY Large language models Tools & APIs OpenAI · Anthropic · Our own models Legacy systems · Data platform Authz IAM, Identity, Certs Agent Gateway: Unified entry point (A2A, OpenAI API, MCP, AG UI) AI Gateway: LLM abstraction – swap models without changing the code Tool Gateway: MCP proxy for controlled tool access Agent Runtime: Kubernetes operator for the agent lifecycle Guardrails: Enforcement of access policies (local or remote) at all three gateways Testbench: Testing agents prior to rollout using Testkube and RAGAS, PromptFoo, et.al Observability: via OpenTelemetry – Logs, Metrics and Traces
  15. From uncontrolled proliferation of agents to an Agentic Control Plane

    with the Agentic Layer FRONT END Chat · Language · Companion · Workflow Engine Agentic Layer. An open-source stack, which makes the control plane a concrete, production-ready solution. AGENTIC LAYER UNDER WRITING SALES INPUT MANAGEMENT DAMAGE PROCESSING CUSTOMERS SERVICE Risk Agent Consultancy Assistant Classification Agent Regulatory Agent Dialogue Agent ACCESS Vendor-neutral Switch between LLM providers or cloud services – without having to modify an agent. CONTROL GUARDRAILS Modular Each component can be used individually — there’s no need to choose one way or the other. Agent Gateway Tool Gateway AI Gateway Guardrails TRANSPARENCY GOVERNANCE OPERATIONS QUALITY Observability Compliance & Audit Runtime Testbench Security & Guardrails Policies are centralised and interchangeable — not hard-coded into every agent. Testability No agent without a passed exam — even after every update, not just at go-live. PLATFORM & INFRASTRUCTURE Kubernetes · Core systems · Data platform · Services Open Source The code and roadmap are in your hands — not the provider’s. The control plane is the blueprint. The agentic layer is the implementation.
  16. Let´s connect! Mario-Leander Reimer Managing Director [email protected] QAware GmbH |

    Aschauer Straße 30 | 81549 München | GF: Dr. Josef Adersberger, Michael Stehnken, Michael Rohleder, Mario-Leander Reimer| Niederlassungen in München, Mainz, Rosenheim, Darmstadt | +49 89 232315-0 | [email protected] | www.qaware.de

More Decks by M.-Leander Reimer

Other Decks in Technology