Von der Idee zur Wirkung: Architektur und Aufbau einer Agentic AI-Plattform #InfoDaysSA

Unternehmen erkennen zunehmend die strategische Bedeutung von Künstlicher Intelligenz (KI), doch die Erstellung und Bereitstellung von KI-Lösungen in großem Maßstab ist mit erheblichen Herausforderungen verbunden. Eine KI-Plattform vereinheitlicht und vereinfacht den gesamten KI-Lebenszyklus. Sie verfügt über einen Cloud-basierten, generischen Kern, der sich um übergreifende Aspekte wie Compliance, MLOps und Monitoring kümmert. Modulare Adapter sorgen für eine nahtlose Integration in die bestehende IT-Infrastruktur.

Dieser Vortrag bietet eine praktische Schritt-für-Schritt-Anleitung für die Architektur und den Aufbau einer unternehmensweiten Plattform. Wir zeigen welche Cloud-native und Open-Source-Technologien dabei helfen, eine hochautomatisierte und skalierbare Umgebung zu schaffen.

Transcript

1. qaware.de From idea to impact. Architecture and Design of an Agentic AI platform Mario-Leander Reimer mario-leander.reimer@qaware.de @LeanderReimer @qaware #CloudNativeNerd #gerneperdude
2. 2 Mario-Leander Reimer Managing Director | CTO @LeanderReimer #cloudnativenerd #qaware #gernperDude
3. Productivity ... as a strategic imperative. A lack of productivity can be recognised by 1. Non-marketable prices 2. Large backlog of unprocessed orders 3. Low conversion rate 4. Low customer and employee satisfaction Shortage of skilled labour due to demographic change and changing work motivators Competition Marketable prices even when new challengers enter the market Automation Replace or extend human labour with automation Assistance Make human labour more productive Optimisation Recognising potential for improvements in existing processes and workflows
4. Better Assistance and Process Automation with Agentic AI
5. Why do we need an AI platform?
6. "According to Gartner, 80% of AI PoCs fail on their way into productive use." https://www.qaware.de/ki-vom-proof-of-concept-poc-zur-entwicklung/
7. The 80% Fallacy of AI projects. 7 QAware Juan Pablo Bottaro, LinkedIn Engineering Blog
8. Key challenges: technology, models and tools, scaling. Source: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2023-generative-ais-breakout-year ▪ Different challenges are seen depending on the maturity of the group ▪ AI newcomers often underestimate the complexity of technologies, models and tools ▪ Production and scaling challenges often hinder production readiness ▪ High cognitive load and lack of expertise are also drivers for failing projects 8
9. "Too much cognitive load will become a bottleneck for fast flow and high productivity for many DevOps teams." Team Topologies: Organizing Business and Technology Teams for Fast Flow
10. Platform engineering is the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era. Platform engineers provide an integrated product most often referred to as an “Internal Developer Platform” covering the operational necessities of the entire lifecycle of an application. https://platformengineering.org/blog/what-is-platform-engineering
11. An example reference architecture for an IDP. Developer Control Plane Integration and Delivery Plane Monitoring and Logging Plane Security Plane IDE Service Catalog / API Catalog Developer Portal Application Source Code Infrastructure & Platform Source Code Observability Secrets & Identity Manager CI Pipeline Registry CD Pipeline Resource Plane Compute Data Integration Networking Platform Orchestrator Certificates & Encryption GitOps https://humanitec.com/reference-architectures
12. qaware.de Agentic AI
13. qaware.de ... and we have the perfect surfboard! The logical continuation: a. From applications to microservices to AI agents b. From on-prem to cloud platforms to AI platforms
14. Micro-Agent GenAI Usage Prompts, Flow control Tools (MCP) Antwort enthält Aufrufe an OpenAI API ❏ Clear responsibility ❏ Vertical in terms of expertise ❏ manageably large ❏ potentially reusable Micro-Agent A2A AI agents will be implemented according to the microservice architecture paradigm. … … … Tool Server Business Logic LLM, LAM, SLM, domain-specific foundation models ? SSE HTTP
15. Our proposal for an AI Platform Reference Architecture
16. Platform Plane Observability Operability Resource Plane Compute Data Integration Security Delivery FinOps Integration & Delivery Plane Quality Plane Data Plane Model Plane Compliance Plane Service Plane User Serving Plane Access Plane / APIs Orchestration Plane Data Modelling Plane
18. Compliance Plane Integration & Delivery Plane Service Plane Platform Plane Operability Resource Plane Compute Data: Local SSD Integration Security Delivery FinOps Quality Plane Data Plane Model Plane User Serving Plane Access Plane Data Modelling Pl.
19. qaware/k8s-native-ai-platform
20. https://www.agentic-layer.ai/ https://www.agentic-layer.ai
21. The technical backbone for smart workloads.
22. agentic-layer/
23. AI native application and system landscapes of tomorrow and thus hypothesis of how companies will change digitally Core system (with data and transaction sovereignty) Other centralised systems A core system as SaaS with low vertical integration and with high standard fidelity ("clean core") & stability. Customised extensions only via individual software. Conversational user interfaces Chat systems as primary user interfaces of employees and customers Data platform (usually data lakehouse technology plus metadata management) AI & automation platform (usually Agentic Automation, GenAI, ML and Process Mining) Development platform (usually DevOps/IDP with cloud technology) The IT becomes a platform provider, which enables business departments to create vertical solutions - while maintaining a high level of compliance, security, production maturity and operational stability. Applications Assistants Agents Customised or ready-made vertical solutions that are developed under the responsibility of business departments as AI-accelerated composable applications on underlying platforms APIs Workflows Analysis Customisations Customisations
24. QAware GmbH | Aschauer Straße 30 | 81549 München | GF: Dr. Josef Adersberger, Michael Stehnken, Michael Rohleder, Mario-Leander Reimer Niederlassungen in München, Mainz, Rosenheim, Darmstadt | +49 89 232315-0 | info@qaware.de The next step? Let's talk! Mario-Leander Reimer Managing Director, CTO mario-leander.reimer@qaware.de +49 151 61314748