IT-Tage 2024: LLMs in der Praxis: Top Patterns & Lösungen für nahtlose Integration

Erfahrungen aus der Generative-AI-Praxis: In diesem Vortrag präsentiert Christian Weyer konkrete Patterns & Lösungen für die Integration von Large Language Models (LLMs) in eigene Softwarelösungen. Wichtige Themen wie Semantic Routing, RAG, Structured Output oder Observability werden mit Code-Beispielen illustriert. Es erwartet Entwickler und Architekten ein pragmatischer Einblick zur Umsetzung in eigenen Projekten.

Transcript

1. LLMs in der Praxis: Top Patterns & Lösungen für nahtlose Integration Christian Weyer | Co-Founder & CTO | Thinktecture AG | christian.weyer@thinktecture.com
2. Language Models Empower software to understand and generate semantically rich human language, transforming it into text or structured data for both humans and machines. Embedding Models Enable software to capture semantic meaning by encoding human language into numerical vector representations, facilitating understanding, comparison, and retrieval for both humans and machines. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration 2 🫱 🫲
3. § Language models are always part of end-to-end architectures § Client apps (Web, desktop, mobile, etc.) § Services with APIs § Databases § etc. § LLMs / SLMs enable human language as a first-class citizen 🤯 § Extending access to our software LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration End-to-end with LLMs & SLMs 3 Clients Services LLMs Desktop Web Mobile Service A Service B Service C API Gateway Monitoring LLM 1 LLM 2
4. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Classical UIs – strong UX for certain use cases 4
5. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration EXTENDED END-TO-END SOLUTIONS 5
6. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN SEMANTIC GUARDING & ROUTING 6
7. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Semantics-based decisions for user queries 7 Guarding (e.g. prompt injection) Routing (selecting correct target) “Lorem ipsum…?” Semantic Engine (Embedding Model, Fine-tuned Language Model) Target RAG 1 Target Structured Output & API Call Target RAG 2
8. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN RAG (RETRIEVAL-AUGMENTED GENERATION) 8
9. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Talk to your data Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Results Question Answ er LLM 9 Embedding Model Embedding Model 💡 Indexing / Embedding Question Answering .md, .docx, .pdf etc. “Lorem ipsum…?” 💡 Vector DB
10. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN STRUCTURED OUTPUT 10
11. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Structured data from unstructured input for API calling 11 “OK, when is my colleague CW available for a two- days workshop?” System Prompt (with employee data) + Schema / Function Calling (for structured output) (Internal) (Web) API
12. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN OBSERVABILITY 12
13. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Getting insights: Traces & more 13
14. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration END-TO-END SOLUTION ILLUSTRATED 14
15. Semantic routing LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Talk to your systems 15 Web App / Watch App Speech-to-Text Internal Gateway (Python FastAPI) LLM / SLM Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data (Function calling) Generate response with availability Response Response with experts availability 🔉 Speech-to-text for response Response audio Internal Business API (node.js – veeeery old) Query Availability API Availability When is CL…? CL will be…
16. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Recap: Top patterns & solutions 16 RAG (Retrieval-Augmented Generation) Structured Output Semantic Guarding & Routing Observability
17. Thank you! Christian Weyer https://thinktecture.com/christian-weyer christian.weyer@thinktecture.com
18. § Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for AI § Google GDE for Web Technologies christian.weyer@thinktecture.com @christianweyer https://www.thinktecture.com LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Christian Weyer Co-Founder & CTO @ Thinktecture AG 18