LLMs in der Praxis: Top Patterns & Lösungen für Integration in Ihre Anwendungen Christian Weyer | Co-Founder & CTO | Thinktecture AG | [email protected] 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen Our journey 2 Lightweight RAG Structured Output / Tool Calling AI models for your software Semantic Routing Observability 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen AI MODELS FOR YOUR SOFTWARE 3 

Language Models understand and generate semantically rich human language, transforming it into text or structured data for both humans and machines. ⚠ Non-deterministic: same input can lead to different outputs. Embedding Models capture semantic meaning by encoding human language into numerical vector representations, facilitating understanding, comparison, and retrieval for both humans and machines. ✅ Deterministic: same input always results in the same embedding. LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen 4 🫱 🫲 

§ Language & embedding models part of end-to-end architectures § Embedding models can be run locally § Optimized for CPU § Language models still hard to run locally § High GPU power § High VRAM § High memory bandwidth LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen API-based AI model integrations 5 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen Classical applications & UIs 6 API-based data Document-based data 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen Language-enabled “UIs” – Talk-to-TT 7 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen 8 C4 system context diagram § Various tech stacks § Docker-based distributed system 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen PATTERN LIGHTWEIGHT RAG [RETRIEVAL-AUGMENTED GENERATION] 9 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen “Talk to your Data” Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Results Question Answ er w / sources LLM Embedding Model Embedding Model 💡 Indexing / Embedding Question Answering .md, .docx, .pdf etc. “Lorem ipsum…?” 💡 Vector DB 10 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen PATTERN STRUCTURED OUTPUT 11 

§ Integration is being standardized with MCP LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen Structured data from unstructured input For calling APIs / tools 12 “OK, when is my colleague CW available for a two- days workshop?” System Prompt (with employee data) + Schema / Function Calling (for structured output) (Internal) Web API Availability business logic 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen PATTERN SEMANTIC ROUTING 13 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen Semantics-based decisions for user interactions Guarding (e.g. prompt injection) Routing (selecting correct target) “Lorem ipsum…?” Target RAG Target API Call Target … something else … Fine-tuned Language Model Embedding Model 14 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen PATTERN / SOLUTION OBSERVABILITY 15 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen Things can get… overwhelming 16 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen End-to-end tracing 17 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen 18 AI solutions are 10% AI and 100% software engineering. 18 

LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen YOUR journey 19 AI Models for your software Lightweight RAG Semantic Routing Observability Structured Output / Tool Calling 

Thank you! Christian Weyer https://thinktecture.com/christian-weyer [email protected] 20 

§ Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft MVP for AI § Google GDE for Web AI § [email protected] § https://www.thinktecture.com LLMs in der Praxis Top Patterns & Lösungen für Integration in Ihre Anwendungen Christian Weyer Co-Founder & CTO @ Thinktecture AG 21