LLMs in der Praxis: Top Patterns & Lösungen für nahtlose Integration Christian Weyer | Co-Founder & CTO | Thinktecture AG | [email protected]

Language Models Empower software to understand and generate semantically rich human language, transforming it into text or structured data for both humans and machines. Embedding Models Enable software to capture semantic meaning by encoding human language into numerical vector representations, facilitating understanding, comparison, and retrieval for both humans and machines. LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration 2 🫱 🫲

§ Language models are always part of end-to-end architectures § Client apps (Web, desktop, mobile, etc.) § Services with APIs § Databases § etc. § LLMs / SLMs enable human language as a first-class citizen 🤯 § Extending access to our software LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration End-to-end with LLMs & SLMs 3 Clients Services LLMs Desktop Web Mobile Service A Service B Service C API Gateway Monitoring LLM 1 LLM 2

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Classical UIs – strong UX for certain use cases 4

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration EXTENDED END-TO-END SOLUTIONS 5

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN SEMANTIC GUARDING & ROUTING 6

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Semantics-based decisions for user queries 7 Guarding (e.g. prompt injection) Routing (selecting correct target) “Lorem ipsum…?” Semantic Engine (Embedding Model, Fine-tuned Language Model) Target RAG 1 Target Structured Output & API Call Target RAG 2

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN RAG (RETRIEVAL-AUGMENTED GENERATION) 8

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Talk to your data Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Results Question Answ er LLM 9 Embedding Model Embedding Model 💡 Indexing / Embedding Question Answering .md, .docx, .pdf etc. “Lorem ipsum…?” 💡 Vector DB

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN STRUCTURED OUTPUT 10

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Structured data from unstructured input for API calling 11 “OK, when is my colleague CW available for a two- days workshop?” System Prompt (with employee data) + Schema / Function Calling (for structured output) (Internal) (Web) API

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration PATTERN OBSERVABILITY 12

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Getting insights: Traces & more 13

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration END-TO-END SOLUTION ILLUSTRATED 14

Semantic routing LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Talk to your systems 15 Web App / Watch App Speech-to-Text Internal Gateway (Python FastAPI) LLM / SLM Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data (Function calling) Generate response with availability Response Response with experts availability 🔉 Speech-to-text for response Response audio Internal Business API (node.js – veeeery old) Query Availability API Availability When is CL…? CL will be…

LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Recap: Top patterns & solutions 16 RAG (Retrieval-Augmented Generation) Structured Output Semantic Guarding & Routing Observability

Thank you! Christian Weyer https://thinktecture.com/christian-weyer [email protected]

§ Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for AI § Google GDE for Web Technologies [email protected] @christianweyer https://www.thinktecture.com LLMs in der Praxis Top Patterns & Lösungen für nahtlose Integration Christian Weyer Co-Founder & CTO @ Thinktecture AG 18