Semantic AI Language & Embedding Models hand-in-hand Christian Weyer | Co-Founder & CTO | Thinktecture AG | [email protected] 

Semantic AI Language & Embedding Models hand-in-hand LLM- ALL-THE-THINGS? 

Language Models understand and generate semantically rich human language, transforming it into text or structured data for both humans and machines. ⚠ Non-deterministic: same input can lead to different outputs. Embedding Models capture semantic meaning by encoding human language into numerical vector representations, facilitating understanding, comparison, and retrieval for both humans and machines. ✅ Deterministic: same input always results in the same embedding. Semantic AI Language & Embedding Models hand-in-hand 🫱 🫲 Semantic AI Generative AI 3 

Semantic AI Language & Embedding Models hand-in-hand MODELS FOR OUR SOFTWARE 

Semantic AI Language & Embedding Models hand-in-hand Classical applications & UIs 5 

Semantic AI Language & Embedding Models hand-in-hand Language-enabled “UIs” 6 

Semantic AI Language & Embedding Models hand-in-hand SEMANTIC AI PATTERN LOCAL RAG [RETRIEVAL-AUGMENTED GENERATION] 

Semantic AI Language & Embedding Models hand-in-hand “Talk to your data” Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Results Question Answ er LLM Embedding Model Embedding Model 💡 Indexing / Embedding Question Answering .md, .docx, .pdf etc. “Lorem ipsum…?” 💡 Vector DB 8 

§ Frameworks § LangChain § Fastembed § Lightweight & efficient for generating text embeddings § Embedding model § jinaai/jina-embeddings-v2-base-de (local, no GPU required) § Vector store § PostgreSql (pgvector) vector store § LLM/SLM § Llama 3.3 70B on Cerebras (very fast) Semantic AI Language & Embedding Models hand-in-hand Technical implementation – Local RAG 9 

Semantic AI Language & Embedding Models hand-in-hand SEMANTIC AI PATTERN STRUCTURED OUTPUT 

Semantic AI Language & Embedding Models hand-in-hand Structured data from unstructured input – e.g. for API calling “OK, when is my colleague CW available for a two- days workshop?” System Prompt (with employee data) + Schema / Function Calling (for structured output) Web API Availability business logic 11 

§ Frameworks § Pydantic § Instructor § Methodology § Schema with JSON Mode (or Function Calling) § SLM/LLM § Llama 3.3 70B on Cerebras (very fast) Semantic AI Language & Embedding Models hand-in-hand Technical implementation – Structured Output 12 

Semantic AI Language & Embedding Models hand-in-hand SEMANTIC AI PATTERN SEMANTIC GUARDING & ROUTING 

Semantic AI Language & Embedding Models hand-in-hand Semantics-based decisions for user interactions Guarding (e.g. prompt injection) Routing (selecting correct target) “Lorem ipsum…?” Target RAG 1 Target Structured Output & API Call Target … something else … Fine-tuned Language Model Embedding Model 14 

Guarding § Frameworks § llm-guard § HuggingFace Transformers § Model § deepset/deberta-v3-base- injection (local, no GPU required) Routing § Frameworks § semantic-routing § Fastembed § Embedding model § BAAI/bge-small-en-v1.5 (local, no GPU required) § Vector store § PostgreSql (pgvector) Semantic AI Language & Embedding Models hand-in-hand Technical implementation – Semantic Guarding & Routing 15 

Language Models Embedding Models Semantic AI Language & Embedding Models hand-in-hand 🫱 🫲 Semantic AI 16 

Thank you! Christian Weyer https://thinktecture.com/christian-weyer [email protected] 17 

§ Technology catalyst § AI-powered solutions § Pragmatic end-to-end architectures § Microsoft Regional Director § Microsoft MVP for AI § Google GDE for Web AI [email protected] https://www.thinktecture.com Semantic AI Language & Embedding Models hand-in-hand Christian Weyer Co-Founder & CTO @ Thinktecture AG 18 

Semantic AI Language & Embedding Models hand-in-hand END-TO-END SOLUTION ILLUSTRATED 19 

Semantic routing Semantic AI Language & Embedding Models hand-in-hand "Talk to your systems"(for Availability info) Web App / Watch App Speech-to-Text Internal Gateway (Python FastAPI) LLM / SLM Text-to-Speech Transcribe spoken text Transcribed text Check for experts availability with text Extract { experts, booking times } from text Structured JSON data (Function calling) Generate response with availability Response Response with experts availability 🔉 Speech-to-text for response Response audio Internal Business API (node.js – veeeery old) Query Availability API Availability When is CL…? CL will be… 20