Semantic AI Why Embeddings Might Matter More Than LLMs Christian Weyer | Co-Founder & CTO | Thinktecture AG | [email protected] 

Semantic AI Why Embeddings Might Matter More Than LLMs AI MODELS FOR OUR SOFTWARE 2 

Semantic AI Why Embeddings Might Matter More Than LLMs Classical applications & UIs 3 API-based data Document-based data 

Semantic AI Why Embeddings Might Matter More Than LLMs Language-enabled “UIs” – Talk-to-TT 4 

Semantic AI Why Embeddings Might Matter More Than LLMs LLM ALL-THE-THINGS? 5 

Language Models understand and generate semantically rich human language, transforming it into text or structured data for both humans and machines. ⚠ Non-deterministic: same input can lead to different outputs. Embedding Models capture semantic meaning by encoding human language into numerical vector representations, facilitating understanding, comparison, and retrieval for both humans and machines. ✅ Deterministic: same input always results in the same embedding. Semantic AI Why Embeddings Might Matter More Than LLMs 6 🫱 🫲 Semantic AI Generative AI 

Semantic AI Why Embeddings Might Matter More Than LLMs SCENARIO LIGHTWEIGHT RAG [RETRIEVAL-AUGMENTED GENERATION] 7 

Semantic AI Why Embeddings Might Matter More Than LLMs Talking to documents (Retrieval-augmented generation) Cleanup & Split Text Embedding Question Text Embedding Save Query Relevant Results Question Answ er w / sources LLM Embedding Model Embedding Model 💡 Indexing / Embedding Question Answering .md, .docx, .pdf etc. “What should I do…?” Vector DB 8 

§ Python Frameworks § LangChain § FastEmbed § Lightweight & efficient for generating text embeddings § Embedding model § jinaai/jina-embeddings-v2-base-de (local, no GPU req) – 768 dims § Vector store § PostgreSql (pgvector) vector store § LLM/SLM § Llama 3.3 70B on Cerebras (very fast) Semantic AI Why Embeddings Might Matter More Than LLMs Technical implementation – Lightweight RAG 9 

Semantic AI Why Embeddings Might Matter More Than LLMs SCENARIO STRUCTURED OUTPUT 10 

§ Tools integration is being standardized with MCP Semantic AI Why Embeddings Might Matter More Than LLMs Talking to APIs (Function / Tool calling) 11 “When is CW available for a two-days workshop?” System Prompt (+ employee data) + Schema (for structured output) Web API Availability business logic 

§ Python Frameworks § Pydantic § Instructor § Methodology § Schema with JSON Mode (opt. Function Calling) § SLM / LLM § Llama 3.3 70B on Cerebras (very fast) Semantic AI Why Embeddings Might Matter More Than LLMs Technical implementation – Structured Output 12 

Semantic AI Why Embeddings Might Matter More Than LLMs SCENARIO SEMANTIC GUARDING & ROUTING 13 

Semantic AI Why Embeddings Might Matter More Than LLMs Semantics-based decisions for user interactions Guarding (e.g. prompt injection) Routing (selecting correct target) “Lorem ipsum…?” Target RAG Target API Call Target … something else … Fine-tuned Language Model Embedding Model 14 

Guarding § Python Frameworks § llm-guard § HuggingFace Transformers § NLP / NLU model § deepset/ deberta-v3-base-injection (local, no GPU req.) Routing § Python Frameworks § semantic-routing § FastEmbed § Embedding model § intfloat/ multilingual-e5-large (local, no GPU req.) – 1024 dims § Vector store § PostgreSql (pgvector) Semantic AI Why Embeddings Might Matter More Than LLMs Technical implementation – Semantic Guarding & Routing 15 

Semantic AI Why Embeddings Might Matter More Than LLMs RECAP 16 

Model type Core function Output type Embeddings role & relevance Embedding Models Encode meaning as vectors Deterministic vectors Embeddings are the product Language Models (SLMs / LLMs) Generate, transform, and understand language Human-readable text or structured data Embeddings are the foundation NLP / NLU Models Classify, decide, or detect patterns Labels, scores, decisions Embeddings are the backbone Semantic AI Why Embeddings Might Matter More Than LLMs Models, models, models – not without embeddings 17 

Thank you! Christian Weyer [email protected]