Stop Answering Today's Questions with Yesterday's Data: Low-Latency RAG with Kafka and Flink

Delivered at Arc of AI Conference 2026 - Austin, Texas.

Your shiny, new, cutting-edge RAG microservice is only as smart as its context. And if that context is refreshed by a slow, batch-driven job, your AI is essentially answering today’s critical questions by consulting yesterday’s equivalent of a stale newspaper.

It’s time to transition your RAG architecture from batch dependence to streaming certainty. Let’s discuss a “streams-first” approach to building data pipelines with fresh context. We’re using Apache Kafka and Apache Flink to build the always-on knowledge backbone your RAG microservices deserve.

We’ll focus on the foundational engineering practices that guarantee reliability and access to real-time data:

* Kafka as the data substrate: Data streams based on a fault-tolerant, high-throughput source of truth to capture every critical change across your organization.
* Flink’s Real-Time Prep: Leveraging Flink for stateless transformation, stateful contextual enrichment and streamlined chunking—performing the heavy lifting as data arrives.
* Production-Grade Guardrails: Implementing crucial patterns like Exactly-Once Semantics (EOS) for data consistency and establishing a Dead Letter Queue (DLQ) strategy for reliable error handling.

Join this session for a discussion of the core data principles needed to build truly resilient RAG microservices where the knowledge base is always measured in seconds, not days.

Transcript

1. SandonJacobs STOP ANSWERING TODAY’S QUESTIONS WITH YESTERDAY’S DATA
2. SandonJacobs HOW DID WE GET HERE?
4. SandonJacobs LESSONS OF THE LAST DECADE 2016: Microservices 2026: Agentic RAG Engineering Lesson
5. SandonJacobs THE SINGLE SOURCE OF TRUTH VS EVENTUAL CONSISTENCY THE HALLUCINATION WINDOW & THE E-WORD
6. SandonJacobs THE HALLUCINATION WINDOW Time T: Batch Job Compete T+15: Batch job runs T+12: Noted… T+8: Well that’ s interesting T+5: Critical Event T+10: Seems important
7. STUFF IS HAPPENING…
8. AND PEOPLE/SYSTEMS WANNA KNOW ABOUT IT… EVENTS!!!
9. SandonJacobs I HAVE TO ASK NOTIFY ME
10. SandonJacobs WHERE WE’RE GOING This is a DATA STREAMING talk… …as it relates to building Agentic RAG microservices. Apache Ka f ka, as the data substrate. Apache Flink, as the processing engine. Modern microservices (agentic applications) should be event-driven.
11. SandonJacobs SANDON JACOBS Senior Developer Advocate, Con f luent Raleigh, NC 25+ years experience, mostly JVM languages 12+ years in data streaming, dating back to Ka f ka 0.8 Let’s talk about software delivery - development, testing, CI/CD, infrastructure as code Indigenous American (citizen of the Waccamaw-Siouan Tribe of North Carolina)
12. SandonJacobs THE DATA SUBSTRATE
19. SandonJacobs ACTIVE INTELLIGENCE
20. SandonJacobs Real-Time Transformation & Data Cleaning Stateful Stream Processing (Retaining Revelant History) Memory Compacted State Stream-to-Stream Joins Product ID Product ID (Joining Global Context) On-the- f ly Feature Engineering Raw Numbers Temperature Metrics `HeatingAnomaly`: TRUE …from `Data at Rest`… …to `Context in Flight`…
21. SandonJacobs DON’T!!!
22. SandonJacobs CLAIM CHECK PATTERN
23. SandonJacobs CLAIM AND CHUNK Object Storage Smart Chunking Embedding Model Vector DB Normalized Vector Stream
25. SandonJacobs Trend? Anomaly? Spike? Problem? Attack? Healthy? PERSPECTIVE
27. SandonJacobs FROM THINKING TO ACTING
29. SandonJacobs LET’S LAND THE PLANE
30. SandonJacobs EFFECTIVE AGENTS HAVE FRESH DATA AI-READY SERVICES ARE EVENT-DRIVEN
31. SandonJacobs THANKS FOR BEING HERE