Modular RAG Architectures with Java and Spring AI

Retrieval-Augmented Generation (RAG) is a foundational approach to enhancing LLMs by integrating them with targeted, domain-specific data. This session will provide a hands-on guide to designing and implementing modular RAG architectures using Java and Spring AI. Through practical examples, you will gain insights into building production-ready RAG workflows that support both current and future use cases.

The session will cover five key stages:
- Data Indexing: Methods for structuring, chunking, and indexing documents, improving the efficiency and accuracy of downstream retrieval tasks.
- Query Analysis: Techniques for transforming and enhancing queries in the pre-retrieval phase to improve relevance and precision during retrieval.
- Retrieval: Strategies for sourcing data from multiple repositories, using agent-based routing to select relevant data sources, and combining, re-ranking, or filtering results to deliver the most contextually relevant information for generation.
- Augmentation: Methods for augmenting user prompts with retrieved content, including advanced techniques for contextualization, citation, and summarization.
- Evaluation: Metrics and evaluation methods to assess document relevance and model output accuracy, as well as iterative feedback loops for refining RAG workflows to meet quality standards.
Additionally, the session will address important considerations for observability and developer experience in building RAG workflows.

Transcript

1. Thomas Vitale Spring I/O 23rd May 2025 Modular RAG Architectures with Java and @thomasvitale.com
2. Systematic Thomas Vitale @thomasvitale.com
3. Application Inference Service Consum es LLM s Architecture Database Reads/writes data Observability Platform Exports telem etry Spring Boot Application Vaadin Spring AI Arconia @thomasvitale.com
4. Arconia Dev Services and OpenTelemetry arconia dev gradle bootRun mvn spring-boot:run @thomasvitale.com arconia.io
5. Chatbot @thomasvitale.com “Legacy software companies adding an AI chatbot to their product" @andykreed
6. Sequential RAG @thomasvitale.com
7. Prompt Stuffing Inference Service Request Response Augmenting Prompts with Context Answer Application @thomasvitale.com Question Context
8. Retrieval Augmented Generation Inference Service Request Response Question Answer Application Augment with Context Web Search Engines Search Engine HTTP API @thomasvitale.com Context
9. Sequential RAG Retrieval Response Query Augmented Query @thomasvitale.com Generation Naive
10. Ingestion Pipeline @thomasvitale.com Data Preparation Document Reader Reads from Writes To Document Transformer Document Writer
11. Retrieval Augmented Generation Inference Service Request Response Question Answer Application Augment with Context Vector Stores Vector Store Semantic Search @thomasvitale.com
12. Sequential RAG @thomasvitale.com Advanced Pre- Retrieval Query Retrieval Generation Response
13. Sequential RAG Retrieval @thomasvitale.com Generation Advanced Pre- Retrieval Query Response Post- Retrieval
14. Branching RAG @thomasvitale.com
15. Join Branching RAG @thomasvitale.com Post-Retrieval Join Generation Response Retrieval Retrieval Retrieval Expand Pre- Retrieval Query
16. Conditional RAG @thomasvitale.com
17. Conditional RAG @thomasvitale.com Query Routing Generation Response Retrieval routes to Generation Response Retrieval routes to Generation Response Retrieval routes to Router Query
18. Tools @thomasvitale.com
19. Retrieval Augmented Generation Inference Service Request Response Question Answer Application Augment with Context Prompt Augmentation with Retrieved Context Source Query @thomasvitale.com
20. Tools Inference Service Request Tool Calling Question Response Answer Application API Tool Call Tool Execution Tool Call Request Tool Call Response @thomasvitale.com
21. Agentic RAG @thomasvitale.com
22. Agentic RAG @thomasvitale.com Orchestration Query Response Generation Retrieval Tool 2 uses Generation Retrieval Tool 1 uses LLM uses Agent
23. 1979 IBM
24. Modular RAG https://github.com/ThomasVitale/modular-rag https://github.com/ThomasVitale/llm-apps-java-spring-ai Thomas Vitale @thomasvitale.com thomasvitale.com Architectures with Java and