Cape Town 2025 RAG vs Fine-Tuning: Choosing the Right Approach for Your GenAI Apps Abed Matini Senior Backend Developer, Ogilvy 

Agenda ● What is Fine-Tuning? What is RAG? ● Search in RAG: Keyword, Semantic, Hybrid ● Live Demo ● RAG vs. Fine-Tuning ● Advanced RAG Techniques 

Why vanilla LLMs fall short ● Knowledge cutoff; no access to private data ● Hallucinations and lack of citations ● Compliance, auditability, and governance needs ● Rapidly changing information ● Cost and maintenance in real apps 

LLMs are already powerful What LLMs don’t know ✓ Summarizing text × Private databases and internal docs ✓ Generating and rewriting content × Hard-to-access or siloed information ✓ Code completion and transformation × Real-time updates and fresh content ✓ Reasoning (with limits) × Organization-specific terminology & style 

What is Fine-Tuning? Fine-tuning involves further training a pre-trained LLM on a taskspecific dataset, fine-tuning helps it specialize in a particular area. • Requires labeled data and training infrastructure 

Fine-Tuning Key Benefits ● Further trains model weights for specific tasks or styles ● Improves consistency, structured outputs, domain vocabulary ● Improved accuracy, enhanced specificity, ● Reduced hallucinations & bias. 

What is RAG? Retrieval-Augmented Generation (RAG) is a framework that enhances LLMs by connecting them to external knowledge sources. 

RAG Pipeline Overview Response + Citations Prompt Retriever Augmented Prompt Knowledge Base LLM 

RAG Key Benefits ● Access to up-to-date information ● Reduce hallucinations ● Ability to use private or domain-specific data ● Transparent source attribution (Grounded) ● Cost-effective (no retraining needed) 

RAG Components LLM Knowledge Base Generates answers from the augmented prompt The content pool: docs, sites, databases, APIs. Retriever Searches the knowledge base to fetch the most relevant context for a query. 

RAG Workflow Chunking Embedding Storing • Ingest and chunk content (overlap/semantic) • Embed chunks (choose embedding model) • Store embeddings (vector DB) • Retrieve top_k relevant chunks • Rerank or compress context • Generate answer with citations Retrieval Reranking Generation 

No content

No content

RAG Workflow Chunking Embedding Storing Retrieval Reranking Generation Break the source content into smaller, manageable sections for easier processing. 

RAG Workflow Chunking Embedding Storing Retrieval Reranking Generation Convert each section into a numerical representation that captures its meaning. 

RAG Workflow Chunking Embedding Storing Retrieval Reranking Generation Save these embeddings in a vector database for efficient semantic search. 

RAG Workflow Chunking Embedding Storing Retrieval Reranking Find and return the most relevant sections based on a query. Knowledge base Generation 

RAG Workflow Chunking Embedding Storing Retrieval Reranking Reorder the retrieved sections to prioritize the most relevant ones. Generation 

RAG Workflow Chunking Embedding Storing Retrieval Use the selected sections to produce the final answer. Reranking Generation 

Search Methods Keyword Search Semantic Search Hybrid Search ✓ Precise matching ✓ Understands meaning ✓ Combines both ✓ Uses algorithms like ✓ Uses vector ✓ Metadata Filtering BM25, TF-IDF ✓ Fast × embeddings ✓ Finds conceptually Struggles with synonyms related content × May miss exact terms ✓ Balances precision and recall ✓ More comprehensive results ✓ Best approach for most RAG systems 

Semantic Search Similar Sentences Pet 

Hybrid Search 1. Result A 1. Result A 2. Result B 2. Result B 3. Result C Keyword Search 50 result each! 3. Result C Metadata Filter 1. Result C 1. Result C Retriever 2. Result B 2. Result B Semantic Search 3. Result X Metadata Filter 3. Result X Top_k results 

Search Methods Examples ● Query: “renewal policy for enterprise contracts” ● Keyword finds “renewal” sections ● Semantic finds “contract extension” and “rollover” ● Hybrid returns both; filters to source=“policy-docs”, region=“ZA” 

No content

Cape Town DEMO 

Fine-Tuning vs RAG: Comparison Advantages Disadvantages - Training takes time Fine-Tuning - Matches org-style with unsupervised training - Requires expertise - Can align with compliance & proprietary data - Not available for all models - No source references - Higher risk of hallucination - Fast setup, no fine-tuning needed - Can use new documents in minutes RAG - Managed solutions available - Provides source references - Lower hallucination risk - Not great at summarizing whole documents 

Decision Framework 

Cape Town Start with Rag for custom document Q&A. Use Fine-tuning if you need tasks like summarization or ogspecific style. 

Advanced Rag Techniques Metadata Filtering Hypothetical Question Generation Multi-Source Retrieval Reranking Prompt Optimization Self-Evaluation 

Key Takeaways ● RAG grounds LLMs with real, fresh knowledge and citations ● Fine-tuning improves style/structure and task performance ● Hybrid approach often wins in production ● ADK makes RAG implementations faster and maintainable ● Evaluate, monitor, and iterate continuously 

Thank You https://www.linkedin.com/in/matini https://google.github.io/adk-doc