RAGs powered by Google Search (By: Shajeel Afzal) - DevFest Lahore 2024

Talk by Shajeel Afzal (https://www.linkedin.com/in/shajeelafzal/) at DevFest Lahore 2024 by GDG Lahore.

Transcript

1. Proprietary Proprietary Shajeel Afzal, Founder, UpAlerts RAGs powered by Google Search
2. Proprietary Table of Contents What is RAG? Semantic search is not similarity search Beyond semantic search Real-world challenges of RAG How to get started 01 02 03 04 05
3. Proprietary What is RAG? Retrieval Augmented Generation 01
4. Proprietary Challenge: LLM has limited knowledge, causing hallucinations User LLM Docs What is video localized narratives? The docs are too many to read. Just going to guess...
5. Proprietary Solution: RAG; Use retrieval backend to enhance LLM's capability and reliability User LLM Retrieval Docs What is video localized narratives? Hey Search, find papers on video localized narratives!
6. Proprietary Various RAG techniques in the wild Text Chunking 01 02 03 04 05... Query Expansion Hybrid Search Knowledge Graph Reranking etc... Why not just use Google Search? It has them all. I need these Advanced RAGs to find right information!
7. Proprietary Vertex AI Search: Google Search on Vertex AI Extraction & Summarization Collection (web, files, DBs, connectors, etc.) Parsing Search Chunking & Annotation Tokenizing & Embedding Token & Vector Indexing Collect Build Serve Vertex AI Search
8. Proprietary Semantic search is not just similarity search 02
9. Confidential + Proprietary Proprietary
10. Proprietary The question is not the answer User LLM Retrieval Docs Why is the sky blue? Find similar text with "Why is the sky blue"! There's no such text... The scattering of sunlight causes the blue color
11. Proprietary How Google Search solved this with RankBrain and Neural matching “Neural matching helps us understand fuzzier representations of concepts in queries and pages, and match them to one another. It looks at an entire query or page rather than just keywords, developing a better understanding of the underlying concepts represented in them. Take the search “insights how to manage a green,” for example. If a friend asked you this, you’d probably be stumped. But with neural matching, we’re able to make sense of it. By looking at the broader representations of concepts in the query — management, leadership, personality and more — neural matching can decipher that this searcher is looking for management tips based on a popular, color-based personality guide.” —How AI powers great search results
12. Proprietary Google Search technology assets for Information Retrieval RankBrain Neural matching ScaNN TPU Deep Learning based semantic search and reranking for billions of users since 2015 Deep Learning technology to learn the relationship between query and docs since 2018 One of the largest and fastest vector search Infrastructure Empowers major Google services for search and recommendation AI processor for delivering semantic search to billions since 2015 Millisecond latency at a reasonable cost
13. Proprietary Example: Google Merchandise Store dataset With a popular keyword-based search engine... Warm clothing for winter? Keyword search
14. Proprietary Example: Google Merchandise Store dataset Warm clothing for winter? With Vertexi AI Search... Generative AI Summary: You can find warm clothing for winter at the Google Merchandise Store [1, 2, 5]. Some of the items include: Google Vail Unisex Grey Puffer Jacket, Google Denali Unisex Puffer Vest, Google Denali Womens Puffer Vest, Super G Tahoe Unisex Black Puffer Vest, Super G Glacier Unisex Puffer Jacket, Super G Glacier Womens Puffer Jacket [5]. Neural matching
15. Proprietary Place Image Here Tensor Processing Unit (TPU): To deliver semantic search for billions of users Google has been at the forefront of semantic search for a decade Google started development of TPU in 2013 to deliver a production-grade semantic search to billions of Google Search users The first TPU was deployed to Google Search production serving infrastructure in 2015.
16. From text to multimodal, Search to recommendation Contents and business data LLM Custom Embeddings
17. Proprietary Generative Recommendation with Vertex AI Search + LLM reasoning With Vertexi AI Search + LLM reasoning Type-specific queries: Warm winter jackets, Cozy knitwear, Thermal leggings, Waterproof snow boots Activity-specific queries: Ski clothing, Winter running gear, Work-appropriate winter outfits, Cozy homewear Style-specific queries: Cashmere sweaters, Puffer vests, Statement scarves, Athleisure-inspired winter looks What would be the best queries to find cross-sell items for this user? Generate personalized queries
18. Proprietary
19. Proprietary Beyond Semantic Search: Hybrid search, reranking, filtering, and extraction 03
20. Proprietary Keyword search with 25+ years of experience: Hybrid search in Vertex AI Search Query Rewriting Keyword Search Ranking & Filtering Vector Search Search Results Snippets & Summaries "Dinosaur keyholder" "You can find a Chrome Dino Keychain for $8.00. It's a cute keychain that you can take on adventures with you."
21. Proprietary Query understanding and rewriting, Keyword + semantic search "dinosaur keyholder" With Vertexi AI Search... • Word stemming and spell correction • Adding related words and synonyms • Removing unimportant words • Annotating important entities with Knowledge Graph • Keyword search + Semantic search Let's also find dino and keychain, and maybe interested in lanyard and pin?
22. Proprietary Deep re-ranking: two-stage retrieval
23. Proprietary Dozens of signals for reranking on each query
24. Proprietary Why Extraction is needed? User LLM Vertex AI Search Relevant doc What is video localized narratives? I got the doc, but it has 20 pages... Too long to read
25. Proprietary Smart content extraction with Vertex AI Search This is so handy! Extracting most relevant part
26. Proprietary Summarization and multi-turn search with Vertex AI Search No need to build a RAG by myself! Built-in LLM Summarized text
27. Proprietary Google-quality document processing Document Parsing Generating Embeddings Token Indexing Vector Indexing Element-level Annotation Text Segmentation Segment-level Annotation Tokenization Doc
28. Proprietary Document and query annotation with Knowledge Graph
29. Proprietary Real-world challenges of RAG Enterprise data integration, RAG dev process 04
30. Proprietary Collect scattered enterprise data: Blended search
31. Proprietary Collect scattered enterprise data: Connectors
32. Proprietary Building a RAG from scratch v. Fully managed Project "Building a Tower of RAG" Data scientists ML engineers Data engineers Search engineers DevOps MLOps SREs years
33. Proprietary
34. Proprietary
35. Proprietary
36. Proprietary How to get started with RAG on Vertex AI Search 05
37. Proprietary How to get started • LangChain-based samples and documents: RAG sample notebooks using Vertex AI Search, PaLM, and LangChain and Vertex AI Search support in LangChain • Grounding in Vertex AI: provides a quick and easy way for grounding • Check Grounding API provides a grounding score for an answer candidate • Vertex AI Conversation-based grounding: Vertex AI Search and Conversation: search with follow-ups • How to use custom embedding with Vertex AI Search • Vertex AI Search and Conversation product page • Get started with Vertex AI Search • Vertex AI Search sample notebooks on GitHub Gen AI repo • Video: Harnessing the power of generative AI to deliver next-gen search experiences
38. Proprietary Thank you.