Nov 2024 - Impact of Vector Search: Unraveling Purpose-built vs. Traditional Databases for Gen AI Applications

In this presentation, Atita aims to illuminate the transformative influence of vector search on generative AI (Gen AI) applications by comparing purpose-built and traditional databases integrated with vector capabilities. Their goal is to showcase the pivotal role of database selection in the success of Gen AI applications, emphasizing the importance of prioritizing quality over convenience.

Through a detailed examination of the complexities and operational limitations faced by traditional databases adapting to vector search, speaker highlights how purpose-built solutions offer unparalleled efficiency and reliability in meeting the evolving demands of Gen AI applications. Atita hopes that their audience will gain a deep understanding of the impact of vector databases on the performance and scalability of Gen AI applications, enabling them to make informed decisions when choosing between purpose-built and traditional databases for vector search.

Ultimately, this presentation seeks to clarify the need for the adoption of advanced database solutions that can empower Gen AI applications to deliver real-time responsiveness, contextual relevance, and optimal performance in a rapidly evolving digital landscape.

More info - https://events.pinetool.ai/3254/#sessions/107999?referrer%5Bpathname%5D=%2Fsessions&referrer%5Bsearch%5D=&referrer%5Btitle%5D=Sessions

Transcript

1. Impact of Vector Search for Gen AI Applications Atita Arora AI Solution Architect
2. About me Started 2008 - Computer Applications - Strategic Business Management - Vector / Semantic Search - Language analysis - Information retrieval Opensource Committer Loves to travel, eat, cook Mom of 2 boys
3. Raise your hand if you are… - Using generative AI in your projects? - Planning to use generative AI in your projects? - Building a generative AI application for a specific domain?
4. Trivia Alert !!
5. Our Ever-Growing Data Hoard Ref : 1 Exabyte =1,000,000,000GB
6. Why One-Size-Fits-All AI Falls Short for Enterprises
7. Structured vs Unstructured Data use-cases
8. SQL Like Inverted Index
9. Information Retrieval Techniques* Open Source LLM/ Multimodal IR/ Multilingual IR 2023Present Semantic Search Word Embeddings) 2018 Learning to Rank LTR 2017 Named Entity Recognition 2015 Personalization 2013 2010 Natural Language Processing Intent Using language processing techniques 2000 Text Analysis Using Synonyms, Stemming, Lemmatization Pre 2000 Pattern search , Exact search Mostly driven by databases The Evolution of 2011 Multi-word Synonyms Initial implementation and later developed into its advance form Closed Source LLM Multilingual IR 2022 *Arranged by peak popularity wrt applications in IR systems
10. Challenges of using Text / Keyword based techniques ONLY Actual user query intent/context is still a mystery! Variety of attributes Vocabulary mismatch Multimodal Content (Text, Image, Audio, Video) Missing standardization …And many more domain level challenges!!
11. Discovery of common language!
12. The magic of !! A high dimensional array of numbers that captures the context & semantic meaning An object is known by company it keeps !
13. The magic of !! • In our example the word ‘right’ has a different meanings in each sentence
14. Vector Search An essential part of the AI Transformation SEARCH SYSTEMS RAG / INFORMATION ASSISTANTS ANOMALY DETECTION RECOMMENDATIONS NN Encoders + Vector Database INDUSTRIES Gig Economy ECommerce Law-Tech Fashion Biometrics Med-Tech Anti-fraud Agriculture Manufacturing Streaming Services Ed-Tech Marketplaces Media & News HRTech Ad-Tech Online Dating
15. Vector Search powered by HNSW HNSW is a powerful graph-based indexing algorithm that most vector application rely on to organize vectors for fast and efficient search.
16. Trivia - 1 What does HNSW stands for ? - Hierarchical Navigable Small World
17. Trivia - 2 Can you name popular libraries used for approximate nearest neighbor searches? - FAISS - NMSLIB - ANNOY
18. Vector Search support as an add-on
19. What is a vector database? • A vector database is a specialized data management system tailored for high-dimensional vector data. • Designed for efficient indexing, and retrieval of vectors. • Utilizes innovative indexing techniques and optimized storage solutions. • Features scalable architectures to handle growing data demands. • Delivers exceptional query performance while ensuring cost efficiency. • Essential for managing complex datasets across diverse modalities (e.g., text, images, videos).
20. MTEB Leaderboard* *Dated 19.11.24
21. Memory consumption - support for high dimensional data • Size of the vector usually varies from 384 to 30522 dimensions. • Basic math: Number of dimensions: 1536 (OpenAI embedding model) Size of each dimension (float32): 4 bytes Total size: 1536×4 = 6144 bytes ÷ 1024 = 6 KB and a million such points - 6 GB ** A 300-character text at 1 byte per character (UTF-8) is only 300 bytes
22. High Compute - support for operations with vectors • Vector operations are computationally intensive • Datasets Volume - millions / billions of vectors. • Demand for near real-time vector indexing and retrieval
23. Performance Aspect • Leverage memory to enhance search performance • Need specialized ways to optimize memory usage and improve search performance for vectors
24. Quantization Techniques - save memory and improve processing efficiency. • Leverage memory to enhance search performance • Need specialized ways to optimize memory usage and improve search performance
25. Operation Scale - support for high volume / lower latency Custom sharding
26. Speed Precision Memory Pick two
27. Enterprise-Ready, Massive-Scale Vector Search Technology for the Next AI Generation Performance Centric Quick and easy to start Resource optimization focussed All embeddings supported OOTB Fully Open Source project Scalability Oriented Most Loved open-source vector search database > 10 K+ Adopters Worldwide > 7 M+ Downloads > 30 K+ Community Members
28. Fastest Vector Search Engine Up to 10 times faster according to benchmarks
29. Performance Centric Advanced one-stage Filtering ⬢ Purpose built for handling high-dimensional and complexity of billion scale vectors. ⬢ Support for dense , sparse and multivector vectors Exceptional performance and handling of vectors Store any kind of data including geolocation along with vectors and filter search results. Resource Optimization focussed Effective on-disk storage options and low level hardware optimization. Memory Maps and storage optimizations Up to 64x Memory usage reduction and x40 search speed up*. Quantization - Scalar, Product, and unique Binary Quantization
30. Advanced Scalability Custom sharding Scaling options to support data growth and performance. Scaling Achieved through Payload-based sharding Deployment flexibility Zero downtime update of vectors in case of model switch (blue-green deployment) Zero downtime segregation and dynamic read scaling Read/Write ⬢ Horizontal ⬢ Vertical
32. Flavours of Vector Search • Dense embeddings • Sparse embeddings • Multivector representations • Hybrid / Combination
33. Vector databases empower your AI journey, enabling faster innovation and smarter solutions that outpace traditional bolt-on approaches !!
34. Qdrant Prefetch (All in one) https://github.com /qdrant/qdrant-rag -eval/blob/master/ workshop-rag-eva l-oxford-llm2024/n otebook/advance d_hybrid_search_ qdrant_rag.ipynb
35. Trivia - 3 What advanced document retrieval model leverages Vision Language Models (VLMs) to index and retrieve information from complex documents by generating multi-vector representations of both text and images? - COLPALI - COLQWEN
36. Register now : https://try.qdrant.tech/colpali-webinar Struggling to retrieve complex, visually rich PDFs? ColPali has you covered with efficient document retrieval using Vision Language Models. What’s on the agenda: ✅ How ColPali Enhances Document Retrieval ✅ Boosting ColPali with Qdrant's Binary Quantization ✅ Using ColPali as a Reranker ✅ ColPali in RAG/Vision RAG Applications
38. Thank you !! A free forever 1GB cluster included for trying out. No credit card required. Feel free to reach us at : info@qdrant.com
39. “There are many open-source and licensed vector databases available today. A few of them are Qdrant, Pinecone, Faiss, Weaviate etc. We opted for Qdrant for our use case due to its robust pre-filtering capabilities and efficient payload indexing support.” -Anshu Aditya, Meesho , Dailymotion Testimonials & Results