June 2023 - Vectorize Your Open Source Search Engine

Neural search (a.k.a. Vector search) has rewritten the standards of information retrieval in many different domains.
Vector search can help you gather a better understanding of the user query intent, drive product recommendations, search across different source data (text, images, audio, video), deliver better results, improve personalization and create a more successful user experience. Vector search goes beyond keywords to harvest the potential of graphs and embeddings to match users to the intended document, product, job, picture, song, or video.
As fascinating as this may sound it's easy to find ourselves lost in the deluge of new information.
If you're struggling to get started, understand what vector search can bring to the party, add cool new models such as OpenAI models and want to avoid common pitfalls, this talk is for you.

Video here : https://www.youtube.com/watch?v=U7PQNyeQrXQ

Transcript

1. www.opensourceconnections.com Vectorizing your Opensource Search Engine opensourceconnections.com.com Atita Arora
2. www.opensourceconnections.com About me - Language analysis - Information retrieval - Computer Applications - Strategic Business Management 2 X Started Opensource
3. www.opensourceconnections.com Who are ? Our comprehensive trainings, expert consulting and impactful community initiatives have empowered numerous individuals and teams, enabling them to enhance their proficiency in search relevance and drive business success. 3 Conferences We Host >4000 Relevance Slack community members 118 Search-related Open Source Projects 3 Books on Search
4. www.opensourceconnections.com Vector Search is buzzin'
5. www.opensourceconnections.com What the heck is Vector Search? • Vector embedding encodes an object in machine language • The idea of encoding is to capture the semantic meaning of object : • Vector search involves converting a "source object" into a vector embedding using a desired transformer, then index and search through the vector embeddings. iphone with transparent backcover transparent backcover for iphone Text , Image, Audio , Video
6. www.opensourceconnections.com Conventional Challenges of a Search Engine 1. Context in queries ( language , long tail ) 2. Vocabulary mismatch 3. Missing standardization 4. Multimodal Content (Text , Image, Audio , Video) 5. Trends and Popularity 6. Variety of attributes …………………..…And there are certainly many more !!
7. www.opensourceconnections.com How can Vectors help ? • Fix recall issues caused by: ◦ long-tail queries ◦ zero hits due to vocabulary mismatch ◦ zero hits due to multilingual search • Multimodal (Use text , images , audio , video) • Also exposes a lot of potential in : ◦ Marketing and Promotions - similar users ◦ Recommendations and Suggestions - similar products • Boosting precision by leveraging meaning that goes beyond the individual words themselves and focuses on the context and intent behind the language used.
8. www.opensourceconnections.com From Keywords to Vectors - Implementing Vector Search in Your Search Engine 1. Vector Extraction * Data gather * Config changes & fields selection * Model Selection * Adding Vectors to the data 2. Conversion and Transformation * Encoding Service - Conversion of text query -> Text / Image Vector * Transformation of Search API to accommodate Vectors 3. Optimization for Relevancy Improvements and Efficiency * Offline Experimentation for relevancy improvements (Quepid) * Fine-tuning and performance optimizations VECTORIZE
9. www.opensourceconnections.com Keep it Simple : From Theory to Practice • Identify a designated business use-case to try Vector Search • Exemplify the value with a POC: ◦ Using existing Lucene-based search ◦ Leverage pre-existing models (from Hugging Face) • Experiment to prove improvements • Focus on analytical understanding • Leverage Chorus as a foundational framework to expedite your initial progress.
10. www.opensourceconnections.com User Interface Search Engine Offline Experimentation Search management
11. www.opensourceconnections.com Implementation Architecture Vector Extraction Conversion & Transformation
12. www.opensourceconnections.com Query : Projector Screen <- Results with default algo Results with match by text vector ->
13. www.opensourceconnections.com Query : smartwatch by sumsung <- Results with default algo Results with match by image vector ->
14. www.opensourceconnections.com Is my search quality improving ? Addressing : Optimization for Relevancy Improvements and Efficiency
15. www.opensourceconnections.com Why Invest in an Offline Lab? • Increasing experiment rate correlates with better search. • Offline experimentation is a magnitude faster & cheaper than an online A/B test • Allows you to make bigger changes without correspondingly greater risk. • Forces you to have repeatable measurements of search quality. Check out Quepid, an open source tool for offline testing. Enabling Experimentation for measuring improvements Offline Lab Source : P Fries , Haystack 2018
16. www.opensourceconnections.com Is my search quality improving (nDCG@10)? Query Type / Algo Exact Product Type Symptoms Feature Thematic Compatibility Slang/ Acronym Typos Avg nDCG@10 BM25 0.76 0.47 0.44 0.44 0.04 0.78 0.34 0.15 0.44 Match by Text Vector 0.26 0.87 0.59 0.67 0.63 1 0.88 0.90 0.70 Match by Image Vector 0.50 0.73 0.50 0.72 0.66 0.80 0.84 0.64 0.66 Boost by Text Vector 0.55 0.65 0.62 0.66 0.10 1 0.85 0.92 0.66 Boost by Image Vector 0.58 0.67 0.50 0.65 0 0.80 0.84 0.78 0.60 Hybrid
17. www.opensourceconnections.com Caveats • Transformer model suitable for the business case • Limitations of the models • Dimensions are not free - Infrastructural changes • Fast and scalable vector encoding service • KNN consistently retrieves a 'N' results, even when the relevance to the query may be relatively weak.
18. www.opensourceconnections.com Other things to experiment with • Fine-tuned model support • Combining Vectors with managed searches* • Vector based Re-ranking model • Combining scores from text + image search results *Promoted searches , legal binding , brand conflicts etc
19. www.opensourceconnections.com Query : organising stuff for office Results with default algo Results with match by image model Results with fine-tuned image model Getting Better through Fine Tuning.. Fine-tuned Model courtesy : Jina.ai
20. www.opensourceconnections.com • Diagram generated : https://app.diagrams.net/ • Query Types https://baymard.com/blog/ecommerce-search-query-types • Model Courtesy https://huggingface.co/models • More Vector and Relevance stuff https://opensourceconnections.com/blog • Chorus ◦ Solr - https://github.com/querqy/chorus. ◦ Elastic - https://github.com/querqy/chorus-elasticsearch-edition References
21. www.opensourceconnections.com Thank you !