Vald: OSS ANN Nearest Neighbor Dense Vector Search Engine - Introduction and Case Studies -

Yusuke Kato (Yahoo! JAPAN / Technology Incubation, CTO / Software Engineer)
Kiichiro Yukawa (Yahoo! JAPAN / Technology Incubation, CTO / Software Engineer)

https://tech-verse.me/ja/sessions/172
https://tech-verse.me/en/sessions/172
https://tech-verse.me/ko/sessions/172

Transcript

2. Agenda !Vector Search Overview !Trend of Vector Search !Vald !Use Case of Vald !Demo
3. Yusuke Kato / Yahoo Japan Corporation - Technology Incubation, CTO - Vald Founder/Certified OSS Developer (Vald) - Joined as a new graduate in 2016. Belongs to the company-wide troubleshooting SWAT team and is involved in multiple development projects. Vald development began in 2019. I'm also the PO of the technology domain of the R&D organization Technology Incubation and the PO of the in-house Vald team. - Camping/Badminton/Home server operation/!/" - GitHub: @kpango - Twitter: @kpang0 Self-Introduction: Yusuke Kato Self-Introduction
4. Vector Search
5. What is Vector Search? It is a method of searching for similar vectors from indexed vectors by expressing the similarity of unstructured data such as images and sounds in a vector space and calculating the distance between vectors. In the case of high-dimension vectors, k Nearest Neighbor (kNN) increases the number of calculations, so it is common to search using Approximate Nearest Neighbor (ANN). !Data formats that can be handled: text, image, video, audio, binary, etc. !Range of applications: Similarity search, recommendations, data analysis, etc. !Usage examples: Google image search, YouTube, Spotify, etc.
6. Components of Vector Search Embedding Algorithm Index Store BERT, ML models NGT, ScaNN, Faiss In-Memory, Storage
7. Vector Search Flow Architecture of Vector Search
8. Indexing Data Flow Architecture of Vector Search
9. Search Data Flow Architecture of Vector Search
10. Trend of Vector Search
11. Vector Search Libraries SPTAG NGT HNSW Faiss Annoy ScaNN
12. Vector Search Engines Pinecone Milvus Weaviate Qdrant Vertex AI Matching Engine Vald Standalone Service Cloud Service
13. Vector Search Engines Pinecone Milvus Weaviate Qdrant Vertex AI Matching Engine Vald Standalone Service Cloud Service
14. Vald
15. What is Vald? !A distributed fast approximate nearest neighbor dense vector search engine for OSS running on Kubernetes !Enables fast searching from billion-scale vector data !Flexibly customizable according to user needs !Multilingual support !Technologies used
16. Main Features Distributed Index Extensibility Replication Horizontal Scalability Auto Backup Asynchronous Indexing
17. Architecture
18. Vald LB Gateway !Vald Main Gateway !Resource-based load balancer !Manage index replicas !Aggregate and Re-Rank search results
19. Vald Discoverer ! Frequent acquisition of the resource status of Vald components and Kubernetes Nodes deployed on Kubernetes ! Sharing resource status with LB Gateway and Index Manager
20. Vald Index Manager !Vald's index management component !Controls agent index timing and prevents "stop of the world"
21. Vald Agent !Index retention of vector data !CRUD processing for vectors !Can be used as a standalone
22. Distributed Index Replication
23. Distributed Index Replication ! Looks at the cluster resources and distributes the index to multiple agents so that the memory is smooth ! Improves search speed by using multiple compute resources ! Prevents index loss due to failures with replicas and multiple backup methods
24. Data Flow Overview Distributed Index Replication
25. Determine Insert Destination Distributed Index Replication
26. Determine Insert Destination Distributed Index Replication
27. Determine Insert Destination Distributed Index Replication
28. Determine Insert Destination Distributed Index Replication
29. Determine Insert Destination Distributed Index Replication
30. Determine Insert Destination Distributed Index Replication
31. Insert Vector to Replica Group Distributed Index Replication
32. Without Resource Based Load Balancing Memory Based Distributed Index
33. With Resource Based Load Balancing Memory Based Distributed Index
34. Automatically Indexing & Backup
35. About Automatically Indexing & Backup Data Flow of Indexing & Backup
36. Search Architecture Overview
37. Search Operation Data Flow
38. Search Operation Data Flow
39. Detailed Aggregation Flow Aggregate Search Result
40. Aggregate Search Result Detailed Aggregation Flow
41. Detailed Aggregation Flow Get search result from each Agent
42. Get search result from each Agent Detailed Aggregation Flow
43. Get search result from each Agent Detailed Aggregation Flow
44. Get search result from each Agent Detailed Aggregation Flow
45. Detailed Aggregation Flow Get search result from each Agent
46. Detailed Aggregation Flow Get search result from each Agent
47. Detailed Aggregation Flow Get search result from each Agent !Only some results can be processed even if the number of Top K or nodes is large !Works well with network latency and noisy neighbor problems !Considers timeout processing, etc.
48. Agenda !Vector Search Overview !Trend of Vector Search !Vald !Use Case of Vald !Demo
49. Kiichiro Yukawa / Yahoo Japan Corporation - Technology Incubation, CTO - Vald PM / Vald CRE - Joined as a new graduate in 2016. Joined the Vald team in 2020 after planning, developing, operating, and maintaining internal systems. Engaged in Vald's development project management and CRE domain. - Tennis, snowboarding, golf, camping, cooking Self-Introduction: Kiichiro Yukawa Self-Introduction
50. Usecase
51. Use Case Examples Use-cases
52. Search Use-cases Acquires object data similar to the object data you want to search for
53. Classification Use-cases Acquires object data similar to the object data you want to search for Categorizes which group (category) the entered object data belongs to
54. Recommendation Use-cases Acquires object data similar to the object data you want to search for Returns the items you want to recommend for the searched object data as search results
55. Analysis Use-cases Acquires object data similar to the object data you want to search for Can be used for anomaly detection, malware detection, etc.
56. !Searches for similar products from product images (Some products) Similar product image search Source of Image: https://store.shopping.yahoo.co.jp/zozo/67501203.html (2022-11-17)
57. !Infers product information from product images when listing Product information inference
58. Similar document image search 「JAPAN SEARCH」 https://jpsearch.go.jp/（operated by the National Library）
59. Similar source code search !Vectorize source code using CodeBert !Matching of recruiting companies and engineers with similarity search of source code Matching engineers and companies with code Engineers Recruiting Manager
60. How to Implement
61. How to Deploy Vald How to deploy the Vald cluster ! Deploy using Helm • Deployable in a few steps • Edit and upgrade clusters via Helm commands ! Deploy using the vald-helm-operator • Operate Vald cluster using Kubectl • Manage so that vald-helm-operator is in an ideal state based on Custom Resource Definition • Recommended for production environments
62. Deployment procedure 1.Prepare the Kubernetes Cluster 2.Add the Vald chart to Helm Repo 3.Create/Modify values.yaml • Vald-only configuration file when deploying with Helm • You will see the default settings for fields not filled in 4.Helm commandでapply Deploying Vald cluster with Helm Deployment the Vald cluster using Helm chart #Description of common setting items defaults: logging: info image: "v1.6.3" ... #Settings for each component gateway: #Configuration of vald-lb-gateway lb: gateway_config: index_replica: 2 agent: #Configuration of vald-agent-ngt ngt: dimension: 784 ... ... $ helm repo add vald https://vald.vdaas.org/charts $ helm install vald vald/vald --values values.yaml
63. Demo
64. Deploy Vald cluster
65. chiVe Demo Pre-Process & Insert
66. chiVe Demo Search
67. chiVe Demo Update & Remove
68. chiVe Demo Word Analogies
69. Clean Up Vald cluster
70. Roadmap
71. l Faiss support l QBG (public beta) 2023/01 2023/02 2023/03 2022/12 Roadmap l Continuous Benchmark tool l Request Mirroring (BCP) l Network Policy l Efficient Search for large top-K l Index correction l Content Search
72. Summary
73. Summary ! Introduction to vector search ! Introduction to Vald and distinctive features ! Presentation of Vald's application domain and application examples ! Demo with ChiVe
74. Contact Us GitHub Web Site Slack
75. We’re waiting your contribution! GitHub
76. Thank you