$30 off During Our Annual Pro Sale. View Details »

Vald: OSS ANN Nearest Neighbor Dense Vector Search Engine - Introduction and Case Studies -

Vald: OSS ANN Nearest Neighbor Dense Vector Search Engine - Introduction and Case Studies -

Yusuke Kato (Yahoo! JAPAN / Technology Incubation, CTO / Software Engineer)
Kiichiro Yukawa (Yahoo! JAPAN / Technology Incubation, CTO / Software Engineer)

https://tech-verse.me/ja/sessions/172
https://tech-verse.me/en/sessions/172
https://tech-verse.me/ko/sessions/172

Tech-Verse2022
PRO

November 17, 2022
Tweet

More Decks by Tech-Verse2022

Other Decks in Technology

Transcript

  1. None
  2. Agenda !Vector Search Overview !Trend of Vector Search !Vald !Use

    Case of Vald !Demo
  3. Yusuke Kato / Yahoo Japan Corporation - Technology Incubation, CTO

    - Vald Founder/Certified OSS Developer (Vald) - Joined as a new graduate in 2016. Belongs to the company-wide troubleshooting SWAT team and is involved in multiple development projects. Vald development began in 2019. I'm also the PO of the technology domain of the R&D organization Technology Incubation and the PO of the in-house Vald team. - Camping/Badminton/Home server operation/!/" - GitHub: @kpango - Twitter: @kpang0 Self-Introduction: Yusuke Kato Self-Introduction
  4. Vector Search

  5. What is Vector Search? It is a method of searching

    for similar vectors from indexed vectors by expressing the similarity of unstructured data such as images and sounds in a vector space and calculating the distance between vectors. In the case of high-dimension vectors, k Nearest Neighbor (kNN) increases the number of calculations, so it is common to search using Approximate Nearest Neighbor (ANN). !Data formats that can be handled: text, image, video, audio, binary, etc. !Range of applications: Similarity search, recommendations, data analysis, etc. !Usage examples: Google image search, YouTube, Spotify, etc.
  6. Components of Vector Search Embedding Algorithm Index Store BERT, ML

    models NGT, ScaNN, Faiss In-Memory, Storage
  7. Vector Search Flow Architecture of Vector Search

  8. Indexing Data Flow Architecture of Vector Search

  9. Search Data Flow Architecture of Vector Search

  10. Trend of Vector Search

  11. Vector Search Libraries SPTAG NGT HNSW Faiss Annoy ScaNN

  12. Vector Search Engines Pinecone Milvus Weaviate Qdrant Vertex AI Matching

    Engine Vald Standalone Service Cloud Service
  13. Vector Search Engines Pinecone Milvus Weaviate Qdrant Vertex AI Matching

    Engine Vald Standalone Service Cloud Service
  14. Vald

  15. What is Vald? !A distributed fast approximate nearest neighbor dense

    vector search engine for OSS running on Kubernetes !Enables fast searching from billion-scale vector data !Flexibly customizable according to user needs !Multilingual support !Technologies used
  16. Main Features Distributed Index Extensibility Replication Horizontal Scalability Auto Backup

    Asynchronous Indexing
  17. Architecture

  18. Vald LB Gateway !Vald Main Gateway !Resource-based load balancer !Manage

    index replicas !Aggregate and Re-Rank search results
  19. Vald Discoverer ! Frequent acquisition of the resource status of

    Vald components and Kubernetes Nodes deployed on Kubernetes ! Sharing resource status with LB Gateway and Index Manager
  20. Vald Index Manager !Vald's index management component !Controls agent index

    timing and prevents "stop of the world"
  21. Vald Agent !Index retention of vector data !CRUD processing for

    vectors !Can be used as a standalone
  22. Distributed Index Replication

  23. Distributed Index Replication ! Looks at the cluster resources and

    distributes the index to multiple agents so that the memory is smooth ! Improves search speed by using multiple compute resources ! Prevents index loss due to failures with replicas and multiple backup methods
  24. Data Flow Overview Distributed Index Replication

  25. Determine Insert Destination Distributed Index Replication

  26. Determine Insert Destination Distributed Index Replication

  27. Determine Insert Destination Distributed Index Replication

  28. Determine Insert Destination Distributed Index Replication

  29. Determine Insert Destination Distributed Index Replication

  30. Determine Insert Destination Distributed Index Replication

  31. Insert Vector to Replica Group Distributed Index Replication

  32. Without Resource Based Load Balancing Memory Based Distributed Index

  33. With Resource Based Load Balancing Memory Based Distributed Index

  34. Automatically Indexing & Backup

  35. About Automatically Indexing & Backup Data Flow of Indexing &

    Backup
  36. Search Architecture Overview

  37. Search Operation Data Flow

  38. Search Operation Data Flow

  39. Detailed Aggregation Flow Aggregate Search Result

  40. Aggregate Search Result Detailed Aggregation Flow

  41. Detailed Aggregation Flow Get search result from each Agent

  42. Get search result from each Agent Detailed Aggregation Flow

  43. Get search result from each Agent Detailed Aggregation Flow

  44. Get search result from each Agent Detailed Aggregation Flow

  45. Detailed Aggregation Flow Get search result from each Agent

  46. Detailed Aggregation Flow Get search result from each Agent

  47. Detailed Aggregation Flow Get search result from each Agent !Only

    some results can be processed even if the number of Top K or nodes is large !Works well with network latency and noisy neighbor problems !Considers timeout processing, etc.
  48. Agenda !Vector Search Overview !Trend of Vector Search !Vald !Use

    Case of Vald !Demo
  49. Kiichiro Yukawa / Yahoo Japan Corporation - Technology Incubation, CTO

    - Vald PM / Vald CRE - Joined as a new graduate in 2016. Joined the Vald team in 2020 after planning, developing, operating, and maintaining internal systems. Engaged in Vald's development project management and CRE domain. - Tennis, snowboarding, golf, camping, cooking Self-Introduction: Kiichiro Yukawa Self-Introduction
  50. Usecase

  51. Use Case Examples Use-cases

  52. Search Use-cases Acquires object data similar to the object data

    you want to search for
  53. Classification Use-cases Acquires object data similar to the object data

    you want to search for Categorizes which group (category) the entered object data belongs to
  54. Recommendation Use-cases Acquires object data similar to the object data

    you want to search for Returns the items you want to recommend for the searched object data as search results
  55. Analysis Use-cases Acquires object data similar to the object data

    you want to search for Can be used for anomaly detection, malware detection, etc.
  56. !Searches for similar products from product images (Some products) Similar

    product image search Source of Image: https://store.shopping.yahoo.co.jp/zozo/67501203.html (2022-11-17)
  57. !Infers product information from product images when listing Product information

    inference
  58. Similar document image search 「JAPAN SEARCH」 https://jpsearch.go.jp/(operated by the National

    Library)
  59. Similar source code search !Vectorize source code using CodeBert !Matching

    of recruiting companies and engineers with similarity search of source code Matching engineers and companies with code Engineers Recruiting Manager
  60. How to Implement

  61. How to Deploy Vald How to deploy the Vald cluster

    ! Deploy using Helm • Deployable in a few steps • Edit and upgrade clusters via Helm commands ! Deploy using the vald-helm-operator • Operate Vald cluster using Kubectl • Manage so that vald-helm-operator is in an ideal state based on Custom Resource Definition • Recommended for production environments
  62. Deployment procedure 1.Prepare the Kubernetes Cluster 2.Add the Vald chart

    to Helm Repo 3.Create/Modify values.yaml • Vald-only configuration file when deploying with Helm • You will see the default settings for fields not filled in 4.Helm commandでapply Deploying Vald cluster with Helm Deployment the Vald cluster using Helm chart #Description of common setting items defaults: logging: info image: "v1.6.3" ... #Settings for each component gateway: #Configuration of vald-lb-gateway lb: gateway_config: index_replica: 2 agent: #Configuration of vald-agent-ngt ngt: dimension: 784 ... ... $ helm repo add vald https://vald.vdaas.org/charts $ helm install vald vald/vald --values values.yaml
  63. Demo

  64. Deploy Vald cluster

  65. chiVe Demo Pre-Process & Insert

  66. chiVe Demo Search

  67. chiVe Demo Update & Remove

  68. chiVe Demo Word Analogies

  69. Clean Up Vald cluster

  70. Roadmap

  71. l Faiss support l QBG (public beta) 2023/01 2023/02 2023/03

    2022/12 Roadmap l Continuous Benchmark tool l Request Mirroring (BCP) l Network Policy l Efficient Search for large top-K l Index correction l Content Search
  72. Summary

  73. Summary ! Introduction to vector search ! Introduction to Vald

    and distinctive features ! Presentation of Vald's application domain and application examples ! Demo with ChiVe
  74. Contact Us GitHub Web Site Slack

  75. We’re waiting your contribution! GitHub

  76. Thank you