Tweet
Tweet

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Agenda !Vector Search Overview !Trend of Vector Search !Vald !Use Case of Vald !Demo

Slide 3

Slide 3 text

Yusuke Kato / Yahoo Japan Corporation - Technology Incubation, CTO - Vald Founder/Certified OSS Developer (Vald) - Joined as a new graduate in 2016. Belongs to the company-wide troubleshooting SWAT team and is involved in multiple development projects. Vald development began in 2019. I'm also the PO of the technology domain of the R&D organization Technology Incubation and the PO of the in-house Vald team. - Camping/Badminton/Home server operation/!/" - GitHub: @kpango - Twitter: @kpang0 Self-Introduction: Yusuke Kato Self-Introduction

Slide 4

Slide 4 text

Vector Search

Slide 5

Slide 5 text

What is Vector Search? It is a method of searching for similar vectors from indexed vectors by expressing the similarity of unstructured data such as images and sounds in a vector space and calculating the distance between vectors. In the case of high-dimension vectors, k Nearest Neighbor (kNN) increases the number of calculations, so it is common to search using Approximate Nearest Neighbor (ANN). !Data formats that can be handled: text, image, video, audio, binary, etc. !Range of applications: Similarity search, recommendations, data analysis, etc. !Usage examples: Google image search, YouTube, Spotify, etc.

Slide 6

Slide 6 text

Components of Vector Search Embedding Algorithm Index Store BERT, ML models NGT, ScaNN, Faiss In-Memory, Storage

Slide 7

Slide 7 text

Vector Search Flow Architecture of Vector Search

Slide 8

Slide 8 text

Indexing Data Flow Architecture of Vector Search

Slide 9

Slide 9 text

Search Data Flow Architecture of Vector Search

Slide 10

Slide 10 text

Trend of Vector Search

Slide 11

Slide 11 text

Vector Search Libraries SPTAG NGT HNSW Faiss Annoy ScaNN

Slide 12

Slide 12 text

Vector Search Engines Pinecone Milvus Weaviate Qdrant Vertex AI Matching Engine Vald Standalone Service Cloud Service

Slide 13

Slide 13 text

Vector Search Engines Pinecone Milvus Weaviate Qdrant Vertex AI Matching Engine Vald Standalone Service Cloud Service

Slide 14

Slide 14 text

Vald

Slide 15

Slide 15 text

What is Vald? !A distributed fast approximate nearest neighbor dense vector search engine for OSS running on Kubernetes !Enables fast searching from billion-scale vector data !Flexibly customizable according to user needs !Multilingual support !Technologies used

Slide 16

Slide 16 text

Main Features Distributed Index Extensibility Replication Horizontal Scalability Auto Backup Asynchronous Indexing

Slide 17

Slide 17 text

Architecture

Slide 18

Slide 18 text

Vald LB Gateway !Vald Main Gateway !Resource-based load balancer !Manage index replicas !Aggregate and Re-Rank search results

Slide 19

Slide 19 text

Vald Discoverer ! Frequent acquisition of the resource status of Vald components and Kubernetes Nodes deployed on Kubernetes ! Sharing resource status with LB Gateway and Index Manager

Slide 20

Slide 20 text

Vald Index Manager !Vald's index management component !Controls agent index timing and prevents "stop of the world"

Slide 21

Slide 21 text

Vald Agent !Index retention of vector data !CRUD processing for vectors !Can be used as a standalone

Slide 22

Slide 22 text

Distributed Index Replication

Slide 23

Slide 23 text

Distributed Index Replication ! Looks at the cluster resources and distributes the index to multiple agents so that the memory is smooth ! Improves search speed by using multiple compute resources ! Prevents index loss due to failures with replicas and multiple backup methods

Slide 24

Slide 24 text

Data Flow Overview Distributed Index Replication

Slide 25

Slide 25 text

Determine Insert Destination Distributed Index Replication

Slide 26

Slide 26 text

Determine Insert Destination Distributed Index Replication

Slide 27

Slide 27 text

Determine Insert Destination Distributed Index Replication

Slide 28

Slide 28 text

Determine Insert Destination Distributed Index Replication

Slide 29

Slide 29 text

Determine Insert Destination Distributed Index Replication

Slide 30

Slide 30 text

Determine Insert Destination Distributed Index Replication

Slide 31

Slide 31 text

Insert Vector to Replica Group Distributed Index Replication

Slide 32

Slide 32 text

Without Resource Based Load Balancing Memory Based Distributed Index

Slide 33

Slide 33 text

With Resource Based Load Balancing Memory Based Distributed Index

Slide 34

Slide 34 text

Automatically Indexing & Backup

Slide 35

Slide 35 text

About Automatically Indexing & Backup Data Flow of Indexing & Backup

Slide 36

Slide 36 text

Search Architecture Overview

Slide 37

Slide 37 text

Search Operation Data Flow

Slide 38

Slide 38 text

Search Operation Data Flow

Slide 39

Slide 39 text

Detailed Aggregation Flow Aggregate Search Result

Slide 40

Slide 40 text

Aggregate Search Result Detailed Aggregation Flow

Slide 41

Slide 41 text

Detailed Aggregation Flow Get search result from each Agent

Slide 42

Slide 42 text

Get search result from each Agent Detailed Aggregation Flow

Slide 43

Slide 43 text

Get search result from each Agent Detailed Aggregation Flow

Slide 44

Slide 44 text

Get search result from each Agent Detailed Aggregation Flow

Slide 45

Slide 45 text

Detailed Aggregation Flow Get search result from each Agent

Slide 46

Slide 46 text

Detailed Aggregation Flow Get search result from each Agent

Slide 47

Slide 47 text

Detailed Aggregation Flow Get search result from each Agent !Only some results can be processed even if the number of Top K or nodes is large !Works well with network latency and noisy neighbor problems !Considers timeout processing, etc.

Slide 48

Slide 48 text

Agenda !Vector Search Overview !Trend of Vector Search !Vald !Use Case of Vald !Demo

Slide 49

Slide 49 text

Kiichiro Yukawa / Yahoo Japan Corporation - Technology Incubation, CTO - Vald PM / Vald CRE - Joined as a new graduate in 2016. Joined the Vald team in 2020 after planning, developing, operating, and maintaining internal systems. Engaged in Vald's development project management and CRE domain. - Tennis, snowboarding, golf, camping, cooking Self-Introduction: Kiichiro Yukawa Self-Introduction

Slide 50

Slide 50 text

Usecase

Slide 51

Slide 51 text

Use Case Examples Use-cases

Slide 52

Slide 52 text

Search Use-cases Acquires object data similar to the object data you want to search for

Slide 53

Slide 53 text

Classification Use-cases Acquires object data similar to the object data you want to search for Categorizes which group (category) the entered object data belongs to

Slide 54

Slide 54 text

Recommendation Use-cases Acquires object data similar to the object data you want to search for Returns the items you want to recommend for the searched object data as search results

Slide 55

Slide 55 text

Analysis Use-cases Acquires object data similar to the object data you want to search for Can be used for anomaly detection, malware detection, etc.

Slide 56

Slide 56 text

!Searches for similar products from product images (Some products) Similar product image search Source of Image: https://store.shopping.yahoo.co.jp/zozo/67501203.html (2022-11-17)

Slide 57

Slide 57 text

!Infers product information from product images when listing Product information inference

Slide 58

Slide 58 text

Similar document image search 「JAPAN SEARCH」 https://jpsearch.go.jp/（operated by the National Library）

Slide 59

Slide 59 text

Similar source code search !Vectorize source code using CodeBert !Matching of recruiting companies and engineers with similarity search of source code Matching engineers and companies with code Engineers Recruiting Manager

Slide 60

Slide 60 text

How to Implement

Slide 61

Slide 61 text

How to Deploy Vald How to deploy the Vald cluster ! Deploy using Helm • Deployable in a few steps • Edit and upgrade clusters via Helm commands ! Deploy using the vald-helm-operator • Operate Vald cluster using Kubectl • Manage so that vald-helm-operator is in an ideal state based on Custom Resource Definition • Recommended for production environments

Slide 62

Slide 62 text

Deployment procedure 1.Prepare the Kubernetes Cluster 2.Add the Vald chart to Helm Repo 3.Create/Modify values.yaml • Vald-only configuration file when deploying with Helm • You will see the default settings for fields not filled in 4.Helm commandでapply Deploying Vald cluster with Helm Deployment the Vald cluster using Helm chart #Description of common setting items defaults: logging: info image: "v1.6.3" ... #Settings for each component gateway: #Configuration of vald-lb-gateway lb: gateway_config: index_replica: 2 agent: #Configuration of vald-agent-ngt ngt: dimension: 784 ... ... $ helm repo add vald https://vald.vdaas.org/charts $ helm install vald vald/vald --values values.yaml

Slide 63

Slide 63 text

Demo

Slide 64

Slide 64 text

Deploy Vald cluster

Slide 65

Slide 65 text

chiVe Demo Pre-Process & Insert

Slide 66

Slide 66 text

chiVe Demo Search

Slide 67

Slide 67 text

chiVe Demo Update & Remove

Slide 68

Slide 68 text

chiVe Demo Word Analogies

Slide 69

Slide 69 text

Clean Up Vald cluster

Slide 70

Slide 70 text

Roadmap

Slide 71

Slide 71 text

l Faiss support l QBG (public beta) 2023/01 2023/02 2023/03 2022/12 Roadmap l Continuous Benchmark tool l Request Mirroring (BCP) l Network Policy l Efficient Search for large top-K l Index correction l Content Search

Slide 72

Slide 72 text

Summary

Slide 73

Slide 73 text

Summary ! Introduction to vector search ! Introduction to Vald and distinctive features ! Presentation of Vald's application domain and application examples ! Demo with ChiVe

Slide 74

Slide 74 text

Contact Us GitHub Web Site Slack

Slide 75

Slide 75 text

We’re waiting your contribution! GitHub

Slide 76

Slide 76 text

Thank you