20250226 NLP colloquium: "SoftMatcha: 10億単語規模コーパス検索のための柔らかくも高速なパターンマッチャー"

Transcript

2. ◼ ⚫ ⚫ ⚫ ⚫ ⚫ ◼ ⚫ ⚫ ⚫
3. ◼ ◼ ◼ ◼
4. ◼ ◼ ◼ ◼
5. ◼ ◼ ◼ ◼
6. 𝑤 ◼ 𝑤 ◼ ⚫ 𝑤 ⚫ 𝑤 ◼ ⚫
7. ◼ ⚫ ◼ ⚫ ◼ ⚫ (Radovanovic+, JMLR2010) ⚫ ▶ Wang+, (arxiv) 2022, “Text Embeddings by Weakly-Supervised Contrastive Pre-training”. Radovanovic+, JMLR 2010, “Hubs in space: Popular nearest neighbors in high-dimensional data”.
8. ◼ ◼ ◼ ◼
9. ◼ ⚫ 𝐩 = 𝑝1 , … , 𝑝𝑀 ∈ Σ∗ ⚫ 𝐭 = 𝑡1 , … , 𝑡𝑁 ∈ Σ∗ ▶ Σ∗ ◼ ⚫ ⚫ ◼ ◼
10. ◼ ⚫ 𝑤 ∈ 𝒱 𝐷 ⚫ 𝐯𝑤 ∈ ℝ𝐷 ≔ 𝑤 ⚫ ▶ cos 𝐯person , 𝐯people > cos 𝐯person , 𝐯bird ▶
11. ◼ ◼ ⚫ ※
12. ◼ ⚫ 𝑡𝑖 = 𝑝𝑗 cos 𝐯𝑡𝑖 , 𝐯𝑝𝑗 ≥ 𝛼 ▶ 𝛼 = 1.0 ※ 𝛼 = 0.7 ◼ ⚫ ⚫
13. 𝑡1 𝑡2 𝑡3 𝑡4 𝑡5 𝑡6 𝑡7 𝑡8 𝑡9 𝑡10 𝑡11 𝑡12 𝑡13 ◼ ◼ ◼ ⚫
14. ◼ 𝒮𝑤 ≔ 𝑣 ∈ 𝒱 cos 𝐯𝑣 ⊤𝐯𝑤 ≥ 𝛼 ⚫ 𝑤 𝒮we 𝒮talk 𝒮about
15. ⇔ 𝒮we , 𝒮talk , 𝒮about ⇔ 𝒮we , 𝒮talk , 𝒮about 𝑖, 𝑖 + 1, 𝑖 + 2 𝑖 𝒮we 𝒮talk 𝒮about
16. 𝒮we 𝒮talk 𝒮about 𝒮we ℳ ℳ ← 1,10,6 𝒮talk ℳ ℳ′ ← 2 − 1, 11 − 1,7 − 1 = 1,10,6 ℳ ← ℳ ∩ ℳ′ = 1,10,6 𝒮about ℳ ℳ′ ← 8 − 2, 12 − 2 = 6,10 ℳ ← ℳ ∩ ℳ′ = 6,10 ℳ
17. 𝒮we 𝒮talk 𝒮about 𝑡1 𝑡2 𝑡3 𝑡4 𝑡5 𝑡6 𝑡7 𝑡8 𝑡9 𝑡10 𝑡11 𝑡12 𝑡13
18. 𝒮𝑝1 𝐼𝒮𝑝1 𝒮𝑝𝑀 𝐼𝒮𝑝𝑀 𝒮𝑝1 𝐼𝒮𝑝1 ℳ ℳ ← 𝐼𝒮𝑝1 𝑘 = 2, … , 𝑀 ℳ′ ← 𝑖 − 𝑘 + 1 𝑖 ∈ 𝐼𝒮𝑝𝑘 ℳ ← ℳ ∩ ℳ′ ℳ 𝐩 = 𝑝1 , … , 𝑝𝑀
19. ◼ ⚫ ▶ ▶ ⚫
20. ◼ ◼ ⚫ (Wang+, 2024) ⚫ (Douze+, 2024) (Malkov & Yashunin, IEEE TPAMI, 2018) ◼ ⚫ ▶ 𝛼 = 0.55 (Pennington+, EMNLP2014) ▶ 𝛼 = 0.50 (Grave+, arXiv:1802.06893) Wang+, arXiv:2402.05672, “Multilingual E5 Text Embeddings: A Technical Report”. Douze+, arXiv:2401.08281, “The Faiss library”. Malkov & Yashunin, IEEE TPAMI, 2018, “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs”. Pennington+, EMNLP2014, “GloVe: Global Vectors for Word Representation”. Grave+, arXiv:1802.06893, “Learning Word Vectors for 157 Languages”.
21. ◼ ⚫ ▶ ⚫
22. ◼ ⚫ ⚫ ◼ ⚫
23. ◼ ⚫ (Crane, IJDL 2023) ⚫ (Bothwell+, EMNLP2023) ◼ Crane, IJDL 2023, “The Perseus Digital Library and the future of libraries.”. Bothwell+, EMNLP2023, “Introducing Rhetorical Parallelism Detection: A New Task with Datasets, Metrics, and Baselines”.
24. ◼ ⚫ ⚫ ⚫ ◼ ◼
25. ◼ ⚫ ◼ ⚫ ▶ ◼ ⚫ ▶ ▶ ⚫ ⚫
26. ◼ ⚫ 𝐼𝒮𝑝𝑘 ▶ ⚫ ◼ ⚫
27. ◼ ◼ ⚫ ⚫ ⚫ 𝑂 1 ▶ ◼ ⚫ ⚫ ⚫ 𝑂 log |𝐵| ▶
28. ◼ ⚫ ⚫ ◼ ⚫ ⚫ ▶ ⚫