Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
データ鮮度を落とさずに安全にReindexしたい
Search
kotaroooo0
February 26, 2025
0
44
データ鮮度を落とさずに安全にReindexしたい
kotaroooo0
February 26, 2025
Tweet
Share
More Decks by kotaroooo0
See All by kotaroooo0
検索エンジン自作入門 Go Conference 2021 Spring
kotaroooo0
17
7.2k
転置インデックスでどう検索しているか
kotaroooo0
0
250
俺の全文検索エンジン(Go製)を作り始めた
kotaroooo0
0
100
ぼくのかんがえたさいきょうのDocker Build
kotaroooo0
0
81
Featured
See All Featured
Site-Speed That Sticks
csswizardry
4
400
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
59k
Optimizing for Happiness
mojombo
376
70k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
12
980
Making the Leap to Tech Lead
cromwellryan
133
9.1k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
133
33k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
Code Reviewing Like a Champion
maltzj
521
39k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
29
1k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Transcript
@kotaroooo0 2025/02/26 Elasticsearch Community in Tokyo 2025 σʔλΛམͱͣ͞ʹ҆શʹ Reindex͍ͨ͠ 1
ࣗݾհ 2 ҆ୡޫଠ @kotaroooo0 גࣜձࣾ10X ωοτεʔύʔͷݕࡧਪનͷ ։ൃӡ༻Λ୲͍ͯ͠·͢
ΞδΣϯμ 3 1. Reindexʹ͍ͭͯ 2. 10XͰͷ͜Ε·ͰͷReindexํ๏ͱ 3. 10XͰͷ৽͍͠Reindexํ๏ 4. ·ͱΊ
Reindexʹ͍ͭͯ • ΠϯσοΫεΛ࠶ߏங͢Δ࡞ۀ • ͳͥඞཁͳͷ͔ʁ • ܗଶૉղੳͷࣙॻߋ৽ͷͨΊ • طଘϑΟʔϧυͷมߋɾআͳͲͷͨΊ •
Ͳ͏ͬͯΔ͔ʁ • ୯७ʹߟ͑ΔͱɺReindex API → ΤΠϦΞεΛΓସ͑ • ͔͠͠ɺΠϯσοΫεΛࠩߋ৽͢Δ߹ɺ͜ͷΓํͰσʔλߋ৽࿙ Ε͕ൃੜͯ͠͠·͏… 4 →
10XͰͷ͜Ε·ͰͷReindexํ๏ લఏ • ϝΠϯͷΠϯσοΫεωοτεʔύʔͷΛݕࡧ͢ΔͨΊͷΠϯσοΫε • σʔλͷཧআͳ͠ • ͓٬͞·͔ΒͷϦΫΤετʹΑΔࢀর͕େྔʹ͋Δ • ߋ৽͕ৗʹ͋Δ
• 3shards, 2replicaͰ1shard 12GB͘Β͍ 5 10XͷΠϯσΩγϯάΞʔΩςΫνϟ
10XͰͷ͜Ε·ͰͷReindexํ๏ 6 1. Reindex APIΛୟ͘(70) 2. ΤΠϦΞεΛΓସ͑Δ 3. ߋ৽࿙ΕͷσʔλΛdb͔Β औಘ͠࠶publish(10)
10XͰͷ͜Ε·ͰͷReindexํ๏ • σʔλ͕མͪΔ͕͍࣌ؒ • Reindex͕1࣌ؒҎ্͔͔Γͦͷؒʹߋ৽͞Εͨσʔλ͕ଟ͍ • ΤΠϦΞεΓସ͑ޙʹσʔλճ෮͢Δͷʹ͕͔͔࣌ؒΔ • σʔλճ෮ͷͨΊͷॲཧ͕ɺdb,
queue, workerʹґଘ͠Τϥʔ͕ൃੜ ͍͢͠ 7
10XͰͷ৽͍͠Reindexํ๏ 8 https://tech.legalforce.co.jp/entry/2021/12/21/190129
10XͰͷ৽͍͠Reindexํ๏ • Reindex APIͰqueryΛॻ͚Δ͜ͱΛͬͨ • updatedAtͰߜΓࠐΊɺ͚ࠩͩReindexͰ͖Δ • Կճ͔Reindex APIΛୟ͚ɺঃʑʹॴཁ͕࣌ؒ͘ͳΓσʔλ͕΄΅མͪ ͳ͘ͳΔͷͰ…?
• ·ͨElasticsearch୯ମͰReindexΛ݁Ͱ͖γϯϓϧʹͳΓোੑ্͕͢Δ 9 → →
10XͰͷ৽͍͠Reindexํ๏ 10
10XͰͷ৽͍͠Reindexํ๏ ͬͯΈͨ݁Ռ • 3ճReindexΛ܁Γฦͯ͠Έͨ • ຊ൪ڥ • 1ճͷReindex 4368ඵ •
2ճͷReindex 261ඵ • 3ճͷReindex 331ඵ • ΤΠϦΞεΓସ͑ޙͷReindex 1ඵ • σʔλ͕མͪΔ͕࣌ؒ΄ͱΜͲͳ͍ʂ • ReindexͷλΠϛϯάʹΑΓॴཁ࣌ؒલޙͦ͠͏ 11 ࢀߟ: ։ൃڥ • 1ճͷReindex 5743ඵ • 2ճͷReindex 1028ඵ • 3ճͷReindex 577ඵ • ΤΠϦΞεΓସ͑ޙͷReindex 603ඵ
10XͰͷ৽͍͠Reindexํ๏ ͠ཧআ͕͋ͬͨΒ…? • ͜ͷํ๏Ͱෆे • Reindex API͚ͩͰʮআ͞Εͨσʔλʯ͑ͳ͍ • Ͳ͏ରԠ͢Δ͔ •
ํ๏1: ཧআʹํࣜΛม͑Δ • ํ๏2: আϩάΛه͓͖ͯ͠࠷ޙʹআ͢Δ • ํ๏3: ΤΠϦΞεΓସ͑ޙʹdbͱElasticsearchΛൺֱ͠ෆཁͳυΩϡϝϯτΛ আ͢Δ 12
·ͱΊ Reindex APIΛෳճୟ͘ํ๏ʹΑΓ • σʔλ͕མͪΔ͕࣌ؒେ෯ʹॖͨ͠ • ElasitcsearchͷΈͰReindexΛྃͰ͖ɺγϯϓϧʹͳΓোੑ্͕ͨ͠ 13 → →