Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
データ鮮度を落とさずに安全にReindexしたい
Search
kotaroooo0
February 26, 2025
0
83
データ鮮度を落とさずに安全にReindexしたい
kotaroooo0
February 26, 2025
Tweet
Share
More Decks by kotaroooo0
See All by kotaroooo0
検索エンジン自作入門 Go Conference 2021 Spring
kotaroooo0
17
7.4k
転置インデックスでどう検索しているか
kotaroooo0
0
310
俺の全文検索エンジン(Go製)を作り始めた
kotaroooo0
0
110
ぼくのかんがえたさいきょうのDocker Build
kotaroooo0
0
87
Featured
See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
131
19k
Faster Mobile Websites
deanohume
309
31k
Testing 201, or: Great Expectations
jmmastey
45
7.6k
Making Projects Easy
brettharned
117
6.3k
Visualization
eitanlees
146
16k
Building an army of robots
kneath
306
46k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
21k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
GraphQLとの向き合い方2022年版
quramy
49
14k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Transcript
@kotaroooo0 2025/02/26 Elasticsearch Community in Tokyo 2025 σʔλΛམͱͣ͞ʹ҆શʹ Reindex͍ͨ͠ 1
ࣗݾհ 2 ҆ୡޫଠ @kotaroooo0 גࣜձࣾ10X ωοτεʔύʔͷݕࡧਪનͷ ։ൃӡ༻Λ୲͍ͯ͠·͢
ΞδΣϯμ 3 1. Reindexʹ͍ͭͯ 2. 10XͰͷ͜Ε·ͰͷReindexํ๏ͱ 3. 10XͰͷ৽͍͠Reindexํ๏ 4. ·ͱΊ
Reindexʹ͍ͭͯ • ΠϯσοΫεΛ࠶ߏங͢Δ࡞ۀ • ͳͥඞཁͳͷ͔ʁ • ܗଶૉղੳͷࣙॻߋ৽ͷͨΊ • طଘϑΟʔϧυͷมߋɾআͳͲͷͨΊ •
Ͳ͏ͬͯΔ͔ʁ • ୯७ʹߟ͑ΔͱɺReindex API → ΤΠϦΞεΛΓସ͑ • ͔͠͠ɺΠϯσοΫεΛࠩߋ৽͢Δ߹ɺ͜ͷΓํͰσʔλߋ৽࿙ Ε͕ൃੜͯ͠͠·͏… 4 →
10XͰͷ͜Ε·ͰͷReindexํ๏ લఏ • ϝΠϯͷΠϯσοΫεωοτεʔύʔͷΛݕࡧ͢ΔͨΊͷΠϯσοΫε • σʔλͷཧআͳ͠ • ͓٬͞·͔ΒͷϦΫΤετʹΑΔࢀর͕େྔʹ͋Δ • ߋ৽͕ৗʹ͋Δ
• 3shards, 2replicaͰ1shard 12GB͘Β͍ 5 10XͷΠϯσΩγϯάΞʔΩςΫνϟ
10XͰͷ͜Ε·ͰͷReindexํ๏ 6 1. Reindex APIΛୟ͘(70) 2. ΤΠϦΞεΛΓସ͑Δ 3. ߋ৽࿙ΕͷσʔλΛdb͔Β औಘ͠࠶publish(10)
10XͰͷ͜Ε·ͰͷReindexํ๏ • σʔλ͕མͪΔ͕͍࣌ؒ • Reindex͕1࣌ؒҎ্͔͔Γͦͷؒʹߋ৽͞Εͨσʔλ͕ଟ͍ • ΤΠϦΞεΓସ͑ޙʹσʔλճ෮͢Δͷʹ͕͔͔࣌ؒΔ • σʔλճ෮ͷͨΊͷॲཧ͕ɺdb,
queue, workerʹґଘ͠Τϥʔ͕ൃੜ ͍͢͠ 7
10XͰͷ৽͍͠Reindexํ๏ 8 https://tech.legalforce.co.jp/entry/2021/12/21/190129
10XͰͷ৽͍͠Reindexํ๏ • Reindex APIͰqueryΛॻ͚Δ͜ͱΛͬͨ • updatedAtͰߜΓࠐΊɺ͚ࠩͩReindexͰ͖Δ • Կճ͔Reindex APIΛୟ͚ɺঃʑʹॴཁ͕࣌ؒ͘ͳΓσʔλ͕΄΅མͪ ͳ͘ͳΔͷͰ…?
• ·ͨElasticsearch୯ମͰReindexΛ݁Ͱ͖γϯϓϧʹͳΓোੑ্͕͢Δ 9 → →
10XͰͷ৽͍͠Reindexํ๏ 10
10XͰͷ৽͍͠Reindexํ๏ ͬͯΈͨ݁Ռ • 3ճReindexΛ܁Γฦͯ͠Έͨ • ຊ൪ڥ • 1ճͷReindex 4368ඵ •
2ճͷReindex 261ඵ • 3ճͷReindex 331ඵ • ΤΠϦΞεΓସ͑ޙͷReindex 1ඵ • σʔλ͕མͪΔ͕࣌ؒ΄ͱΜͲͳ͍ʂ • ReindexͷλΠϛϯάʹΑΓॴཁ࣌ؒલޙͦ͠͏ 11 ࢀߟ: ։ൃڥ • 1ճͷReindex 5743ඵ • 2ճͷReindex 1028ඵ • 3ճͷReindex 577ඵ • ΤΠϦΞεΓସ͑ޙͷReindex 603ඵ
10XͰͷ৽͍͠Reindexํ๏ ͠ཧআ͕͋ͬͨΒ…? • ͜ͷํ๏Ͱෆे • Reindex API͚ͩͰʮআ͞Εͨσʔλʯ͑ͳ͍ • Ͳ͏ରԠ͢Δ͔ •
ํ๏1: ཧআʹํࣜΛม͑Δ • ํ๏2: আϩάΛه͓͖ͯ͠࠷ޙʹআ͢Δ • ํ๏3: ΤΠϦΞεΓସ͑ޙʹdbͱElasticsearchΛൺֱ͠ෆཁͳυΩϡϝϯτΛ আ͢Δ 12
·ͱΊ Reindex APIΛෳճୟ͘ํ๏ʹΑΓ • σʔλ͕མͪΔ͕࣌ؒେ෯ʹॖͨ͠ • ElasitcsearchͷΈͰReindexΛྃͰ͖ɺγϯϓϧʹͳΓোੑ্͕ͨ͠ 13 → →