Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DatastoreからSpannerへのゼロダウンタイム移行
Search
Kanji Yomoda
February 18, 2021
0
110
DatastoreからSpannerへのゼロダウンタイム移行
Kanji Yomoda
February 18, 2021
Tweet
Share
More Decks by Kanji Yomoda
See All by Kanji Yomoda
Elasticstack Terraform Providerの最近のアップデート
kyomo
0
94
Goで作る全文検索エンジンライブラリ
kyomo
0
1k
Terraform Provider開発のノウハウ
kyomo
1
210
Paper reading - ROSE: Robust Caches for Amazon Product Search
kyomo
0
40
Elasticsearch Aggregations
kyomo
0
67
AlloyDB overview
kyomo
0
89
Unsupervised synonym Extraction
kyomo
0
240
pubsub_cli - CLI Tool for Cloud Pub/Sub
kyomo
0
100
DRY & 型安全にテスト用structを初期化しよう
kyomo
1
620
Featured
See All Featured
Building Adaptive Systems
keathley
40
2.4k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
10
1.3k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
6
570
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.8k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.4k
Docker and Python
trallard
44
3.3k
For a Future-Friendly Web
brad_frost
176
9.6k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
Fontdeck: Realign not Redesign
paulrobertlloyd
83
5.4k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
4
430
Being A Developer After 40
akosma
89
590k
StorybookのUI Testing Handbookを読んだ
zakiyama
28
5.5k
Transcript
Datastore͔ΒSpannerͷ θϩμϯλΠϜҠߦ Kanji Yomoda
ࣗݾհ Kanji Yomoda / BIR @k__yomo - ৽ଔͰM3ʹೖࣾ͠ݱࡏ2 - αʔόʔαΠυ(Go)ɺΫϥυΠϯϑϥ(GCP,
Terraform) पΓΛओʹ୲
࣍ 1. Ξϯέʔτ৴γεςϜͱ 2. Datastoreར༻࣌ͷ՝ 3. ҠߦઌબఆɾPoC 4. Ҡߦઃܭ 5.
Ҡߦޙͷ݁Ռ 6. ݱࡏͷӡ༻ɾࠓޙͷ՝
1. Ξϯέʔτ৴γεςϜͱ
1. Ξϯέʔτ৴γεςϜͱ • Ϣʔβʔͷ࠷దͳΞϯέʔτ৴Λͱ͢ΔϚΠΫϩαʔϏε ◦ ୭ʹͲͷνϟωϧ(ΞϯέʔτҰཡϖʔδɺαΠτࠂɺϝʔϧɺαΠτ௨ )ͰͲΕ͘Β͍৴͢Δ͔ • ֤ΞϯέʔτγεςϜͱCloud Pub/SubΛ༻͍ͯඇಉظʹ࿈ܞ
(Ξϯέʔτ৴։࢝ɺճใͷಉظͳͲ) • Go, App Engine, Datastore, Cloud Pub/Sub
1. Ξϯέʔτ৴γεςϜͱ(DBؔ࿈) • m3.comͷτοϓϖʔδͱಉ͘͡Β͍ͷΞΫηεྔ • READϔϏʔ (ϢʔβʔͷαΠτΞΫηε࣌ʹɺճՄೳͳΞϯέʔτΛฦ͢ॲཧ͕ϝΠϯ) • ϢʔβʔͷIDͰσʔλΛҾ͘Ϣʔεέʔε͕ओͳͨΊDatastoreΛॳ࠾༻
1. Ξϯέʔτ৴γεςϜͱ(σʔλಉظ) • Cloud Pub/SubͷPush SubscriptionΛར༻͠ɺ ΞϯέʔτγεςϜ͔Β৴ใճใΛऔಘ • Pub/Subͷϝοηʔδ(Πϕϯτ)ͷड͚ޱશͯԼهΛຬͨ͢ɻ ◦
ႈ ◦ ॱෆಉ => ԿճͲͷॱংͰདྷͯϝοηʔδ͕શͯॲཧ͞Ε͑͢͞Ε݁Ռ߹͕อͨΕΔ
2. ৴γεςϜͰͷDatastoreར༻࣌ͷ՝ 1. ΞϓϦέʔγϣϯͰͷෳࡶɾͳJOIN 2. ैྔ՝ۚͳͷͰɺΞϓϦέʔγϣϯͷεέʔϧʹͬͯྉۚεέʔϧ͢Δ ճආํ๏͋Δ͕ɺͦͪΒख͕͔͔ؒΔ 1. 1ʹΑΔϨΠςϯγ্ঢɺ2ʹΑΔྉۚ૿ՃͷճආͷͨΊʹΩϟογϡґଘ =>
ߴϨΠςϯγɺߴྉۚɾߴ։ൃίετɺΩϟογϡґଘ(Redis) લఏ: εΩʔϚઃܭΛਖ਼نԽͯ͠͠·͍ͬͯͨ
શΞϯέʔτγεςϜͷ৴ / ճͷήʔτΣΠΛ୲͍ͬͯΔͨΊɺ ৴γεςϜ͕ࢭ·ΔͱΞϯέʔτશମ͕ࢭ·Δ => ग़དྷΕ৴γεςϜΛࢭΊͨ͘ͳ͍ 3. Ҡߦ࣌ͷ՝
- ϨΠςϯγͷԼ - ։ൃίετͷԼ - Ωϟογϡґଘ͔Βͷ٫ - + ৴γεςϜΛग़དྷΔ͚ͩࢭΊͳ͍ 3.
ղܾ͍ͨ͠՝·ͱΊ
3. Ҡߦઌબఆ - Cloud Spanner or Cloud SQL(PostgreSQL) - Մ༻ੑ
& εέʔϥϏϦςΟ & ٕज़తઓ => Cloud Spanner
3. Cloud SpannerͷPoC - Datastore & ݁Ռ߹ੑ͔ΒͷҠߦͩͬͨͨΊɺ੍؇ͷํ - ओʹRDBMSͱͷࠩ(εΩʔϚઃܭɺΫϥΠΞϯτϥΠϒϥϦɺSQL)Λ֬ೝ -
ΠϯλʔϦʔϒͷ֓೦ - mutation্ݶ - window͕ؔ͑ͳ͍ - ຊ൪૬ͷσʔλΛ༻ҙ͠ෛՙςετ
4. Ҡߦઃܭ Datastoreͷಉظܦ࿏ͨ͠··ɺ৽ͨʹSpanner༻ͷಉظܦ࿏ΛՃ - ݁Ռ߹ੑʹΑΔθϩμϯλΠϜҠߦ͕Մೳ - DatastoreͷΓ͕͠༰қ
None
None
“ϝοηʔδ͕શͯॲཧ͞Ε͑͢͞Ε࠷৽ͷঢ়ଶ͕อͨΕΔ” ͨΊɺ ↑ͷΑ͏ʹ͢ΔࣄͰɺաڈσʔλಉظதʹߋ৽͕͋ͬͨϨίʔυPub/Subϝο ηʔδ͔ΒͷಉظͰશͯ࠷৽ͷঢ়ଶʹߋ৽͞ΕΔ
1. Spanner༻ͷPub/Sub PushαϒεΫϦϓγϣϯΛՃ (͜ͷ࣌ͰPushઌͷΤϯυϙΠϯτ͕ͳ͍ͨΊɺϝοηʔδཹ) 2. Datastore͔Βͷաڈσʔλશಉظ 3. Spanner༻Pub/Sub PushઌΤϯυϙΠϯτՃΛϦϦʔε (ཹϝοηʔδ͕ॲཧ͞ΕΔ)
4. σʔλͷ߹ੑνΣοΫ 5. SpannerଓΓସ͑ϦϦʔε 4. Ҡߦઃܭ
DI͢ΔϦϙδτϦΛ DatastoreͱSpannerͰΓସ͑ Δ ࣮
4. ҠߦʹͨͬͯؾΛ͚ͭͨ - ग़དྷΔ͚ͩϩʔϧόοΫՄೳͳঢ়ଶΛอͭ (ࠓճϦϦʔεޙ1ϲ݄ޙ΄ͲDatastoreͷಉظซ) - DBҠߦҎ֎ͷ͜ͱΛߦΘͳ͍(ϦϑΝΫλϦϯάͳͲ)
5. Ҡߦޙͷ݁Ռ - θϩμϯλΠϜͰͷҠߦޭ - 400msޙΛ͏Ζ͍͍ͭͯͨϨΠςϯγ͕100msʹ - Ωϟογϡͷґଘ٫ - SQLʹΑͬͯΞϓϦέʔγϣϯͷίʔυ͕ཧ
6. ݱࡏͷӡ༻ - Spannerຊ൪Πϯελϯε(1~3node)ͱqaɾϩʔΧϧ։ൃڞ༗ͷΠϯελϯε(1node) - xo/xoͷSpanner൛Ͱ͋Δcloudspannerecosystem/yoΛར༻ (εΩʔϚ͔ΒGoͷDTOίʔυΛੜ͢ΔϥΠϒϥϦ) - QAڥͷεΩʔϚ ϚΠάϨʔγϣϯwrenchɺຊ൪ख࡞ۀ
- CISpannerͷΤϛϡʔϨʔλʔΛ༻
6. ࠓޙͷ՝ ҆ఆతͳӡ༻ͷཱ֬ - εΩʔϚ ϚΠάϨʔγϣϯͷඋ - Φʔτεέʔϧ - CIͷຊSpannerʹΑΔςετ࣮ߦͷՃ
(ݱࡏCIͰΤϛϡϨʔλͷΈ༻)
·ͱΊ - Datastoreར༻࣌ͷ՝ɾ৴γεςϜͷ͔ΒSpannerͷҠߦΛܾఆ - Cloud Pub/SubͱσʔλಉظͷઃܭΛ׆༻͠ɺ θϩμϯλΠϜͰͷDBҠߦΛ࣮ࢪ - ݱࡏΑΓ҆ఆతͳӡ༻Λࢦͯ͠վળத