Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DatastoreからSpannerへのゼロダウンタイム移行
Search
Kanji Yomoda
February 18, 2021
0
110
DatastoreからSpannerへのゼロダウンタイム移行
Kanji Yomoda
February 18, 2021
Tweet
Share
More Decks by Kanji Yomoda
See All by Kanji Yomoda
Elasticstack Terraform Providerの最近のアップデート
kyomo
0
94
Goで作る全文検索エンジンライブラリ
kyomo
0
1k
Terraform Provider開発のノウハウ
kyomo
1
220
Paper reading - ROSE: Robust Caches for Amazon Product Search
kyomo
0
40
Elasticsearch Aggregations
kyomo
0
68
AlloyDB overview
kyomo
0
90
Unsupervised synonym Extraction
kyomo
0
240
pubsub_cli - CLI Tool for Cloud Pub/Sub
kyomo
0
100
DRY & 型安全にテスト用structを初期化しよう
kyomo
1
630
Featured
See All Featured
Navigating Team Friction
lara
183
15k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
100
18k
Designing for humans not robots
tammielis
250
25k
YesSQL, Process and Tooling at Scale
rocio
172
14k
Typedesign – Prime Four
hannesfritz
41
2.5k
Git: the NoSQL Database
bkeepers
PRO
428
65k
Fireside Chat
paigeccino
35
3.2k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
7.1k
KATA
mclloyd
29
14k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.7k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Transcript
Datastore͔ΒSpannerͷ θϩμϯλΠϜҠߦ Kanji Yomoda
ࣗݾհ Kanji Yomoda / BIR @k__yomo - ৽ଔͰM3ʹೖࣾ͠ݱࡏ2 - αʔόʔαΠυ(Go)ɺΫϥυΠϯϑϥ(GCP,
Terraform) पΓΛओʹ୲
࣍ 1. Ξϯέʔτ৴γεςϜͱ 2. Datastoreར༻࣌ͷ՝ 3. ҠߦઌબఆɾPoC 4. Ҡߦઃܭ 5.
Ҡߦޙͷ݁Ռ 6. ݱࡏͷӡ༻ɾࠓޙͷ՝
1. Ξϯέʔτ৴γεςϜͱ
1. Ξϯέʔτ৴γεςϜͱ • Ϣʔβʔͷ࠷దͳΞϯέʔτ৴Λͱ͢ΔϚΠΫϩαʔϏε ◦ ୭ʹͲͷνϟωϧ(ΞϯέʔτҰཡϖʔδɺαΠτࠂɺϝʔϧɺαΠτ௨ )ͰͲΕ͘Β͍৴͢Δ͔ • ֤ΞϯέʔτγεςϜͱCloud Pub/SubΛ༻͍ͯඇಉظʹ࿈ܞ
(Ξϯέʔτ৴։࢝ɺճใͷಉظͳͲ) • Go, App Engine, Datastore, Cloud Pub/Sub
1. Ξϯέʔτ৴γεςϜͱ(DBؔ࿈) • m3.comͷτοϓϖʔδͱಉ͘͡Β͍ͷΞΫηεྔ • READϔϏʔ (ϢʔβʔͷαΠτΞΫηε࣌ʹɺճՄೳͳΞϯέʔτΛฦ͢ॲཧ͕ϝΠϯ) • ϢʔβʔͷIDͰσʔλΛҾ͘Ϣʔεέʔε͕ओͳͨΊDatastoreΛॳ࠾༻
1. Ξϯέʔτ৴γεςϜͱ(σʔλಉظ) • Cloud Pub/SubͷPush SubscriptionΛར༻͠ɺ ΞϯέʔτγεςϜ͔Β৴ใճใΛऔಘ • Pub/Subͷϝοηʔδ(Πϕϯτ)ͷड͚ޱશͯԼهΛຬͨ͢ɻ ◦
ႈ ◦ ॱෆಉ => ԿճͲͷॱংͰདྷͯϝοηʔδ͕શͯॲཧ͞Ε͑͢͞Ε݁Ռ߹͕อͨΕΔ
2. ৴γεςϜͰͷDatastoreར༻࣌ͷ՝ 1. ΞϓϦέʔγϣϯͰͷෳࡶɾͳJOIN 2. ैྔ՝ۚͳͷͰɺΞϓϦέʔγϣϯͷεέʔϧʹͬͯྉۚεέʔϧ͢Δ ճආํ๏͋Δ͕ɺͦͪΒख͕͔͔ؒΔ 1. 1ʹΑΔϨΠςϯγ্ঢɺ2ʹΑΔྉۚ૿ՃͷճආͷͨΊʹΩϟογϡґଘ =>
ߴϨΠςϯγɺߴྉۚɾߴ։ൃίετɺΩϟογϡґଘ(Redis) લఏ: εΩʔϚઃܭΛਖ਼نԽͯ͠͠·͍ͬͯͨ
શΞϯέʔτγεςϜͷ৴ / ճͷήʔτΣΠΛ୲͍ͬͯΔͨΊɺ ৴γεςϜ͕ࢭ·ΔͱΞϯέʔτશମ͕ࢭ·Δ => ग़དྷΕ৴γεςϜΛࢭΊͨ͘ͳ͍ 3. Ҡߦ࣌ͷ՝
- ϨΠςϯγͷԼ - ։ൃίετͷԼ - Ωϟογϡґଘ͔Βͷ٫ - + ৴γεςϜΛग़དྷΔ͚ͩࢭΊͳ͍ 3.
ղܾ͍ͨ͠՝·ͱΊ
3. Ҡߦઌબఆ - Cloud Spanner or Cloud SQL(PostgreSQL) - Մ༻ੑ
& εέʔϥϏϦςΟ & ٕज़తઓ => Cloud Spanner
3. Cloud SpannerͷPoC - Datastore & ݁Ռ߹ੑ͔ΒͷҠߦͩͬͨͨΊɺ੍؇ͷํ - ओʹRDBMSͱͷࠩ(εΩʔϚઃܭɺΫϥΠΞϯτϥΠϒϥϦɺSQL)Λ֬ೝ -
ΠϯλʔϦʔϒͷ֓೦ - mutation্ݶ - window͕ؔ͑ͳ͍ - ຊ൪૬ͷσʔλΛ༻ҙ͠ෛՙςετ
4. Ҡߦઃܭ Datastoreͷಉظܦ࿏ͨ͠··ɺ৽ͨʹSpanner༻ͷಉظܦ࿏ΛՃ - ݁Ռ߹ੑʹΑΔθϩμϯλΠϜҠߦ͕Մೳ - DatastoreͷΓ͕͠༰қ
None
None
“ϝοηʔδ͕શͯॲཧ͞Ε͑͢͞Ε࠷৽ͷঢ়ଶ͕อͨΕΔ” ͨΊɺ ↑ͷΑ͏ʹ͢ΔࣄͰɺաڈσʔλಉظதʹߋ৽͕͋ͬͨϨίʔυPub/Subϝο ηʔδ͔ΒͷಉظͰશͯ࠷৽ͷঢ়ଶʹߋ৽͞ΕΔ
1. Spanner༻ͷPub/Sub PushαϒεΫϦϓγϣϯΛՃ (͜ͷ࣌ͰPushઌͷΤϯυϙΠϯτ͕ͳ͍ͨΊɺϝοηʔδཹ) 2. Datastore͔Βͷաڈσʔλશಉظ 3. Spanner༻Pub/Sub PushઌΤϯυϙΠϯτՃΛϦϦʔε (ཹϝοηʔδ͕ॲཧ͞ΕΔ)
4. σʔλͷ߹ੑνΣοΫ 5. SpannerଓΓସ͑ϦϦʔε 4. Ҡߦઃܭ
DI͢ΔϦϙδτϦΛ DatastoreͱSpannerͰΓସ͑ Δ ࣮
4. ҠߦʹͨͬͯؾΛ͚ͭͨ - ग़དྷΔ͚ͩϩʔϧόοΫՄೳͳঢ়ଶΛอͭ (ࠓճϦϦʔεޙ1ϲ݄ޙ΄ͲDatastoreͷಉظซ) - DBҠߦҎ֎ͷ͜ͱΛߦΘͳ͍(ϦϑΝΫλϦϯάͳͲ)
5. Ҡߦޙͷ݁Ռ - θϩμϯλΠϜͰͷҠߦޭ - 400msޙΛ͏Ζ͍͍ͭͯͨϨΠςϯγ͕100msʹ - Ωϟογϡͷґଘ٫ - SQLʹΑͬͯΞϓϦέʔγϣϯͷίʔυ͕ཧ
6. ݱࡏͷӡ༻ - Spannerຊ൪Πϯελϯε(1~3node)ͱqaɾϩʔΧϧ։ൃڞ༗ͷΠϯελϯε(1node) - xo/xoͷSpanner൛Ͱ͋Δcloudspannerecosystem/yoΛར༻ (εΩʔϚ͔ΒGoͷDTOίʔυΛੜ͢ΔϥΠϒϥϦ) - QAڥͷεΩʔϚ ϚΠάϨʔγϣϯwrenchɺຊ൪ख࡞ۀ
- CISpannerͷΤϛϡʔϨʔλʔΛ༻
6. ࠓޙͷ՝ ҆ఆతͳӡ༻ͷཱ֬ - εΩʔϚ ϚΠάϨʔγϣϯͷඋ - Φʔτεέʔϧ - CIͷຊSpannerʹΑΔςετ࣮ߦͷՃ
(ݱࡏCIͰΤϛϡϨʔλͷΈ༻)
·ͱΊ - Datastoreར༻࣌ͷ՝ɾ৴γεςϜͷ͔ΒSpannerͷҠߦΛܾఆ - Cloud Pub/SubͱσʔλಉظͷઃܭΛ׆༻͠ɺ θϩμϯλΠϜͰͷDBҠߦΛ࣮ࢪ - ݱࡏΑΓ҆ఆతͳӡ༻Λࢦͯ͠վળத