Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DB移行を支える技術
Search
onigra
October 25, 2013
Technology
12
18k
DB移行を支える技術
MySQL Casual Vol5 LT資料
http://www.zusaar.com/event/1086003
onigra
October 25, 2013
Tweet
Share
More Decks by onigra
See All by onigra
THE GOAL
onigra
3
130
devsumi-2024-summer
onigra
5
2.6k
第一種低層住居専用地域
onigra
0
280
jaws-ug-ecspresso-meetup-20230808
onigra
0
1.9k
ginza-ruby-kaigi-01
onigra
4
1.1k
PHP-CS-FixerとかAtomとか
onigra
1
1.4k
プログラミング初心者でも始められるコミュニティへの参加と貢献
onigra
4
800
Techblog Deep Dive Meetup #1
onigra
0
2k
とある業務オペレーション自動化の話
onigra
0
1k
Other Decks in Technology
See All in Technology
防災デジタル分野での官民共創の取り組み (2)DIT/CCとD-CERTについて
ditccsugii
0
290
AI時代こそ求められる設計力- AWSクラウドデザインパターン3選で信頼性と拡張性を高める-
kenichirokimura
3
310
カンファレンスに託児サポートがあるということ / Having Childcare Support at Conferences
nobu09
1
560
コンテキストエンジニアリング入門〜AI Coding Agent作りで学ぶ文脈設計〜
kworkdev
PRO
1
840
AWS Top Engineer、浮いてませんか? / As an AWS Top Engineer, Are You Out of Place?
yuj1osm
2
210
Vibe Coding Year in Review. From Karpathy to Real-World Agents by Niels Rolland, CEO Paatch
vcoisne
0
130
フレームワークを意識させないワークショップづくり
keigosuda
0
160
リセラー企業のテクサポ担当が考える、生成 AI 時代のトラブルシュート 2025
kazzpapa3
1
160
プロポーザルのコツ ~ Kaigi on Rails 2025 初参加で3名の登壇を実現 ~
naro143
1
240
GoでもGUIアプリを作りたい!
kworkdev
PRO
0
140
[Keynote] What do you need to know about DevEx in 2025
salaboy
0
170
やる気のない自分との向き合い方/How to Deal with Your Unmotivated Self
sanogemaru
0
500
Featured
See All Featured
Side Projects
sachag
455
43k
The Cost Of JavaScript in 2023
addyosmani
55
9k
4 Signs Your Business is Dying
shpigford
185
22k
Into the Great Unknown - MozCon
thekraken
40
2.1k
GraphQLの誤解/rethinking-graphql
sonatard
73
11k
Leading Effective Engineering Teams in the AI Era
addyosmani
5
400
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.2k
A better future with KSS
kneath
239
18k
Embracing the Ebb and Flow
colly
88
4.8k
Rebuilding a faster, lazier Slack
samanthasiow
84
9.2k
A Tale of Four Properties
chriscoyier
161
23k
A designer walks into a library…
pauljervisheath
209
24k
Transcript
MySQLͷҠߦΛࢧ͑Δٕज़ 1310݄25༵ۚ
Yudai @nekogeruge_987 WEBܥاۀۈ DB(RDBMS, NoSQL) Ruby Chef Fluentd https://github.com/onigra 1310݄25༵ۚ
ࠓͷ݄̍ʹ IBM DB2͔Β MySQL5.5ʹ Ҡߦͨ͠Λ͠·͢ 1310݄25༵ۚ
نײ 1310݄25༵ۚ
allཧ εΩʔϚ 13 ςʔϒϧ 330 ૯Ϩίʔυ 321,906,470 1ςʔϒϧ͋ͨΓͷϨίʔυ 975,474 Ұ൪Ϩίʔυ͕ଟ͍ςʔϒϧ
18,054,990 13ӡ༻͞ΕͯΔ جװγεςϜɺαʔϏεܥDB܈ JavaɺPHPɺVB6ͱ͔… euc-jp -> utf-8 1310݄25༵ۚ
݁Ռɺϯԯԁͷ ܦඅൃੜΛ્ࢭ 1310݄25༵ۚ
۩ମతͳ ࡞ۀ༰ 1310݄25༵ۚ
• DB2ͷϨίʔυΛCSVΤΫεϙʔτ • MySQLLOAD INFILE • /var/lib/mysql ΛslaveͷαʔόҠͯ͠෮ݩʢColdBackupʣ • ϨϓϦέʔγϣϯઃఆ
• FederatedΤϯδϯΛ༻͢ΔಛघͳϨϓϦέʔγϣϯͷߏங (MySQL -> DB2 ͷϨϓϦέʔγϣϯΛ͢Δඞཁ͕͋ͬͨͨΊ) • ͜ΕΒΛShellScript(bash)ͷόονͰߦ͏ 1310݄25༵ۚ
Time Limit 4࣌ؒ ※ΦϑϥΠϯϝϯς8࣌ؒͷ͏ͪ લ4࣌ؒͰྃ͢Δඞཁ͕͋Δ 1310݄25༵ۚ
ରࡦ 1310݄25༵ۚ
1. LOAD࣌ؒͷॖΛਤΔ 1310݄25༵ۚ
•Bulk Insert < LOAD INFILE < Cold Backup •Slow-logɺBin-log ͳͲࢭΊΕΔϩάࢭΊΔ
•INDEXΛDROP -> LOAD -> INDEXషΔ (ςʔϒϧʹΑΓૣ͔ͬͨΓɺషͬͨ··ͱ ͦΜͳʹ͕ࠩແ͔ͬͨΓ) • csvετϨʔδΤϯδϯΛ͏ͱͪΐͬͺΒ͍͚͠Ͳ͏ ·͍͔͘ͳ͔ͬͨʢΓ͔ͨѱ͔ͬͨͷ͔ʣ 1310݄25༵ۚ
͜ͷลάάͬͨΒ ׂͱग़ͯ͘Δ͕ɺ શମͷ࡞ۀͰݟΔͱ େ͖ͳॖʹ ͳΒͳ͔ͬͨ 1310݄25༵ۚ
2.δϣϒϑϩʔͷ࠷దԽ ʢJenkinsͷ׆༻ʣ 1310݄25༵ۚ
1310݄25༵ۚ
Jenkins Build Flow Plugin https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin 1310݄25༵ۚ
Build Pipeline Plugin͕ڧྗʹͳͬͨײ͡ Job FlowΛGroovyͰॻ͚Δ ฒྻ࣮ߦ(Parallel) ϦτϥΠ(Retry) ྫ֎ͷัଊ(guard / rescue)
ͳͲͷػೳΛඋ͍͑ͯΔ 1310݄25༵ۚ
࠷্Ґ parallel ( { build(“order” ) }, { build(“user”) },
{ build(“master”) } ) user build (“export”) build (“rsync”) build (“load”) build (“mysqld stop”) build (“rsync”) parallel ( { build(“master mysqld start”) }, { build(“slave mysqld start”) } ) 1310݄25༵ۚ
ͪͳΈʹɺ͜ͷҊ݅ͷޙ Tivoliͱ͔JP1ͱ͔Γ·ͨ͠ ·ͩۀքྺͰݴ͏ͱएखͳΜͰ… 1310݄25༵ۚ
σʔλҠߦΛCI͢ΔΑ͏ͳײ͡Ͱ ຖ·Θ͢ ࠷ऴతʹ1ΫϦοΫͰҠߦ͕ྃ͢ΔBuild FlowΛ࡞ ↓ ͦΕΛຖ࣮ߦ͠ɺԿճޭͤ͞Δ ↓ Ҡߦຊ൪ʹର͢Δڪා৺Λແ͘͢ 1310݄25༵ۚ
̍ΫϦοΫσʔλҠߦ ʙ͍ͭ·ͰखͰDBҠߦͯ͠ΔΜͰ͔͢ʁʙ ※ @ryuzee͞ΜͷΦϚʔδϡͰ͢ http://www.slideshare.net/Ryuzee/devsumia 1310݄25༵ۚ
શࣗಈԽͰ͖͔ͨ ۃͳɺ࡞ۀऀ͕͍ͳ͍ʢԿ͠ͳ͍ʣ σʔλҠߦͰ͖Δؾ͕͢Δ 1310݄25༵ۚ
༨ஊ δϣϒεέδϡʔϥʔͷΦʔϓϯιʔειϑτΣΞͬͯΠέͯΔͷແ͍ΜͰ͔͢Ͷʁ δϣϒεέδϡʔϥʔͬͯΤϯλʔϓϥΠζք۾Ͱॏๅ͞Εͯͦ͏͔ͩΒ WEBͷਓୡڵຯແ͍ʁ DWHͱ͔݁ߏ͏ͱࢥ͏ΜͰ͚͢Ͳ jobschedulerͳΔάάϥϏϦςΟͷ͍ΦʔϓϯιʔεͷιϑτΣΞ͚͋ͬͨͲɺ ใগͳ͍ 1310݄25༵ۚ
͜ΕͰ·ͩؒʹ߹Θͳ͍ 1310݄25༵ۚ
3. σʔλసૹ࣌ؒͷॖ (Cold Backup) 1310݄25༵ۚ
/var/lib/mysql ԼͷશϑΝΠϧΛ ฒྻͰrsync͢Δ http://qiita.com/nekogeruge_987/ items/ed1bd8704b25b1f509ec 1310݄25༵ۚ
ҙ σΟϨΫτϦߏΛઌʹίϐʔઌʹίϐʔ͓͔ͯ͠ͳ͍ͱrsync͕͚͜Δ ωοτϫʔΫͷଳҬͷଠ͞ʹґଘ͍ͯ͠ΔͷͰɺଳҬ͕ڱ͍ͱ͋·ΓޮՌ͕ແ͍ ϑΝΠϧͷϓϩηεΛىಈ͢ΔͨΊɺ100ສݸϑΝΠϧ͕͋Δͱ100ສϓϩηε͕ͨͪ͋Δ ྃޙʹී௨ͷrsyncΛ࣮ߦ͠ɺ͕ࠩແ͍͔ΛνΣοΫ͢Δͱϕλʔ 1310݄25༵ۚ
1࣌ؒఔ͔͔ͬͯͨͷ͕ 20ఔͰྃ͢ΔΑ͏ʹ ͳΓ·ͨ͠ 1310݄25༵ۚ
൪֎ɿࠓࢥ͏ͱΓ͔ͨͬͨ͜ͱ ShellScripͷUnitTestΛॻ͘ 1310݄25༵ۚ
࣌ShellScript΄΅ॻ͍ͨࣄແ͍ ͍ͭͰʹςετॻ͍ͨࣄແ͍ ࣄલͷҠߦͰΘΕͯͨίʔυΛ͍ճ͢ ࣌ؒແ͔ͬͨͷͰແઅૢʹScript͕૿͍͑ͯ͘ ࠓshunit2ͱ͍͏ShellScriptͷTestingFrameworkͰςετॻ͍ͯΔ https://speakerdeck.com/yudaisuzuki/shell-script-testing-framework-shunit2 1310݄25༵ۚ
Thanks!! https://twitter.com/nekogeruge_987 http://onigra.github.io/ 1310݄25༵ۚ