Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
DB移行を支える技術 社内勉強会用
Search
Yudai Suzuki
October 25, 2013
0
87
DB移行を支える技術 社内勉強会用
Yudai Suzuki
October 25, 2013
Tweet
Share
More Decks by Yudai Suzuki
See All by Yudai Suzuki
コメントの書き方について
yudaisuzuki
1
190
Shell Script Testing Framework shunit2
yudaisuzuki
6
3.5k
Featured
See All Featured
Stop Working from a Prison Cell
hatefulcrawdad
266
19k
Faster Mobile Websites
deanohume
299
30k
Web Components: a chance to create the future
zenorocha
305
41k
Ruby is Unlike a Banana
tanoku
96
10k
Become a Pro
speakerdeck
PRO
11
4.5k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
120
39k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
352
28k
Design by the Numbers
sachag
274
18k
Bash Introduction
62gerente
604
210k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
14
1.5k
GitHub's CSS Performance
jonrohan
1025
450k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
274
13k
Transcript
MySQLͷҠߦΛࢧ͑Δٕज़ 1310݄25༵ۚ
ࠓͷ݄̍ʹ IBM DB2͔Β MySQL5.5ʹ Ҡߦͨ͠Λ͠·͢ 1310݄25༵ۚ
نײ 1310݄25༵ۚ
allཧ εΩʔϚ 13 ςʔϒϧ 330 ૯Ϩίʔυ 321,906,470 1ςʔϒϧ͋ͨΓͷϨίʔυ 975,474 Ұ൪Ϩίʔυ͕ଟ͍ςʔϒϧ
18,054,990 13ӡ༻͞ΕͯΔ جװγεςϜɺαʔϏεܥDB܈ JavaɺPHPɺVB6ͱ͔… euc-jp -> utf-8 1310݄25༵ۚ
݁Ռɺϯԯԁͷ ܦඅൃੜΛ્ࢭ 1310݄25༵ۚ
۩ମతͳ ࡞ۀ༰ 1310݄25༵ۚ
• DB2ͷϨίʔυΛCSVΤΫεϙʔτ • MySQLLOAD INFILE • /var/lib/mysql ΛslaveͷαʔόҠͯ͠෮ݩʢColdBackupʣ • ϨϓϦέʔγϣϯઃఆ
• FederatedΤϯδϯΛ༻͢ΔಛघͳϨϓϦέʔγϣϯͷߏங (MySQL -> DB2 ͷϨϓϦέʔγϣϯΛ͢Δඞཁ͕͋ͬͨͨΊ) • ͜ΕΒΛShellScript(bash)ͷόονͰߦ͏ 1310݄25༵ۚ
Time Limit 4࣌ؒ ※ΦϑϥΠϯϝϯς8࣌ؒͷ͏ͪ લ4࣌ؒͰྃ͢Δඞཁ͕͋Δ 1310݄25༵ۚ
ରࡦ 1310݄25༵ۚ
1. LOAD࣌ؒͷॖΛਤΔ 1310݄25༵ۚ
•Bulk Insert < LOAD INFILE < Cold Backup •Slow-logɺBin-log ͳͲࢭΊΕΔϩάࢭΊΔ
•INDEXΛDROP -> LOAD -> INDEXషΔ (ςʔϒϧʹΑΓૣ͔ͬͨΓɺషͬͨ··ͱ ͦΜͳʹ͕ࠩແ͔ͬͨΓ) • csvετϨʔδΤϯδϯΛ͏ͱͪΐͬͺΒ͍͚͠Ͳ͏ ·͍͔͘ͳ͔ͬͨʢΓ͔ͨѱ͔ͬͨͷ͔ʣ 1310݄25༵ۚ
͜ͷลάάͬͨΒ ׂͱग़ͯ͘Δ͕ɺ શମͷ࡞ۀͰݟΔͱ େ͖ͳॖʹ ͳΒͳ͔ͬͨ 1310݄25༵ۚ
2.δϣϒϑϩʔͷ࠷దԽ ʢJenkinsͷ׆༻ʣ 1310݄25༵ۚ
1310݄25༵ۚ
Jenkins Build Flow Plugin https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin 1310݄25༵ۚ
Build Pipeline Plugin͕ڧྗʹͳͬͨײ͡ Job FlowΛGroovyͰॻ͚Δ ฒྻ࣮ߦ(Parallel) ϦτϥΠ(Retry) ྫ֎ͷัଊ(guard / rescue)
ͳͲͷػೳΛඋ͍͑ͯΔ 1310݄25༵ۚ
࠷্Ґ parallel ( { build(“order” ) }, { build(“user”) },
{ build(“master”) } ) user build (“export”) build (“rsync”) build (“load”) build (“mysqld stop”) build (“rsync”) parallel ( { build(“master mysqld start”) }, { build(“slave mysqld start”) } ) 1310݄25༵ۚ
ͪͳΈʹɺ͜ͷҊ݅ͷޙ Tivoliͱ͔JP1ͱ͔Γ·ͨ͠ ·ͩۀքྺͰݴ͏ͱएखͳΜͰ… 1310݄25༵ۚ
σʔλҠߦΛCI͢ΔΑ͏ͳײ͡Ͱ ຖ·Θ͢ ࠷ऴతʹ1ΫϦοΫͰҠߦ͕ྃ͢ΔBuild FlowΛ࡞ ↓ ͦΕΛຖ࣮ߦ͠ɺԿճޭͤ͞Δ ↓ Ҡߦຊ൪ʹର͢Δڪා৺Λແ͘͢ 1310݄25༵ۚ
̍ΫϦοΫσʔλҠߦ ʙ͍ͭ·ͰखͰDBҠߦͯ͠ΔΜͰ͔͢ʁʙ ※ @ryuzee͞ΜͷΦϚʔδϡͰ͢ http://www.slideshare.net/Ryuzee/devsumia 1310݄25༵ۚ
શࣗಈԽͰ͖͔ͨ ۃͳɺ࡞ۀऀ͕͍ͳ͍ʢԿ͠ͳ͍ʣ σʔλҠߦͰ͖Δؾ͕͢Δ 1310݄25༵ۚ
༨ஊ δϣϒεέδϡʔϥʔͷΦʔϓϯιʔειϑτΣΞͬͯΠέͯΔͷແ͍ΜͰ͔͢Ͷʁ δϣϒεέδϡʔϥʔͬͯΤϯλʔϓϥΠζք۾Ͱॏๅ͞Εͯͦ͏͔ͩΒ WEBͷਓୡڵຯແ͍ʁ DWHͱ͔݁ߏ͏ͱࢥ͏ΜͰ͚͢Ͳ jobschedulerͳΔάάϥϏϦςΟͷ͍ΦʔϓϯιʔεͷιϑτΣΞ͚͋ͬͨͲɺ ใগͳ͍ 1310݄25༵ۚ
͜ΕͰ·ͩؒʹ߹Θͳ͍ 1310݄25༵ۚ
3. σʔλసૹ࣌ؒͷॖ (Cold Backup) 1310݄25༵ۚ
/var/lib/mysql ԼͷશϑΝΠϧΛ ฒྻͰrsync͢Δ http://qiita.com/nekogeruge_987/ items/ed1bd8704b25b1f509ec 1310݄25༵ۚ
ҙ σΟϨΫτϦߏΛઌʹίϐʔઌʹίϐʔ͓͔ͯ͠ͳ͍ͱrsync͕͚͜Δ ωοτϫʔΫͷଳҬͷଠ͞ʹґଘ͍ͯ͠ΔͷͰɺଳҬ͕ڱ͍ͱ͋·ΓޮՌ͕ແ͍ ϑΝΠϧͷϓϩηεΛىಈ͢ΔͨΊɺ100ສݸϑΝΠϧ͕͋Δͱ100ສϓϩηε͕ͨͪ͋Δ ྃޙʹී௨ͷrsyncΛ࣮ߦ͠ɺ͕ࠩແ͍͔ΛνΣοΫ͢Δͱϕλʔ 1310݄25༵ۚ
1࣌ؒఔ͔͔ͬͯͨͷ͕ 20ఔͰྃ͢ΔΑ͏ʹ ͳΓ·ͨ͠ 1310݄25༵ۚ
൪֎ɿࠓࢥ͏ͱΓ͔ͨͬͨ͜ͱ ShellScripͷUnitTestΛॻ͘ 1310݄25༵ۚ
࣌ShellScript΄΅ॻ͍ͨࣄແ͍ ͍ͭͰʹςετॻ͍ͨࣄແ͍ લͷҠߦͰΘΕͯͨίʔυΛ͍ճ͢ ࣌ؒແ͔ͬͨͷͰແઅૢʹScript͕૿͍͑ͯ͘ ৭ʑऩू͔ͭͳ͘ͳͬͯ͘Δײ͕͋͡Δ 1310݄25༵ۚ