Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
hbstyle-2014-12-11
rrreeeyyy
December 11, 2014
Technology
8
2.7k
hbstyle-2014-12-11
MySQL 5.6.10 以下に存在する レプリケーションのバグと原因特定について (Heartbeats 社内勉強会 2014/12/11)
rrreeeyyy
December 11, 2014
Tweet
Share
More Decks by rrreeeyyy
See All by rrreeeyyy
SRE の歩き方・進め方 / sre-walk-through-procedure
rrreeeyyy
0
6.4k
「信頼性」を保ちつつ大規模サービスをリニューアルする / cookpad-tech-kitchen-service-embedded-sres
rrreeeyyy
11
10k
Cookpad and Prometheus
rrreeeyyy
6
19k
SRE-Lounge-8-Cookpad-Microservice-Architecture-Overview
rrreeeyyy
5
4.5k
A survey of anomaly detection methodologies for web system
rrreeeyyy
5
870
エンジニアリングをちゃんとやる あるいは 人類の平和 について / wsa02-rrreeeyyy
rrreeeyyy
13
2.6k
「自立」したWebシステムを創る。自分の好きなことをする世界を目指して。/ ipsj-one-2018-rrreeeyyy
rrreeeyyy
3
1.5k
Web サービスの信頼性と運用の自動化について / iot40-rrreeeyyy
rrreeeyyy
12
6.9k
SRE at Cookpad
rrreeeyyy
1
1k
Other Decks in Technology
See All in Technology
230125 モニターマウントLT ITガジェット翁(Ryu.Cyber)さん
comucal
PRO
0
4.6k
SignalR を使ったアプリケーション開発をより快適に!
nenonaninu
0
610
03_ユーザビリティテスト
kouzoukaikaku
0
330
re:Invent2022 前後の Amazon EventBridge のアップデートを踏まえつつ、情シスの仕事をより楽しくしたい話。 / EventBridge for Information Systems Department
_kensh
2
700
ChatGPT for Hacking
anugrahsr
0
4.3k
Periodic Multi-Agent Path Planning
hziwara
0
110
Logbii(ログビー) 会社紹介
logbii
0
120
Cloudflare Workersで動くOG画像生成器
aiji42
1
490
もし本番ネットワークをまるごと仮想環境に”コピー”できたらうれしいですか? / janog51
corestate55
0
360
AI Builderについて
miyakemito
0
890
組織に対してSREを適用するとどうなるか
kuniim
2
700
ユーザーテストガイドライン VERSION 2.0
kouzoukaikaku
0
1.2k
Featured
See All Featured
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
6
840
Facilitating Awesome Meetings
lara
33
4.6k
For a Future-Friendly Web
brad_frost
166
7.8k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
500
130k
Writing Fast Ruby
sferik
613
58k
It's Worth the Effort
3n
177
26k
Ruby is Unlike a Banana
tanoku
93
9.5k
Mobile First: as difficult as doing things right
swwweet
213
7.8k
JazzCon 2018 Closing Keynote - Leadership for the Reluctant Leader
reverentgeek
175
9.1k
Designing on Purpose - Digital PM Summit 2013
jponch
108
5.9k
Support Driven Design
roundedbygravity
88
8.9k
Fireside Chat
paigeccino
16
1.8k
Transcript
MySQL ϨϓϦέʔγϣϯͷόάͱ ݪҼಛఆʹ͍ͭͯ YOSHIKAWA Ryota hbstyle-2014-12-11 1
ࣄ • MySQL (5.5.32) ʹͯɺϨϓϦέʔγϣϯͷෆ۩߹͕ൃੜ • DDL શͯө͞ΕΔ • DML
ಛఆͷςʔϒϧʹͯө͞Εͳ͍ • Slave_(IO|SQL)_Running ڞʹ Yes • Τϥʔແ͠ • MySQL 5.6.10 Ҏલશ͕ͯ͜ͷόάͷର YOSHIKAWA Ryota hbstyle-2014-12-11 2
ݪҼ • MasterDB ͷ table_id ͕ UINT_MAX Λ͑ͨ߹ʹൃੜ • SlaveDB
ͷ table_id ͕Φʔόʔϑϩʔ • MasterDB ͕͍࣋ͬͯΔ table_id ͱ, ϨϓϦέʔγϣϯͰୡ͞ΕΔ table_id ʹෆ߹͕ൃੜ • binlog ͕ Row|Mixed ͷϨϓϦέʔγϣϯͰӨڹ • ৄ͘͠ http://bugs.mysql.com/bug.php?id=67352 YOSHIKAWA Ryota hbstyle-2014-12-11 3
ݪҼ (ଓ͖) • Master ͕ཧ͢Δ table_id (table_map_id) ulong •
64bit Linux ڥͰ 18446744073709551615 ͕࠷େ • Slave ͕ཧ͢Δ table_id uint • 64bit Linux ڥͰ 4294967295 ͕࠷େ • CREATE/DROP/TRUNCATE Ͱ table_id ͕૿Ճ͍ͯ͘͠ • ෆ߹͕ى͖Δͱ Slave σʔλΛߋ৽Ͱ͖ͳ͘ͳΔ YOSHIKAWA Ryota hbstyle-2014-12-11 4
• σόοάϩά # Master Ͱcheck_repl ςʔϒϧͷॻ͖ࠐΈΛͨ࣌͠ͷ Slave ͷσόοάϩά table_mapping::set_table(ulong,TABLE*): enter:
table_id: 20497567 table: 0x3d361f0 (check_repl) table_mapping::set_table(ulong,TABLE*): info: tid 20497567 -> table 0x3d361f0 (check_repl) table_mapping::get_table(ulong): enter: table_id: 4315464863 table_mapping::get_table(ulong): info: tid 4315464863 is not mapped! # ... 20497567 = 4315464863 - (4294967295 + 1) # ... (4294967295 ... UINT_MAX) • ιʔείʔυ * table_map_id > # grep -n "ulong table_map_id" sql/table.h > 645: ulong table_map_id; /* for row-based replication */ * table_id > # grep -n "uint table_id" sql/table.h > 1457: uint table_id; /* table id (from binlog) for opened table */ YOSHIKAWA Ryota hbstyle-2014-12-11 5
Ͳ͏ͬͯಛఆ͔ͨ͠ YOSHIKAWA Ryota hbstyle-2014-12-11 6
Day1 YOSHIKAWA Ryota hbstyle-2014-12-11 7
ਪଌʹΑΔԾઆݕূͷ܁Γฦ͠ • ਓ͕ؒ૾Ͱ͖Δൣғͷόάͩͱ͜ͷํ๏͕ߴ(ܦݧతʹ) • ಉ༷ͷݱ(slave ߋ৽ෆՄ)Λ google Ͱͻͨ͢Βݕࡧɾݕূ • ਓؒͷ૾Ͱ͖ͳ͍ൣғͷόάͩͱϋϚΔ
• લྫͷݟ͕ݕࡧͮ͠Β͍/ແ͍߹ಛʹ • ܦݧΛੵΉࣄͰൣғΛ͛Δ͜ͱՄೳͳؾ͢Δ YOSHIKAWA Ryota hbstyle-2014-12-11 8
Day2 YOSHIKAWA Ryota hbstyle-2014-12-11 9
ࣄͷ࠶ݱɾΓ͚ • ࣗͷݕࡧɾਪଌग़དྷΔϨϕϧΛ͍͑ͯΔࣄΛೝࣝ • ຊ൪ػͷ binlog ΛோΊͯոͦ͠͏ͳૢ࡞Λચ͍ग़͢ • Slave ͷσʔλΛݟͯԿ࣌ࠒʹෆ߹͕ى͖͔ͨ֬ೝ
• ෆ۩߹͕ى͖ͯΔ DB ͰࣄͷΓ͚ • σʔλϑΝΠϧͷ༰ྔຊʹ૿͍͑ͯͳ͍͔ • ϦϨʔϩάʹॻ͖ࠐΈ͕͋Δ͔... YOSHIKAWA Ryota hbstyle-2014-12-11 10
Day3 YOSHIKAWA Ryota hbstyle-2014-12-11 11
࠶ݱࡁΈڥͰͷσόοά • ো͕ى͖͍ͯͨ Master ʹผͷ Slave Λܨ͙ͱ࠶ݱग़དྷͨ • Slave ʹؔΘΒͣো͕ى͖Δ
=> Master ͓͔ͦ͠͏ • Slave ͷ MySQL Λ Debug ༗ޮͰϏϧυͯ͠ܨ͍ͰΈΔ cmake . -DWITH_DEBUG=1 -DMYSQL_MAINTAINER_MODE=0 • ਖ਼ৗͳ Master ʹܨ͍ͩ࣌ͷϩάͱҟৗͳϩάΛݟൺΔ • ϩάͱιʔείʔυΛൺֱͯ͠Ͳ͜ͰԿ͕ى͖ͯΔ͔Ѳ YOSHIKAWA Ryota hbstyle-2014-12-11 12
ʹཱͬͨπʔϧ • mysqlbinlog (binlog, relaylog ղੳ) • tcpdump & Percona
Toolkit (ύέοτɾσʔλෆ߹ௐࠪ) • icdiff (֤छϩάͷൺֱ) • gdb (MySQL ͷσόοά) • vim & ctags & Unite grep ... (ιʔείʔυϦʔσΟϯά) YOSHIKAWA Ryota hbstyle-2014-12-11 13
ʹཱͬͨݟ (1) • hbstyle Ͱͷݟ • @indigo13love ͞Μͷੲͷൃද • sql/slave.cc
ϨϓϦέʔγϣϯͷݟ • ܭࢉػՊֶͷݟ • MySQL Packet ͷϔομ͙Β͍ͳΒಡΊΔ • ֶ෦ͷ࣌ʹύέοτோΊͨݟ YOSHIKAWA Ryota hbstyle-2014-12-11 14
ʹཱͬͨݟ (2) • ιʔείʔυϦʔσΟϯάͷݟ • ֶ෦࢛Ͱ৭ΜͳιϑτΣΞͷιʔείʔυಡΜͩݟ • Apache, Nginx, Keepalived,
LVS ... • ϏϏΒͳ͍͜ͱ • ϩάͱಥ͖߹ΘͤͯಡΉ • πʔϧΛָͬͯͯ͠ಡΉ YOSHIKAWA Ryota hbstyle-2014-12-11 15
[demo] πʔϧͷ͍ํͱ͔ιʔεಡΈͱ͔ͪΐͬͱ͚ͩ YOSHIKAWA Ryota hbstyle-2014-12-11 16
ল YOSHIKAWA Ryota hbstyle-2014-12-11 17
Don't guess, measure! — Robert C. Pike YOSHIKAWA Ryota hbstyle-2014-12-11
18
ল • ਪଌ͢ΔͳɺܭଌͤΑ • ύϑΥʔϚϯε͚ͩͰͳ͘όάಉ͡ • (ఆͷ)؆୯ͳόάΛ૬खʹͯ͠ΔͱΕΔ͜ͱ͋Δ • ࣄɾݪҼΓ͚ɾιʔείʔυ͕શͯ •
ϩάΛݟΔɾιʔείʔυΛݟΔ YOSHIKAWA Ryota hbstyle-2014-12-11 19
• νϟοτͱ͔ͷݺͼ͔͚Ͱڠྗͯͩͬͨ͘͠͞օ͞Μ͋Γ͕ ͱ͏͍͟͝·ͨ͠ • ಛʹ @saito @takamoto @hosokoshi @matsuu ...
• ͬͱݕࡧྗ͕ߴ͚ΕҰൃͰͨͲΓண͍͔ͨ͠Εͳ͍ • http://bugs.mysql.com/bug.php?id=67352 • ͱ͍͑ MySQL ʹͪΐͬͱৄ͘͠ͳΕͯΑ͔ͬͨ YOSHIKAWA Ryota hbstyle-2014-12-11 20