Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ISUCON3 - ISUCON Makers Casual Talks
Search
FUJIWARA Shunichiro
December 10, 2014
Programming
3
810
ISUCON3 - ISUCON Makers Casual Talks
FUJIWARA Shunichiro
December 10, 2014
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
alecthomas/kong はいいぞ
fujiwara3
6
1.6k
ecspressoの設計思想に至る道 / sekkeinight2025
fujiwara3
12
2.7k
さくらのIaaS基盤のモニタリングとOpenTelemetry/OSC Hokkaido 2025
fujiwara3
3
1.1k
監視のこれまでとこれから/sakura monitoring seminar 2025
fujiwara3
11
5.2k
k6による負荷試験 入門から日常的な実践まで/Re:TechTalk #01
fujiwara3
2
130
困難を「一般解」で解く
fujiwara3
10
3.8k
「隙間家具OSS」に至る道/Fujiwara Tech Conference 2025
fujiwara3
7
13k
alecthomas/kong はいいぞ / kamakura.go#7
fujiwara3
1
1.1k
ISUCONに強くなるかもしれない日々の過ごしかた/Findy ISUCON 2024-11-14
fujiwara3
11
1.4k
Other Decks in Programming
See All in Programming
Updates on MLS on Ruby (and maybe more)
sylph01
1
160
サイトを作ったらNFCタグキーホルダーを爆速で作れ!
yuukis
0
710
開発チーム・開発組織の設計改善スキルの向上
masuda220
PRO
17
9.2k
Microsoft Orleans, Daprのアクターモデルを使い効率的に開発、デプロイを行うためのSekibanの試行錯誤 / Sekiban: Exploring Efficient Development and Deployment with Microsoft Orleans and Dapr Actor Models
tomohisa
0
220
物語を動かす行動"量" #エンジニアニメ
konifar
14
5.6k
ワープロって実は計算機で
pepepper
2
1.4k
RDoc meets YARD
okuramasafumi
4
160
【第4回】関東Kaggler会「Kaggleは執筆に役立つ」
mipypf
0
940
Portapad紹介プレゼンテーション
gotoumakakeru
1
130
旅行プランAIエージェント開発の裏側
ippo012
1
500
エンジニアのための”最低限いい感じ”デザイン入門
shunshobon
0
130
ソフトウェアテスト徹底指南書の紹介
goyoki
1
120
Featured
See All Featured
The Cost Of JavaScript in 2023
addyosmani
53
8.9k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
51
5.5k
BBQ
matthewcrist
89
9.8k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Become a Pro
speakerdeck
PRO
29
5.5k
How to Think Like a Performance Engineer
csswizardry
26
1.8k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Embracing the Ebb and Flow
colly
87
4.8k
Speed Design
sergeychernyshev
32
1.1k
Navigating Team Friction
lara
189
15k
Agile that works and the tools we love
rasmusluckow
330
21k
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
Transcript
ISUCON'Makers'Casual'Talks ISUCON'3 2014/12/10&@fujiwara
About&me @fujiwara ISUCON'1,'2'༏উ'(fujiwara) ISUCON'3'ग़ ໘ന๏ਓΧϠοΫ
ग़ͷܦҢ ISUCON'1,'2'fujiwara࿈ʂ 2ͷ࠙ձʮग़͍ͯͩ͘͠͞Αʯʮ͍ʂʯ CROSS%2013%@941͞Μʹ͓ئ͍͞ΕΔ
Timeline 2013/1&CROSS&2013 2013/4'ΧϠοΫٕज़෦৽ଔݚमʮࣾISUCONʯ 2013/6'@941͞ΜͱϛʔςΟϯά 2013/7/31'ग़࡞ઓձٞ 2013/9/21'YAPC::Asia'2013'ʮࣾISUCONͷͭ͘Γ͔ͨʯൃද 2013/10/5,6)ISUCON)3)ΦϯϥΠϯ༧બ 2013/11/9'ISUCON'3'ຊબ
ISUCON'1,2'ͱ'3'ͷҧ͍ ΦϯϥΠϯ༧બ 2ճɺผͷΛ࡞Δ
ࣾISUCON ৽ଔݚमͷҰ ओʹαʔόαΠυΛΔΤϯδχΞ͚ HTTPΛֶͿ$→$WebΞϓϦΛ࡞Δ ࡞ͬͨΞϓϦͰISUCON! github.com/kayac/newbie2training
ࣾISUCON ݚमͷҰͷͨΊૉͳग़ • ੩తϑΝΠϧΛΞϓϦαʔό͕৴ • ΠϯσοΫε͕ுΒΕ͍ͯͳ͍ςʔϒϧ • ϧʔϓதͰΫΤϦ͕ඈͿ ੩తϑΝΠϧΛwebαʔό৴$+$ΠϯσοΫεΛదʹுΔ$+$ϧ ʔϓΫΤϦΛJOINʹॻ͖͑
→$είΞ20ഒ
ࣾISUCONͰͷܦݧ • Ր༵"WebAppॻ͘ • ਫ༵"ςετॻ͘ • ༵"ϕϯνϚʔΫॻ͘ • @acidlemon"ʹղ͍ͯΒ͏ •
༵ۚே͔Β"ISUCON (னؒݚमͷߨࢣۀ)
ࣾISUCONͰͷল • ్தͰϕϯνϚʔΫϗετ"㱻"֤VMͷଳҬةͳ͍ • ଳҬ͕͢ΔͱείΞ͕͔ࠩͳ͘ͳΔ • ్தͰ"Accept-Encoding: gzip"Ճ
2013/7/31 ग़࡞ઓձٞ !@fujiwara! !@acidlemon !@typester! !@songmu
memo ը૾ϦαΠζ ಉ࣌ଓܥɺνϟοτɺϨΠςϯγ push͔ΒͷҰ੪ΞΫηεΛΤϥʔʹ͠ͳ͍Ͱ͑Δ ಉใ RedisͬͯΔ͚Ͳ͍ํѱ͍ mongo git href="" PostgreSQLೖΕ͓ͯ͘
ຊબʹ࠾༻ • ը૾ϦαΠζ • ಉ࣌ଓܥɺϨΠςϯγ • Ұ੪ΞΫηε ༧બʮຊબͷωλόϨʹͳΒͳ͍ʯͷ
10/5,6'ΦϯϥΠϯ༧બ
༧બग़༰ Github'GistͷΑ͏ͳͷ ϩάΠϯػೳɻCookieʹΑΔηογϣϯҡ͕࣋ඞཁ ϓϥΠϕʔτߘػೳɻϓϥΠϕʔτઃఆ͞ΕͨͷϩάΠϯ ͍ͯ͠ΔϢʔβࣗʹ͔͠ݟ͑ͳ͍ MarkdownܗࣜͰߘ ʴ!᠘తͳઃఆ
༧બͷҙਤ ໌Β͔ʹຊબग़ʹ࣮ྗͷΓͳ͍νʔϜΛ;Δ͍མͱ͢ ×"੩తϑΝΠϧΛΞϓϦαʔό͕৴ ×"ΠϯσοΫε͕ͳ͍ςʔϒϧ ×"ϧʔϓΫΤϦ ×"Markdownมʹ֎෦ίϚϯυىಈ ී௨ͷWebΞϓϦͱͯ͠μϝͳ࣮ΛͤΕߴείΞʹ ʴ$Ϙʔφεͱͯ͠ʮ᠘ʯΛΫϦΞͯ͠Ճ
༧બ४උ ʙ!9/17!ग़ͷϓϩτλΠϓGoͷϕϯνΫϥΠΞϯτԾ࣮ 9/25ʙ%@acidlemon%ʹղ͍ͯΒ͍͡ΊΔ 9/30%/recent/*%ΛՃ 10/2,3'Ruby,'Python,'Go,'Node,'PHP'Ҡ২ 10/5,6'ISUCON3'༧બ
༧બ ͔ࣗΒIRCͰαϙʔτ
༧બͷʮ᠘ʯ
MySQL&InnoDB&memcached&plugin MySQL&5.6ͷ৽ػೳ • memcached(protocolͰMySQLͱ௨৴Ͱ͖Δ • port(11211(Λ(Listen • ΞϓϦέʔγϣϯ͔Βηογϣϯอଘʹ༻ • port(11212(Ͱී௨ͷ(memcached(ىಈͯ͋͠Δ
• ॳظঢ়ଶͰ༻͍ͯ͠ͳ͍
ҙຯͷͳ͍!CSS!link <link rel="stylesheet" href="/"> • ϕϯνϚʔΧʔϦϯΫΛḷͬͯ"/"ʹϦΫΤετΛඈ͢ • ͔͠͠είΞ"0 • PageViewͰͳ͍ͱ͍͏ѻ͍
• HTML͔Βআ͢ΔͱϦΫΤετ͠ͳ͘ͳΔ
ҰൃΞτͰͳ͍ݮํࣜ fail=3·Ͱݮͳ͠ failͷࣗ%&(ͨͱ͑&fails=6&ͳΒ& )&Λݮ fail%>=%13%Ͱ% %ݮʹͳΔͷͰείΞ%0
༧બͷল • είΞϦϯάͷৄࡉ͕ެ։͞Ε͍ͯͳ͔ͬͨ • /recent/*"ʹର͢ΔϕϯνϚʔΧʔͷνΣοΫ͕͔ͬͨ • ͋ͱ͔ΒՃͨ͠෦ • ͜͜Λ"Cache"͢ΔͱΑ͘ճΔ
༧બϕϯνϚʔΧʔ • Go • ࢀՃऀ͕ࣗ༝ʹͳΔϗετͰಈ࡞͢ΔͨΊɺղੳ͞Εʹ͍͘ Α͏ʹ • --workload$Φϓγϣϯॳಋೖ • ISUCON$1,2ͰௐͰ͖ͳ͔ͬͨ
༧બϕϯνϚʔΧʔ • ੩తϑΝΠϧ͚ͩΞΫηε͢Δ"sta&c"worker • ϦΫΤετ͝ͱʹ"10ms"wait • /recent/*"͚ͩΞΫηε͢Δ"recent"worker • ϩάΠϯɺߘɺӾཡɺϩάΞτɺӾཡΛ܁Γฦ͢"main" worker
༧બϕϯνϚʔΧʔ !͕߹ܭ!worker! w=1$>$ $(sta*c=1,$recent=1,$main=6) w=2$>$ $(sta*c=2,$recent=2,$main=14) w=3$>$ $(sta*c=3,$recent=3,$main=28)
༧બϕϯνϚʔΧʔͷʮ݀ʯ recent&worker& • wait&ͳ͠ʹճΔ • දࣔཁૉ͕ߋ৽͞Ε͍ͯΔ͔Ͳ͏͔ͷνΣοΫΛͯ͠ͳ ͍ʂʂʂ /recent/*!ΛΩϟογϡͯ͠ߴʹฦ͢ͱ!είΞ↑↑
༧બͷল είΞࢉग़ͷৄࡉެ։͢͠ աͳʮ᠘ʯΑ͘ͳ͍ ϕϯνϚʔΧʔͷ݀Ͱ͖ΔݶΓ௵͢͠
11/9$ຊબ
ຊબग़ʹͨͬͯ ʮ᠘ʯΊΑ͏ ෳαʔόΛ༗ޮʹ͑Δग़ʹ͠Α͏
ຊબ୲ !@fujiwara!ɺϕϯνϚʔΫɺείΞूܭγεςϜ !@acidlemon!ࣄલղɺϕϯνϚʔΫνΣοΫɺΦʔϓχϯ άɺϨΪϡϨʔγϣϯ !@handlename:!nodeҠ২ɺಈ࡞֬ೝ !@mix3:!GoҠ২ɺಈ࡞֬ೝ !@nobu_ohta:!RubyҠ২ɺPythonҠ২ !@m0t0k1ch1:!PHPҠ২ɺ෧σβΠϯ
ຊબ४උ 10/17%ΩοΫΦϑ 10/29%ग़ͱϕϯνϚʔΧʔϓϩτλΠϓͰղ͖࢝ΊΔ%ଟݴޠҠ ২։࢝ 11/5%5ߏͰղ͘,%ຊ൪༻αʔόࠐΈ 11/7%ຊ൪αʔό্Ͱςετ։࢝ 11/8%݁Ռूܭը໘࡞ɺσόοά 11/9%ISUCON3%ຊબ
ຊબग़༰ ը૾ߘ͕Ͱ͖ΔSNS HTMLΛฦ͢෦ͳ͘ɺJSON)APIͷΈ ϢʔβΛϑΥϩʔͰ͖ΔɻϑΥϩʔ͍ͯ͠ΔϢʔβͷߘ͕ྲྀΕ ͯ͘Δ Long%polling%API%͋Γ ϕϯνϚʔΧʔը૾Λߘͨ͠ΓJSONΛऔಘͨ͠Γ͢Δ
είΞࢉग़ํ๏ Լهͷ!(a)+(b)+(c)!͕૯߹είΞ (a)$ߘεϐʔυ$=$ϨεϙϯελΠϜ͕͍΄ͲߴείΞ (b)$λΠϜϥΠϯө$=$ߘͨ͠ΤϯτϦͷใ͕ө͞Ε Δͷ͕͍΄ͲߴείΞ (c)$ͦͷଞϦΫΤετ$=$1
είΞࢉग़ํ๏ ϨεϙϯελΠϜʹґଘ͢ΔείΞҎԼͷܭࢉࣜʹΑΔ 5"sec"="1 1"sec"="3.3 100"msec"="6.64 10"msec"="9.9
ຊબͷϙΠϯτ 5ͷαʔόΛͲ͏͏͔ ॳظঢ়ଶͰ1ͷΈʹΞϓϦέʔγϣϯ͕ىಈ͍ͯ͠Δ 2ʙ4ʮۭʯ(OSɺϛυϧΣΞͷॳظઃఆͷΈ) ଳҬ1͋ͨΓ • eth0&:&100Mbps&(ϕϯνϚʔΧʔ͕ΞΫηε) • eth1&:&1Gbps&(ࣗ༝ʹ͑Δ)
ຊબͷϙΠϯτ ը૾ͷϦαΠζΛ͍ͭߦ͏͔ ॳظঢ়ଶϦΫΤετ͝ͱʹຖճಈతੜ 1. ߘ࣌ʹ#S,#M,#L#ͷ3αΠζ#+#ΦϦδφϧΛશͯੜ͠อଘ 2. ߘ࣌ΦϦδφϧͷΈอଘ͠ɺαϜωΠϧӾཡ࣌ʹੜ ※!શͯͷαϜωΠϧ͕ϦΫΤετ͞ΕΔΘ͚Ͱͳ͍
ຊબͷϙΠϯτ ը૾ͷϦαΠζΛԿͰߦ͏͔ ॳظ࣮!ImageMagick!convert!ίϚϯυΛ(֎෦)࣮ߦ • ը૾ϥΠϒϥϦʹΑͬͯม݁Ռʹࠩҟ͕ͰΔ • ը૾Λൺֱ͢ΔπʔϧΛఏڙɻϦϑΝϨϯε࣮ͱͷޡ͕ࠩҰ ఆҎԼͰͳ͍ͱfail
ຊબͷϙΠϯτ Long%poll%API%ΛͲ͏ߴԽ͢Δ͔ ॳظ࣮2ඵsleepͯ͠ฦͤΔͷ͕͋Δ͔νΣοΫ • ฏۉϨεϙϯελΠϜ͕1ඵʹͳΔ • ԿΒ͔ͷΠϕϯτ௨Λར༻ͯ͠ߴʹөͰ͖ΔͱߴείΞ
ຊબͷϙΠϯτ ଳҬΛ͍͔ʹຒΊΔ͔ ϕϯνϚʔΧʔ࠷େ!100Mbps! !5!=!500Mbps!Λ͑Δ Ͱ͖Δ͚ͩଟ͘ͷϗετ͔Β৴͢ΔͱଳҬΛ༗ޮʹ͑Δ
ຊબϕϯνϚʔΧʔ Perl%(Furl%+%AnyEvent) post:&ΞΠίϯͱΤϯτϦΛPOSTɺαϜωΠϧΛGETͯ֬͠ೝ view:&ը૾ΛGET&(0.5&sec&sleep) check:&ϢʔβݖݶʹΑΓݟ͑Δݟ͑ͳ͍ͷݖݶνΣοΫ crawl:&·ͣϩάΠϯ͔ͯ͠ΒλΠϜϥΠϯ(Long&poll)ʹுΓ͘ ?meline:&λΠϜϥΠϯʹுΓ͖ɺྲྀΕ͖ͯͨϢʔβΛfollowɺ ը૾ΛGETͳͲ 6"process(post,"view,"check,"crawl"֤×1"5meline"×"2)"/"workload
ຊબϕϯνϚʔΧʔͷଳҬ੍ޚ LVS$Ͱ$localhost:[νʔϜʹରԠ͢Δport]$͔Βϕϯν૬खʹ$NAT ϕϯνϚʔΫϦΫΤετ࣌ʹ!NAT!ઌͷෳϗετΛ!on!/!off QoS$github.com/matsumoto1r/qos1control tc#ͷ#wrapper.#100Mbps#ʹ੍ݶ
ຊબͷল ଟগͷτϥϒϧ͋ͬͨͷͷ͓͓ΉͶָ͠ΜͰ͑ͨ • ϕϯνϚʔΧʔͷෳΓସ͑όά • Pythonॳظ࣮Ͱ·Εʹ(Fail(͢Δ • tmpfile(ͷ(flush(λΠϛϯά ࠷ޙ!Fail!͢ΔνʔϜ͕ଟ͔ͬͨͷ͕೦
None