Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ウェブアプリケーションフレームワークの向こう側 / The other side of Web...
Search
Yuuki Tsubouchi (yuuk1)
August 22, 2015
Technology
1
590
ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks
はてなインターン2015 インフラ講義
Yuuki Tsubouchi (yuuk1)
August 22, 2015
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
AIスパコン「さくらONE」の オブザーバビリティ / Observability for AI Supercomputer SAKURAONE
yuukit
2
470
AIスパコン「さくらONE」のLLM学習ベンチマークによる性能評価 / SAKURAONE LLM Training Benchmarking
yuukit
2
760
とあるSREの博士「過程」 / A Certain SRE’s Ph.D. Journey
yuukit
11
4.6k
eBPFを用いたAIネットワーク監視システム論文の実装 / eBPF Japan Meetup #4
yuukit
3
1.4k
クラウドのテレメトリーシステム研究動向2025年
yuukit
4
1.1k
博士論文公聴会: Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining / PhD Defence
yuukit
1
270
博士学位論文予備審査 / Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining
yuukit
1
2.2k
MetricSifter:クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
310
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
15k
Other Decks in Technology
See All in Technology
「タコピーの原罪」から学ぶ間違った”支援” / the bad support of Takopii
piyonakajima
0
150
20251027_findyさん_音声エージェントLT
almondo_event
2
460
.NET 10のBlazorの期待の新機能
htkym
0
110
頭部ふわふわ浄酔器
uyupun
0
110
Kubernetes self-healing of your workload
hwchiu
0
560
もう外には出ない。より快適なフルリモート環境を目指して
mottyzzz
13
11k
ラスベガスの歩き方 2025年版(re:Invent 事前勉強会)
junjikoide
0
340
serverless team topology
_kensh
3
240
生成AI時代のPythonセキュリティとガバナンス
abenben
0
140
20251027_マルチエージェントとは
almondo_event
1
450
デザインとエンジニアリングの架け橋を目指す OPTiMのデザインシステム「nucleus」の軌跡と広げ方
optim
0
120
オブザーバビリティが育むシステム理解と好奇心
maruloop
2
1.3k
Featured
See All Featured
Rails Girls Zürich Keynote
gr2m
95
14k
Unsuck your backbone
ammeep
671
58k
Building Better People: How to give real-time feedback that sticks.
wjessup
369
20k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.6k
The Cost Of JavaScript in 2023
addyosmani
55
9.1k
The Power of CSS Pseudo Elements
geoffreycrofte
80
6k
Music & Morning Musume
bryan
46
6.9k
RailsConf 2023
tenderlove
30
1.3k
Leading Effective Engineering Teams in the AI Era
addyosmani
7
640
Typedesign – Prime Four
hannesfritz
42
2.8k
The Cult of Friendly URLs
andyhume
79
6.6k
Keith and Marios Guide to Fast Websites
keithpitt
411
23k
Transcript
ΣϒΞϓϦέʔγϣϯ ϑϨʔϜϫʔΫͷ͜͏ଆ ͯͳΠϯλʔϯ Πϯϑϥߨٛ dΠϯλʔϯͷ͜ΖʹΓ͔ͨͬͨΠϯϑϥͷd
JEZ@VVLJ γεςϜϓϥοτϑΥʔϜ෦ ͯͳΠϯλʔϯ
ߨٛͷ͡Ίʹ wޙաఔʹ͚ͯɺͯͳͷຊ൪γεςϜʹ͍ͭ ͯཧղΛਂΊ͍ͯͩ͘͞ wͯͳʹґଘ͠ͳ͍ͳΔ͘ҰൠతͳΛ৺͕ ͚͍ͯ·͢ wࣈҎ֎ެ։ͯ͠Α͍༰Ͱ͢ w՝͋Γ·ͤΜɻؾܰʹௌ͍͍ͯͩ͘͞
None
ΣϒΞϓϦέʔγϣϯ ϑϨʔϜϫʔΫ
63-3PVUFS $POUSPMMFS .PEFM .Z42- #SPXTFS HTTP Request HTTP Response 8"'Ͱίʔυॻ͍ͯΔͱ͖ͷΠϝʔδ
ΣϒΞϓϦέʔγϣϯ ͷϒϥοΫϘοΫε
ʮϞσϧͷ෦͍͢͝Πϯϑϥ͕ ͦͷઌʹ͋Δʯ
͍͢͝Πϯϑϥ
ͯͳϒϩά
.BDLFSFM
None
8"' Πϯϑϥ
ΈΜͳ͕ݺΜͰΔ Πϯϑϥͬͯ
ΞϓϦέʔγϣϯΤϯδχΞ ͔ΒΈͨΠϯϑϥ wΞϓϦέʔγϣϯࣗମͲͷΑ͏ʹͯ͠ಈ͍͍ͯΔͷ͔ w8FCαʔό 1SFGPSL σʔλϕʔεଓͷҡ࣋ wΞϓϦέʔγϣϯͲͷΑ͏ʹͯ͠σʔλϕʔεʹΞΫ ηε͍ͯ͠Δͷ͔ wεϨʔϒ
γϟʔσΟϯά wϦΫΤετΞϓϦέʔγϣϯʹͲͷΑ͏ʹͯ͠౸ୡ͢ Δͷ͔ wϦόʔεϓϩΩγ
8FCγεςϜͷجຊ
8FCγεςϜͷجຊ w)551ϦΫΤετΛड͚ͯϨεϙϯεΛฦ͢ wෳͷϢʔβ͕ಉ࣌ʹϦΫΤετΛ͛͏Δ wϨΠςϯγͱεϧʔϓοτΛಉ࣌ʹୡ͢Δ ඞཁ͕͋Δ wͲΜͳʹෳࡶͳ8FCγεςϜͰجຊಉ͡
ΞϓϦέʔγϣϯͲͷ Α͏ʹಈ͍͍ͯΔͷ͔
ΞϓϦέʔγϣϯαʔό w8FCαʔϏεʹ͓͍ͯγεςϜͷத৺ͱͳΔαʔό wΞϓϦέʔγϣϯϩδοΫ͜͜Ͱ࣮ߦ͞ΕΔ wҎԼͷॲཧΛ࣮ߦ͢Δ8FCαʔό wϦΫΤετड৴ w)551ϦΫΤετͷղੳ wσʔλϕʔεΞΫηεΛؚΉΞϓϦέʔγϣϯϩδοΫͷ ࣮ߦ w)551Ϩεϙϯεͷߏங wϨεϙϯεૹ৴
ΞϓϦέʔγϣϯαʔόͱݴޠ w ੲωοτϫʔΫ)551ͷϔομॲཧ͚ͩ "QBDIF $ݴޠ ͰΒ͍ͤͯͨ w $(*ͱ͔NPE@IPHFͱ͔ͷಾٕज़ͰແཧΓ$ ݴޠͷ8FCαʔόͷ্ʹ1FSMͳͲΛಈ͔͢ w
ࠓͰΞϓϦέʔγϣϯίʔυͱಉ͡ݴޠͷ8FCαʔ όΛ͏ w 1FSMͳΒ4UBSMFU1MBDL 4DBMBͳΒ+FUUZͳͲ
Ͳ͏ͬͯωοτϫʔΫ௨৴ ͢Δͷ͔ w ωοτϫʔΫ௨৴جຊతʹ04Χʔωϧ͕Δ w -JOVYͷ߹γεςϜίʔϧΛͬͯ04Χʔωϧͷ ػೳΛωοτϫʔΫ௨৴ػೳΛݺͼग़͢ w TPDLFU
CJOE MJTUFO BDDFQU w جຊແݶϧʔϓͭͭ͠ɺΧʔωϧ͔Βσʔλड৴ Πϕϯτ͕͖ͨΒϦΫΤετॲཧʹҠΔ
ΫϥΠΞϯτ αʔό socket(2) bind(2) listen(2) accept(2) connect(2) write(2) (ϒϩοΫ͢Δ) socket(2)
“GET / HTTP1.1\nHost: TCP 3-way handshake “HTTP/1.1 200 OK\n…” read(2) (accept͔ΒΔ) read(2) (ϒϩοΫ͢Δ) (ϒϩοΫ͢Δ) (ϒϩοΫ͢Δ) (read͔ΒΔ) ϦΫΤετॲཧ write(2) (read͔ΒΔ) read(2) (ϒϩοΫ͢Δ) close(2) লུ
8FCαʔόΛෳͷ ΫϥΠΞϯτʹରԠ͢Δ
worker! process worker! process worker! process worker! process parent! process
Client accept loop accept loop accept loop accept loop fork fork fork fork Client Client Client 1SFGPSLํࣜ
thread thread thread thread Client accept loop accept loop accept
loop accept loop Client Client Client εϨουϓʔϧํࣜ Process
Client Client Client Client main! thread select loop data! base
non-blocking I/O Πϕϯτۦಈํࣜ
w 1FSM1SFGPSL 4DBMB +BWB εϨουϓʔϧ ͕ଟ͍ w Ϛϧνϓϩηε 1SFGPSL ͔ϚϧνεϨου
εϨο υϓʔϧ ͔Ͱάϩʔόϧมͷѻ͍͕͔ΘΔ w ϚϧνεϨουͰάϩʔόϧมͷϩοΫڝ߹ͯ͠ ٧·ΔͳͲ w ٯʹϚϧνϓϩηεͳΒෳͷίϯςΩετ͔Βڞ ௨ͷΦϒδΣΫτΛࢀর͢Δ͜ͱ͋·Γͳ͍ ʢϑΝΠϧͳͲ04ͷϦιʔεผʣ
ΞϓϦέʔγϣϯ ͲͷΑ͏ʹͯ͠σʔλϕʔε ʹΞΫηε͍ͯ͠Δͷ͔
σʔλϕʔεͱͷଓ
͍ͭଓ։࢝͢Δͷ͔ w 1FSM w %#*DPOOFDUΛݺΜͩͱ͖ w େɺ࣮ࡍʹϦΫΤετ͕ͱΜͰ͖ͯσʔλϕʔ εΞΫηε͢Δஈ֊Ͱଓ͢Δ w 4DBMB
w ΞϓϦέʔγϣϯͷىಈ࣌ʹଓࡁΈͷ%#Φϒ δΣΫτ͕࡞͞ΕΔ
ଓͷҡ࣋ʁ w σʔλϕʔεͷଓίετ w 5$1ͷXBZϋϯυγΣʔΫ w %#ͷϋϯυγΣʔΫ w %#ͷೝূ w
%#ଆͷଓ༻ϓϩηεεϨουͷੜίετ w ଓ͕ଟ͍ͱ%#αʔόͷෛՙ͕ߴ͍ w ଓࡁΈͷιέοτΛ͍·Θ͍ͨ͠
ଓͷӬଓԽ w ଓ w ϦΫΤετॲཧதͷΈଓΛҡ࣋͢Δ w ECIΦϒδΣΫτΛίϯςΩετΦϒδΣΫτͳ ͲʹΩϟογϡ͢ΕΑ͍ w ৗ࣌ଓ
w ECIΦϒδΣΫτΛάϩʔόϧมͳͲʹΩϟο γϡ͢Δ w ίωΫγϣϯϓʔϧΛ͏
ଓͷӬଓԽͷϋϚΓϙΠϯτ w ଓ͞·͟·ͳཁҼͰ్தͰΕΔ͜ͱ͕͋Δ w %#͕͓ͪͯϑΣΠϧΦʔόͨ͠ w ωοτϫʔΫো w ECIΦϒδΣΫτউखʹ࠶ଓͯ͘͠Εͳ͍ w
4DPQF$POUBJOFS%#* %#*Y)BOEMFSΛ ͏ w ϩʔυόϥϯα͔ΒॏΈௐͰ%#αʔϏεΞ τͰ͖ͳ͍ແཧΓΔ
8FCαʔϏεߏ
Proxy App Database request response request response transaction w ϦόʔεϓϩΩγ/HJOY
"QBDIF w ΞϓϦέʔγϣϯ1FSM 4DBMB w σʔλϕʔε.Z42- 1PTUHSF42-
ͳͥෳͷαʔόΛ͏ ͷ͔
ෳͷαʔόΛ͏ཧ༝ wγεςϜͷՄ༻ੑͷ୲อ wෛՙࢄ wૄ݁߹Խ
Մ༻ੑͷ୲อ wαʔόཧతʹյΕͨΓաෛՙΦϖϨʔγϣϯϛε Ͱఀࢭ wελϯόΠػʹॲཧΛҾ͖ܧ͙ʢϑΣΠϧΦʔόεΠο νΦʔόʣ͕දత wΞΫςΟϒɾελϯόΠؒͰૄ௨ࢹ w૬ख͕མͪͯͨΒ͕ࣗΞΫςΟϒʹͳΔ w7*1 Ծ*1 ͰଓઌΛมߋ͢Δ
wϛυϧΣΞଆͰϞʔυΓସ͑ͳͲ͕ඞཁͳ͜ͱ wLFFQBMJWFE)FBSUCFBUͳͲͷπʔϧ
ෛՙࢄ wϩʔυόϥϯαΛલஈʹஔ͍ͯৼΓ͚ wಉׂ͡ͷαʔόΛฒΔ wϩʔυόϥϯαʹ7*1ͳͲͷΤϯυϙΠϯτ͕ ͍͍ͭͯΔ wΞϓϦέʔγϣϯ͔Βͦ͜Λࢀর͢Δ͚ͩ wઐ༻ͷϩʔυόϥϯα-74 )"1SPYZ wϦόʔεϓϩΩγͰ༻͢Δ͜ͱ͋Δ wԽ݉Ͷඋ͍͑ͯΔ
ૄ݁߹Խ wಉ͡ϗετʹෳͷϛυϧΣΞΛಉډͤ͞Δ ͜ͱͰ͖Δ w࣮ࡍʹ wͷΓ͚ͷ͢͠͞ wෛՙࢄ wׂ͕ҟͳΔ͔Βͱ͍ͬͯɺશ෦͚ͨ΄͏͕ Α͍Θ͚Ͱͳ͍ wαʔό͕૿͑͗͢Δ߹ಉډݕ౼
Proxy App Database request response request response transaction w ϦόʔεϓϩΩγ/HJOY
"QBDIF w ΞϓϦέʔγϣϯ1FSM 4DBMB w σʔλϕʔε.Z42- 1PTUHSF42-
ϦόʔεϓϩΩγ
w άϩʔόϧ͔ΒϦΫΤετΛड͚͚Δαʔό w ϦΫΤετͷ༰ʹ͋Θͤͯ w ޙଓͷαʔόʹϦΫΤετͨ͠Γ w ΞΫηε੍ޚͨ͠Γ w ੩తϑΝΠϧΛฦͨ͠Γ͢Δ
w ੍ޚՄೳϙΠϯτΛ૿ͯ͠ɺύϑΥʔϚϯε্ ͤͨ͞ΓɺγεςϜશମͷӡ༻ޮΛ্͛Δ Proxy App Database request response request response transaction
Proxy bot༻app request response user༻app UserAgent: ~Googlebot w CPU༻ͱVTFS༻Λ6"Ͱఆͯ͠ৼΓ͚Δ w
োͷӨڹൣғΛݶఆ͍ͨ͠ w CPUͱVTFSͰνϡʔχϯά༰Λม͑ͨΓ͢Δ w QSFGPSLܕͰಛʹޮ͖͍͢
ΞϓϦέʔγϣϯ
w ͍͍ͩͨطग़ w άϩʔόϧͳϦΫΤετΞϓϦαʔόͰड͚ͯ ͍͚ͳ͍ͱ͍͏Θ͚Ͱͳ͍ w 4UBSMFU+FUUZΑΓɺ/HJOY"QBDIFͷ΄͏͕ ֎ʹެ։͢Δ8FCαʔόͱͯ҆͠৺ w มͳϔομ৯Θ͞ΕΔͱ͔
w খ͞ͳαʔϏεͩͱΞϓϦαʔόͱϦόʔεϓϩΩ γΛಉډͤ͞Δ͜ͱ͋Δ Proxy App Database request response request response transaction
σʔλϕʔε
w ͯͳϒϩάنͳΒϚελ̍Ͱࡹ͚Δ w ςʔϒϧσʔλͷΑ͘ࢀর͞ΕΔ෦͕ϝϞϦʹͬ ͍ͯͯɺదʹΠϯσοΫεΛΔ w ελϯόΠϚελͷ༻ҙ͠.)"ͷΑ͏ͳΈͰϑΣ ΠϧΦʔόͤ͞Δ w ϚελͷεέʔϧΞοϓʹݶք͕དྷͨ
w SFBE͕ݶքࢀর༻εϨʔϒΛ༻ҙ w XSJUF͕ݶքਫฏׂ Proxy App Database request response request response transaction
slave Load! Balancer master slave slave App ࢀরΫΤϦͷΈ ߋ৽ΫΤϦத৺ SELECT
INSERT/UPDATE ࢀর༻εϨʔϒ
master A App ਫฏׂ γϟʔσΟϯά master B master C ϢʔβIDΛ
3Ͱׂͬͨ༨͕0 ༨͕1 ༨͕2
ϫϯϥϯΫ্ͷ γεςϜΛߏங͢Δ
Ωϟογϡ
w $16ͷΩϟογϡϝϞϦ - -ΩϟογϡͳͲʣ w 04ͷϖʔδΩϟογϡ w .Z42- *OOP%# ͷόοϑΝϓʔϧ
w ࢄΩϟογϡαʔό NFNDBDIFE w Ωϟογϡػೳ͖ͭϦόʔεϓϩΩγ 7BSOJTI 4RVJE
Proxy App Database Squid/ Varnisn memcached
ඈͼಓ۩ͱͯ͠ͷ ,74/P42-
w 3FEJT NFNDBDIFE .POHP%#ͳͲ w 3%#.4ΑΓ͍σʔλϕʔε w ετϨʔδͱͯ͠ͷ҆৺ײྼΔ͔͠Εͳ͍ w ো࣌ͷσʔλ߹ੑ͕ෆ҆
w ϦϨʔγϣφϧϞσϧͷΑ͏ͳཧతʹཱ֬͞Εͨσʔ λϞσϧͰͳ͍ w ύϑΥʔϚϯε্ͷͨΊʹิॿతʹ༻͢Δ͜ͱ͕ଟ͍ w .BDLFSFMͰ࣌ܥྻσʔλϕʔεͱͯ͠(SBQIJUFΛ ͍ͬͯΔ
ඇಉظॲཧ
w ϝʔϧ5XJUUFSͷߘͳͲɺ֎෦γεςϜ͕མ ͪͯҾ͖ͣΒΕͯ͠·͏ͷΛ͙ w QSFGPSLͷϓϩηεͷେ෦͕োதͷ֎෦γε ςϜΛୟ͍͍ͯΔͱɺࣗͷαʔϏεམͪΔ w ֎෦γεςϜʹґଘͨ͠Γɺ࣮ߦ͕͍ॲཧΛ͋ͱͰ ࣮ߦ͍ͨ͠ඇಉظॲཧ w
ͯͳͰδϣϒΩϡʔΛ͏ w ઐ༻ͷ.Z42-ʹδϣϒΛೖ w ผαʔόͰಈ͘ϫʔΧʔ͕.Z42-ʹϙʔϦϯά
Proxy App db-master workerdb worker db-slave
όονॲཧ
w ຖேϢʔβʹϝʔϧΛ৴͍ͨ͠ɺఆظతʹϗοτΤ ϯτϦΛߋ৽͍ͨ͠ͳͲ w ઐ༻ͷόοναʔόͰDSPOʹεΫϦϓτΛஔ͘ w όονॲཧσʔλϕʔεͷ༰Λूܭ͢ΔΑ͏ͳॏ ͍ΫΤϦ͕͛ΒΕΔ͜ͱ͕ଟ͍ w όονॲཧ༻ͷ%#εϨʔϒΛ༻ҙͯ͠ɺͦͬͪʹ
͚Δ
ͯͳϒϩά
.BDLFSFM
ʮࡸ૿ͤʁʯ
؆୯ʹ૿ͯ͠ղܾͰ͖ΔΑ͏ ͳΞʔΩςΫνϟΛߟ͑Δͷ͕ ͨͷ͍͠
ࢀߟจݙ
None
wʮ͢ΔαʔϏεʯ w IUUQCMPHIBSVLBTBOKQFOUSZ wʮΫϥυͱΦϯϓϨʯ w IUUQCMPHIBSVLBTBOKQFOUSZ
·ͱΊ wΞϓϦέʔγϣϯαʔόΛத৺ʹݱͷαʔό ߏͱͦͷצͲ͜ΖΛհ wͳͥෳͷαʔόΛ͏ͷ͔ w8FCαʔϏεߏ 1SPYZ"QQ%# wΩϟογϡɺ/P42-ɺඇಉظॲཧɺόον
͍͞͝ʹ wΞϓϦέʔγϣϯΤϯδχΞ͔ΒΦϖ ϨʔγϣϯΤϯδχΞʹస͢Δਓ ҙ֎ͱଟ͍ w͢͜͠Ͱ8FCͷΠϯϑϥʹڵຯΛ ͬͯΒ͑Εͱࢥ͍·͢