Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks
Yuuki Tsubouchi (yuuk1)
August 22, 2015
Technology
1
240
ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks
はてなインターン2015 インフラ講義
Yuuki Tsubouchi (yuuk1)
August 22, 2015
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
yuukit
4
1.6k
yuukit
1
760
yuukit
0
76
yuukit
0
1.2k
yuukit
0
880
yuukit
0
310
yuukit
5
810
yuukit
0
110
yuukit
0
50
Other Decks in Technology
See All in Technology
gamella
3
1.4k
you
0
270
grapecity_dev
0
130
eayedi
1
120
cfisch3r
1
310
imdigitallab
0
110
mmarukaw
0
1.8k
yosuke_matsuura
PRO
0
170
hashhub
3
15k
azara
1
820
grapecity_dev
0
170
1stship
0
240
Featured
See All Featured
lauravandoore
437
28k
stephaniewalter
260
11k
nonsquared
81
3.4k
hursman
107
9.2k
sstephenson
145
12k
philhawksworth
192
8.8k
mthomps
39
2.3k
shlominoach
176
7.5k
productmarketing
5
710
bkeepers
321
53k
tenderlove
53
3.5k
erikaheidi
14
4.3k
Transcript
ΣϒΞϓϦέʔγϣϯ ϑϨʔϜϫʔΫͷ͜͏ଆ ͯͳΠϯλʔϯ Πϯϑϥߨٛ dΠϯλʔϯͷ͜ΖʹΓ͔ͨͬͨΠϯϑϥͷd
JEZ@VVLJ γεςϜϓϥοτϑΥʔϜ෦ ͯͳΠϯλʔϯ
ߨٛͷ͡Ίʹ wޙաఔʹ͚ͯɺͯͳͷຊ൪γεςϜʹ͍ͭ ͯཧղΛਂΊ͍ͯͩ͘͞ wͯͳʹґଘ͠ͳ͍ͳΔ͘ҰൠతͳΛ৺͕ ͚͍ͯ·͢ wࣈҎ֎ެ։ͯ͠Α͍༰Ͱ͢ w՝͋Γ·ͤΜɻؾܰʹௌ͍͍ͯͩ͘͞
None
ΣϒΞϓϦέʔγϣϯ ϑϨʔϜϫʔΫ
63-3PVUFS $POUSPMMFS .PEFM .Z42- #SPXTFS HTTP Request HTTP Response 8"'Ͱίʔυॻ͍ͯΔͱ͖ͷΠϝʔδ
ΣϒΞϓϦέʔγϣϯ ͷϒϥοΫϘοΫε
ʮϞσϧͷ෦͍͢͝Πϯϑϥ͕ ͦͷઌʹ͋Δʯ
͍͢͝Πϯϑϥ
ͯͳϒϩά
.BDLFSFM
None
8"' Πϯϑϥ
ΈΜͳ͕ݺΜͰΔ Πϯϑϥͬͯ
ΞϓϦέʔγϣϯΤϯδχΞ ͔ΒΈͨΠϯϑϥ wΞϓϦέʔγϣϯࣗମͲͷΑ͏ʹͯ͠ಈ͍͍ͯΔͷ͔ w8FCαʔό 1SFGPSL σʔλϕʔεଓͷҡ࣋ wΞϓϦέʔγϣϯͲͷΑ͏ʹͯ͠σʔλϕʔεʹΞΫ ηε͍ͯ͠Δͷ͔ wεϨʔϒ
γϟʔσΟϯά wϦΫΤετΞϓϦέʔγϣϯʹͲͷΑ͏ʹͯ͠౸ୡ͢ Δͷ͔ wϦόʔεϓϩΩγ
8FCγεςϜͷجຊ
8FCγεςϜͷجຊ w)551ϦΫΤετΛड͚ͯϨεϙϯεΛฦ͢ wෳͷϢʔβ͕ಉ࣌ʹϦΫΤετΛ͛͏Δ wϨΠςϯγͱεϧʔϓοτΛಉ࣌ʹୡ͢Δ ඞཁ͕͋Δ wͲΜͳʹෳࡶͳ8FCγεςϜͰجຊಉ͡
ΞϓϦέʔγϣϯͲͷ Α͏ʹಈ͍͍ͯΔͷ͔
ΞϓϦέʔγϣϯαʔό w8FCαʔϏεʹ͓͍ͯγεςϜͷத৺ͱͳΔαʔό wΞϓϦέʔγϣϯϩδοΫ͜͜Ͱ࣮ߦ͞ΕΔ wҎԼͷॲཧΛ࣮ߦ͢Δ8FCαʔό wϦΫΤετड৴ w)551ϦΫΤετͷղੳ wσʔλϕʔεΞΫηεΛؚΉΞϓϦέʔγϣϯϩδοΫͷ ࣮ߦ w)551Ϩεϙϯεͷߏங wϨεϙϯεૹ৴
ΞϓϦέʔγϣϯαʔόͱݴޠ w ੲωοτϫʔΫ)551ͷϔομॲཧ͚ͩ "QBDIF $ݴޠ ͰΒ͍ͤͯͨ w $(*ͱ͔NPE@IPHFͱ͔ͷಾٕज़ͰແཧΓ$ ݴޠͷ8FCαʔόͷ্ʹ1FSMͳͲΛಈ͔͢ w
ࠓͰΞϓϦέʔγϣϯίʔυͱಉ͡ݴޠͷ8FCαʔ όΛ͏ w 1FSMͳΒ4UBSMFU1MBDL 4DBMBͳΒ+FUUZͳͲ
Ͳ͏ͬͯωοτϫʔΫ௨৴ ͢Δͷ͔ w ωοτϫʔΫ௨৴جຊతʹ04Χʔωϧ͕Δ w -JOVYͷ߹γεςϜίʔϧΛͬͯ04Χʔωϧͷ ػೳΛωοτϫʔΫ௨৴ػೳΛݺͼग़͢ w TPDLFU
CJOE MJTUFO BDDFQU w جຊແݶϧʔϓͭͭ͠ɺΧʔωϧ͔Βσʔλड৴ Πϕϯτ͕͖ͨΒϦΫΤετॲཧʹҠΔ
ΫϥΠΞϯτ αʔό socket(2) bind(2) listen(2) accept(2) connect(2) write(2) (ϒϩοΫ͢Δ) socket(2)
“GET / HTTP1.1\nHost: TCP 3-way handshake “HTTP/1.1 200 OK\n…” read(2) (accept͔ΒΔ) read(2) (ϒϩοΫ͢Δ) (ϒϩοΫ͢Δ) (ϒϩοΫ͢Δ) (read͔ΒΔ) ϦΫΤετॲཧ write(2) (read͔ΒΔ) read(2) (ϒϩοΫ͢Δ) close(2) লུ
8FCαʔόΛෳͷ ΫϥΠΞϯτʹରԠ͢Δ
worker! process worker! process worker! process worker! process parent! process
Client accept loop accept loop accept loop accept loop fork fork fork fork Client Client Client 1SFGPSLํࣜ
thread thread thread thread Client accept loop accept loop accept
loop accept loop Client Client Client εϨουϓʔϧํࣜ Process
Client Client Client Client main! thread select loop data! base
non-blocking I/O Πϕϯτۦಈํࣜ
w 1FSM1SFGPSL 4DBMB +BWB εϨουϓʔϧ ͕ଟ͍ w Ϛϧνϓϩηε 1SFGPSL ͔ϚϧνεϨου
εϨο υϓʔϧ ͔Ͱάϩʔόϧมͷѻ͍͕͔ΘΔ w ϚϧνεϨουͰάϩʔόϧมͷϩοΫڝ߹ͯ͠ ٧·ΔͳͲ w ٯʹϚϧνϓϩηεͳΒෳͷίϯςΩετ͔Βڞ ௨ͷΦϒδΣΫτΛࢀর͢Δ͜ͱ͋·Γͳ͍ ʢϑΝΠϧͳͲ04ͷϦιʔεผʣ
ΞϓϦέʔγϣϯ ͲͷΑ͏ʹͯ͠σʔλϕʔε ʹΞΫηε͍ͯ͠Δͷ͔
σʔλϕʔεͱͷଓ
͍ͭଓ։࢝͢Δͷ͔ w 1FSM w %#*DPOOFDUΛݺΜͩͱ͖ w େɺ࣮ࡍʹϦΫΤετ͕ͱΜͰ͖ͯσʔλϕʔ εΞΫηε͢Δஈ֊Ͱଓ͢Δ w 4DBMB
w ΞϓϦέʔγϣϯͷىಈ࣌ʹଓࡁΈͷ%#Φϒ δΣΫτ͕࡞͞ΕΔ
ଓͷҡ࣋ʁ w σʔλϕʔεͷଓίετ w 5$1ͷXBZϋϯυγΣʔΫ w %#ͷϋϯυγΣʔΫ w %#ͷೝূ w
%#ଆͷଓ༻ϓϩηεεϨουͷੜίετ w ଓ͕ଟ͍ͱ%#αʔόͷෛՙ͕ߴ͍ w ଓࡁΈͷιέοτΛ͍·Θ͍ͨ͠
ଓͷӬଓԽ w ଓ w ϦΫΤετॲཧதͷΈଓΛҡ࣋͢Δ w ECIΦϒδΣΫτΛίϯςΩετΦϒδΣΫτͳ ͲʹΩϟογϡ͢ΕΑ͍ w ৗ࣌ଓ
w ECIΦϒδΣΫτΛάϩʔόϧมͳͲʹΩϟο γϡ͢Δ w ίωΫγϣϯϓʔϧΛ͏
ଓͷӬଓԽͷϋϚΓϙΠϯτ w ଓ͞·͟·ͳཁҼͰ్தͰΕΔ͜ͱ͕͋Δ w %#͕͓ͪͯϑΣΠϧΦʔόͨ͠ w ωοτϫʔΫো w ECIΦϒδΣΫτউखʹ࠶ଓͯ͘͠Εͳ͍ w
4DPQF$POUBJOFS%#* %#*Y)BOEMFSΛ ͏ w ϩʔυόϥϯα͔ΒॏΈௐͰ%#αʔϏεΞ τͰ͖ͳ͍ແཧΓΔ
8FCαʔϏεߏ
Proxy App Database request response request response transaction w ϦόʔεϓϩΩγ/HJOY
"QBDIF w ΞϓϦέʔγϣϯ1FSM 4DBMB w σʔλϕʔε.Z42- 1PTUHSF42-
ͳͥෳͷαʔόΛ͏ ͷ͔
ෳͷαʔόΛ͏ཧ༝ wγεςϜͷՄ༻ੑͷ୲อ wෛՙࢄ wૄ݁߹Խ
Մ༻ੑͷ୲อ wαʔόཧతʹյΕͨΓաෛՙΦϖϨʔγϣϯϛε Ͱఀࢭ wελϯόΠػʹॲཧΛҾ͖ܧ͙ʢϑΣΠϧΦʔόεΠο νΦʔόʣ͕දత wΞΫςΟϒɾελϯόΠؒͰૄ௨ࢹ w૬ख͕མͪͯͨΒ͕ࣗΞΫςΟϒʹͳΔ w7*1 Ծ*1 ͰଓઌΛมߋ͢Δ
wϛυϧΣΞଆͰϞʔυΓସ͑ͳͲ͕ඞཁͳ͜ͱ wLFFQBMJWFE)FBSUCFBUͳͲͷπʔϧ
ෛՙࢄ wϩʔυόϥϯαΛલஈʹஔ͍ͯৼΓ͚ wಉׂ͡ͷαʔόΛฒΔ wϩʔυόϥϯαʹ7*1ͳͲͷΤϯυϙΠϯτ͕ ͍͍ͭͯΔ wΞϓϦέʔγϣϯ͔Βͦ͜Λࢀর͢Δ͚ͩ wઐ༻ͷϩʔυόϥϯα-74 )"1SPYZ wϦόʔεϓϩΩγͰ༻͢Δ͜ͱ͋Δ wԽ݉Ͷඋ͍͑ͯΔ
ૄ݁߹Խ wಉ͡ϗετʹෳͷϛυϧΣΞΛಉډͤ͞Δ ͜ͱͰ͖Δ w࣮ࡍʹ wͷΓ͚ͷ͢͠͞ wෛՙࢄ wׂ͕ҟͳΔ͔Βͱ͍ͬͯɺશ෦͚ͨ΄͏͕ Α͍Θ͚Ͱͳ͍ wαʔό͕૿͑͗͢Δ߹ಉډݕ౼
Proxy App Database request response request response transaction w ϦόʔεϓϩΩγ/HJOY
"QBDIF w ΞϓϦέʔγϣϯ1FSM 4DBMB w σʔλϕʔε.Z42- 1PTUHSF42-
ϦόʔεϓϩΩγ
w άϩʔόϧ͔ΒϦΫΤετΛड͚͚Δαʔό w ϦΫΤετͷ༰ʹ͋Θͤͯ w ޙଓͷαʔόʹϦΫΤετͨ͠Γ w ΞΫηε੍ޚͨ͠Γ w ੩తϑΝΠϧΛฦͨ͠Γ͢Δ
w ੍ޚՄೳϙΠϯτΛ૿ͯ͠ɺύϑΥʔϚϯε্ ͤͨ͞ΓɺγεςϜશମͷӡ༻ޮΛ্͛Δ Proxy App Database request response request response transaction
Proxy bot༻app request response user༻app UserAgent: ~Googlebot w CPU༻ͱVTFS༻Λ6"Ͱఆͯ͠ৼΓ͚Δ w
োͷӨڹൣғΛݶఆ͍ͨ͠ w CPUͱVTFSͰνϡʔχϯά༰Λม͑ͨΓ͢Δ w QSFGPSLܕͰಛʹޮ͖͍͢
ΞϓϦέʔγϣϯ
w ͍͍ͩͨطग़ w άϩʔόϧͳϦΫΤετΞϓϦαʔόͰड͚ͯ ͍͚ͳ͍ͱ͍͏Θ͚Ͱͳ͍ w 4UBSMFU+FUUZΑΓɺ/HJOY"QBDIFͷ΄͏͕ ֎ʹެ։͢Δ8FCαʔόͱͯ҆͠৺ w มͳϔομ৯Θ͞ΕΔͱ͔
w খ͞ͳαʔϏεͩͱΞϓϦαʔόͱϦόʔεϓϩΩ γΛಉډͤ͞Δ͜ͱ͋Δ Proxy App Database request response request response transaction
σʔλϕʔε
w ͯͳϒϩάنͳΒϚελ̍Ͱࡹ͚Δ w ςʔϒϧσʔλͷΑ͘ࢀর͞ΕΔ෦͕ϝϞϦʹͬ ͍ͯͯɺదʹΠϯσοΫεΛΔ w ελϯόΠϚελͷ༻ҙ͠.)"ͷΑ͏ͳΈͰϑΣ ΠϧΦʔόͤ͞Δ w ϚελͷεέʔϧΞοϓʹݶք͕དྷͨ
w SFBE͕ݶքࢀর༻εϨʔϒΛ༻ҙ w XSJUF͕ݶքਫฏׂ Proxy App Database request response request response transaction
slave Load! Balancer master slave slave App ࢀরΫΤϦͷΈ ߋ৽ΫΤϦத৺ SELECT
INSERT/UPDATE ࢀর༻εϨʔϒ
master A App ਫฏׂ γϟʔσΟϯά master B master C ϢʔβIDΛ
3Ͱׂͬͨ༨͕0 ༨͕1 ༨͕2
ϫϯϥϯΫ্ͷ γεςϜΛߏங͢Δ
Ωϟογϡ
w $16ͷΩϟογϡϝϞϦ - -ΩϟογϡͳͲʣ w 04ͷϖʔδΩϟογϡ w .Z42- *OOP%# ͷόοϑΝϓʔϧ
w ࢄΩϟογϡαʔό NFNDBDIFE w Ωϟογϡػೳ͖ͭϦόʔεϓϩΩγ 7BSOJTI 4RVJE
Proxy App Database Squid/ Varnisn memcached
ඈͼಓ۩ͱͯ͠ͷ ,74/P42-
w 3FEJT NFNDBDIFE .POHP%#ͳͲ w 3%#.4ΑΓ͍σʔλϕʔε w ετϨʔδͱͯ͠ͷ҆৺ײྼΔ͔͠Εͳ͍ w ো࣌ͷσʔλ߹ੑ͕ෆ҆
w ϦϨʔγϣφϧϞσϧͷΑ͏ͳཧతʹཱ֬͞Εͨσʔ λϞσϧͰͳ͍ w ύϑΥʔϚϯε্ͷͨΊʹิॿతʹ༻͢Δ͜ͱ͕ଟ͍ w .BDLFSFMͰ࣌ܥྻσʔλϕʔεͱͯ͠(SBQIJUFΛ ͍ͬͯΔ
ඇಉظॲཧ
w ϝʔϧ5XJUUFSͷߘͳͲɺ֎෦γεςϜ͕མ ͪͯҾ͖ͣΒΕͯ͠·͏ͷΛ͙ w QSFGPSLͷϓϩηεͷେ෦͕োதͷ֎෦γε ςϜΛୟ͍͍ͯΔͱɺࣗͷαʔϏεམͪΔ w ֎෦γεςϜʹґଘͨ͠Γɺ࣮ߦ͕͍ॲཧΛ͋ͱͰ ࣮ߦ͍ͨ͠ඇಉظॲཧ w
ͯͳͰδϣϒΩϡʔΛ͏ w ઐ༻ͷ.Z42-ʹδϣϒΛೖ w ผαʔόͰಈ͘ϫʔΧʔ͕.Z42-ʹϙʔϦϯά
Proxy App db-master workerdb worker db-slave
όονॲཧ
w ຖேϢʔβʹϝʔϧΛ৴͍ͨ͠ɺఆظతʹϗοτΤ ϯτϦΛߋ৽͍ͨ͠ͳͲ w ઐ༻ͷόοναʔόͰDSPOʹεΫϦϓτΛஔ͘ w όονॲཧσʔλϕʔεͷ༰Λूܭ͢ΔΑ͏ͳॏ ͍ΫΤϦ͕͛ΒΕΔ͜ͱ͕ଟ͍ w όονॲཧ༻ͷ%#εϨʔϒΛ༻ҙͯ͠ɺͦͬͪʹ
͚Δ
ͯͳϒϩά
.BDLFSFM
ʮࡸ૿ͤʁʯ
؆୯ʹ૿ͯ͠ղܾͰ͖ΔΑ͏ ͳΞʔΩςΫνϟΛߟ͑Δͷ͕ ͨͷ͍͠
ࢀߟจݙ
None
wʮ͢ΔαʔϏεʯ w IUUQCMPHIBSVLBTBOKQFOUSZ wʮΫϥυͱΦϯϓϨʯ w IUUQCMPHIBSVLBTBOKQFOUSZ
·ͱΊ wΞϓϦέʔγϣϯαʔόΛத৺ʹݱͷαʔό ߏͱͦͷצͲ͜ΖΛհ wͳͥෳͷαʔόΛ͏ͷ͔ w8FCαʔϏεߏ 1SPYZ"QQ%# wΩϟογϡɺ/P42-ɺඇಉظॲཧɺόον
͍͞͝ʹ wΞϓϦέʔγϣϯΤϯδχΞ͔ΒΦϖ ϨʔγϣϯΤϯδχΞʹస͢Δਓ ҙ֎ͱଟ͍ w͢͜͠Ͱ8FCͷΠϯϑϥʹڵຯΛ ͬͯΒ͑Εͱࢥ͍·͢