ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks

ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks

はてなインターン2015 インフラ講義

A658ec7f1badf73819dfa501165016c1?s=128

Yuuki Tsubouchi (yuuk1)

August 22, 2015
Tweet

Transcript

  1. ΢ΣϒΞϓϦέʔγϣϯ ϑϨʔϜϫʔΫͷ޲͜͏ଆ ͸ͯͳΠϯλʔϯ Πϯϑϥߨٛ dΠϯλʔϯͷ͜Ζʹ஌Γ͔ͨͬͨΠϯϑϥͷ࿩d

  2. JEZ@VVLJ γεςϜϓϥοτϑΥʔϜ෦ ͸ͯͳΠϯλʔϯ

  3. ߨٛͷ͸͡Ίʹ wޙ൒աఔʹ޲͚ͯɺ͸ͯͳͷຊ൪γεςϜʹ͍ͭ ͯཧղΛਂΊ͍ͯͩ͘͞ w͸ͯͳʹґଘ͠ͳ͍ͳΔ΂͘Ұൠతͳ࿩Λ৺͕ ͚͍ͯ·͢ w਺ࣈҎ֎͸ެ։ͯ͠Α͍಺༰Ͱ͢ w՝୊͸͋Γ·ͤΜɻؾܰʹௌ͍͍ͯͩ͘͞

  4. None
  5. ΢ΣϒΞϓϦέʔγϣϯ ϑϨʔϜϫʔΫ

  6. 63-3PVUFS $POUSPMMFS .PEFM .Z42- #SPXTFS HTTP Request HTTP Response 8"'Ͱίʔυॻ͍ͯΔͱ͖ͷΠϝʔδ

  7. ΢ΣϒΞϓϦέʔγϣϯ ͷϒϥοΫϘοΫε

  8. ʮϞσϧͷ෦෼͸͍͢͝Πϯϑϥ͕ ͦͷઌʹ͋Δʯ

  9. ͍͢͝Πϯϑϥ

  10. ͸ͯͳϒϩά

  11. .BDLFSFM

  12. None
  13. 8"' Πϯϑϥ

  14. ΈΜͳ͕ݺΜͰΔ Πϯϑϥͬͯ

  15. ΞϓϦέʔγϣϯΤϯδχΞ ͔ΒΈͨΠϯϑϥ wΞϓϦέʔγϣϯࣗମ͸ͲͷΑ͏ʹͯ͠ಈ͍͍ͯΔͷ͔ w8FCαʔό 1SFGPSL σʔλϕʔε઀ଓͷҡ࣋  wΞϓϦέʔγϣϯ͸ͲͷΑ͏ʹͯ͠σʔλϕʔεʹΞΫ ηε͍ͯ͠Δͷ͔ wεϨʔϒ

    γϟʔσΟϯά  wϦΫΤετ͸ΞϓϦέʔγϣϯʹͲͷΑ͏ʹͯ͠౸ୡ͢ Δͷ͔ wϦόʔεϓϩΩγ
  16. 8FCγεςϜͷجຊ

  17. 8FCγεςϜͷجຊ w)551ϦΫΤετΛड͚ͯϨεϙϯεΛฦ͢ wෳ਺ͷϢʔβ͕ಉ࣌ʹϦΫΤετΛ౤͛͏Δ wϨΠςϯγͱεϧʔϓοτΛಉ࣌ʹୡ੒͢Δ ඞཁ͕͋Δ wͲΜͳʹෳࡶͳ8FCγεςϜͰ΋جຊ͸ಉ͡

  18. ΞϓϦέʔγϣϯ͸Ͳͷ Α͏ʹಈ͍͍ͯΔͷ͔

  19. ΞϓϦέʔγϣϯαʔό w8FCαʔϏεʹ͓͍ͯγεςϜͷத৺ͱͳΔαʔό wΞϓϦέʔγϣϯϩδοΫ͸͜͜Ͱ࣮ߦ͞ΕΔ wҎԼͷॲཧΛ࣮ߦ͢Δ8FCαʔό wϦΫΤετड৴ w)551ϦΫΤετͷղੳ wσʔλϕʔεΞΫηεΛؚΉΞϓϦέʔγϣϯϩδοΫͷ ࣮ߦ w)551Ϩεϙϯεͷߏங wϨεϙϯεૹ৴

  20. ΞϓϦέʔγϣϯαʔόͱݴޠ w ੲ͸ωοτϫʔΫ΍)551ͷϔομॲཧ͚ͩ "QBDIF $ݴޠ Ͱ΍Β͍ͤͯͨ w $(*ͱ͔NPE@IPHFͱ͔ͷಾٕज़Ͱແཧ΍Γ$ ݴޠͷ8FCαʔόͷ্ʹ1FSMͳͲΛಈ͔͢ w

    ࠓͰ͸ΞϓϦέʔγϣϯίʔυͱಉ͡ݴޠͷ8FCαʔ όΛ࢖͏ w 1FSMͳΒ4UBSMFU1MBDL 4DBMBͳΒ+FUUZͳͲ
  21. Ͳ͏΍ͬͯωοτϫʔΫ௨৴ ͢Δͷ͔ w ωοτϫʔΫ௨৴͸جຊతʹ04Χʔωϧ͕΍Δ w -JOVYͷ৔߹γεςϜίʔϧΛ࢖ͬͯ04Χʔωϧͷ ػೳΛωοτϫʔΫ௨৴ػೳΛݺͼग़͢ w TPDLFU 

    CJOE  MJTUFO  BDDFQU   w جຊ͸ແݶϧʔϓͭͭ͠ɺΧʔωϧ͔Βσʔλड৴ Πϕϯτ͕͖ͨΒϦΫΤετॲཧʹҠΔ
  22. ΫϥΠΞϯτ αʔό socket(2) bind(2) listen(2) accept(2) connect(2) write(2) (ϒϩοΫ͢Δ) socket(2)

    “GET / HTTP1.1\nHost: TCP 3-way handshake “HTTP/1.1 200 OK\n…” read(2) (accept͔Β໭Δ) read(2) (ϒϩοΫ͢Δ) (ϒϩοΫ͢Δ) (ϒϩοΫ͢Δ) (read͔Β໭Δ) ϦΫΤετॲཧ write(2) (read͔Β໭Δ) read(2) (ϒϩοΫ͢Δ) close(2) লུ
  23. 8FCαʔόΛෳ਺ͷ ΫϥΠΞϯτʹରԠ͢Δ

  24. worker! process worker! process worker! process worker! process parent! process

    Client accept loop accept loop accept loop accept loop fork fork fork fork Client Client Client 1SFGPSLํࣜ
  25. thread thread thread thread Client accept loop accept loop accept

    loop accept loop Client Client Client εϨουϓʔϧํࣜ Process
  26. Client Client Client Client main! thread select loop data! base

    non-blocking I/O Πϕϯτۦಈํࣜ
  27. w 1FSM͸1SFGPSL 4DBMB +BWB ͸εϨουϓʔϧ ͕ଟ͍ w Ϛϧνϓϩηε 1SFGPSL ͔ϚϧνεϨου

    εϨο υϓʔϧ ͔Ͱάϩʔόϧม਺ͷѻ͍͕͔ΘΔ w ϚϧνεϨουͰάϩʔόϧม਺ͷϩοΫڝ߹ͯ͠ ٧·ΔͳͲ w ٯʹϚϧνϓϩηεͳΒෳ਺ͷίϯςΩετ͔Βڞ ௨ͷΦϒδΣΫτΛࢀর͢Δ͜ͱ͸͋·Γͳ͍ ʢϑΝΠϧͳͲ04ͷϦιʔε͸ผʣ
  28. ΞϓϦέʔγϣϯ͸ ͲͷΑ͏ʹͯ͠σʔλϕʔε ʹΞΫηε͍ͯ͠Δͷ͔

  29. σʔλϕʔεͱͷ઀ଓ

  30. ͍ͭ઀ଓ։࢝͢Δͷ͔ w 1FSM w %#*DPOOFDUΛݺΜͩͱ͖ w େ఍ɺ࣮ࡍʹϦΫΤετ͕ͱΜͰ͖ͯσʔλϕʔ εΞΫηε͢Δஈ֊Ͱ઀ଓ͢Δ w 4DBMB

    w ΞϓϦέʔγϣϯͷىಈ࣌ʹ઀ଓࡁΈͷ%#Φϒ δΣΫτ͕࡞੒͞ΕΔ
  31. ઀ଓͷҡ࣋ʁ w σʔλϕʔεͷ઀ଓίετ w 5$1ͷXBZϋϯυγΣʔΫ w %#૚ͷϋϯυγΣʔΫ w %#૚ͷೝূ w

    %#ଆͷ઀ଓ༻ϓϩηεεϨουͷੜ੒ίετ w ઀ଓ਺͕ଟ͍ͱ%#αʔόͷෛՙ͕ߴ͍ w ઀ଓࡁΈͷιέοτΛ࢖͍·Θ͍ͨ͠
  32. ઀ଓͷӬଓԽ w ౎౓઀ଓ w ϦΫΤετॲཧதͷΈ઀ଓΛҡ࣋͢Δ w ECIΦϒδΣΫτΛίϯςΩετΦϒδΣΫτͳ ͲʹΩϟογϡ͢Ε͹Α͍ w ৗ࣌઀ଓ

    w ECIΦϒδΣΫτΛάϩʔόϧม਺ͳͲʹΩϟο γϡ͢Δ w ίωΫγϣϯϓʔϧΛ࢖͏
  33. ઀ଓͷӬଓԽͷϋϚΓϙΠϯτ w ઀ଓ͸͞·͟·ͳཁҼͰ్தͰ੾ΕΔ͜ͱ͕͋Δ w %#͕͓ͪͯϑΣΠϧΦʔόͨ͠ w ωοτϫʔΫো֐ w ECIΦϒδΣΫτ͸উखʹ࠶઀ଓͯ͘͠Εͳ͍ w

    4DPQF$POUBJOFS%#* %#*Y)BOEMFSΛ ࢖͏ w ϩʔυόϥϯα͔ΒॏΈௐ੔Ͱ%#୆αʔϏεΞ΢ τͰ͖ͳ͍ແཧ΍Γ੾Δ
  34. 8FCαʔϏε૚ߏ੒

  35. Proxy App Database request response request response transaction w ϦόʔεϓϩΩγ/HJOY

    "QBDIF w ΞϓϦέʔγϣϯ1FSM 4DBMB w σʔλϕʔε.Z42- 1PTUHSF42-
  36. ͳͥෳ਺ͷαʔόΛ࢖͏ ͷ͔

  37. ෳ਺ͷαʔόΛ࢖͏ཧ༝ wγεςϜͷՄ༻ੑͷ୲อ wෛՙ෼ࢄ wૄ݁߹Խ

  38. Մ༻ੑͷ୲อ wαʔό͸෺ཧతʹյΕͨΓաෛՙ΍ΦϖϨʔγϣϯϛε Ͱఀࢭ wελϯόΠػʹॲཧΛҾ͖ܧ͙ʢϑΣΠϧΦʔόεΠο νΦʔόʣ͕୅දత wΞΫςΟϒɾελϯόΠؒͰૄ௨؂ࢹ w૬ख͕མͪͯͨΒࣗ෼͕ΞΫςΟϒʹͳΔ w7*1 Ծ૝*1 Ͱ઀ଓઌΛมߋ͢Δ

    wϛυϧ΢ΣΞଆͰϞʔυ੾Γସ͑ͳͲ͕ඞཁͳ͜ͱ΋ wLFFQBMJWFE)FBSUCFBUͳͲͷπʔϧ
  39. ෛՙ෼ࢄ wϩʔυόϥϯαΛલஈʹஔ͍ͯৼΓ෼͚ wಉ͡໾ׂͷαʔόΛฒ΂Δ wϩʔυόϥϯαʹ͸7*1ͳͲͷΤϯυϙΠϯτ͕ ͍͍ͭͯΔ wΞϓϦέʔγϣϯ͔Β͸ͦ͜Λࢀর͢Δ͚ͩ wઐ༻ͷϩʔυόϥϯα-74 )"1SPYZ wϦόʔεϓϩΩγͰ୅༻͢Δ͜ͱ΋͋Δ w৑௕Խ΋݉Ͷඋ͍͑ͯΔ

  40. ૄ݁߹Խ wಉ͡ϗετʹෳ਺ͷϛυϧ΢ΣΞΛಉډͤ͞Δ ͜ͱ΋Ͱ͖Δ w࣮ࡍʹ͸ w໰୊ͷ੾Γ෼͚ͷ͠΍͢͞ wෛՙ෼ࢄ w໾ׂ͕ҟͳΔ͔Βͱ͍ͬͯɺશ෦෼͚ͨ΄͏͕ Α͍Θ͚Ͱ͸ͳ͍ wαʔό਺͕૿͑͗͢Δ৔߹͸ಉډ΋ݕ౼

  41. Proxy App Database request response request response transaction w ϦόʔεϓϩΩγ/HJOY

    "QBDIF w ΞϓϦέʔγϣϯ1FSM 4DBMB w σʔλϕʔε.Z42- 1PTUHSF42-
  42. ϦόʔεϓϩΩγ

  43. w άϩʔόϧ͔ΒϦΫΤετΛ௚઀ड͚෇͚Δαʔό w ϦΫΤετͷ಺༰ʹ͋Θͤͯ w ޙଓͷαʔόʹϦΫΤετͨ͠Γ w ΞΫηε੍ޚͨ͠Γ w ੩తϑΝΠϧΛฦͨ͠Γ͢Δ

    w ੍ޚՄೳϙΠϯτΛ૿΍ͯ͠ɺύϑΥʔϚϯε޲্ ͤͨ͞ΓɺγεςϜશମͷӡ༻ޮ཰Λ্͛Δ Proxy App Database request response request response transaction
  44. Proxy bot༻app request response user༻app UserAgent: ~Googlebot w CPU༻ͱVTFS༻Λ6"Ͱ൑ఆͯ͠ৼΓ෼͚Δ w

    ো֐ͷӨڹൣғΛݶఆ͍ͨ͠ w CPUͱVTFSͰνϡʔχϯά಺༰Λม͑ͨΓ͢Δ w QSFGPSLܕͰ͸ಛʹޮ͖΍͍͢
  45. ΞϓϦέʔγϣϯ

  46. w ͍͍ͩͨطग़ w άϩʔόϧͳϦΫΤετ͸ΞϓϦαʔόͰड͚ͯ͸ ͍͚ͳ͍ͱ͍͏Θ͚Ͱ͸ͳ͍ w 4UBSMFU΍+FUUZΑΓɺ/HJOY"QBDIFͷ΄͏͕ ֎ʹެ։͢Δ8FCαʔόͱͯ͠͸҆৺ w มͳϔομ৯Θ͞ΕΔͱ͔

    w খ͞ͳαʔϏεͩͱΞϓϦαʔόͱϦόʔεϓϩΩ γΛಉډͤ͞Δ͜ͱ΋͋Δ Proxy App Database request response request response transaction
  47. σʔλϕʔε

  48. w ͸ͯͳϒϩάن໛ͳΒϚελ̍୆Ͱࡹ͚Δ w ςʔϒϧσʔλͷΑ͘ࢀর͞ΕΔ෦෼͕ϝϞϦʹ৐ͬ ͍ͯͯɺద੾ʹΠϯσοΫεΛ͸Δ w ελϯόΠϚελͷ༻ҙ͠.)"ͷΑ͏ͳ࢓૊ΈͰϑΣ ΠϧΦʔόͤ͞Δ w ϚελͷεέʔϧΞοϓʹݶք͕དྷͨ

    w SFBE͕ݶքࢀর༻εϨʔϒΛ༻ҙ w XSJUF͕ݶքਫฏ෼ׂ Proxy App Database request response request response transaction
  49. slave Load! Balancer master slave slave App ࢀরΫΤϦͷΈ ߋ৽ΫΤϦத৺ SELECT

    INSERT/UPDATE ࢀর༻εϨʔϒ
  50. master A App ਫฏ෼ׂ γϟʔσΟϯά master B master C ϢʔβIDΛ

    3Ͱׂͬͨ৒༨͕0 ৒༨͕1 ৒༨͕2
  51. ϫϯϥϯΫ্ͷ γεςϜΛߏங͢Δ

  52. Ωϟογϡ

  53. w $16ͷΩϟογϡϝϞϦ - -ΩϟογϡͳͲʣ w 04ͷϖʔδΩϟογϡ w .Z42- *OOP%# ͷόοϑΝϓʔϧ

    w ෼ࢄΩϟογϡαʔό NFNDBDIFE  w Ωϟογϡػೳ͖ͭϦόʔεϓϩΩγ 7BSOJTI 4RVJE
  54. Proxy App Database Squid/ Varnisn memcached

  55. ඈͼಓ۩ͱͯ͠ͷ ,74/P42-

  56. w 3FEJT NFNDBDIFE .POHP%#ͳͲ w 3%#.4ΑΓ͸΍͍σʔλϕʔε w ετϨʔδͱͯ͠ͷ҆৺ײ͸ྼΔ͔΋͠Εͳ͍ w ো֐࣌ͷσʔλ੔߹ੑ͕ෆ҆

    w ϦϨʔγϣφϧϞσϧͷΑ͏ͳཧ࿦తʹཱ֬͞Εͨσʔ λϞσϧͰ͸ͳ͍ w ύϑΥʔϚϯε޲্ͷͨΊʹิॿతʹ࢖༻͢Δ͜ͱ͕ଟ͍ w .BDLFSFMͰ͸࣌ܥྻσʔλϕʔεͱͯ͠(SBQIJUFΛ ࢖͍ͬͯΔ
  57. ඇಉظॲཧ

  58. w ϝʔϧ΍5XJUUFS΁ͷ౤ߘͳͲ͸ɺ֎෦γεςϜ͕མ ͪͯҾ͖ͣΒΕͯ͠·͏ͷΛ๷͙ w QSFGPSLͷϓϩηεͷେ෦෼͕ো֐தͷ֎෦γε ςϜΛୟ͍͍ͯΔͱɺࣗ෼ͷαʔϏε΋མͪΔ w ֎෦γεςϜʹґଘͨ͠Γɺ࣮ߦ͕஗͍ॲཧΛ͋ͱͰ ࣮ߦ͍ͨ͠ඇಉظॲཧ w

    ͸ͯͳͰ͸δϣϒΩϡʔΛ࢖͏ w ઐ༻ͷ.Z42-ʹδϣϒΛ౤ೖ w ผαʔόͰಈ͘ϫʔΧʔ͕.Z42-ʹϙʔϦϯά
  59. Proxy App db-master workerdb worker db-slave

  60. όονॲཧ

  61. w ຖேϢʔβʹϝʔϧΛ഑৴͍ͨ͠ɺఆظతʹϗοτΤ ϯτϦΛߋ৽͍ͨ͠ͳͲ w ઐ༻ͷόοναʔόͰDSPOʹεΫϦϓτΛஔ͘ w όονॲཧ͸σʔλϕʔεͷ಺༰Λूܭ͢ΔΑ͏ͳॏ ͍ΫΤϦ͕౤͛ΒΕΔ͜ͱ͕ଟ͍ w όονॲཧ༻ͷ%#εϨʔϒΛ༻ҙͯ͠ɺͦͬͪʹ

    ޲͚Δ
  62. ͸ͯͳϒϩά

  63. .BDLFSFM

  64. ʮࡸ૿΍ͤ͹ʁʯ

  65. ؆୯ʹ૿΍ͯ͠ղܾͰ͖ΔΑ͏ ͳΞʔΩςΫνϟΛߟ͑Δͷ͕ ͨͷ͍͠

  66. ࢀߟจݙ

  67. None
  68. wʮ੒௕͢ΔαʔϏεʯ w IUUQCMPHIBSVLBTBOKQFOUSZ wʮΫϥ΢υͱΦϯϓϨʯ w IUUQCMPHIBSVLBTBOKQFOUSZ

  69. ·ͱΊ wΞϓϦέʔγϣϯαʔόΛத৺ʹݱ৔ͷαʔό ߏ੒ͱͦͷצͲ͜ΖΛ঺հ wͳͥෳ਺ͷαʔόΛ࢖͏ͷ͔ w8FCαʔϏε૚ߏ੒ 1SPYZ"QQ%#  wΩϟογϡɺ/P42-ɺඇಉظॲཧɺόον

  70. ͍͞͝ʹ wΞϓϦέʔγϣϯΤϯδχΞ͔ΒΦϖ ϨʔγϣϯΤϯδχΞʹస޲͢Δਓ͸ ҙ֎ͱଟ͍ w͢͜͠Ͱ΋8FCͷΠϯϑϥʹڵຯΛ ΋ͬͯ΋Β͑Ε͹ͱࢥ͍·͢