$30 off During Our Annual Pro Sale. View Details »

ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks

ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks

はてなインターン2015 インフラ講義

Yuuki Tsubouchi (yuuk1)

August 22, 2015
Tweet

More Decks by Yuuki Tsubouchi (yuuk1)

Other Decks in Technology

Transcript

  1. ΢ΣϒΞϓϦέʔγϣϯ
    ϑϨʔϜϫʔΫͷ޲͜͏ଆ
    ͸ͯͳΠϯλʔϯ
    Πϯϑϥߨٛ
    dΠϯλʔϯͷ͜Ζʹ஌Γ͔ͨͬͨΠϯϑϥͷ࿩d

    View Slide

  2. JEZ@VVLJ
    γεςϜϓϥοτϑΥʔϜ෦
    ͸ͯͳΠϯλʔϯ

    View Slide

  3. ߨٛͷ͸͡Ίʹ
    wޙ൒աఔʹ޲͚ͯɺ͸ͯͳͷຊ൪γεςϜʹ͍ͭ
    ͯཧղΛਂΊ͍ͯͩ͘͞
    w͸ͯͳʹґଘ͠ͳ͍ͳΔ΂͘Ұൠతͳ࿩Λ৺͕
    ͚͍ͯ·͢
    w਺ࣈҎ֎͸ެ։ͯ͠Α͍಺༰Ͱ͢
    w՝୊͸͋Γ·ͤΜɻؾܰʹௌ͍͍ͯͩ͘͞

    View Slide

  4. View Slide

  5. ΢ΣϒΞϓϦέʔγϣϯ
    ϑϨʔϜϫʔΫ

    View Slide

  6. 63-3PVUFS
    $POUSPMMFS
    .PEFM
    .Z42-
    #SPXTFS
    HTTP Request
    HTTP Response
    8"'Ͱίʔυॻ͍ͯΔͱ͖ͷΠϝʔδ

    View Slide

  7. ΢ΣϒΞϓϦέʔγϣϯ
    ͷϒϥοΫϘοΫε

    View Slide

  8. ʮϞσϧͷ෦෼͸͍͢͝Πϯϑϥ͕
    ͦͷઌʹ͋Δʯ

    View Slide

  9. ͍͢͝Πϯϑϥ

    View Slide

  10. ͸ͯͳϒϩά

    View Slide

  11. .BDLFSFM

    View Slide

  12. View Slide

  13. 8"' Πϯϑϥ

    View Slide

  14. ΈΜͳ͕ݺΜͰΔ
    Πϯϑϥͬͯ

    View Slide

  15. ΞϓϦέʔγϣϯΤϯδχΞ
    ͔ΒΈͨΠϯϑϥ
    wΞϓϦέʔγϣϯࣗମ͸ͲͷΑ͏ʹͯ͠ಈ͍͍ͯΔͷ͔
    w8FCαʔό 1SFGPSL σʔλϕʔε઀ଓͷҡ࣋
    wΞϓϦέʔγϣϯ͸ͲͷΑ͏ʹͯ͠σʔλϕʔεʹΞΫ
    ηε͍ͯ͠Δͷ͔
    wεϨʔϒ γϟʔσΟϯά
    wϦΫΤετ͸ΞϓϦέʔγϣϯʹͲͷΑ͏ʹͯ͠౸ୡ͢
    Δͷ͔
    wϦόʔεϓϩΩγ

    View Slide

  16. 8FCγεςϜͷجຊ

    View Slide

  17. 8FCγεςϜͷجຊ
    w)551ϦΫΤετΛड͚ͯϨεϙϯεΛฦ͢
    wෳ਺ͷϢʔβ͕ಉ࣌ʹϦΫΤετΛ౤͛͏Δ
    wϨΠςϯγͱεϧʔϓοτΛಉ࣌ʹୡ੒͢Δ
    ඞཁ͕͋Δ
    wͲΜͳʹෳࡶͳ8FCγεςϜͰ΋جຊ͸ಉ͡

    View Slide

  18. ΞϓϦέʔγϣϯ͸Ͳͷ
    Α͏ʹಈ͍͍ͯΔͷ͔

    View Slide

  19. ΞϓϦέʔγϣϯαʔό
    w8FCαʔϏεʹ͓͍ͯγεςϜͷத৺ͱͳΔαʔό
    wΞϓϦέʔγϣϯϩδοΫ͸͜͜Ͱ࣮ߦ͞ΕΔ
    wҎԼͷॲཧΛ࣮ߦ͢Δ8FCαʔό
    wϦΫΤετड৴
    w)551ϦΫΤετͷղੳ
    wσʔλϕʔεΞΫηεΛؚΉΞϓϦέʔγϣϯϩδοΫͷ
    ࣮ߦ
    w)551Ϩεϙϯεͷߏங
    wϨεϙϯεૹ৴

    View Slide

  20. ΞϓϦέʔγϣϯαʔόͱݴޠ
    w ੲ͸ωοτϫʔΫ΍)551ͷϔομॲཧ͚ͩ
    "QBDIF $ݴޠ
    Ͱ΍Β͍ͤͯͨ
    w $(*ͱ͔NPE@IPHFͱ͔ͷಾٕज़Ͱແཧ΍Γ$
    ݴޠͷ8FCαʔόͷ্ʹ1FSMͳͲΛಈ͔͢
    w ࠓͰ͸ΞϓϦέʔγϣϯίʔυͱಉ͡ݴޠͷ8FCαʔ
    όΛ࢖͏
    w 1FSMͳΒ4UBSMFU1MBDL 4DBMBͳΒ+FUUZͳͲ

    View Slide

  21. Ͳ͏΍ͬͯωοτϫʔΫ௨৴
    ͢Δͷ͔
    w ωοτϫʔΫ௨৴͸جຊతʹ04Χʔωϧ͕΍Δ
    w -JOVYͷ৔߹γεςϜίʔϧΛ࢖ͬͯ04Χʔωϧͷ
    ػೳΛωοτϫʔΫ௨৴ػೳΛݺͼग़͢
    w TPDLFU
    CJOE
    MJTUFO
    BDDFQU

    w جຊ͸ແݶϧʔϓͭͭ͠ɺΧʔωϧ͔Βσʔλड৴
    Πϕϯτ͕͖ͨΒϦΫΤετॲཧʹҠΔ

    View Slide

  22. ΫϥΠΞϯτ αʔό
    socket(2) bind(2) listen(2)
    accept(2)
    connect(2)
    write(2)
    (ϒϩοΫ͢Δ)
    socket(2)
    “GET / HTTP1.1\nHost:
    TCP 3-way handshake
    “HTTP/1.1 200 OK\n…”
    read(2)
    (accept͔Β໭Δ)
    read(2)
    (ϒϩοΫ͢Δ)
    (ϒϩοΫ͢Δ)
    (ϒϩοΫ͢Δ)
    (read͔Β໭Δ)
    ϦΫΤετॲཧ
    write(2)
    (read͔Β໭Δ)
    read(2)
    (ϒϩοΫ͢Δ)
    close(2)
    লུ

    View Slide

  23. 8FCαʔόΛෳ਺ͷ
    ΫϥΠΞϯτʹରԠ͢Δ

    View Slide

  24. worker!
    process
    worker!
    process
    worker!
    process
    worker!
    process
    parent!
    process
    Client
    accept loop
    accept loop
    accept loop
    accept loop
    fork
    fork
    fork
    fork
    Client
    Client
    Client
    1SFGPSLํࣜ

    View Slide

  25. thread
    thread
    thread
    thread
    Client
    accept loop
    accept loop
    accept loop
    accept loop
    Client
    Client
    Client
    εϨουϓʔϧํࣜ
    Process

    View Slide

  26. Client
    Client
    Client
    Client
    main!
    thread
    select loop
    data!
    base
    non-blocking
    I/O
    Πϕϯτۦಈํࣜ

    View Slide

  27. w 1FSM͸1SFGPSL 4DBMB +BWB
    ͸εϨουϓʔϧ
    ͕ଟ͍
    w Ϛϧνϓϩηε 1SFGPSL
    ͔ϚϧνεϨου εϨο
    υϓʔϧ
    ͔Ͱάϩʔόϧม਺ͷѻ͍͕͔ΘΔ
    w ϚϧνεϨουͰάϩʔόϧม਺ͷϩοΫڝ߹ͯ͠
    ٧·ΔͳͲ
    w ٯʹϚϧνϓϩηεͳΒෳ਺ͷίϯςΩετ͔Βڞ
    ௨ͷΦϒδΣΫτΛࢀর͢Δ͜ͱ͸͋·Γͳ͍
    ʢϑΝΠϧͳͲ04ͷϦιʔε͸ผʣ

    View Slide

  28. ΞϓϦέʔγϣϯ͸
    ͲͷΑ͏ʹͯ͠σʔλϕʔε
    ʹΞΫηε͍ͯ͠Δͷ͔

    View Slide

  29. σʔλϕʔεͱͷ઀ଓ

    View Slide

  30. ͍ͭ઀ଓ։࢝͢Δͷ͔
    w 1FSM
    w %#*DPOOFDUΛݺΜͩͱ͖
    w େ఍ɺ࣮ࡍʹϦΫΤετ͕ͱΜͰ͖ͯσʔλϕʔ
    εΞΫηε͢Δஈ֊Ͱ઀ଓ͢Δ
    w 4DBMB
    w ΞϓϦέʔγϣϯͷىಈ࣌ʹ઀ଓࡁΈͷ%#Φϒ
    δΣΫτ͕࡞੒͞ΕΔ

    View Slide

  31. ઀ଓͷҡ࣋ʁ
    w σʔλϕʔεͷ઀ଓίετ
    w 5$1ͷXBZϋϯυγΣʔΫ
    w %#૚ͷϋϯυγΣʔΫ
    w %#૚ͷೝূ
    w %#ଆͷ઀ଓ༻ϓϩηεεϨουͷੜ੒ίετ
    w ઀ଓ਺͕ଟ͍ͱ%#αʔόͷෛՙ͕ߴ͍
    w ઀ଓࡁΈͷιέοτΛ࢖͍·Θ͍ͨ͠

    View Slide

  32. ઀ଓͷӬଓԽ
    w ౎౓઀ଓ
    w ϦΫΤετॲཧதͷΈ઀ଓΛҡ࣋͢Δ
    w ECIΦϒδΣΫτΛίϯςΩετΦϒδΣΫτͳ
    ͲʹΩϟογϡ͢Ε͹Α͍
    w ৗ࣌઀ଓ
    w ECIΦϒδΣΫτΛάϩʔόϧม਺ͳͲʹΩϟο
    γϡ͢Δ
    w ίωΫγϣϯϓʔϧΛ࢖͏

    View Slide

  33. ઀ଓͷӬଓԽͷϋϚΓϙΠϯτ
    w ઀ଓ͸͞·͟·ͳཁҼͰ్தͰ੾ΕΔ͜ͱ͕͋Δ
    w %#͕͓ͪͯϑΣΠϧΦʔόͨ͠
    w ωοτϫʔΫো֐
    w ECIΦϒδΣΫτ͸উखʹ࠶઀ଓͯ͘͠Εͳ͍
    w 4DPQF$POUBJOFS%#* %#*Y)BOEMFSΛ
    ࢖͏
    w ϩʔυόϥϯα͔ΒॏΈௐ੔Ͱ%#୆αʔϏεΞ΢
    τͰ͖ͳ͍ແཧ΍Γ੾Δ

    View Slide

  34. 8FCαʔϏε૚ߏ੒

    View Slide

  35. Proxy App Database
    request
    response
    request
    response
    transaction
    w ϦόʔεϓϩΩγ/HJOY "QBDIF
    w ΞϓϦέʔγϣϯ1FSM 4DBMB
    w σʔλϕʔε.Z42- 1PTUHSF42-

    View Slide

  36. ͳͥෳ਺ͷαʔόΛ࢖͏
    ͷ͔

    View Slide

  37. ෳ਺ͷαʔόΛ࢖͏ཧ༝
    wγεςϜͷՄ༻ੑͷ୲อ
    wෛՙ෼ࢄ
    wૄ݁߹Խ

    View Slide

  38. Մ༻ੑͷ୲อ
    wαʔό͸෺ཧతʹյΕͨΓաෛՙ΍ΦϖϨʔγϣϯϛε
    Ͱఀࢭ
    wελϯόΠػʹॲཧΛҾ͖ܧ͙ʢϑΣΠϧΦʔόεΠο
    νΦʔόʣ͕୅දత
    wΞΫςΟϒɾελϯόΠؒͰૄ௨؂ࢹ
    w૬ख͕མͪͯͨΒࣗ෼͕ΞΫςΟϒʹͳΔ
    w7*1 Ծ૝*1
    Ͱ઀ଓઌΛมߋ͢Δ
    wϛυϧ΢ΣΞଆͰϞʔυ੾Γସ͑ͳͲ͕ඞཁͳ͜ͱ΋
    wLFFQBMJWFE)FBSUCFBUͳͲͷπʔϧ

    View Slide

  39. ෛՙ෼ࢄ
    wϩʔυόϥϯαΛલஈʹஔ͍ͯৼΓ෼͚
    wಉ͡໾ׂͷαʔόΛฒ΂Δ
    wϩʔυόϥϯαʹ͸7*1ͳͲͷΤϯυϙΠϯτ͕
    ͍͍ͭͯΔ
    wΞϓϦέʔγϣϯ͔Β͸ͦ͜Λࢀর͢Δ͚ͩ
    wઐ༻ͷϩʔυόϥϯα-74 )"1SPYZ
    wϦόʔεϓϩΩγͰ୅༻͢Δ͜ͱ΋͋Δ
    w৑௕Խ΋݉Ͷඋ͍͑ͯΔ

    View Slide

  40. ૄ݁߹Խ
    wಉ͡ϗετʹෳ਺ͷϛυϧ΢ΣΞΛಉډͤ͞Δ
    ͜ͱ΋Ͱ͖Δ
    w࣮ࡍʹ͸
    w໰୊ͷ੾Γ෼͚ͷ͠΍͢͞
    wෛՙ෼ࢄ
    w໾ׂ͕ҟͳΔ͔Βͱ͍ͬͯɺશ෦෼͚ͨ΄͏͕
    Α͍Θ͚Ͱ͸ͳ͍
    wαʔό਺͕૿͑͗͢Δ৔߹͸ಉډ΋ݕ౼

    View Slide

  41. Proxy App Database
    request
    response
    request
    response
    transaction
    w ϦόʔεϓϩΩγ/HJOY "QBDIF
    w ΞϓϦέʔγϣϯ1FSM 4DBMB
    w σʔλϕʔε.Z42- 1PTUHSF42-

    View Slide

  42. ϦόʔεϓϩΩγ

    View Slide

  43. w άϩʔόϧ͔ΒϦΫΤετΛ௚઀ड͚෇͚Δαʔό
    w ϦΫΤετͷ಺༰ʹ͋Θͤͯ
    w ޙଓͷαʔόʹϦΫΤετͨ͠Γ
    w ΞΫηε੍ޚͨ͠Γ
    w ੩తϑΝΠϧΛฦͨ͠Γ͢Δ
    w ੍ޚՄೳϙΠϯτΛ૿΍ͯ͠ɺύϑΥʔϚϯε޲্
    ͤͨ͞ΓɺγεςϜશମͷӡ༻ޮ཰Λ্͛Δ
    Proxy App Database
    request
    response
    request
    response
    transaction

    View Slide

  44. Proxy
    bot༻app
    request
    response
    user༻app
    UserAgent: ~Googlebot
    w CPU༻ͱVTFS༻Λ6"Ͱ൑ఆͯ͠ৼΓ෼͚Δ
    w ো֐ͷӨڹൣғΛݶఆ͍ͨ͠
    w CPUͱVTFSͰνϡʔχϯά಺༰Λม͑ͨΓ͢Δ
    w QSFGPSLܕͰ͸ಛʹޮ͖΍͍͢

    View Slide

  45. ΞϓϦέʔγϣϯ

    View Slide

  46. w ͍͍ͩͨطग़
    w άϩʔόϧͳϦΫΤετ͸ΞϓϦαʔόͰड͚ͯ͸
    ͍͚ͳ͍ͱ͍͏Θ͚Ͱ͸ͳ͍
    w 4UBSMFU΍+FUUZΑΓɺ/HJOY"QBDIFͷ΄͏͕
    ֎ʹެ։͢Δ8FCαʔόͱͯ͠͸҆৺
    w มͳϔομ৯Θ͞ΕΔͱ͔
    w খ͞ͳαʔϏεͩͱΞϓϦαʔόͱϦόʔεϓϩΩ
    γΛಉډͤ͞Δ͜ͱ΋͋Δ
    Proxy App Database
    request
    response
    request
    response
    transaction

    View Slide

  47. σʔλϕʔε

    View Slide

  48. w ͸ͯͳϒϩάن໛ͳΒϚελ̍୆Ͱࡹ͚Δ
    w ςʔϒϧσʔλͷΑ͘ࢀর͞ΕΔ෦෼͕ϝϞϦʹ৐ͬ
    ͍ͯͯɺద੾ʹΠϯσοΫεΛ͸Δ
    w ελϯόΠϚελͷ༻ҙ͠.)"ͷΑ͏ͳ࢓૊ΈͰϑΣ
    ΠϧΦʔόͤ͞Δ
    w ϚελͷεέʔϧΞοϓʹݶք͕དྷͨ
    w SFBE͕ݶքࢀর༻εϨʔϒΛ༻ҙ
    w XSJUF͕ݶքਫฏ෼ׂ
    Proxy App Database
    request
    response
    request
    response
    transaction

    View Slide

  49. slave
    Load!
    Balancer
    master
    slave
    slave
    App
    ࢀরΫΤϦͷΈ
    ߋ৽ΫΤϦத৺
    SELECT
    INSERT/UPDATE
    ࢀর༻εϨʔϒ

    View Slide

  50. master A
    App
    ਫฏ෼ׂ γϟʔσΟϯά

    master B
    master C
    ϢʔβIDΛ
    3Ͱׂͬͨ৒༨͕0
    ৒༨͕1
    ৒༨͕2

    View Slide

  51. ϫϯϥϯΫ্ͷ
    γεςϜΛߏங͢Δ

    View Slide

  52. Ωϟογϡ

    View Slide

  53. w $16ͷΩϟογϡϝϞϦ - -ΩϟογϡͳͲʣ
    w 04ͷϖʔδΩϟογϡ
    w .Z42- *OOP%#
    ͷόοϑΝϓʔϧ
    w ෼ࢄΩϟογϡαʔό NFNDBDIFE

    w Ωϟογϡػೳ͖ͭϦόʔεϓϩΩγ 7BSOJTI
    4RVJE

    View Slide

  54. Proxy App Database
    Squid/
    Varnisn
    memcached

    View Slide

  55. ඈͼಓ۩ͱͯ͠ͷ
    ,74/P42-

    View Slide

  56. w 3FEJT NFNDBDIFE .POHP%#ͳͲ
    w 3%#.4ΑΓ͸΍͍σʔλϕʔε
    w ετϨʔδͱͯ͠ͷ҆৺ײ͸ྼΔ͔΋͠Εͳ͍
    w ো֐࣌ͷσʔλ੔߹ੑ͕ෆ҆
    w ϦϨʔγϣφϧϞσϧͷΑ͏ͳཧ࿦తʹཱ֬͞Εͨσʔ
    λϞσϧͰ͸ͳ͍
    w ύϑΥʔϚϯε޲্ͷͨΊʹิॿతʹ࢖༻͢Δ͜ͱ͕ଟ͍
    w .BDLFSFMͰ͸࣌ܥྻσʔλϕʔεͱͯ͠(SBQIJUFΛ
    ࢖͍ͬͯΔ

    View Slide

  57. ඇಉظॲཧ

    View Slide

  58. w ϝʔϧ΍5XJUUFS΁ͷ౤ߘͳͲ͸ɺ֎෦γεςϜ͕མ
    ͪͯҾ͖ͣΒΕͯ͠·͏ͷΛ๷͙
    w QSFGPSLͷϓϩηεͷେ෦෼͕ো֐தͷ֎෦γε
    ςϜΛୟ͍͍ͯΔͱɺࣗ෼ͷαʔϏε΋མͪΔ
    w ֎෦γεςϜʹґଘͨ͠Γɺ࣮ߦ͕஗͍ॲཧΛ͋ͱͰ
    ࣮ߦ͍ͨ͠ඇಉظॲཧ
    w ͸ͯͳͰ͸δϣϒΩϡʔΛ࢖͏
    w ઐ༻ͷ.Z42-ʹδϣϒΛ౤ೖ
    w ผαʔόͰಈ͘ϫʔΧʔ͕.Z42-ʹϙʔϦϯά

    View Slide

  59. Proxy App db-master
    workerdb
    worker
    db-slave

    View Slide

  60. όονॲཧ

    View Slide

  61. w ຖேϢʔβʹϝʔϧΛ഑৴͍ͨ͠ɺఆظతʹϗοτΤ
    ϯτϦΛߋ৽͍ͨ͠ͳͲ
    w ઐ༻ͷόοναʔόͰDSPOʹεΫϦϓτΛஔ͘
    w όονॲཧ͸σʔλϕʔεͷ಺༰Λूܭ͢ΔΑ͏ͳॏ
    ͍ΫΤϦ͕౤͛ΒΕΔ͜ͱ͕ଟ͍
    w όονॲཧ༻ͷ%#εϨʔϒΛ༻ҙͯ͠ɺͦͬͪʹ
    ޲͚Δ

    View Slide

  62. ͸ͯͳϒϩά

    View Slide

  63. .BDLFSFM

    View Slide

  64. ʮࡸ૿΍ͤ͹ʁʯ

    View Slide

  65. ؆୯ʹ૿΍ͯ͠ղܾͰ͖ΔΑ͏
    ͳΞʔΩςΫνϟΛߟ͑Δͷ͕
    ͨͷ͍͠

    View Slide

  66. ࢀߟจݙ

    View Slide

  67. View Slide

  68. wʮ੒௕͢ΔαʔϏεʯ
    w IUUQCMPHIBSVLBTBOKQFOUSZ
    wʮΫϥ΢υͱΦϯϓϨʯ
    w IUUQCMPHIBSVLBTBOKQFOUSZ

    View Slide

  69. ·ͱΊ
    wΞϓϦέʔγϣϯαʔόΛத৺ʹݱ৔ͷαʔό
    ߏ੒ͱͦͷצͲ͜ΖΛ঺հ
    wͳͥෳ਺ͷαʔόΛ࢖͏ͷ͔
    w8FCαʔϏε૚ߏ੒ 1SPYZ"QQ%#

    wΩϟογϡɺ/P42-ɺඇಉظॲཧɺόον

    View Slide

  70. ͍͞͝ʹ
    wΞϓϦέʔγϣϯΤϯδχΞ͔ΒΦϖ
    ϨʔγϣϯΤϯδχΞʹస޲͢Δਓ͸
    ҙ֎ͱଟ͍
    w͢͜͠Ͱ΋8FCͷΠϯϑϥʹڵຯΛ
    ΋ͬͯ΋Β͑Ε͹ͱࢥ͍·͢

    View Slide