ウェブアプリケーションフレームワークの向こう側 / The other side of Web Application Frameworks

はてなインターン2015 インフラ講義

Transcript

1. ΢ΣϒΞϓϦέʔγϣϯ
   ϑϨʔϜϫʔΫͷ޲͜͏ଆ
   ͸ͯͳΠϯλʔϯ
   Πϯϑϥߨٛ
   dΠϯλʔϯͷ͜Ζʹ஌Γ͔ͨͬͨΠϯϑϥͷ࿩d
   

   View Slide

2. JEZ@VVLJ
   γεςϜϓϥοτϑΥʔϜ෦
   ͸ͯͳΠϯλʔϯ
   

   View Slide

3. ߨٛͷ͸͡Ίʹ
   wޙ൒աఔʹ޲͚ͯɺ͸ͯͳͷຊ൪γεςϜʹ͍ͭ
   ͯཧղΛਂΊ͍ͯͩ͘͞
   w͸ͯͳʹґଘ͠ͳ͍ͳΔ΂͘Ұൠతͳ࿩Λ৺͕
   ͚͍ͯ·͢
   w਺ࣈҎ֎͸ެ։ͯ͠Α͍಺༰Ͱ͢
   w՝୊͸͋Γ·ͤΜɻؾܰʹௌ͍͍ͯͩ͘͞
   

   View Slide

4. View Slide

5. ΢ΣϒΞϓϦέʔγϣϯ
   ϑϨʔϜϫʔΫ
   

   View Slide

6. 63-3PVUFS
   $POUSPMMFS
   .PEFM
   .Z42-
   #SPXTFS
   HTTP Request
   HTTP Response
   8"'Ͱίʔυॻ͍ͯΔͱ͖ͷΠϝʔδ
   

   View Slide

7. ΢ΣϒΞϓϦέʔγϣϯ
   ͷϒϥοΫϘοΫε
   

   View Slide

8. ʮϞσϧͷ෦෼͸͍͢͝Πϯϑϥ͕
   ͦͷઌʹ͋Δʯ
   

   View Slide

9. ͍͢͝Πϯϑϥ
   

   View Slide

10. ͸ͯͳϒϩά
    

    View Slide

11. .BDLFSFM
    

    View Slide

12. View Slide

13. 8"' Πϯϑϥ
    

    View Slide

14. ΈΜͳ͕ݺΜͰΔ
    Πϯϑϥͬͯ
    

    View Slide

15. ΞϓϦέʔγϣϯΤϯδχΞ
    ͔ΒΈͨΠϯϑϥ
    wΞϓϦέʔγϣϯࣗମ͸ͲͷΑ͏ʹͯ͠ಈ͍͍ͯΔͷ͔
    w8FCαʔό 1SFGPSL σʔλϕʔε઀ଓͷҡ࣋
    wΞϓϦέʔγϣϯ͸ͲͷΑ͏ʹͯ͠σʔλϕʔεʹΞΫ
    ηε͍ͯ͠Δͷ͔
    wεϨʔϒ γϟʔσΟϯά
    wϦΫΤετ͸ΞϓϦέʔγϣϯʹͲͷΑ͏ʹͯ͠౸ୡ͢
    Δͷ͔
    wϦόʔεϓϩΩγ
    

    View Slide

16. 8FCγεςϜͷجຊ
    

    View Slide

17. 8FCγεςϜͷجຊ
    w)551ϦΫΤετΛड͚ͯϨεϙϯεΛฦ͢
    wෳ਺ͷϢʔβ͕ಉ࣌ʹϦΫΤετΛ౤͛͏Δ
    wϨΠςϯγͱεϧʔϓοτΛಉ࣌ʹୡ੒͢Δ
    ඞཁ͕͋Δ
    wͲΜͳʹෳࡶͳ8FCγεςϜͰ΋جຊ͸ಉ͡
    

    View Slide

18. ΞϓϦέʔγϣϯ͸Ͳͷ
    Α͏ʹಈ͍͍ͯΔͷ͔
    

    View Slide

19. ΞϓϦέʔγϣϯαʔό
    w8FCαʔϏεʹ͓͍ͯγεςϜͷத৺ͱͳΔαʔό
    wΞϓϦέʔγϣϯϩδοΫ͸͜͜Ͱ࣮ߦ͞ΕΔ
    wҎԼͷॲཧΛ࣮ߦ͢Δ8FCαʔό
    wϦΫΤετड৴
    w)551ϦΫΤετͷղੳ
    wσʔλϕʔεΞΫηεΛؚΉΞϓϦέʔγϣϯϩδοΫͷ
    ࣮ߦ
    w)551Ϩεϙϯεͷߏங
    wϨεϙϯεૹ৴
    

    View Slide

20. ΞϓϦέʔγϣϯαʔόͱݴޠ
    w ੲ͸ωοτϫʔΫ΍)551ͷϔομॲཧ͚ͩ
    "QBDIF $ݴޠ
    Ͱ΍Β͍ͤͯͨ
    w $(*ͱ͔NPE@IPHFͱ͔ͷಾٕज़Ͱແཧ΍Γ$
    ݴޠͷ8FCαʔόͷ্ʹ1FSMͳͲΛಈ͔͢
    w ࠓͰ͸ΞϓϦέʔγϣϯίʔυͱಉ͡ݴޠͷ8FCαʔ
    όΛ࢖͏
    w 1FSMͳΒ4UBSMFU1MBDL 4DBMBͳΒ+FUUZͳͲ
    

    View Slide

21. Ͳ͏΍ͬͯωοτϫʔΫ௨৴
    ͢Δͷ͔
    w ωοτϫʔΫ௨৴͸جຊతʹ04Χʔωϧ͕΍Δ
    w -JOVYͷ৔߹γεςϜίʔϧΛ࢖ͬͯ04Χʔωϧͷ
    ػೳΛωοτϫʔΫ௨৴ػೳΛݺͼग़͢
    w TPDLFU
    CJOE
    MJTUFO
    BDDFQU
    
    w جຊ͸ແݶϧʔϓͭͭ͠ɺΧʔωϧ͔Βσʔλड৴
    Πϕϯτ͕͖ͨΒϦΫΤετॲཧʹҠΔ
    

    View Slide

22. ΫϥΠΞϯτ αʔό
    socket(2) bind(2) listen(2)
    accept(2)
    connect(2)
    write(2)
    (ϒϩοΫ͢Δ)
    socket(2)
    “GET / HTTP1.1\nHost:
    TCP 3-way handshake
    “HTTP/1.1 200 OK\n…”
    read(2)
    (accept͔Β໭Δ)
    read(2)
    (ϒϩοΫ͢Δ)
    (ϒϩοΫ͢Δ)
    (ϒϩοΫ͢Δ)
    (read͔Β໭Δ)
    ϦΫΤετॲཧ
    write(2)
    (read͔Β໭Δ)
    read(2)
    (ϒϩοΫ͢Δ)
    close(2)
    লུ
    

    View Slide

23. 8FCαʔόΛෳ਺ͷ
    ΫϥΠΞϯτʹରԠ͢Δ
    

    View Slide

24. worker!
    process
    worker!
    process
    worker!
    process
    worker!
    process
    parent!
    process
    Client
    accept loop
    accept loop
    accept loop
    accept loop
    fork
    fork
    fork
    fork
    Client
    Client
    Client
    1SFGPSLํࣜ
    

    View Slide

25. thread
    thread
    thread
    thread
    Client
    accept loop
    accept loop
    accept loop
    accept loop
    Client
    Client
    Client
    εϨουϓʔϧํࣜ
    Process
    

    View Slide

26. Client
    Client
    Client
    Client
    main!
    thread
    select loop
    data!
    base
    non-blocking
    I/O
    Πϕϯτۦಈํࣜ
    

    View Slide

27. w 1FSM͸1SFGPSL 4DBMB +BWB
    ͸εϨουϓʔϧ
    ͕ଟ͍
    w Ϛϧνϓϩηε 1SFGPSL
    ͔ϚϧνεϨου εϨο
    υϓʔϧ
    ͔Ͱάϩʔόϧม਺ͷѻ͍͕͔ΘΔ
    w ϚϧνεϨουͰάϩʔόϧม਺ͷϩοΫڝ߹ͯ͠
    ٧·ΔͳͲ
    w ٯʹϚϧνϓϩηεͳΒෳ਺ͷίϯςΩετ͔Βڞ
    ௨ͷΦϒδΣΫτΛࢀর͢Δ͜ͱ͸͋·Γͳ͍
    ʢϑΝΠϧͳͲ04ͷϦιʔε͸ผʣ
    

    View Slide

28. ΞϓϦέʔγϣϯ͸
    ͲͷΑ͏ʹͯ͠σʔλϕʔε
    ʹΞΫηε͍ͯ͠Δͷ͔
    

    View Slide

29. σʔλϕʔεͱͷ઀ଓ
    

    View Slide

30. ͍ͭ઀ଓ։࢝͢Δͷ͔
    w 1FSM
    w %#*DPOOFDUΛݺΜͩͱ͖
    w େ఍ɺ࣮ࡍʹϦΫΤετ͕ͱΜͰ͖ͯσʔλϕʔ
    εΞΫηε͢Δஈ֊Ͱ઀ଓ͢Δ
    w 4DBMB
    w ΞϓϦέʔγϣϯͷىಈ࣌ʹ઀ଓࡁΈͷ%#Φϒ
    δΣΫτ͕࡞੒͞ΕΔ
    

    View Slide

31. ઀ଓͷҡ࣋ʁ
    w σʔλϕʔεͷ઀ଓίετ
    w 5$1ͷXBZϋϯυγΣʔΫ
    w %#૚ͷϋϯυγΣʔΫ
    w %#૚ͷೝূ
    w %#ଆͷ઀ଓ༻ϓϩηεεϨουͷੜ੒ίετ
    w ઀ଓ਺͕ଟ͍ͱ%#αʔόͷෛՙ͕ߴ͍
    w ઀ଓࡁΈͷιέοτΛ࢖͍·Θ͍ͨ͠
    

    View Slide

32. ઀ଓͷӬଓԽ
    w ౎౓઀ଓ
    w ϦΫΤετॲཧதͷΈ઀ଓΛҡ࣋͢Δ
    w ECIΦϒδΣΫτΛίϯςΩετΦϒδΣΫτͳ
    ͲʹΩϟογϡ͢Ε͹Α͍
    w ৗ࣌઀ଓ
    w ECIΦϒδΣΫτΛάϩʔόϧม਺ͳͲʹΩϟο
    γϡ͢Δ
    w ίωΫγϣϯϓʔϧΛ࢖͏
    

    View Slide

33. ઀ଓͷӬଓԽͷϋϚΓϙΠϯτ
    w ઀ଓ͸͞·͟·ͳཁҼͰ్தͰ੾ΕΔ͜ͱ͕͋Δ
    w %#͕͓ͪͯϑΣΠϧΦʔόͨ͠
    w ωοτϫʔΫো֐
    w ECIΦϒδΣΫτ͸উखʹ࠶઀ଓͯ͘͠Εͳ͍
    w 4DPQF$POUBJOFS%#* %#*Y)BOEMFSΛ
    ࢖͏
    w ϩʔυόϥϯα͔ΒॏΈௐ੔Ͱ%#୆αʔϏεΞ΢
    τͰ͖ͳ͍ແཧ΍Γ੾Δ
    

    View Slide

34. 8FCαʔϏε૚ߏ੒
    

    View Slide

35. Proxy App Database
    request
    response
    request
    response
    transaction
    w ϦόʔεϓϩΩγ/HJOY "QBDIF
    w ΞϓϦέʔγϣϯ1FSM 4DBMB
    w σʔλϕʔε.Z42- 1PTUHSF42-
    

    View Slide

36. ͳͥෳ਺ͷαʔόΛ࢖͏
    ͷ͔
    

    View Slide

37. ෳ਺ͷαʔόΛ࢖͏ཧ༝
    wγεςϜͷՄ༻ੑͷ୲อ
    wෛՙ෼ࢄ
    wૄ݁߹Խ
    

    View Slide

38. Մ༻ੑͷ୲อ
    wαʔό͸෺ཧతʹյΕͨΓաෛՙ΍ΦϖϨʔγϣϯϛε
    Ͱఀࢭ
    wελϯόΠػʹॲཧΛҾ͖ܧ͙ʢϑΣΠϧΦʔόεΠο
    νΦʔόʣ͕୅දత
    wΞΫςΟϒɾελϯόΠؒͰૄ௨؂ࢹ
    w૬ख͕མͪͯͨΒࣗ෼͕ΞΫςΟϒʹͳΔ
    w7*1 Ծ૝*1
    Ͱ઀ଓઌΛมߋ͢Δ
    wϛυϧ΢ΣΞଆͰϞʔυ੾Γସ͑ͳͲ͕ඞཁͳ͜ͱ΋
    wLFFQBMJWFE)FBSUCFBUͳͲͷπʔϧ
    

    View Slide

39. ෛՙ෼ࢄ
    wϩʔυόϥϯαΛલஈʹஔ͍ͯৼΓ෼͚
    wಉ͡໾ׂͷαʔόΛฒ΂Δ
    wϩʔυόϥϯαʹ͸7*1ͳͲͷΤϯυϙΠϯτ͕
    ͍͍ͭͯΔ
    wΞϓϦέʔγϣϯ͔Β͸ͦ͜Λࢀর͢Δ͚ͩ
    wઐ༻ͷϩʔυόϥϯα-74 )"1SPYZ
    wϦόʔεϓϩΩγͰ୅༻͢Δ͜ͱ΋͋Δ
    w৑௕Խ΋݉Ͷඋ͍͑ͯΔ
    

    View Slide

40. ૄ݁߹Խ
    wಉ͡ϗετʹෳ਺ͷϛυϧ΢ΣΞΛಉډͤ͞Δ
    ͜ͱ΋Ͱ͖Δ
    w࣮ࡍʹ͸
    w໰୊ͷ੾Γ෼͚ͷ͠΍͢͞
    wෛՙ෼ࢄ
    w໾ׂ͕ҟͳΔ͔Βͱ͍ͬͯɺશ෦෼͚ͨ΄͏͕
    Α͍Θ͚Ͱ͸ͳ͍
    wαʔό਺͕૿͑͗͢Δ৔߹͸ಉډ΋ݕ౼
    

    View Slide

41. Proxy App Database
    request
    response
    request
    response
    transaction
    w ϦόʔεϓϩΩγ/HJOY "QBDIF
    w ΞϓϦέʔγϣϯ1FSM 4DBMB
    w σʔλϕʔε.Z42- 1PTUHSF42-
    

    View Slide

42. ϦόʔεϓϩΩγ
    

    View Slide

43. w άϩʔόϧ͔ΒϦΫΤετΛ௚઀ड͚෇͚Δαʔό
    w ϦΫΤετͷ಺༰ʹ͋Θͤͯ
    w ޙଓͷαʔόʹϦΫΤετͨ͠Γ
    w ΞΫηε੍ޚͨ͠Γ
    w ੩తϑΝΠϧΛฦͨ͠Γ͢Δ
    w ੍ޚՄೳϙΠϯτΛ૿΍ͯ͠ɺύϑΥʔϚϯε޲্
    ͤͨ͞ΓɺγεςϜશମͷӡ༻ޮ཰Λ্͛Δ
    Proxy App Database
    request
    response
    request
    response
    transaction
    

    View Slide

44. Proxy
    bot༻app
    request
    response
    user༻app
    UserAgent: ~Googlebot
    w CPU༻ͱVTFS༻Λ6"Ͱ൑ఆͯ͠ৼΓ෼͚Δ
    w ো֐ͷӨڹൣғΛݶఆ͍ͨ͠
    w CPUͱVTFSͰνϡʔχϯά಺༰Λม͑ͨΓ͢Δ
    w QSFGPSLܕͰ͸ಛʹޮ͖΍͍͢
    

    View Slide

45. ΞϓϦέʔγϣϯ
    

    View Slide

46. w ͍͍ͩͨطग़
    w άϩʔόϧͳϦΫΤετ͸ΞϓϦαʔόͰड͚ͯ͸
    ͍͚ͳ͍ͱ͍͏Θ͚Ͱ͸ͳ͍
    w 4UBSMFU΍+FUUZΑΓɺ/HJOY"QBDIFͷ΄͏͕
    ֎ʹެ։͢Δ8FCαʔόͱͯ͠͸҆৺
    w มͳϔομ৯Θ͞ΕΔͱ͔
    w খ͞ͳαʔϏεͩͱΞϓϦαʔόͱϦόʔεϓϩΩ
    γΛಉډͤ͞Δ͜ͱ΋͋Δ
    Proxy App Database
    request
    response
    request
    response
    transaction
    

    View Slide

47. σʔλϕʔε
    

    View Slide

48. w ͸ͯͳϒϩάن໛ͳΒϚελ̍୆Ͱࡹ͚Δ
    w ςʔϒϧσʔλͷΑ͘ࢀর͞ΕΔ෦෼͕ϝϞϦʹ৐ͬ
    ͍ͯͯɺద੾ʹΠϯσοΫεΛ͸Δ
    w ελϯόΠϚελͷ༻ҙ͠.)"ͷΑ͏ͳ࢓૊ΈͰϑΣ
    ΠϧΦʔόͤ͞Δ
    w ϚελͷεέʔϧΞοϓʹݶք͕དྷͨ
    w SFBE͕ݶքࢀর༻εϨʔϒΛ༻ҙ
    w XSJUF͕ݶքਫฏ෼ׂ
    Proxy App Database
    request
    response
    request
    response
    transaction
    

    View Slide

49. slave
    Load!
    Balancer
    master
    slave
    slave
    App
    ࢀরΫΤϦͷΈ
    ߋ৽ΫΤϦத৺
    SELECT
    INSERT/UPDATE
    ࢀর༻εϨʔϒ
    

    View Slide

50. master A
    App
    ਫฏ෼ׂ γϟʔσΟϯά
    
    master B
    master C
    ϢʔβIDΛ
    3Ͱׂͬͨ৒༨͕0
    ৒༨͕1
    ৒༨͕2
    

    View Slide

51. ϫϯϥϯΫ্ͷ
    γεςϜΛߏங͢Δ
    

    View Slide

52. Ωϟογϡ
    

    View Slide

53. w $16ͷΩϟογϡϝϞϦ - -ΩϟογϡͳͲʣ
    w 04ͷϖʔδΩϟογϡ
    w .Z42- *OOP%#
    ͷόοϑΝϓʔϧ
    w ෼ࢄΩϟογϡαʔό NFNDBDIFE
    
    w Ωϟογϡػೳ͖ͭϦόʔεϓϩΩγ 7BSOJTI
    4RVJE
    
    

    View Slide

54. Proxy App Database
    Squid/
    Varnisn
    memcached
    

    View Slide

55. ඈͼಓ۩ͱͯ͠ͷ
    ,74/P42-
    

    View Slide

56. w 3FEJT NFNDBDIFE .POHP%#ͳͲ
    w 3%#.4ΑΓ͸΍͍σʔλϕʔε
    w ετϨʔδͱͯ͠ͷ҆৺ײ͸ྼΔ͔΋͠Εͳ͍
    w ো֐࣌ͷσʔλ੔߹ੑ͕ෆ҆
    w ϦϨʔγϣφϧϞσϧͷΑ͏ͳཧ࿦తʹཱ֬͞Εͨσʔ
    λϞσϧͰ͸ͳ͍
    w ύϑΥʔϚϯε޲্ͷͨΊʹิॿతʹ࢖༻͢Δ͜ͱ͕ଟ͍
    w .BDLFSFMͰ͸࣌ܥྻσʔλϕʔεͱͯ͠(SBQIJUFΛ
    ࢖͍ͬͯΔ
    

    View Slide

57. ඇಉظॲཧ
    

    View Slide

58. w ϝʔϧ΍5XJUUFS΁ͷ౤ߘͳͲ͸ɺ֎෦γεςϜ͕མ
    ͪͯҾ͖ͣΒΕͯ͠·͏ͷΛ๷͙
    w QSFGPSLͷϓϩηεͷେ෦෼͕ো֐தͷ֎෦γε
    ςϜΛୟ͍͍ͯΔͱɺࣗ෼ͷαʔϏε΋མͪΔ
    w ֎෦γεςϜʹґଘͨ͠Γɺ࣮ߦ͕஗͍ॲཧΛ͋ͱͰ
    ࣮ߦ͍ͨ͠ඇಉظॲཧ
    w ͸ͯͳͰ͸δϣϒΩϡʔΛ࢖͏
    w ઐ༻ͷ.Z42-ʹδϣϒΛ౤ೖ
    w ผαʔόͰಈ͘ϫʔΧʔ͕.Z42-ʹϙʔϦϯά
    

    View Slide

59. Proxy App db-master
    workerdb
    worker
    db-slave
    

    View Slide

60. όονॲཧ
    

    View Slide

61. w ຖேϢʔβʹϝʔϧΛ഑৴͍ͨ͠ɺఆظతʹϗοτΤ
    ϯτϦΛߋ৽͍ͨ͠ͳͲ
    w ઐ༻ͷόοναʔόͰDSPOʹεΫϦϓτΛஔ͘
    w όονॲཧ͸σʔλϕʔεͷ಺༰Λूܭ͢ΔΑ͏ͳॏ
    ͍ΫΤϦ͕౤͛ΒΕΔ͜ͱ͕ଟ͍
    w όονॲཧ༻ͷ%#εϨʔϒΛ༻ҙͯ͠ɺͦͬͪʹ
    ޲͚Δ
    

    View Slide

62. ͸ͯͳϒϩά
    

    View Slide

63. .BDLFSFM
    

    View Slide

64. ʮࡸ૿΍ͤ͹ʁʯ
    

    View Slide

65. ؆୯ʹ૿΍ͯ͠ղܾͰ͖ΔΑ͏
    ͳΞʔΩςΫνϟΛߟ͑Δͷ͕
    ͨͷ͍͠
    

    View Slide

66. ࢀߟจݙ
    

    View Slide

67. View Slide

68. wʮ੒௕͢ΔαʔϏεʯ
    w IUUQCMPHIBSVLBTBOKQFOUSZ
    wʮΫϥ΢υͱΦϯϓϨʯ
    w IUUQCMPHIBSVLBTBOKQFOUSZ
    

    View Slide

69. ·ͱΊ
    wΞϓϦέʔγϣϯαʔόΛத৺ʹݱ৔ͷαʔό
    ߏ੒ͱͦͷצͲ͜ΖΛ঺հ
    wͳͥෳ਺ͷαʔόΛ࢖͏ͷ͔
    w8FCαʔϏε૚ߏ੒ 1SPYZ"QQ%#
    
    wΩϟογϡɺ/P42-ɺඇಉظॲཧɺόον
    

    View Slide

70. ͍͞͝ʹ
    wΞϓϦέʔγϣϯΤϯδχΞ͔ΒΦϖ
    ϨʔγϣϯΤϯδχΞʹస޲͢Δਓ͸
    ҙ֎ͱଟ͍
    w͢͜͠Ͱ΋8FCͷΠϯϑϥʹڵຯΛ
    ΋ͬͯ΋Β͑Ε͹ͱࢥ͍·͢
    

    View Slide