Upgrade to Pro — share decks privately, control downloads, hide ads and more …

負荷分散技術を選ぶときに考えること

 負荷分散技術を選ぶときに考えること

ペパボはてな技術大会(京都)発表資料

masayoshi

July 04, 2016
Tweet

More Decks by masayoshi

Other Decks in Technology

Transcript

  1. ෛՙ෼ࢄٕज़Λ
    બͿͱ͖ʹߟ͑Δ͜ͱ
    ϖύϘ͸ͯͳٕज़େձ ژ౎

    גࣜձࣾ͸ͯͳ
    JENBTBZPTIJ

    View full-size slide

  2. ຊ೔ͷൃද
    • ࣗݾ঺հ
    • ෛՙ෼ࢄͷ֓ཁ
    • αʔό಺Ϧιʔεຖͷෛՙ෼ࢄ
    • ϩʔϧຖʹෛՙ෼ࢄ
    • αʔόຖͷෛՙ෼ࢄ
    • ෛՙ෼ࢄٕज़ͷબͼํͱ·ͱΊ

    View full-size slide

  3. ID: masayoshi
    Twitter: yoyogidesaiz
    ৽ଔ 2016೥4݄
    ΢ΣϒΦϖϨʔγϣϯΤϯδχΞ

    View full-size slide

  4. • େֶ
    • ωοτϫʔΫܥͷݚڀ
    • SDN(Software-Defined Networking)
    • ΞϧόΠτͰαʔό؂ࢹۀ຿
    • ࢓ࣄ
    • ωοτϫʔΫ͕த৺
    • αʔϏε͕Քಇ͍ͯ͠Δαʔό΋ݟΔ

    View full-size slide

  5. 2
    ෛՙ෼ࢄٕज़ͷ֓ཁ

    View full-size slide

  6. %BUB$FOUFS
    %BUB$FOUFS
    6TFS
    1SPYZ
    8FC
    -PBE#BMBODFS
    4FSWFS
    8FC
    %#
    ଎͍ ஗͍

    View full-size slide

  7. ෛՙ෼ࢄͷछྨ
    • αʔό಺Ϧιʔεͷෛՙ෼ࢄ
    • ϩʔϧ͝ͱͷෛՙ෼ࢄ
    • αʔό͝ͱͷෛՙ෼ࢄ

    View full-size slide

  8. 3
    αʔό಺Ϧιʔεͷ
    ෛՙ෼ࢄ

    View full-size slide

  9. %BUB$FOUFS
    %BUB$FOUFS
    6TFS
    1SPYZ
    8FC
    -PBE#BMBODFS
    4FSWFS
    8FC
    %#
    ଎͍ ஗͍

    View full-size slide

  10. αʔό಺Ϧιʔεͷෛՙ෼ࢄٕज़
    • ϋʔυ΢ΣΞػೳ
    • Hyper-ThreadingɺRAID0ͳͲ
    • ϋʔυ΢ΣΞΛҙࣝͨ͠OSͷෛՙ෼ࢄػೳ
    • NUMAɺRSS/RFS/RPSͳͲ
    • OSΛҙࣝͨ͠ΞϓϦέʔγϣϯͷ࡞Γํ
    • fork()ɺclone()ͳͲͷγεςϜίʔϧ

    View full-size slide

  11. ϝϞϦίϯτϩʔϥ
    /6."/PEF
    ϝϞϦίϯτϩʔϥ
    /6."/PEF
    σʔλ

    View full-size slide

  12. ϓϩηε
    εϨου
    ϓϩηε
    GPSL

    DMPOF

    ϓϩηε
    TFMFDU

    View full-size slide

  13. ϓϩηε
    εϨου
    ϓϩηε
    GPSL

    DMPOF

    ϓϩηε
    TFMFDU

    • fork(), clone()
    • task_tܕߏ଄ମΛෳ੡͢Δ
    • task_tܕߏ଄ମΛεέδϡʔϦϯάͯ͠CPUʹׂ౰
    • select(), epoll()
    • ϑΝΠϧσΟεΫϦϓλ(fd)Λ؂ࢹ͢Δ(IOଟॏԽ)
    • ಡΈॻ͖Ͱ͖Δfd͕͋Ε͹fdΛฦ͢

    View full-size slide

  14. Α͋͘Δ͜ͱ
    top - 13:41:45 up 88 days, 15:26, 1 user, load average: 0.79, 0.78, 0.77
    Tasks: 117 total, 2 running, 115 sleeping, 0 stopped, 0 zombie
    %Cpu0 : 95.9 us, 0.0 sy, 0.0 ni, 3.1 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0 st
    %Cpu1 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
    %Cpu2 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
    %Cpu3 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
    %Cpu4 : 0.0 us, 1.0 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
    %Cpu5 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
    %Cpu6 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
    %Cpu7 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st

    View full-size slide

  15. Ϧιʔεຖͷෛՙ෼ࢄٕज़
    • ϋʔυ΢ΣΞػೳͱϋʔυ΢ΣΞΛҙࣝͨ͠OSػೳ
    • ରԠ͍ͯ͠Ε͹ػೳΛONʹ͢Δ͚ͩ(͓खܰ)
    • ྑ͘΋ѱ͘΋Ӆṭ͞Ε͍ͯΔ(ௐࠪ͸ࠔ೉)
    • ΞϓϦέʔγϣϯ࣮૷
    • OSɺγεςϜίʔϧ΁ͷཧղ
    • ಛʹϛυϧ΢ΣΞͷ࣮૷Λҙࣝ͢Δ
    • OS͕ϦιʔεΛ෼ࢄ͠΍͍͢ίʔυΛॻ͘(೉͍͠)

    View full-size slide

  16. 4
    ϩʔϧຖͰͷ෼ࢄ

    View full-size slide

  17. %BUB$FOUFS
    %BUB$FOUFS
    6TFS
    1SPYZ
    8FC
    -PBE#BMBODFS
    4FSWFS
    8FC
    %#
    ଎͍ ஗͍

    View full-size slide

  18. 6TFS
    1SPYZ 8FC %#
    ଎͍ ஗͍
    %#.4
    ΫΤϦΩϟογϡ
    ಈతϖʔδ
    Ωϟογϡ
    ੩తϑΝΠϧ

    View full-size slide

  19. ϩʔϧຖͷෛՙ෼ࢄٕज़
    • Ϣʔβʹ͍ۙͱ͜ΖͰฦ͍ͯ͘͠
    • جຊతʹ͸Ωϟογϡ
    • ࢖͍Ͳ͜ΖΛݟۃΊΔ
    • ΩϟογϡʹཔΓա͗ͳ͍
    • Ωϟογϡ͕ώοτ͠ͳͯ͘ഁ୼
    • ߏ੒΍࣮૷͕ແཧ͠ա͗ͳ͍ఔ౓ʹ࢖͏

    View full-size slide

  20. 5
    αʔόຖͷෛՙ෼ࢄ

    View full-size slide

  21. %BUB$FOUFS
    %BUB$FOUFS
    6TFS
    1SPYZ
    8FC
    -PBE#BMBODFS
    4FSWFS
    8FC
    %#
    ଎͍ ஗͍

    View full-size slide

  22. αʔόຖͷෛՙ෼ࢄ
    • ෛՙ෼ࢄٕज़Λ࣮ݱ͢Δʹ͸…
    • ෼ࢄγεςϜͷߟ͑ํΛࢀߟʹ͢Δ
    • ౰વͱ͍͑͹౰વ͕ͩ…
    • ಛʹαʔόͷෛՙ෼ࢄͰ͸ద༻͠΍͍͢

    View full-size slide

  23. • ن໛ಁաੑ
    • εέʔϥϏϦςΟ
    • Ґஔಁաੑ
    • Ϧιʔεͷ৔ॴΛҙࣝͤͣಉ͡ํ๏ͰΞΫηε
    • Կ͔͠Βͷ໊લղܾͷ࢓૊Έ͕ඞཁ
    • ো֐ಁաੑ
    • ো֐͕ൃੜͯ͠΋มΘΓͳ͘ΞΫηεՄೳ
    • ো֐Λݕ஌͢Δ࢓૊Έ͕ඞཁ

    View full-size slide

  24. DNSϥ΢ϯυϩϏϯ
    • ൺֱత༰қʹҐஔಁաੑΛ࣮ݱ
    • ෼ࢄઌϊʔυͷϔϧενΣοΫ͸ͳ͍
    • ΫϥΠΞϯτ࣮૷͕ීٴ
    • ΫϥΠΞϯτͷ઀ଓઌΛ௚઀෼ࢄͤ͞Δ͜ͱ͕Մೳ
    FYBNQMFDPN


    View full-size slide

  25. ϩʔυόϥϯα
    • L4ͳΒIP,ϙʔτͰҐஔಁաੑ͕͋Δ
    • ෼ࢄઌϊʔυͷϔϧενΣοΫ͕Մೳ
    • ωοτϫʔΫͷ੍ݶΛड͚Δ



    View full-size slide

  26. Proxy
    • HAProxyͱ͔NginxͳͲ͕ίωΫγϣϯΛऴ୺͠෼ࢄ
    • ऴ୺͠ͳ͍L7ϩʔυόϥϯα΋͋Δ͕ॲཧෛՙ͕ߴ͍
    • ϔϧενΣοΫ΋Ͱ͖ɺҐஔಁաੑ΋ߴ͍
    IUUQXXXFYBNQMFDPN


    1SPYZ

    View full-size slide

  27. ෼ࢄઌϊʔυͷϔϧενΣοΫ
    • ϔϧενΣοΫ
    • Proxy,LBͳͲ͕෼ࢄઌϊʔυΛ؂ࢹ
    • ໰୊͕͋ΔϊʔυΛ෼ࢄઌϊʔυϦετ͔Β࡟আ
    IUUQXXXFYBNQMFDPN


    1SPYZ

    View full-size slide

  28. ෼ࢄઌϊʔυͷϔϧενΣοΫ
    • ϔϧενΣοΫ
    • Proxy,LBͳͲ͕෼ࢄઌϊʔυΛ؂ࢹ
    • ໰୊͕͋ΔϊʔυΛ෼ࢄઌϊʔυϦετ͔Β࡟আ
    IUUQXXXFYBNQMFDPN


    1SPYZ
    Ͳ͏΍ͬͯ
    ໰୊ͬͯԿ
    ϔϧενΣοΫ͢Δ૷ஔ͕
    ނোͨ͠Β

    View full-size slide

  29. ϔϧενΣοΫ೉͍͠໰୊
    • ࣮૷ɺઃܭڞʹ೉͍͠
    • ͲͷϨΠϠʔ·Ͱ࣮૷͢Δ͔
    • ICMPͰL3νΣοΫ?
    • L7ͰHTTPνΣοΫ?

    View full-size slide

  30. ϔϧενΣοΫ೉͍͠໰୊
    • Ԡ౴͸ਖ਼͍͠ͷ͔?
    • OKͱฦ͍ͯ͠Δϊʔυ͸ຊ౰ʹOK͔ʁԠ౴͸ਖ਼
    ͍͠ͷ͔ʁ
    • Ϗβϯνϯক܉໰୊
    • Gossip Protocol
    • ϥϯμϜʹྡ઀ϊʔυΛબͼ৘ใΛ఻͑Δ
    • ϥϯμϜʹબΜͩෳ਺ϊʔυ͔ΒϔϧενΣοΫ

    View full-size slide

  31. ϩʔυόϥϯαͷ଱ো֐ੑ
    • VRRPͱ͔Α͘࢖ΘΕΔ
    • ৑௕Խٕज़Ͱ͋Γɺෛՙ෼ࢄٕज़Ͱ͸ͳ͍
    • Ұ୆͸ϗοτελϯόΠঢ়ଶ(ॲཧ͸͍ͯ͠ͳ͍)
    7331

    View full-size slide

  32. ΫϥελγεςϜ
    • ༷ʑͳύλʔϯ͕͋Δ͕ྫ͑͹…
    • ෛՙ෼ࢄΛ͢ΔϦʔμ
    • ͦΕҎ֎ͷϊʔυʹॲཧΛ෼ࢄ
    • Ϧʔμ͕ނো࣌͸৽ͨͳϦʔμΛબग़
    • ίϯηϯαεΞϧΰϦζϜ

    View full-size slide

  33. ෛՙ෼ࢄٕज़ͷબͼํͱ
    ·ͱΊ

    View full-size slide

  34. ෛՙ෼ࢄٕज़ͷબͿͱ͖ʹߟ͑Δ͜ͱ
    • ෛՙ෼ࢄٕज़͸૊Έ߹Θͤͯ࢖͏
    • ඞཁҎ্ʹෳࡶʹ͠ͳ͍
    • ෛՙ෼ࢄ͕ඞཁͳͷ͔࠶ߟ͢Δ

    View full-size slide

  35. ෛՙ෼ࢄٕज़͸૊Έ߹Θͤͯ࢖͏
    • αʔό಺ͷෛՙ෼ࢄٕज़
    • RFSΛ༗ޮʹͯ͠soft irqΛϚϧνίΞε
    έʔϧ
    • auto NUMAΛ༗ޮʹ͢Δ
    • Prefork,eventܕϛυϧ΢ΣΞͰίωΫγϣ
    ϯΛ͞͹͘

    View full-size slide

  36. ෛՙ෼ࢄٕज़͸૊Έ߹Θͤͯ࢖͏
    • ϩʔϧຖͷෛՙ෼ࢄٕज़
    • ProxyͰ੩తϑΝΠϧΛฦ͢
    • ΩϟογϡαʔόʹಈతϖʔδΛΩϟογϡͤ͞Δ
    • ώοτ͢Ε͹ฦ͢
    • ώοτ͠ͳ͍ͳΒDBʹ໰͍߹ΘͤΔ
    • DBαʔόͰ͸ΫΤϦΩϟογϡΛ༗ޮʹ͢Δ

    View full-size slide

  37. ෛՙ෼ࢄٕज़͸૊Έ߹Θͤͯ࢖͏
    • αʔόෛՙ෼ࢄٕज़
    • LBΛಋೖ͢Δ
    • Ґஔಁաੑͷ֬อ
    • DNS
    • ো֐ಁաੑͷ֬อ
    • Gossip ProrocolͰϔϧενΣοΫͱϊʔυ؅ཧ
    • ίϯηϯαεΞϧΰϦζϜͰϦʔμબग़

    View full-size slide

  38. ඞཁҎ্ʹෳࡶʹ͠ͳ͍
    • ෳࡶͰ͋Δ͜ͱ͸ѱ
    • ίετͷ૿Ճ
    • αʔόඅ༻ͷ૿Ճ
    • ؅ཧίετͷ૿Ճ
    • ো֐ൃੜ཰ͷ૿Ճ

    View full-size slide

  39. ෛՙ෼ࢄ͕ඞཁͳͷ͔࠶ߟ͢Δ
    • ෛՙΛݮΒ͢
    • ΞϧΰϦζϜɺSQLΫΤϦվળ
    • վળग़དྷͨͱ͖ͷݟฦΓ͕େ͖͍
    • ॲཧ଎౓Λ͋͛Δ
    • ίετ͕ݟ߹͏ͳΒεέʔϧΞοϓ
    • ଴ͪߦྻཧ࿦
    • ͦΕͰ΋ͩΊͳΒෛՙΛ෼ࢄ͢Δ

    View full-size slide

  40. ·ͱΊ
    • ෛՙ෼ࢄٕज़͸૊Έ߹ΘͤΔ
    • Ϧιʔεɺϩʔϧɺαʔό
    • ֤ʑͷ෼ࢄٕज़ͷಁաੑΛߟ͑Δ
    • Ґஔಁաੑɺো֐ಁաੑͳͲ
    • ෛՙ෼ࢄΛ͢΂͖͔Λߟ͑Δ
    • ෛՙ෼ࢄ͠ͳͯ͘ࡁΉͳΒͦΕ͕Ұ൪

    View full-size slide