Upgrade to Pro — share decks privately, control downloads, hide ads and more …

こんなこともあろうかと、 サーバーは予め増やして置いた 〜アクセス頻度予測を用いてサーバーを自動増減する「計画的スケーリング」その後〜 / LINE Developer Meetup #32

こんなこともあろうかと、 サーバーは予め増やして置いた 〜アクセス頻度予測を用いてサーバーを自動増減する「計画的スケーリング」その後〜 / LINE Developer Meetup #32

LINE Developer Meetup #32 https://line.connpass.com/event/83597/

Ryo Takaishi

April 25, 2018
Tweet

More Decks by Ryo Takaishi

Other Decks in Technology

Transcript

  1. ∁ੴྒ / GMO Pepabo, Inc.
    2018-04-24 LINE Developer Meetup #32
    ʙΞΫηεස౓༧ଌΛ༻͍ͯαʔόʔΛࣗಈ૿ݮ͢ΔʮܭըతεέʔϦϯάʯͦͷޙʙ
    ͜Μͳ͜ͱ΋͋Ζ͏͔ͱɺ
    αʔόʔ͸༧Ί૿΍ͯ͠ஔ͍ͨ

    View Slide

  2. ιϑτ΢ΣΞΤϯδχΞ
    ∁ੴ ྒ / @r_takaishi
    https://repl.info/
    TAKAISHI Ryo
    ٕज़෦ ٕज़ج൫νʔϜ

    View Slide

  3. ΩϟύγςΟϓϥϯχϯά

    View Slide

  4. μ΢ϯ͠ͳ͍ෛՙΛ͞͹͘

    ແବͳ͘ద੾ͳϦιʔεΛ഑ஔ͢Δ

    View Slide

  5. !5
    • ϐʔΫλΠϜʹ͋Θͤͯ୆਺Λܾఆ
    • ϝσΟΞ࿐ग़ɾΩϟϯϖʔϯ࣌ʹ͸ݸผʹ૿ઃ
    ΞϓϦέʔγϣϯαʔόʔͷ৔߹@minne

    View Slide

  6. !6
    ϐʔΫλΠϜʹ͋Θͤͯ୆਺Λઃఆ͢Δ৔߹










    ࣌ؒ

    ΞΫηεස౓ʢਖ਼نԽࡁʣ αʔόʔ୆਺

    View Slide

  7. ΞΫηε͕গͳ͍࣌ؒ͸αʔόʔ
    ୆਺Λ͍͍ײ͡ʹݮΒ͍ͨ͠

    View Slide

  8. ΞΫηε͕ଟ͍࣌ؒ͸ͦΕΛࡹ͚
    ΔΑ͏ͳαʔόʔ୆਺ʹ͍ͨ͠

    View Slide

  9. ෛՙʹԠͯ͡
    Φʔτεέʔϧ͢Ε͹͍͍

    View Slide

  10. !10
    • ϐʔΫ࣌ͷෛՙ͸ॠؒతʹൃੜ͢ΔͨΊɺݕ஌ͯ͠
    ͔ΒͷεέʔϧΞ΢τͰ͸ؒʹ߹Θͳ͍
    • αʔόʔͷىಈʙαʔϏεΠϯʹ͔͔ΔλΠϜϥά
    • Ծ૝Ϛγϯͷىಈ
    • ىಈ࣌ʹ࠷৽ͷΞϓϦέʔγϣϯίʔυΛσϓϩΠ
    ෛՙΛݕ஌ͯ͠Φʔτεέʔϧ͢Δख๏

    View Slide

  11. ͳ͔ͳ͔೉͍͠ʜ

    View Slide

  12. ͱࢥ͍ͬͯͨΒɺ೥ͷ಄ʹ

    View Slide

  13. ͍͍΋ͷ͋ΔΑ…ʂ

    View Slide

  14. ࡾ୐ݚڀॴһʹΑΔ
    ఏҊख๏

    View Slide

  15. ΞΫηεස౓༧ଌΛ༻͍ͨ
    ܭըతεέʔϦϯά

    View Slide

  16. ͜Ε·Ͱ஝ੵ͖ͯͨ͠
    ΞΫηεϩάΛ࢖ͬͯ
    ະདྷͷΞΫηεස౓Λ༧ଌ͢Δ

    View Slide

  17. ࣄલʹΞΫηεස౓͕Θ͔Ε͹ɺ
    ඞཁͳαʔόʔ୆਺΋Θ͔Δ

    View Slide

  18. • minneͰ͸ΞΫηεϩάɾߦಈϩάΛอଘ͍ͯ͠Δ
    • ྫ͑͹աڈ1ϲ݄ؒɺ1࣌ؒ୯ҐͷΞΫηεස౓͕Θ͔Δ
    • ͜ͷ࣌ܥྻσʔλΛ༻͍ͯɺະདྷͷΞΫηεස౓Λ༧ଌ͢Δ
    • ࣄલʹΞΫηεස౓͕Θ͔Ε͹ɺॲཧՄೳͳ୆਺ʹ૿ݮͰ͖Δ
    !18
    Ͳ͏͍͏΋ͷ͔ͬ͘͟Γ

    View Slide

  19. • minneͰ͸ΞΫηεϩάɾߦಈϩάΛอଘ͍ͯ͠Δ
    • ྫ͑͹աڈ1ϲ݄ؒɺ1࣌ؒ୯ҐͷΞΫηεස౓͕Θ͔Δ
    • ͜ͷ࣌ܥྻσʔλΛ༻͍ͯɺະདྷͷΞΫηεස౓Λ༧ଌ͢Δ
    • ࣄલʹΞΫηεස౓͕Θ͔Ε͹ɺॲཧՄೳͳ୆਺ʹ૿ݮͰ͖Δ
    !19
    Ͳ͏͍͏΋ͷ͔ͬ͘͟Γ
    ࠓ೔࿩͢ͷ͸ɺಛʹ͜ͷ෦෼

    View Slide

  20. !20
    ϩάऩू෦෼͕ؾʹͳΔํ͸εϥΠυΛνΣοΫʂ
    IUUQTTQFBLFSEFDLDPNNPOPDISPNFHBOFQFQBCPMPHJOGSBTUSVDUVSFCJHGPPU

    View Slide

  21. !21
    ༧ଌʹ͍ͭͯؾʹͳΔํ͸ϖύݚϒϩάΛνΣοΫʂ
    IUUQTSBOEQFQBCPDPNBSUJDMFJPUNJZBLFZ

    View Slide

  22. ͱ͍͏Θ͚Ͱ
    ࡞͍ͬͯͧ͘

    View Slide

  23. !23

    ΞΫηεස౓༧ଌ܅ʢϖύݚʣ

    αʔόʔ୆਺ܭࢉɾ૿ݮ܅ʢminneʣ
    ඞཁͳίϯϙʔωϯτ͸Կ͔ʁ

    View Slide

  24. !24
    ͜Ε͕ܭըతεέʔϦϯάͷߏ੒ͩʂ
    cron
    Redis
    ΞΫηεස౓༧ଌ܅
    (Sidekiq)
    IaaS API Endpoint
    ༧ଌΞΫηε਺ͷऔಘ
    ఆظ࣮ߦ
    αʔόʔ୆਺ܭࢉɾ૿ݮ܅
    (hakata)
    ඞཁαʔόʔ୆਺Λܭࢉɾ૿ݮ
    ༧૝ΞΫηεස౓Λอଘ

    View Slide

  25. !25
    ܭըతεέʔϦϯάͷྲྀΕ
    ࣌ ࣌
    ࣌ ࣌


    ຖ೔0࣌ա͗ɿ
    1೔෼ͷΞΫηεස౓Λ༧ଌ͢ΔΑ
    ຖ࣌40෼࣌ա͗ɿ
    ࣍ͷ1࣌ؒͷΞΫηεස౓༧ଌ͔Βαʔόʔ୆਺Λ૿ݮ͢ΔΑ

    View Slide

  26. • ༧ଌΞΫηεස౓(ϦΫΤετ਺/࣌)Λϕʔεͱ͢Δ
    • αʔόʔ͕ৗʹҰఆͷεϧʔϓοτ(rpm)Λอͭ୆਺Λ໨ࢦ͢
    • εϧʔϓοτظ଴஋(rpm)͸NewRelicͷϝτϦΫε͔Βࢉग़
    !26
    ඞཁαʔόʔ਺ͷܭࢉ
    ඞཁ୆਺༧ଌΞΫηεස౓εϧʔϓοτظ଴஋

    View Slide

  27. !27
    • ༧ଌΞΫηεස౓ɿ60,000 ϦΫΤετ/࣌
    • εϧʔϓοτظ଴஋ɿ1,000 rpm
    αʔόʔ୆਺ܭࢉͷྫ
    ඞཁ୆਺༧ଌΞΫηεස౓εϧʔϓοτظ଴஋


    View Slide

  28. • ʮΞΫηεස౓༧ଌʯͱʮϦιʔεྔܭࢉɾ૿ݮʯͷ2͕ͭඞཁ
    • ΞΫηεස౓༧ଌ͸1೔ʹ1ճ
    • Ϧιʔε૿ݮͷλΠϛϯά͸1࣌ؒʹ1ճ
    • Ϧιʔεྔܭࢉ͸༧ଌ஋ͱεϧʔϓοτظ଴஋Λݩʹܭࢉ
    !28
    ܭըతεέʔϦϯάͷઃܭ·ͱΊ

    View Slide

  29. ධՁɾ՝୊

    View Slide

  30. !30
    ΞϓϦέʔγϣϯαʔόʔͷCPU࢖༻཰
    ܭըతεέʔϧɿແޮ
    ܭըతεέʔϧɿ༗ޮ

    View Slide

  31. !31
    ΞϓϦέʔγϣϯαʔόʔ୆਺ͷભҠ
    ܭըతεέʔϧɿແޮ ܭըతεέʔϧɿ༗ޮ

    View Slide

  32. ݁ߏ͍͍͔Μ͡ʹಈ͍͕ͨʜ

    View Slide

  33. ͍͔ͭ͘՝୊΋

    View Slide

  34. !34
    • ༧ଌͷਫ਼౓௿ԼʹΑΔϦιʔεෆ଍
    • ༧ଌʹݱΕͳ͍ΞΫηε΁ͷରԠ
    • εέʔϧΞ΢τ͕ؒʹ߹Θͳ͍έʔε
    ՝୊

    View Slide

  35. ͜͜·Ͱ͸೥ͷ࿩

    View Slide

  36. ผ࡞ۀͷͨΊɺ
    Ұ࣌తʹఀࢭ͍ͯͨ͠

    View Slide

  37. ݄ࠒ͔Βվળʹணख

    View Slide

  38. εέʔϧΞ΢τ
    ͕ؒʹ߹Θͳ͍໰୊

    View Slide

  39. !39
    • αʔόʔىಈ࣌ʹΞϓϦέʔγϣϯͷίʔυΛऔಘɾ
    ల։͓ͯ͠ΓɺϘτϧωοΫͱͳ͍ͬͯΔ
    • ࠷৽Πϝʔδ ≠ ࠷৽ΞϓϦ
    • ΞϓϦͷϥΠϑαΠΫϧ ≠ αʔόʔͷϥΠϑαΠΫϧ
    ͳͥɺεέʔϧΞ΢τ͕ؒʹ߹Θͳ͍ͷ͔ʁ

    View Slide

  40. !40
    ϥΠϑαΠΫϧ
    CentOS minimal www www
    build
    minimal image
    build
    www image
    image lifecycle server lifecycle
    install puppet
    apply puppet
    deploy minne-app
    launch
    www server
    www
    deploy minne-app
    terminate
    www server
    deleted
    release
    minne-app

    View Slide

  41. !41
    • ࠷৽ΞϓϦ = ࠷৽ΠϝʔδΛ໨ࢦ͢
    • ϦϦʔε࣌ʹΠϝʔδΛϏϧυ͠ɺ࠷৽ͷΞϓϦΛ
    ؚΉΑ͏ʹ͢Δ
    ϦϦʔε࣌ʹΠϝʔδΛϏϧυ͢Δ

    View Slide

  42. !42
    • baseΠϝʔδͷϏϧυ
    • wwwΠϝʔδͷϏϧυ
    • wwwΠϝʔδͷݕࠪ
    • αʔόʔͱͯ͠ىಈ͢Δͷ͔ʁ
    • ىಈͨ͠ޙɺͪΌΜͱ֤ϓϩηε͕ಈ͍ͯΞΫηεͰ͖Δ͔ʁ
    ΠϝʔδϏϧυʹඞཁͳ΋ͷ͸…ʁ

    View Slide

  43. !43
    ͍͍͔Μ͡ʹΠϝʔδΛϏϧυͯ͘͠ΕΔ΍ͭ
    hakata
    workflow
    - build
    - push
    - verify
    notify
    run
    खݩ͔Βssh࣮ͯ͠ߦ
    ΞϓϦͷϦϦʔε࣌ʹϏϧυ
    IaaCͷϦϦʔε࣌ʹϏϧυ

    View Slide

  44. ࣮૷ͯ͠ɺಈ࡞Λ֬ೝ
    ࠷৽ΞϓϦ࠷৽Πϝʔδ✨

    View Slide

  45. ܭըతεέʔϦϯάΛ࠶Քಇͯ͠ɺ
    վળ͞Ε͔ͨݕূͨ͠

    View Slide

  46. ܭըతεέʔϦϯάΛ࠶Քಇͯ͠ɺ
    վળ͞Ε͔ͨݕূͨ͠

    View Slide

  47. !47
    • ΠϝʔδͷࣗಈϏϧυʹࢥͬͨΑΓ͕͔͔࣌ؒͬͨ
    • ࠶Քಇ͠Α͏ͱͨ͠Β͍Ζ͍Ζಈ͔ͣbug fix
    ࠶Քಇ͸͜ͷMeetupʹؒʹ߹Θͣ…

    View Slide

  48. ࠓޙʹ͝ظ଴͍ͩ͘͞ʂ

    View Slide

  49. ·ͱΊ

    View Slide

  50. !50
    • ΞΫηεස౓༧ଌΛ༻͍ɺΞΫηε͕૿͑ΔલʹΦʔ
    τεέʔϧ
    • ϖύݚͱ࿈ܞɾ෼୲͠ɺ࣮ݱ
    • ࢼݧӡ༻͠ɺ՝୊Λચ͍ग़ͨ͠
    • ͕ɺվળ·Ͱ͸Ͱ͖ͣ…
    ·ͱΊ

    View Slide

  51. Α͔ͬͨ͜ͱ

    View Slide

  52. !52
    • ϖύݚͱ࿈ܞ͠ɺ৽ٕज़Λ࣮ࡍͷαʔϏεʹಋೖ
    • Φʔτεέʔϧʹ͍ͭͯͷ஌ݟΛಘΔ͜ͱ͕Ͱ͖ͨ
    Α͔ͬͨ͜ͱ

    View Slide

  53. !53
    • ࢓૊Έͷ൚༻Խ
    • ࠓճ͸IaaS্Ͱ࡞͕ͬͨɺ࢓૊Έࣗମ͸ଞͰ΋࢖͑Δ
    • ྫɿKubernetesͷΑ͏ͳίϯςφج൫΁ͷల։
    • ༧ଌʹݱΕͳ͍ΞΫηε΁ͷରԠ
    • ෛՙΛτϦΨʔͱͨ͠εέʔϧΞ΢τ
    ࠓޙ΍͍͖͍ͬͯͨ͜ͱ

    View Slide

  54. We are hiring!
    ϗεςΟϯάαʔϏεͰ৽ٕज़Λ׆༻
    ΢ΣϒαʔϏεͷ4JUF3FMJBCJMJUZ&OHJOFFSJOH
    શࣾతͳٕज़ج൫Λ͕ͬͭΓ
    ࣄۀΛࠩผԽͰ͖Δٕज़Λ࡞Γग़͍ͨ͠ FUDʜ

    View Slide