Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SRE vs ラマダン

SRE vs ラマダン

Lightning talk at chukenweb

More Decks by Takayuki WATANABE (渡辺 喬之)

Other Decks in Technology

Transcript

  1. 43&WTϥϚμϯ
    DIVLFOXFC

    View full-size slide

  2. • ౉ลڤ೭ [Takayuki Watanabe]
    • takanabe
    • @takanabe_w
    • ΫοΫύουגࣜձࣾ
    • ΠϯϑϥετϥΫνϟʔ෦ SRE άϧʔϓ
    • ओʹGlobal αʔϏεͷΠϯϑϥͷ։ൃͱӡ༻Λ୲౰
    ࣗݾ঺հ

    View full-size slide

  3. ΫοΫύουͷάϩʔόϧల։ʹͯ SRE ͕ܦݧͨ͠
    ”ϥϚμϯ”ͱ͍͏Πϕϯτ͕ΫοΫύουͷαʔϏε
    Πϯϑϥʹ༩͑ΔӨڹʹ͍ͭͯ
    ࠓ೔ͷ͓࿩

    View full-size slide

  4. https://cookpad.com
    ࠃ಺ͷΫοΫύουͷτοϓϖʔδ

    View full-size slide

  5. (MPCBMαʔϏεͷઆ໌

    View full-size slide

  6. (MPCBMαʔϏεͷઆ໌
    ΫοΫύουͷ Global αʔϏε

    (ҎԼ Global αʔϏε)

    View full-size slide

  7. (MPCBMαʔϏεͷઆ໌
    21 ݴޠɾ67 ΧࠃҎ্Λର৅ʹ
    αʔϏεΛల։
    https://cookpad.com/us
    https://cookpad.com/id
    ɾ
    ɾ
    ɾ

    View full-size slide

  8. (MPCBMαʔϏεͷઆ໌
    ੈքதͷΠϕϯτͷӨڹΛड͚Δ

    View full-size slide

  9. (MPCBMαʔϏεͷઆ໌
    Global αʔϏεʹͱͬͯҰେΠϕϯτ
    ͷҰͭʹͳ͍ͬͯΔͷ͕ɺ

    View full-size slide

  10. ϥϚμϯͬͯͳʹʁ
    • ώδϡϥྐྵͱ͸ΠεϥϜࣾձͰ࢖ΘΕΔྺ๚
    • ϥϚμϯ͸ώδϡϥྐྵʹ͓͚Δ9݄ͷ͜ͱ
    • அ৯ͷ͜ͱͰ͸ͳ͍
    • ϥϚμϯظؒத͸ ೔ͷग़͔Β೔຅·ͰͷؒɺϜεϦϜ
    ͷٛ຿ͷҰͭͱͯ͠அ৯͕ߦΘΕΔ
    • ΠεϥϜྐྵ͸ଠӄྐྵ
    • ϥϚμϯ͸ଠཅྐྵ͔ΒݟΔͱຖ೥11೔΄Ͳૣ·Δ

    View full-size slide

  11. ϥϚμϯظؒதͷϢʔβϦΫΤετ਺ͷมԽ

    View full-size slide

  12. ظؒதͷϦΫΤετ਺ͷมԽ
    ϥϚμϯظؒதͷϢʔβϦΫΤετ਺ͷมԽ
    !!

    View full-size slide

  13. ΫοΫύουʹͱͬͯͷϥϚμϯ
    • ϥϚμϯظؒதɺ೔຅ޙ͸ՈͰීஈΑΓྉཧΛ͢Δਓ͕૿͑Δ
    • ΫοΫύουΛ஌Βͳ͍ΠεϥϜݍͷਓʑ͕αʔϏεΛར༻
    • ৽نϢʔβ֫ಘͷػձ
    • ΞΫςΟϒϢʔβ਺૿Ճ
    • ϨγϐͷӾཡ਺ɾ౤ߘ਺͕૿Ճ
    • ϓϥοτϑΥʔϜͱͯ͠ͷັྗUP
    • ϓϨϛΞϜαʔϏεར༻ऀͷ૿Ճ
    • ऩӹͷ૿Ճ

    View full-size slide

  14. 43&ʹͱͬͯͷϥϚμϯ
    • զʑ͸அ৯Λ͠ͳ͍
    • Ϣʔβ͔ΒͷϦΫΤετ਺͕͋Δ೔ಥવ਺ഒʹͳΓͦΕ͕Ұϲ݄ଓ͘
    • αʔόͷෛՙ૿Ճ
    • ීஈ૝ఆ͍ͯ͠ͳ͍ࣄ৅ͷൃੜ
    • ෛՙ͕֬อ͍ͯ͠ΔαʔόϦιʔεΛ৯͍ਚ͘͢
    • αʔόͷϩά͕ٸ૿ͯ͠σΟεΫΛຒΊਚ͘͢
    • ීஈ໰୊ͳ͍εϩʔΫΤϦͰDB͕٧·Δ
    • etc …

    View full-size slide

  15. ීஈൃੜ͠ͳ͍αʔόΞϥʔτ͕ൃੜ͕ͪ͠

    View full-size slide

  16. 43&WTϥϚμϯ
    • ϥϚμϯΛ
    • Ұ෦ͷٕज़ελοΫͷೖΕସ͑
    • खಈͰରԠ͍ͯͨ͠ΦϖϨʔγϣϯΛࣗಈԽ
    • αʔό܈ͷΩϟύγςΟͷݟ௚͠
    • ܭըతʹϓϩμΫτ։ൃνʔϜͱ࿈ܞ
    • ύϑΥʔϚϯεͷվળґཔ
    • ظؒதʹ࣮ࢪ͢Δ͖ΘͲ͍ࢪࡦͷαϙʔτ
    • Ϣʔβʹීஈͱಉ͡ʹΑ͏ʹαʔϏεΛఏڙ͢ΔͨΊʹͨ͜͠ͱ
    • ͜ΕΒΛ࣮ࢪ͢Δ͜ͱͰظؒதͷ on call ରԠΛݮΒ͢

    View full-size slide

  17. Ұ෦ͷٕज़ελοΫͷೖΕସ͑
    • Database
    • MySQL on Amazon RDS (Magnetic) ͔ΒAmazon
    Aurora ʹશ໘ೖΕସ͑
    • CDN
    • Akamai͔Β Fastly ΁ͷ੾Γସ͑
    • ΞϓϦέʔγϣϯͷ։ൃ؀ڥ
    • ৽نΞϓϦͷ։ൃ͸ Docker + ECS Λར༻
    ˠαʔϏεͷϨεϙϯελΠϜ΍Մ༻ੑͷվળ

    View full-size slide

  18. ΦϖϨʔγϣϯͷࣗಈԽ
    • αʔόϦιʔεͷෆ଍Λճආ͢Δ
    • Φʔτεέʔϧର৅ͷαʔόΛ֦େ
    • ෛՙʹԠͯࣗ͡ಈతʹαʔόΛεέʔϧΞ΢τ
    • haproxy ΍ nginx ͳͲͷίϯϑΟά΋αʔόͷݱঢ়ʹ
    ߹Θͤͯࣗಈߋ৽(consul + consul-template)
    • ϩάͷٸ૿ʹΑΔσΟεΫͷṧഭͱͦͷରԠΛճආ
    • logrotate Λ࢖ͬͯσΟεΫʹϩάΛ࢒ͣ͞ɺ೔࣌Ͱ S3 ʹ͢
    ͙ʹόοΫΞοϓ͢ΔํࣜΛ࠾༻

    View full-size slide

  19. αʔό܈ͷΩϟύγςΟͷݟ௚͠

    View full-size slide

  20. ϓϩμΫτ։ൃνʔϜͱͷ࿈ܞ

    View full-size slide

  21. ϓϩμΫτ։ൃνʔϜͱͷ࿈ܞ
    • ύϑΥʔϚϯεͷϘτϧωοΫΛ෼ੳͯ͠վળ
    • μογϡϘʔυͷڞ༗
    • ίʔυϨϕϧͰϘτϧωοΫͷڞ༗ɺվળ

    View full-size slide

  22. ظؒதʹ࣮ࢪ͢Δ͖ΘͲ͍ࢪࡦͷαϙʔτ
    • ྫ͑͹ɺങऩͨ͠αΠτͷσʔλΛΫοΫύουͷσʔλϕʔεʹϝ
    ϯςφϯεແ͠ͰϚΠάϨʔγϣϯ͢Δ࣌
    • 2016೥͸͜ΕʹΑΓϥϚμϯظؒதʹ௕࣌ؒͷαʔϏεఀࢭ͕2
    ճൃੜ
    • ࠓ೥΋ϥϚμϯظؒʹσʔλϚΠάϨʔγϣϯͷ༧ఆ͕͋ͬͨ
    • σʔλҠߦํ๏ͷܭը΍ίʔυϨϏϡʔʹ΋ࢀՃ
    • αʔόϦιʔε΍εϩʔΫΤϦϩάͳͲͷ֤छ৘ใΛϞχλϦϯ
    ά͢ΔμογϡϘʔυͷఏڙ
    4FFBMTPIUUQTXXXZPVUVCFDPNXBUDI WQ&9;QV),Z+D

    View full-size slide

  23. 43&WTϥϚμϯͷ݁Ռ

    View full-size slide

  24. 43&WTϥϚμϯͷ݁Ռ
    େ͖ͳ໰୊΋ͳ͘ϥϚμϯظؒऴྃ!

    View full-size slide

  25. ॴײ
    • ϥϚμϯΛ࢝Ίͱͯ͠αʔϏεͷ Global ల։ͳΒͰ͸ͷ՝
    ୊΍໘ന͞ͱૺ۰͢Δ
    • େมͳ͜ͱ΋ଟ͍͕ɺֶͼ΋ଟ͍
    • ָ͠Έͳ͕Β৐Γӽ͍͑ͨ
    • XXX ʹ޲͚ͯΩϟύγςΟͷݟ௚͕͠ඞཁΈ͍ͨͳՔಇ͕
    ൃੜ͠ͳ͍Α͏ʹγεςϜΛߏஙɾվળ͍ͯ͘͜͠ͱ͕ٻ
    ΊΒΕ͍ͯΔ(౰ͨΓલ͕ͩ)

    View full-size slide

  26. ·ͱΊ
    • GlobalαʔϏεʹͱͬͯϥϚμϯ͸େࣄͳΠϕϯτ
    • ීஈͱൺֱͯ͠਺ഒͷن໛ͰϢʔβ਺͕૿͑Δ
    • SRE͸1ϲ݄ଓ͘ߴෛՙΛ৐Γ੾ΔͨΊͷࢪࡦΛଧͭ
    • ෆ଍͍ͯ͠ΔϦιʔε͸଍͢
    • ࣗಈԽ
    • ϓϩμΫτ։ൃνʔϜͱͷ࿈ܞ
    • ࠓޙ͸ಛʹରࡦΛ͠ͳ͍Ͱ৐Γ੾ΕΔΑ͏ʹ͢Δͷ͕ॏཁ

    View full-size slide