Upgrade to Pro — share decks privately, control downloads, hide ads and more …

よりUXに近いSLI・SLOの運用による可用性の再設計

nade
May 15, 2022

 よりUXに近いSLI・SLOの運用による可用性の再設計

nade

May 15, 2022
Tweet

More Decks by nade

Other Decks in Programming

Transcript

  1. ΑΓUXʹ͍ۙSLIɾSLOͷӡ
    ༻ʹΑΔՄ༻ੑͷ࠶ઃܭ
    Ӭ໺ Ұഅ

    View Slide

  2. ࣗݾ঺հ
    ● 2019/4ʙ λοϓϧʹiOSΤϯδχΞͱͯ͠
    δϣΠϯ


    ● 2021೥ΑΓSREνʔϜʹϞόΠϧΤϯδχ
    Ξͱͯ͠ॴଐ


    ● ݱࡏ͸αʔϏεͷ৴པੑΛߴΊΔϛογϣ
    ϯʹऔΓ૊ΜͰ͍·͢
    Ӭ໺ Ұഅ

    View Slide

  3. λοϓϧͷαʔϏεʹ͍ͭͯ
    https://tapple.me/


    2014೥5݄ʹϦϦʔεͨ͠ࠃ಺࠷େن໛ͷϚονϯάΞϓϦͰ͢ɻ1,000छҎ্ͷσʔτϓϥϯ͔Β૬ख
    Λืूͯ͠ϚονϯάͰ͖Δʮ͓Ͱ͔͚ػೳʯΛఏڙ͠ɺ೥ऩɾ৬ۀͳͲͷ৚͚݅ͩͰͳ͘ɺझຯΛ͖ͬ
    ͔͚ʹࣗ෼Β͘͠աͤ͝Δύʔτφʔͱग़ձ͑Δͷ͕ಛ௃Ͱɺ20୅ͷஉঁΛத৺ʹࢧ࣋Λड͚͍ͯ·͢ɻ

    View Slide

  4. ● SREνʔϜʹϞ
    バ
    ΠϧΤϯ
    ジ
    χΞ(iOSɺAndroid֤1ਓ)
    が
    ઐଐ
    で
    ॴଐ͢Δܗ


    ● ΑΓϢʔβʔʹ͍ۙ෦෼ͰͷܭଌɺBFFͷਪਐͱ͍ͬͨόοΫΤϯυSRE͚ͩͰ͸खΛ৳͹ͮ͠Β
    ͍෦෼ʹ΋ΞδϦςΟߴ͘औΓ૊ΊΔΑ͏ʹ͢Δ໨త


    ● Πϯϑϥ΍όοΫΤϯυͷઐ໳ੑͷ֫ಘ΋
    SREνʔϜͷMobileΤϯδχΞʹ͍ͭͯ

    View Slide

  5. ࠓճͷൃද಺༰
    SLI / SLOΛϞόΠϧΞϓϦέʔγϣϯଆͰܭଌ͞ΕΔࢦඪΛ΋ͱʹ࠶ઃܭ͠ɺΑΓUXʹ͍ۙࢦඪΛ໨ࢦ
    ͢औΓ૊Έͷ঺հ


    ● ৴པੑΛճ෮ͤ͞ΔͨΊͷΞΫγϣϯ΍ɺΠϯγσϯτ΁ͷରॲΛఆྔతʹߦ͑ΔΑ͏ʹ


    ● ৴པੑΛఆྔతʹ೺ѲͰ͖Δ͜ͱͰͷ҆৺ײ


    ● ࣄۀ൑அ΍༏ઌ౓ͷίϛϡχέʔγϣϯʹ͓͚Δڞ௨ݴޠʹ

    View Slide

  6. ैདྷͷSLI/SLOͷӡ༻

    View Slide

  7. όοΫΤϯυ


    ● APIαʔόʔ, BFFαʔόʔͦΕͧΕͷϨΠςϯγΛSLIͱͯ͠ར༻


    ● ϞχλϦϯάɺΞϥʔτڞʹDataDogͰ؅ཧ


    ● ҧ൓࣌΍ɺΤϥʔόδΣοτΛஶ͘͠ফඅ͢Δ৔߹͸ଈ࣌ো֐ରԠ
    ैདྷͷSLI/SLOͷӡ༻ — όοΫΤϯυ —
    https://developers.cyberagent.co.jp/blog/archives/20354/

    View Slide

  8. ΫϥΠΞϯτ


    ΫϥΠΞϯτଆͰ͸ಠࣗͷSLIΛӡ༻


    ● Ϋϥογϡ཰ɺύʔεΤϥʔ਺ɺϑϨʔϜϨʔτ
    ैདྷͷSLI/SLOͷӡ༻ — ΫϥΠΞϯτ —

    View Slide

  9. ● Firebase Performance, CrashlyticsͷϩάΛར༻͠ɺϕϩγςΟΞϥʔτΛར༻


    ● Data PortalͰμογϡϘʔυ؅ཧ


    ● ҧ൓࣌͸Өڹͷൣғɺػೳͷॏཁ౓ʹԠͯ͡ΞΫγϣϯΛܾఆ

    ैདྷͷSLI/SLOͷӡ༻ — ΫϥΠΞϯτ —
    ΠϯύΫτ ΞΫγϣϯ
    High
    ࢪࡦΛͱΊͯͰ΋νʔϜશମͰՄೳͳݶΓϦιʔεΛͭ͗ࠐΈɺ


    ϗοτϑΟοΫε΋ݕ౼
    Middle
    ୲౰ͷΤϯδχΞΛΞαΠϯ্ͨ͠Ͱɺ࣍ճϦϦʔεͰඞͣରԠɻ


    ඞཁͰ͋Ε͹ࢪࡦͷख΋ͱΊΔ
    Low
    ୲౰ͷΤϯδχΞΛΞαΠϯ্ͨ͠Ͱɺ


    ։ൃϦιʔε͕ۭ͖࣍ୈదٓରԠ͢Δ

    View Slide

  10. SLO͕αʔϏεͷՄ༻ੑͷ൑அ࣠ʹͳ͍ͬͯͳ͍



    1. emergencyϨϕϧͷΞϥʔτ͕ൃՐ


    2. ΤϯδχΞ͕֤ࣗӨڹൣғʢػೳɺϢʔβʔ਺ʣௐࠪ


    3. ΞΫγϣϯܾఆʢ͓஌ΒͤɺTwitterࠂ஌ɺ޿ࠂఀࢭɺϝϯςΠϯʣ



    1. emergencyϨϕϧͷΞϥʔτ͕ൃՐ


    2. SLI/SLOμογϡϘʔυͰӨڹൣғ֬ೝ


    3. ӨڹͷͰ͍ͯΔSLIɺΤϥʔόδΣοτͷফඅ͔Β࣍ͷΞΫγϣϯΛܾఆ
    ैདྷͷSLI/SLOͷӡ༻ — ՝୊ᶃ —
    ຊདྷͷཧ૝
    ݱࡏͷӡ༻

    View Slide

  11. όοΫΤϯυɺΫϥΠΞϯτʹดͨ͡ࢦඪͦΕͧΕͷΈͰ͸Մ༻ੑΛอূͰ͖ͳ͍




    αʔόʔͷSLO͕ຬͨ͞Ε͍ͯͯ΋ɺ


    ● ૬खͷϝοηʔδ͕ݟΕ͍ͯͳ͍

    → ΫϥΠΞϯτଆͰͷύʔεΤϥʔ͕ݪҼͰϨεϙϯεΛड৴Ͱ͖ͯͳ͍


    ● ৽نొ࿥Ͱ͖ͳ͍

    → ֎෦SNSଆͰো֐ൃੜɺΦϯϘʔσΟϯάϑϩʔ͕ແݶϧʔϓ



    ● ϝοηʔδը໘Λ։͍ͨϢʔβʔͷ○%͕ϝοηʔδͷӾཡʹ੒ޭ͍ͯ͠Δঢ়ଶ͔Θ͔Δ


    ● ৽نొ࿥Λ։࢝ͨ͠Ϣʔβʔͷ○%͕৽نొ࿥ʹ੒ޭ͍ͯ͠Δ͔Θ͔Δ
    ैདྷͷSLI/SLOͷӡ༻ — ՝୊ᶄ —
    ຊདྷͷཧ૝
    ݱࡏͷӡ༻

    View Slide

  12. ໨ࢦ͢ཧ૝ঢ়ଶ
    ● Ϣʔβʔͷظ଴͢ΔUXΛΧόʔ͢Δࢦඪઃఆ

    ● SLI / SLOͷҧ൓ʹΑΔϢʔβʔӨڹ೺ѲΛ༰қʹ

    ● ϏδωεɺΫϥΠΞϯτɺαʔόʔؒͰαʔϏεঢ়ଶͷڞ௨ݴޠΛ࣋ͭ

    View Slide

  13. ΑΓUXʹ͍ۙSLIɾSLOͷઃܭ

    View Slide

  14. Ϣʔβʔͷظ଴͢ΔUXΛΧόʔ͢ΔSLIઃఆ
    ϝϯόʔ͔ΒʮαʔϏεͷՄ༻ੑʯͷٵ্͍͛


    ● औΓ͍ͨࢦඪɺఆٛɺܭଌํ๏ɺѱԽͨ͠৔߹ʹଛͳΘΕΔUXΛώΞϦϯά


    ● ϑΥʔΧε͢ΔࢦඪɺίΞͳମݧΛ࢈ΜͰ͍ΔUXΛܾఆ͢Δ


    ϚΠΫϩαʔϏε౳ͷυϝΠϯϞσϧͱ߹கͤ͞Δ

    View Slide

  15. ϓϥοτϑΥʔϜͷఏڙ͢Δࢦඪͷ࠾༻
    Apple, Google͕σΟϕϩούʔʹఏڙ͢ΔSLI


    ● ϓϥοτϑΥʔϜଆͷߟ͑Δྑ͍ΞϓϦͷఆٛ


    ● ΞϓϦετΞ΍OSϨϕϧͰӨڹ͕͋Δ͜ͱ΋


    ○ ASOʢApp Store Optimizationʣ


    ○ ༏ઌͯ͠ΞϯΠϯετʔϧΛଅ͞ΕΔ


    ○ ΞϓϦͷόοΫάϥ΢ϯυͰͷੜଘ࣌ؒ


    ● λοϓϧͰ͸ԼهͷࢦඪΛSLIͱͯ͠࠾༻


    ○ ΞϓϦαΠζ


    ○ શମΫϥογϡ཰ʢANRʣ


    ○ ىಈ࣌ؒ
    https://developers.cyberagent.co.jp/blog/archives/20354/

    View Slide

  16. SLIҧ൓Λ࠶ݱͨ͠ΞϓϦΛ࢖ͬͨSLOܾΊ
    ● աڈͷΠϯγσϯτ࣌ͷࢦඪ΍ɺࣄલώΞϦϯά͔Βɺ͖͍͠஋ʹ͋ͨΓΛ͚ͭΔ

    ● σόοάػೳͰৄࡉͳ͖͍͠஋Λௐ੔ՄೳͳΞϓϦΛ༻ҙ͠ɺ৮Γͳ͕Βٞ࿦

    View Slide

  17. ࠶ఆٛͨ͠SLI
    όοΫΤϯυʢܧଓʣ


    ● ϨΠςϯγʔ

    ΫϥΠΞϯτʢʹϓϥοτϑΥʔϜࢦඪʣ


    ● ىಈ࣌ؒɺશମΫϥογϡ཰ɺΞϓϦαΠζ

    ίΞػೳʢʹػೳ͝ͱͷՄ༻ੑʣ


    ● ϩάΠϯɺ৽نొ࿥ɺΧʔυϑϦοΫɺ

    ϝοηʔδɺ͓Ͱ͔͚ɺWish

    Մ༻ੑΛ΋ͱʹͨ͠SLIΛઃఆ



    ڞ௨


    Ϋϥογϡ / ANR


    ػೳ͝ͱͷΩʔϝτϦΫε


    ϝοηʔδ: ಡΈࠐΈ଎౓


    ΧʔυϑϦοΫ: ࠶ϑϦοΫՄೳʹͳΔ·Ͱͷ࣌ؒ

    ɾɾɾ

    View Slide

  18. ΫϥΠΞϯτଆͰͷSLI / SLOͷܭଌ

    View Slide

  19. ΫϥΠΞϯτଆͰͷSLI / SLOͷܭଌ
    1. ·ͣ͸Firebase (+BigQuery) ʹ׬݁ͯ͠ϛχϚϜʹՔಇ


    ● ػೳతʹे෼ͳ͜ͱΛݕূ


    2. Monitoring, DashboardΛDataDogʹҠߦʢ͜Ε͔Βʣ


    ● Ξϥʔτͷࣗ༝౓Λ্͛Δ


    ● ଞͷϩάͰར༻ͯ͠ΔαʔϏεͱ߹ΘͤΔ

    View Slide

  20. ΫϥΠΞϯτଆͰͷϞχλϦϯάπʔϧͷݕ౼
    ͻͱ·ͣྉۚ໘Λߟྀ͠FirebaseΛར༻͠ɺʮӡ༻Λ։࢝ → ՝୊͕͋Ε͹ϒϥογϡΞοϓʯͷํ਑ʹ

    1. Firebase Crashlytics / Performance + BigQuery


    ● Firebaseࣗମ͸ແྉͷͨΊBigQueryͷQuery, Active StorageͷྉۚͷΈ


    ● Crashlytics, Performance͸طଘͰ΋ར༻͍ͯ͠ΔͨΊಋೖίετ͕௿͍


    ● BigQueryʹ஝ੵ͢Δ͜ͱͰൺֱతࣗ༝౓͸ߴ͘ར༻Ͱ͖Δ


    2. DataDog RUM


    ● طଘͷϞχλϦϯάπʔϧͱͯ͠DataDogΛར༻͍ͯ͠ΔͨΊπʔϧΛ౷ҰͰ͖Δ


    ● ηογϣϯ୯Ґͷ՝ۚମܥͷͨΊɺࠓճ͸ྉۚ໘Ͱஅ೦


    3. New Relic One


    ● όοΫΤϯυͰDataDogΛར༻͍ͯ͠Δ͜ͱɺྉۚ໘Λഎܠʹஅ೦

    View Slide

  21. FirebaseͰߏ੒͢ΔϛχϚϜͳܭଌϑϩʔ
    Firebase ( Crashlytics + Performance ) ͷΈͰϛχϚϜͳϑϩʔ͸࣮ݱՄೳ


    ● BigQueryʹExportʢϦΞϧλΠϜ΋Մೳʣ


    ● BigQuery্ʹμογϡϘʔυදࣔ༻ͷதؒςʔϒϧ࡞੒


    ● Data PortalͰμογϡϘʔυදࣔ
    )LUHEDVH
    3HUIRUPDQFH
    )LUHEDVH
    &UDVKO\WLFV

    8VHU
    &UDVK
    .H\0HWULFV
    *RRJOH'DWD3RUWDO
    $XWR([SRUW
    %LJ4XHU\
    ,QWHUPHGLDWHWDEOH
    %LJ4XHU\

    View Slide

  22. Ξϥʔτͷઃఆͷࣗ༝౓
    Firebase Performance


    ● ࡞੒ͨ͠ࢦඪ͝ͱʹΞϥʔτͷઃఆ͕Մೳ


    ● ͖͍͠஋Λ50ʙ95ύʔηϯλΠϧͷࢦఆ͕Մೳ


    Firebase Crashlytics


    ● ௚ۙ1࣌ؒͷηογϣϯʹର͢ΔΫϥογϡ཰


    ● ͖͍͠஋Λ0.1%ʙ1.0%ͰίϯτϩʔϧՄೳ

    View Slide

  23. Ϋϥογϡϩάͷ෼ྨ
    https://firebase.google.com/docs/crashlytics/customize-crash-reports
    import UIKit


    import FirebaseCrashlytics


    final class LoginViewController: UIViewController {




    // ը໘Λදࣔ͢ΔλΠϛϯάͰCrashlyticsʹΧελϜΩʔΛઃఆ


    override func viewWillAppear(_ animated: Bool) {


    super.viewWillAppear(animated)


    Crashlytics.crashlytics().setCustomValue("login", forKey: "domain")


    }


    iOSΞϓϦͰͷϩάΠϯը໘ʹର͢Δ࣮૷ྫ
    ֤ػೳͷը໘දࣔ࣌ʹCrashlyticsʹΧελϜΩʔΛઃఆ͢Δ


    ● ΧελϜΩʔΛ΋ͱʹBigQuery (Data Portal) ্ͰػೳυϝΠϯ͝ͱͷΫϥογϡΛूܭ


    ● ػೳ͝ͱͷΞϥʔτ͸ઃఆͰ͖ͳ͍ͨΊɺVelocity Alert → μογϡϘʔυͰػೳӨڹ֬ೝ

    View Slide

  24. ࠓޙʣϞχλϦϯάΛDataDogʹ౷Ұ
    ҎԼͷ؍఺͔ΒDataDogʹू໿͍ͯ͘͠


    ● ϞχλϦϯάͷू໿


    ● Ϋϥογϡͷػೳ͝ͱͷΞϥʔτ
    )LUHEDVH
    3HUIRUPDQFH
    )LUHEDVH
    &UDVKO\WLFV
    $XWR([SRUW
    %LJ4XHU\ 4XHU\ SXVK
    IRUZRUGWRGDWDGRJ
    &ORXG)XQFWLRQV
    'DWD'RJ
    &ORXG
    3XE6XE
    WULJJHU

    View Slide

  25. ӡ༻ʹ͍ͭͯ

    View Slide

  26. ӡ༻ͷมԽ
    Πϯγσϯτൃੜ࣌


    ● PMɺόοΫΤϯυɺΫϥΠΞϯτϝϯόʔશһ͕ϢʔβʔӨڹΛ൑அ͢ΔҰ࣌৘ใͱͯ͠׆༻


    ○ ΤϯδχΞͷௐࠪΛ଴ͨͣͯ͠ωΫετΞΫγϣϯΛಈ͖ग़ͤΔΑ͏ʹ

    ྫʣશϢʔβʔ΁ͷ͓͠ΒͤɺTwitterࠂ஌ɺ޿ࠂͷఀࢭ


    ● ِཅੑͷ௿Լ


    ○ ಛఆͷ৚݅ԼʢΩϟϯϖʔϯதͷΈ౳ʣʹൃੜ͢ΔύʔεΤϥʔΛ༏ઌ౓ߴ͘ΞϥʔτΛ͋͛
    ΒΕΔΑ͏ʹ

    View Slide

  27. ӡ༻ͷมԽ
    ϦϦʔεޙͷϝτϦΫε֬ೝ࣌ʹόοΫΤϯυɺΫϥΠΞϯτͷڞ௨ݴޠʹ



    ● όοΫΤϯυɺΫϥΠΞϯτͦΕͧΕ͕׬શʹ෼͔Εͯ֬ೝ


    ● ֬ೝՄೳͳϝτϦΫεɺμογϡϘʔυΛҰ௨Γ໢ཏతʹ֬ೝ


    ● όοΫΤϯυɺΫϥΠΞϯτڞ௨ͷSLI / SLO μογϡϘʔυΛ֬ೝ


    ○ ௐࠪͷ৔߹͸SLIΛڞ௨ݴޠͱͯ͠ίϛϡχέʔγϣϯ


    ● ҧ൓͕͋ͬͨ৔߹ʹؔ࿈ͷϝτϦΫεɺμογϡϘʔυΛ֬ೝ


    ○ SLI / SLO΋ಉ࣌ʹߦ͏
    ݱࡏͷӡ༻
    ैདྷͷӡ༻

    View Slide

  28. ࠓޙͷల๬
    ● ·ͩ·ͩӡ༻࢝ΊͨͯͳͷͰɺࠓޙ΋஌ݟΛஷΊ͍ͯ͘


    ● ࢦඪͦͷ΋ͷͷܧଓతͳϒϥογϡΞοϓ


    ○ ࢦඪͷ͖͍͠஋ɺλʔήοτʢ90% → 50%ʣ


    ○ ΤϥʔόδΣοτ


    ○ ػೳͷΩʔϝτϦΫεͷର৅


    ● DataDogͰҰؾ௨؏ͯ͠ϞχλϦϯάɺμογϡϘʔυΛ؅ཧͰ͖ΔΑ͏ʹ͢Δ


    ● ૊৫಺ͰͷSLI / SLOͷਁಁ


    ○ ݱঢ়·ͩ·ͩΤϯδχΞҎ֎ʹ͸ਁಁͰ͖ͯͳ͍

    View Slide

  29. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠

    View Slide