ΑΓUXʹ͍ۙSLIɾSLOͷӡ ༻ʹΑΔՄ༻ੑͷ࠶ઃܭ Ӭ໺ Ұഅ

ࣗݾ঺հ ● 2019/4ʙ λοϓϧʹiOSΤϯδχΞͱͯ͠ δϣΠϯ ● 2021೥ΑΓSREνʔϜʹϞόΠϧΤϯδχ Ξͱͯ͠ॴଐ ● ݱࡏ͸αʔϏεͷ৴པੑΛߴΊΔϛογϣ ϯʹऔΓ૊ΜͰ͍·͢ Ӭ໺ Ұഅ

λοϓϧͷαʔϏεʹ͍ͭͯ https://tapple.me/ 2014೥5݄ʹϦϦʔεͨ͠ࠃ಺࠷େن໛ͷϚονϯάΞϓϦͰ͢ɻ1,000छҎ্ͷσʔτϓϥϯ͔Β૬ख Λืूͯ͠ϚονϯάͰ͖Δʮ͓Ͱ͔͚ػೳʯΛఏڙ͠ɺ೥ऩɾ৬ۀͳͲͷ৚͚݅ͩͰͳ͘ɺझຯΛ͖ͬ ͔͚ʹࣗ෼Β͘͠աͤ͝Δύʔτφʔͱग़ձ͑Δͷ͕ಛ௃Ͱɺ20୅ͷஉঁΛத৺ʹࢧ࣋Λड͚͍ͯ·͢ɻ

● SREνʔϜʹϞ バ ΠϧΤϯ ジ χΞ(iOSɺAndroid֤1ਓ) が ઐଐ で ॴଐ͢Δܗ ● ΑΓϢʔβʔʹ͍ۙ෦෼ͰͷܭଌɺBFFͷਪਐͱ͍ͬͨόοΫΤϯυSRE͚ͩͰ͸खΛ৳͹ͮ͠Β ͍෦෼ʹ΋ΞδϦςΟߴ͘औΓ૊ΊΔΑ͏ʹ͢Δ໨త ● Πϯϑϥ΍όοΫΤϯυͷઐ໳ੑͷ֫ಘ΋ SREνʔϜͷMobileΤϯδχΞʹ͍ͭͯ

ࠓճͷൃද಺༰ SLI / SLOΛϞόΠϧΞϓϦέʔγϣϯଆͰܭଌ͞ΕΔࢦඪΛ΋ͱʹ࠶ઃܭ͠ɺΑΓUXʹ͍ۙࢦඪΛ໨ࢦ ͢औΓ૊Έͷ঺հ ● ৴པੑΛճ෮ͤ͞ΔͨΊͷΞΫγϣϯ΍ɺΠϯγσϯτ΁ͷରॲΛఆྔతʹߦ͑ΔΑ͏ʹ ● ৴པੑΛఆྔతʹ೺ѲͰ͖Δ͜ͱͰͷ҆৺ײ ● ࣄۀ൑அ΍༏ઌ౓ͷίϛϡχέʔγϣϯʹ͓͚Δڞ௨ݴޠʹ

ैདྷͷSLI/SLOͷӡ༻

όοΫΤϯυ ● APIαʔόʔ, BFFαʔόʔͦΕͧΕͷϨΠςϯγΛSLIͱͯ͠ར༻ ● ϞχλϦϯάɺΞϥʔτڞʹDataDogͰ؅ཧ ● ҧ൓࣌΍ɺΤϥʔόδΣοτΛஶ͘͠ফඅ͢Δ৔߹͸ଈ࣌ো֐ରԠ ैདྷͷSLI/SLOͷӡ༻ — όοΫΤϯυ — https://developers.cyberagent.co.jp/blog/archives/20354/

ΫϥΠΞϯτ ΫϥΠΞϯτଆͰ͸ಠࣗͷSLIΛӡ༻ ● Ϋϥογϡ཰ɺύʔεΤϥʔ਺ɺϑϨʔϜϨʔτ ैདྷͷSLI/SLOͷӡ༻ — ΫϥΠΞϯτ —

● Firebase Performance, CrashlyticsͷϩάΛར༻͠ɺϕϩγςΟΞϥʔτΛར༻ ● Data PortalͰμογϡϘʔυ؅ཧ ● ҧ൓࣌͸Өڹͷൣғɺػೳͷॏཁ౓ʹԠͯ͡ΞΫγϣϯΛܾఆ 
 ैདྷͷSLI/SLOͷӡ༻ — ΫϥΠΞϯτ — ΠϯύΫτ ΞΫγϣϯ High ࢪࡦΛͱΊͯͰ΋νʔϜશମͰՄೳͳݶΓϦιʔεΛͭ͗ࠐΈɺ ϗοτϑΟοΫε΋ݕ౼ Middle ୲౰ͷΤϯδχΞΛΞαΠϯ্ͨ͠Ͱɺ࣍ճϦϦʔεͰඞͣରԠɻ ඞཁͰ͋Ε͹ࢪࡦͷख΋ͱΊΔ Low ୲౰ͷΤϯδχΞΛΞαΠϯ্ͨ͠Ͱɺ ։ൃϦιʔε͕ۭ͖࣍ୈదٓରԠ͢Δ

SLO͕αʔϏεͷՄ༻ੑͷ൑அ࣠ʹͳ͍ͬͯͳ͍ 
 
 
 1. emergencyϨϕϧͷΞϥʔτ͕ൃՐ 2. ΤϯδχΞ͕֤ࣗӨڹൣғʢػೳɺϢʔβʔ਺ʣௐࠪ 3. ΞΫγϣϯܾఆʢ͓஌ΒͤɺTwitterࠂ஌ɺ޿ࠂఀࢭɺϝϯςΠϯʣ 
 
 
 1. emergencyϨϕϧͷΞϥʔτ͕ൃՐ 2. SLI/SLOμογϡϘʔυͰӨڹൣғ֬ೝ 3. ӨڹͷͰ͍ͯΔSLIɺΤϥʔόδΣοτͷফඅ͔Β࣍ͷΞΫγϣϯΛܾఆ ैདྷͷSLI/SLOͷӡ༻ — ՝୊ᶃ — ຊདྷͷཧ૝ ݱࡏͷӡ༻

όοΫΤϯυɺΫϥΠΞϯτʹดͨ͡ࢦඪͦΕͧΕͷΈͰ͸Մ༻ੑΛอূͰ͖ͳ͍ 
 
 αʔόʔͷSLO͕ຬͨ͞Ε͍ͯͯ΋ɺ ● ૬खͷϝοηʔδ͕ݟΕ͍ͯͳ͍ 
 → ΫϥΠΞϯτଆͰͷύʔεΤϥʔ͕ݪҼͰϨεϙϯεΛड৴Ͱ͖ͯͳ͍ ● ৽نొ࿥Ͱ͖ͳ͍ 
 → ֎෦SNSଆͰো֐ൃੜɺΦϯϘʔσΟϯάϑϩʔ͕ແݶϧʔϓ 
 
 
 ● ϝοηʔδը໘Λ։͍ͨϢʔβʔͷ○%͕ϝοηʔδͷӾཡʹ੒ޭ͍ͯ͠Δঢ়ଶ͔Θ͔Δ ● ৽نొ࿥Λ։࢝ͨ͠Ϣʔβʔͷ○%͕৽نొ࿥ʹ੒ޭ͍ͯ͠Δ͔Θ͔Δ ैདྷͷSLI/SLOͷӡ༻ — ՝୊ᶄ — ຊདྷͷཧ૝ ݱࡏͷӡ༻

໨ࢦ͢ཧ૝ঢ়ଶ ● Ϣʔβʔͷظ଴͢ΔUXΛΧόʔ͢Δࢦඪઃఆ 
 ● SLI / SLOͷҧ൓ʹΑΔϢʔβʔӨڹ೺ѲΛ༰қʹ 
 ● ϏδωεɺΫϥΠΞϯτɺαʔόʔؒͰαʔϏεঢ়ଶͷڞ௨ݴޠΛ࣋ͭ

ΑΓUXʹ͍ۙSLIɾSLOͷઃܭ

Ϣʔβʔͷظ଴͢ΔUXΛΧόʔ͢ΔSLIઃఆ ϝϯόʔ͔ΒʮαʔϏεͷՄ༻ੑʯͷٵ্͍͛ ● औΓ͍ͨࢦඪɺఆٛɺܭଌํ๏ɺѱԽͨ͠৔߹ʹଛͳΘΕΔUXΛώΞϦϯά ● ϑΥʔΧε͢ΔࢦඪɺίΞͳମݧΛ࢈ΜͰ͍ΔUXΛܾఆ͢Δ ϚΠΫϩαʔϏε౳ͷυϝΠϯϞσϧͱ߹கͤ͞Δ

ϓϥοτϑΥʔϜͷఏڙ͢Δࢦඪͷ࠾༻ Apple, Google͕σΟϕϩούʔʹఏڙ͢ΔSLI ● ϓϥοτϑΥʔϜଆͷߟ͑Δྑ͍ΞϓϦͷఆٛ ● ΞϓϦετΞ΍OSϨϕϧͰӨڹ͕͋Δ͜ͱ΋ ○ ASOʢApp Store Optimizationʣ ○ ༏ઌͯ͠ΞϯΠϯετʔϧΛଅ͞ΕΔ ○ ΞϓϦͷόοΫάϥ΢ϯυͰͷੜଘ࣌ؒ ● λοϓϧͰ͸ԼهͷࢦඪΛSLIͱͯ͠࠾༻ ○ ΞϓϦαΠζ ○ શମΫϥογϡ཰ʢANRʣ ○ ىಈ࣌ؒ https://developers.cyberagent.co.jp/blog/archives/20354/

SLIҧ൓Λ࠶ݱͨ͠ΞϓϦΛ࢖ͬͨSLOܾΊ ● աڈͷΠϯγσϯτ࣌ͷࢦඪ΍ɺࣄલώΞϦϯά͔Βɺ͖͍͠஋ʹ͋ͨΓΛ͚ͭΔ 
 ● σόοάػೳͰৄࡉͳ͖͍͠஋Λௐ੔ՄೳͳΞϓϦΛ༻ҙ͠ɺ৮Γͳ͕Βٞ࿦

࠶ఆٛͨ͠SLI όοΫΤϯυʢܧଓʣ ● ϨΠςϯγʔ 
 ΫϥΠΞϯτʢʹϓϥοτϑΥʔϜࢦඪʣ ● ىಈ࣌ؒɺશମΫϥογϡ཰ɺΞϓϦαΠζ 
 ίΞػೳʢʹػೳ͝ͱͷՄ༻ੑʣ ● ϩάΠϯɺ৽نొ࿥ɺΧʔυϑϦοΫɺ 
 ϝοηʔδɺ͓Ͱ͔͚ɺWish 
 Մ༻ੑΛ΋ͱʹͨ͠SLIΛઃఆ 
 ڞ௨ Ϋϥογϡ / ANR ػೳ͝ͱͷΩʔϝτϦΫε ϝοηʔδ: ಡΈࠐΈ଎౓ ΧʔυϑϦοΫ: ࠶ϑϦοΫՄೳʹͳΔ·Ͱͷ࣌ؒ 
 ɾɾɾ

ΫϥΠΞϯτଆͰͷSLI / SLOͷܭଌ

ΫϥΠΞϯτଆͰͷSLI / SLOͷܭଌ 1. ·ͣ͸Firebase (+BigQuery) ʹ׬݁ͯ͠ϛχϚϜʹՔಇ ● ػೳతʹे෼ͳ͜ͱΛݕূ 2. Monitoring, DashboardΛDataDogʹҠߦʢ͜Ε͔Βʣ ● Ξϥʔτͷࣗ༝౓Λ্͛Δ ● ଞͷϩάͰར༻ͯ͠ΔαʔϏεͱ߹ΘͤΔ

ΫϥΠΞϯτଆͰͷϞχλϦϯάπʔϧͷݕ౼ ͻͱ·ͣྉۚ໘Λߟྀ͠FirebaseΛར༻͠ɺʮӡ༻Λ։࢝ → ՝୊͕͋Ε͹ϒϥογϡΞοϓʯͷํ਑ʹ 
 1. Firebase Crashlytics / Performance + BigQuery ● Firebaseࣗମ͸ແྉͷͨΊBigQueryͷQuery, Active StorageͷྉۚͷΈ ● Crashlytics, Performance͸طଘͰ΋ར༻͍ͯ͠ΔͨΊಋೖίετ͕௿͍ ● BigQueryʹ஝ੵ͢Δ͜ͱͰൺֱతࣗ༝౓͸ߴ͘ར༻Ͱ͖Δ 2. DataDog RUM ● طଘͷϞχλϦϯάπʔϧͱͯ͠DataDogΛར༻͍ͯ͠ΔͨΊπʔϧΛ౷ҰͰ͖Δ ● ηογϣϯ୯Ґͷ՝ۚମܥͷͨΊɺࠓճ͸ྉۚ໘Ͱஅ೦ 3. New Relic One ● όοΫΤϯυͰDataDogΛར༻͍ͯ͠Δ͜ͱɺྉۚ໘Λഎܠʹஅ೦

FirebaseͰߏ੒͢ΔϛχϚϜͳܭଌϑϩʔ Firebase ( Crashlytics + Performance ) ͷΈͰϛχϚϜͳϑϩʔ͸࣮ݱՄೳ ● BigQueryʹExportʢϦΞϧλΠϜ΋Մೳʣ ● BigQuery্ʹμογϡϘʔυදࣔ༻ͷதؒςʔϒϧ࡞੒ ● Data PortalͰμογϡϘʔυදࣔ )LUHEDVH 3HUIRUPDQFH )LUHEDVH &UDVKO\WLFV 8VHU &UDVK .H\0HWULFV *RRJOH'DWD3RUWDO $XWR([SRUW %LJ4XHU\ ,QWHUPHGLDWHWDEOH %LJ4XHU\

Ξϥʔτͷઃఆͷࣗ༝౓ Firebase Performance ● ࡞੒ͨ͠ࢦඪ͝ͱʹΞϥʔτͷઃఆ͕Մೳ ● ͖͍͠஋Λ50ʙ95ύʔηϯλΠϧͷࢦఆ͕Մೳ Firebase Crashlytics ● ௚ۙ1࣌ؒͷηογϣϯʹର͢ΔΫϥογϡ཰ ● ͖͍͠஋Λ0.1%ʙ1.0%ͰίϯτϩʔϧՄೳ

Ϋϥογϡϩάͷ෼ྨ https://firebase.google.com/docs/crashlytics/customize-crash-reports import UIKit import FirebaseCrashlytics final class LoginViewController: UIViewController { // ը໘Λදࣔ͢ΔλΠϛϯάͰCrashlyticsʹΧελϜΩʔΛઃఆ override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) Crashlytics.crashlytics().setCustomValue("login", forKey: "domain") } iOSΞϓϦͰͷϩάΠϯը໘ʹର͢Δ࣮૷ྫ ֤ػೳͷը໘දࣔ࣌ʹCrashlyticsʹΧελϜΩʔΛઃఆ͢Δ ● ΧελϜΩʔΛ΋ͱʹBigQuery (Data Portal) ্ͰػೳυϝΠϯ͝ͱͷΫϥογϡΛूܭ ● ػೳ͝ͱͷΞϥʔτ͸ઃఆͰ͖ͳ͍ͨΊɺVelocity Alert → μογϡϘʔυͰػೳӨڹ֬ೝ

ࠓޙʣϞχλϦϯάΛDataDogʹ౷Ұ ҎԼͷ؍఺͔ΒDataDogʹू໿͍ͯ͘͠ ● ϞχλϦϯάͷू໿ ● Ϋϥογϡͷػೳ͝ͱͷΞϥʔτ )LUHEDVH 3HUIRUPDQFH )LUHEDVH &UDVKO\WLFV $XWR([SRUW %LJ4XHU\ 4XHU\ SXVK IRUZRUGWRGDWDGRJ &ORXG)XQFWLRQV 'DWD'RJ &ORXG 3XE6XE WULJJHU

ӡ༻ʹ͍ͭͯ

ӡ༻ͷมԽ Πϯγσϯτൃੜ࣌ ● PMɺόοΫΤϯυɺΫϥΠΞϯτϝϯόʔશһ͕ϢʔβʔӨڹΛ൑அ͢ΔҰ࣌৘ใͱͯ͠׆༻ ○ ΤϯδχΞͷௐࠪΛ଴ͨͣͯ͠ωΫετΞΫγϣϯΛಈ͖ग़ͤΔΑ͏ʹ 
 ྫʣશϢʔβʔ΁ͷ͓͠ΒͤɺTwitterࠂ஌ɺ޿ࠂͷఀࢭ ● ِཅੑͷ௿Լ ○ ಛఆͷ৚݅ԼʢΩϟϯϖʔϯதͷΈ౳ʣʹൃੜ͢ΔύʔεΤϥʔΛ༏ઌ౓ߴ͘ΞϥʔτΛ͋͛ ΒΕΔΑ͏ʹ

ӡ༻ͷมԽ ϦϦʔεޙͷϝτϦΫε֬ೝ࣌ʹόοΫΤϯυɺΫϥΠΞϯτͷڞ௨ݴޠʹ 
 ● όοΫΤϯυɺΫϥΠΞϯτͦΕͧΕ͕׬શʹ෼͔Εͯ֬ೝ ● ֬ೝՄೳͳϝτϦΫεɺμογϡϘʔυΛҰ௨Γ໢ཏతʹ֬ೝ ● όοΫΤϯυɺΫϥΠΞϯτڞ௨ͷSLI / SLO μογϡϘʔυΛ֬ೝ ○ ௐࠪͷ৔߹͸SLIΛڞ௨ݴޠͱͯ͠ίϛϡχέʔγϣϯ ● ҧ൓͕͋ͬͨ৔߹ʹؔ࿈ͷϝτϦΫεɺμογϡϘʔυΛ֬ೝ ○ SLI / SLO΋ಉ࣌ʹߦ͏ ݱࡏͷӡ༻ ैདྷͷӡ༻

ࠓޙͷల๬ ● ·ͩ·ͩӡ༻࢝ΊͨͯͳͷͰɺࠓޙ΋஌ݟΛஷΊ͍ͯ͘ ● ࢦඪͦͷ΋ͷͷܧଓతͳϒϥογϡΞοϓ ○ ࢦඪͷ͖͍͠஋ɺλʔήοτʢ90% → 50%ʣ ○ ΤϥʔόδΣοτ ○ ػೳͷΩʔϝτϦΫεͷର৅ ● DataDogͰҰؾ௨؏ͯ͠ϞχλϦϯάɺμογϡϘʔυΛ؅ཧͰ͖ΔΑ͏ʹ͢Δ ● ૊৫಺ͰͷSLI / SLOͷਁಁ ○ ݱঢ়·ͩ·ͩΤϯδχΞҎ֎ʹ͸ਁಁͰ͖ͯͳ͍

͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠