Share
Embed
分散アプリケーションの異常の原因を即時に診断するための手法の構想 / Causality Tracing in Distributed Applications
Yuuki Tsubouchi (yuuk1)
May 07, 2020
Transcript
25 slides
-
2 1. എܠͱత 2. ؔ࿈ݚڀ 3. ఏҊख๏ 4. ࣮ݧ༧ఆ 5.
·ͱΊͱࠓޙͷ༧ఆ ࣍ -
6 ɾϦΫΤετ୯ҐͷτϨʔγϯάʹΑΔࠜຊݪҼੳ[1] ɾΞϓϦέʔγϣϯʹܭଌίʔυΛՃ͠ͳ͚ΕͳΒͳ͍ ɾߏཁૉؒͷґଘάϥϑͱϝτϦοΫΛར༻ͨࠜ͠ຊݪҼੳ[2][3] ɾґଘؔͷมԽϫʔΫϩʔυͷมԽ͕͋Δͱɺґଘؔͷநग़ॲ ཧΛΓ͞ͳ͚ΕͳΒͳ͍ => ଈ࣌ੑΛຬͨ͞ͳ͍ ɾ·ͨɺ֤ߏཁૉ͝ͱʹαʔϏεϨϕϧΛද͢ΔࢦඪʢSLOʣ Λઃܭ͠ɺࢹ͢ΔͨΊͷख͕ؒ͋Δ
ઌߦݚڀͷ՝ [1]: H. Jayathilaka, C. Krintz and R. Wolski, Performance monitoring and root cause analysis for cloud-hosted web applications, WWW, pp. 469–478, 2017. [2]: J. Thalheim, A. Rodrigues, I. Akkus, and others, Sieve: Actionable Insights from Monitored Metrics in Distributed Systems, ACM/IFIP/USENIX Middleware, pp.14-27 2017. [3]: J. Lin, C. Pengfei, and Z. Zibin, "Microscope: Pinpoint performance issues with causal graphs in micro-service environments." ICSO, pp.3-20, 2018. -
7 1. ϊΠζͰ͋Δ͜ͱͱߏཁૉ୯ҐͰࢦඪΛઃܭ͢Δखؒͷݮ ↪ αʔϏεΛද͢ΔࢦඪͷΈΛࢹ (Ԡ࣌ؒԠΤϥʔͳͲ) ࢹΞϥʔτΛܖػʹTCPଓґଘάϥϑͱ֤ϊʔυ্ͷϝτϦο ΫΛ୳ࡧ දࢦඪͱ࣌ܥྻతಛ͕૬ؔ͢ΔͷΛݪҼͷީิͱ͢Δ 2.
ଈ࣌ੑ ֤ϊʔυ্ͷશϝτϦοΫΛੳ͢ΔͱͳΔͱॲཧ͕͘ͳΔ ↪ ϝτϦοΫऔಘॲཧΛ֤ϊʔυʹࢄอ࣋ɾ͍߹ΘͤʹΑΓߴ Խ ↪ ࣌ܥྻͷ૬ؔੳॲཧΛGPGPUʹΑΓߴԽ ↪ աڈͷॲཧ݁ՌΛ࠶ར༻͠ɺߴఆ ݚڀͷత -
10 ɾΞϓϦέʔγϣϯͷ֤ϦΫΤετʹࣝผࢠΛׂΓৼΓɼޙଓͷϦ ΫΤετʹຒΊࠐΜ্ͩͰɼޙଓͷϓϩηεൖͤ͞Δ ɾࣝผࢠΛཔΓʹɼҙͷߏཁૉؒͷԆΛܭଌՄೳ ɾར: ΞϓϦέʔγϣϯ෦ͷৄࡉͳ͕Մೳ ɾ՝: ܭଌͷͨΊʹɺΞϓϦέʔγϣϯίʔυʹมߋ͕ඞཁ B. H.
Sigelman, et al., Dapper, a Large-Scale Distributed Systems Tracing Infrastructure, Technical report, Google 2010. ؔ࿈ݚڀᶃ: ϦΫΤετ୯ҐͷτϨʔγϯά -
11 ؔ࿈ݚڀᶄ: Sieve B. H. Sigelman, et al., Dapper, a
Large-Scale Distributed Systems Tracing Infrastructure, Technical report, Google 2010. ɾطଘͷࢹج൫͔ΒΞΫγϣϯՄೳͳώϯτΛಘΔγεςϜΛఏҊ ɾ࣌ܥྻΫϥελੳʹΑΓίϯϙʔωϯτؒͷґଘؔΛࣝผ͢Δ ɾ՝: ߏมߋϫʔΫϩʔυͷมԽʹै͢ΔͨΊʹɺੳε ςοϓΛ࠷ॳ͔ΒΓ͢ඞཁ͕͋Δ [2]: J. Thalheim, A. Rodrigues, I. Akkus, and others, Sieve: Actionable Insights from Monitored Metrics in Distributed Systems, ACM/IFIP/USENIX Middleware, pp.14-27 2017. -
12 ؔ࿈ݚڀᶅ: Microscope ɾϚΠΫϩαʔϏεʹܭଌίʔυΛՃ͢Δ͜ͱͳ͘ɺཁૉؒͷҼՌ ؔάϥϑΛޮతʹߏங͠ɺҟৗͷݪҼΛϦΞϧλΠϜͰਪଌ ɾ՝1: ֤ߏཁૉͷදϝτϦοΫͷมಈΛΈͯߜΓࠐΉ͕ɺͦΕ Ҏ֎ͷݪҼͱͳΔϝτϦοΫީิΛग़ྗ͠ͳ͍ ɾ՝2: ߏཁૉ͝ͱʹදϝτϦοΫͷઃఆͷͨΊʹΞϓϦέʔ
γϣϯ͕ࣝඞཁ [3]: J. Lin, C. Pengfei, and Z. Zibin, "Microscope: Pinpoint performance issues with causal graphs in micro-service environments." ICSO, pp.3-20, 2018. -
15 ఏҊख๏ͷલఏ ɾؔ࿈ݚڀᶅ MicroscopeͷҼՌਪख๏Λϕʔεʹ͢Δ ɾαʔϏεશମͷࢦඪͷΈΛఆٛ͠ɺࢹΞϥʔτΛઃఆ͢Δ ɾϗετ্ͷͯ͢ͷϝτϦοΫ ɾϓϩηεΛϊʔυɺTCPଓΛΤοδͱͨ͠શͳωοτϫʔΫґ ଘάϥϑ[4] Λར༻ͯ͠ɺҟৗͷҼՌؔάϥϑΛߏங͢Δ [4]
௶༎थ, ݹխେ, দຊ྄հ, “Transtracer: ࢄγεςϜʹ͓͚ΔTCP/UDP௨৴ͷऴͷࢹʹΑΔϓϩηεؒґଘؔͷࣗಈ ”, Πϯλʔωοτͱӡ༻ٕज़γϯϙδϜจू, 2019, 64-71 (2019-11-28), 201912݄. -
16 ఏҊख๏ͷϫʔΫϑϩʔʢਤʣ reqs/sec errors/sec latency CPU usage … ݪҼީิϦετ Frontend
Component ֤ϊʔυ ෳͷϝτϦοΫΛͭ 1. දࢦඪͷ ҟৗݕ 2. ґଘάϥϑ͔Βҟৗ ͷҼՌؔάϥϑΛߏங (component, metric, score) (component, metric, score) . . . -
17 ఏҊख๏ͷϫʔΫϑϩʔ 1. SLOͷҧΛݕ 2. ґଘؔάϥϑΛͱʹҼՌਪΛ࣮ߦ ɾϑϩϯτϊʔυ͔Βྡ͢ΔϊʔυΛऔಘ ɾ֤ϊʔυ্ͰαʔϏεશମͷදࢦඪͷ࣌ܥྻಛͱ૬ؔ͢Δϝτ ϦοΫΛอ࣋͢Δ͔Ͳ͏͔Λݕఆ ɾᮢΛ͑ͨ߹֘ϊʔυͱϝτϦοΫͷΛݪҼީิϦετ
ɾશϊʔυΛ୳ࡧ͢Δ·ͰґଘάϥϑΛ୳ࡧ ɾ૬ؔ߹͍ΛείΞͱͯ͠ީิϦετΛϥϯΩϯάԽ 0. ϑϩϯτϊʔυͷදϝτϦοΫࢹͱґଘάϥϑͷऩू