$30 off During Our Annual Pro Sale. View Details »

AIOps研究録―SREのための
システム障害の自動原因診断 / SRE NEXT 2022

AIOps研究録―SREのための
システム障害の自動原因診断 / SRE NEXT 2022

Tweet

More Decks by Yuuki Tsubouchi (yuuk1)

Other Decks in Research

Transcript

  1. AIOpsͷݚڀ ΢ΣϒΦϖϨʔγϣϯɾSRE 2013 2018 2020ʙ ӡ༻σʔλͷޮ཰తͳ؍ଌ๏ͷݚڀ SRE DIVERSITY [Y.Tsubouchi 2021],

    [Y. Tsubouchi 2022] [Y. Tsubouchi 2021]: ௶಺༎थ, ࿬ࡔேਓ, ᖛా݈, দ໦խ޾, খྛོߒ, Ѩ෦ത, দຊ྄հ, HeteroTSDB: ҟछ෼ࢄKVSؒͷࣗಈ֊૚ԽʹΑΔߴੑೳͳ࣌ܥྻσʔ λϕʔε, ৘ใॲཧֶձ࿦จࢽ, Vol.62, No.3, pp.818-828, 2021೥3݄. [Y. Tsubouchi 2022]: Yuuki Tsubouchi, Masayoshi Furukawa, Ryosuke Matsumoto, Low Overhead TCP/UDP Socket-based Tracing for Discovering Network Services Dependencies, Journal of Information Processing, Vol.30, pp.260-268, 2022.
  2. 6 HAL 9000 ʰ2001 ೥Ӊ஦ͷཱྀʱ 18ষ SREͷͨΊͷػցֶशೖ໳ ͔ΒͷҾ༻ ” ͨͬͨࠓɺAE35Ϣχοτͷো֐Λݕग़͠·ͨ͠ɻ

    ࢲ͸72࣌ؒҎ಺ʹ100%ͷ֬཰Ͱػೳఀࢭ͠·͢ɻ” ― HAL 9000ɺʰ2001 ೥Ӊ஦ͷཱྀʱ “͜ͷөը͕ඳ͘ະདྷΛઌݟͷ໌Λ΋ͬͯߏ૝ͨ͠ͷ͸Ξʔ αʔɾCɾΫϥʔΫ(Arthur C. Clarke)ͰɺγεςϜͱϋʔυ΢Σ Ξͷো֐ൃੜΛԿ࣌ؒ΋લʹ༧ଌͰ͖Δ׬શࣗಈԽαʔϏεͱ AI Λ૊Έ߹Θͤ·ͨ͠ɻHAL 9000 ͸ɺཱࣗͨࣗ͠ݾௐ੔ܕͷ ܽ఺͕ͳ͍ػցͱ͍͏ਓྨͷເ(͋Δ͍͸ѱເ)Ͱ͋Γɺਓؒʹ Αͬͯఆٛ͞Εͨ໨ඪΛୡ੒͢ΔͨΊʹɺӉ஦ધͷ৐һͱϛο γϣϯͷ྆ํʹไ࢓͠·͢ɻ”
  3. 7 ɾ1980೥୅ʹ͸ɺωοτϫʔΫ؅ཧʹɺ஌ࣝϕʔεAI΍χϡʔϥϧωοτ ϕʔεAIΛԠ༻͢ΔՄೳੑ͕ٞ࿦͞Ε͍ͯΔ ৘ใγεςϜͷӡ༻ʹAIΛԠ༻͢ΔىݯΛ୳Δ [Cebulka 1989]: Cebulka KD, et al.,

    Applications of arti fi cial intelligence for meeting network management challenges in the 1990s, IEEE GLOBECOM 1989. ɾಛఆͷαʔϏεΛαϙʔτ͢ΔͨΊͷωοτϫʔΫͷॳظઃܭ ɾηϯτϥϧΦϑΟεؒͷઓज़తͳઃඋܭը ɾεΠον͔Βͷϝοηʔδͷ؂ࢹͱ਍அ [Notaro 2021]: Notaro P, et al., A Survey of AIOps Methods for Failure Management. ACM TIST, 2021. ɾ1990೥୅ॳ಄͔ΒΦϯϥΠϯͷιϑτ΢ΣΞ΍ϋʔυ΢ΣΞͷނো༧஌ Ϟσϧ͕͍͔ͭ͘ఏҊ͞Ε͍ͯΔɽͦͷଞͷނো๷ࢭํ๏ͳͲ΋ಉ࣌ظ [Cebulka 1989] [Notaro 2021]
  4. 8 ݱ୅ʹ͓͚ΔAIOpsͷߩݙྖҬ [Notaro ’20]: Notaro, P, Jorge C, and Michael

    G. "A Systematic Mapping Study in AIOps.” ICSOC. Springer, Cham, 2020. [Notaro ’20]: Fig.2 Taxonomy of AIOps as observed in the identified contributions 
 ΑΓసࡌ ো֐؅ཧʹؔ͢Δݚڀ Ϧιʔεͷׂ౰ͳͲͷ 
 ࠷దԽʹؔ͢Δݚڀ
  5. 9 AIOpsͷݚڀྖҬ͝ͱͷ࿦จ਺ [Notaro ’20]: Notaro, P, Jorge C, and Michael

    G. "A Systematic Mapping Study in AIOps.” ICSOC. Springer, Cham, 2020. ɾAIOpsؔ࿈ͷ࿦จ਺ɿ670 ɾ670݅ͷ62.1%͕Failure Managementʢো֐؅ཧʣʹؔ࿈͍ͯ͠Δ ɾো֐༧ଌʢ26.4ˋʣো֐ݕग़ʢ33.7ˋʣݪҼ෼ੳʢ26.7ˋʣ ࿦จ਺͸૿Ճ܏޲
  6. 12 ΞϥʔτετʔϜ γεςϜো֐ݕ஌ޙͷ՝୊ େྔͷϝτϦΫεͷӾཡ CRITICAL: front-end - http_request latency_95 CRITICAL:

    user - latency_95 CRITICAL: user-db_memory_usage CRITICAL: user-db_cpu_user_usage CRITICAL: orders - jvm_heap_memory_usage CRITICAL: orders-db - network_transmit_bytes CRITICAL: payment - http_request_error_5xx CRITICAL: front-end - http_request_latency_50 CRITICAL: front-end - http_request_latency_90 CRITICAL: user-db - cpu_system_usage ೝ஌ෛՙ 
 ૿େ
  7. 13 ঱ঢ়ɿͳʹ͕յΕͨͷ͔ʁ ݪҼɿͳͥյΕͨͷ͔ʁ ͳͥΞϥʔτετʔϜ͕ൃੜ͢Δͷ͔ʁ ঱ঢ়ͱݪҼʹ͸૬ରؔ܎͕͋ΔͨΊɺ۠ผ͢Δ͜ͱ͕࣮͸೉͍͠ ঱ঢ়ͱݪҼ͕ಉ࣌ʹΞϥʔτ͞ΕΔͨΊ } ঱ঢ় ݪҼ 1

    HTTP 500΋͘͠͸400͕ฦ͞Ε͍ͯΔ σʔλϕʔεαʔό͕઀ଓΛڋ൱͍ͯ͠Δ 2 σʔλϕʔεαʔό͕઀ଓΛڋ൱͍ͯ͠Δ σʔλϕʔεαʔόͷ σΟεΫ࢖༻ྔ͕ຬഋͱͳ͍ͬͯΔ 3 σʔλϕʔεαʔόͷ σΟεΫ࢖༻ྔ͕ຬഋͱͳ͍ͬͯΔ ΫΤϦϩάͷϑΝΠϧαΠζ͕ٸ଎ʹ૿Ճ 4 ΫΤϦϩάͷϑΝΠϧαΠζ͕ٸ଎ʹ૿Ճ … ঱ঢ়ͷΈΛΞϥʔτ͢Ε͹Α͍ͷͰ͸ʁ
  8. 14 “Alert symptoms, diagnose causes” SLOʹجͮ͘ΞϥʔςΟϯά [SRWbook 18] Chapter 5

    "Alerting on SLOs", Beyer B, et al., The Site Reliability Workbook: Practical ways to implement SRE. O'Reilly Media, Inc."; 2018. ※ [SRWbook 18] Four Golden Signals / RED Latency, Traffic, Errors, Saturation Rate, Errors, Duration αʔϏεશମͷ঱ঢ়ʹର͢ΔΞϥʔτ AIʹΑΔݪҼ਍அ τϦΨʔ ӡ༻σʔλͷࣗಈղੳ ӡ༻σʔλ ϝτϦΫε ϩά τϨʔε Πϕϯτ SREs
  9. 15 ྨࣅͷண૝Λ΋ͭઌਓ͸͍Δ Chen P, et al., Causeinfer: Automatic and distributed

    performance diagnosis with hierarchical causality graph in large distributed systems. IEEE INFOCOM 2014. CauseInfer (2014) ɾίϯϙʔωϯτ͝ͱʹͷύέοτͷண৴࣌ ࠁ͔Βਪఆ͢ΔTCP஗ԆΛܭଌ ɾ౷ܭతҼՌ୳ࡧ෼໺ͷPCΞϧΰϦζϜʢޙ ड़ʣʹΑΓҼՌάϥϑੜ੒ SLOϝτϦΫεͷҧ൓ݕ஌ΛτϦΨʔͱͯ͠ҼՌਪ࿦ CauseInfer (2014) Fig. 2.ΑΓҰ෦సࡌ Lin J, et al., Microscope: Pinpoint performance issues with causal graphs in micro-service environments. ICSOC, 2018. Microscope (2018) CauseInferʹରͯ͠ɺඇ௨৴ؔ܎ͷґଘΛߟྀ ɾCaudeInferಉ༷ʹҼՌάϥϑΛੜ੒ ɾϚΠΫϩαʔϏεͰ͸ɺ֤αʔϏεͰ Ԡ౴͕࣌ؒܭଌ͞ΕΔ Microscope (2018) Fig. 2. ΑΓసࡌ
  10. ҼՌάϥϑͷܦ࿏ͷϥϯΩϯάग़ྗ ɾAutoMAP (2020): ϚΠΫϩαʔϏε୯ҐͰԠ౴͚࣌ؒͩͰͳ͘7छྨͷϝ τϦΫε͔ΒಘͨݸผͷҼՌάϥϑΛ߹੒ ɾFluxInfer (2020): PCΞϧΰϦζϜͰ͸ͳ͘ɺॏΈ෇͖ແ޲ґଘάϥϑ + ϖʔδϥϯΫ

    ɾMicroCause (2020): PCΞϧΰϦζϜͷ࣌ܥྻͷϥάΛߟྀ͢Δվྑ 16 ྨࣅͷண૝Λ΋ͭઌਓ͸͍Δ ʔ ൃలฤ ݪҼϝτϦΫεͷϥϯΩϯάग़ྗ ɾPatternMatcher (2021): ϝτϦΫεͷҟৗύλʔϯΛCNNͰ෼ྨ ɾFluxRank (2019): ྨࣅͷ࣌ܥྻΛΫϥελϦϯάޙʹϩδεςΟοΫճؼ ͰϥϯΩϯά
  11. 17 ݪҼ਍அ࿦จͷϦετ https://github.com/dreamhomes/RCAPapers ͞ΒͳΔઌߦݚڀͨͪ Notaro P, Cardoso J, Gerndt M.

    A Survey of AIOps Methods for Failure Management. ACM Transactions on Intelligent Systems and Technology (TIST). 2021 Nov 30;12(6):1-45. Lyu Y, Rajbahadur GK, Lin D, Chen B, Jiang ZM. Towards a Consistent Interpretation of AIOps Models. ACM Transactions on Software Engineering and Methodology (TOSEM). 2021 Nov 15;31(1):1-38. Soldani J, Brogi A. Anomaly detection and failure root cause analysis in (micro) service-based cloud applications: A survey. ACM Computing Surveys (CSUR). 2022 Feb 3;55(3):1-39. https://blog.yuuk.io/entry/2020/ieeecloud2020 https://netman.aiops.org/publications/ ਗ਼՚େֶ NETMAN LAB ࠃࡍձٞ IEEE CLOUD AIOpsؔ࿈ͷαʔϕΠ࿦จ
  12. 18 ɾ՝୊̍ ೖྗͱͳΔϝτϦΫεͷछྨ ΍ݸ਺Λ༧Ίࢦఆ͢Δඞཁ͕͋Δɻ ɾ՝୊̎ ࢦఆͨ͠ม਺ʹରͯ͠ݸผʹ ద੾ͳؔ਺ΛબΜͩΓɺνϡʔχϯ ά͢Δඞཁ͕͋Δɻ ɾ՝୊̏ ਪఆ݁Ռͷઆ໌ੑͷͨΊʹ͸

    ϝτϦΫεͷΈͰ͸ෆ଍͢Δɻ ઌߦݚڀͰͷະղܾͳྖҬΛ୳Δ ɾେྔͷϝτϦΫεΛऩूͰ͖Δ Α͏ʹͳͬͨ ɾʹ΋ؔΘΒͣɺͦΕΒͷϝτϦ ΫεΛ׆༻Ͱ͖͍ͯΔ͔ʁ જࡏత໰୊ҙࣝ γεςϜͰ؍ଌ͞Εͨશͯͷ 
 ϝτϦΫεΛೖྗͱͯ͠ 
 ҼՌάϥϑΛੜ੒
  13. ҟछࠞ߹ϝτϦΫεʹରͯ͠ݸผʹԾఆΛஔ͔ͳ͍ 19 શϝτϦΫε͔ΒݪҼ਍அʹ͔͚Δͱॲཧ͕࣌ؒ௕͘ͳΔ ݚڀͷ໰୊ઃఆɿݪҼ਍அͷલॲཧ ো֐ 
 ݕ஌ ݪҼ਍அ ఏҊ ࣌ܥྻͷ

    
 ݸ਺Λ࡟ݮ ௶಺༎थ΄͔, TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλͷ࣍ݩ࡟ݮख๏, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू, 2020೥. ௚ۙͷݻఆ෯ͷ 
 ϝτϦΫεΛऔಘ ਺෼୯Ґͷ࣮ߦ࣌ؒ ΦϑϥΠϯղੳ ϝτϦΫεΛࣄલࢦ ఆͤͣʹɺߴ଎ͳݪ Ҽ਍அ͕Մೳʹ ҼՌάϥϑ 
 ͷੜ੒
  14. 23 ϑΣʔζ̍ɿ୯มྔ࣌ܥྻͷҟৗੑʹண໨ ࣌ܥྻͷҟৗύλʔϯΛ13ύλʔϯʹ෼ྨͨ͠ྫ [PatternMatcher 2021]: Wu C, Zhao N, Wang

    L, Yang X, Li S, Zhang M, Jin X, Wen X, Nie X, Zhang W, Sui K. Identifying Root-Cause Metrics for Incident Diagnosis in Online Service Systems. [PatternMatcher 2021] 30෼ఔ౓ͷ୹ظؒͷղੳͰΑ͍ͨΊɺقઅੑ΍ϦϦʔεʹΑΔਖ਼ৗ ϞʔυͷมԽΛߟྀ͠ͳͯ͘Α͍ɻ ͢Ͱʹো֐ݕग़͞ΕͨޙͳͷͰɺΦϑϥΠϯҟৗݕ஌ͰΑ͍
  15. [PatternMatcher 2021]: Wu C, Zhao N, Wang L, Yang X,

    Li S, Zhang M, Jin X, Wen X, Nie X, Zhang W, Sui K. Identifying Root-Cause Metrics for Incident Diagnosis in Online Service Systems. 24 ඪຊXͱඪຊY͕ಉҰͷ฼ूஂͷ෼෍ΑΓੜ͍ͯ͡ Δ͔Λݕఆ͢Δ 2ඪຊؒͷ෼෍ͷࠩΛΈΔݕఆɿK-Sݕఆ ͏·͍͔͘ͳ͔ͬͨέʔε ɾγϣʔτεύΠΫͷΑ͏ͳݦஶͳ֎Ε஋ΛؚΉ ࣌ܥྻ p஋: 0.11 p஋: 0.51 ɾগ਺ͷ֎Ε஋Ͱ͸ɺ෼෍͕ҧ͏ͱΈͳ͞ΕΔ΄ ͲͰ͸ͳ͍ ࣌ܥྻΛ௨ৗظؒͱςετظؒʹ2෼ׂ͠ɺظؒؒ ͷ෼෍ࠩҟΛݕఆ ʢ[PatternMatcher 2021]Ͱ࠾༻͞Ε͍ͯΔʣ
  16. 25 ɾ௚؍ɿ࣌ܥྻͷಛੑ͕࣌ؒͷܦաͱͱ΋ʹมԽ͠ͳ͍ ɾఆٛɿܥྻͷฏۉ͓Αͼ෼ࢄ͕࣌ؒʹΑΒͣҰఆɼ͔ͭࣗݾڞ෼ࢄ͕࣌ؒ ࠩͷΈʹґଘ͢Δੑ࣭ ࣌ܥྻͷੑ࣭ʮఆৗੑʯͷ͋ͯ͸Ί ɾఆৗੑʹ͋ͯ͸·Βͳ͍ܥྻ͸ҟৗͱΈͳ͢ γϣʔτεύΠΫ 1֊ 
 ࠩ෼

    ͏·͍͔͘ͳ͔ͬͨέʔε ADFݕఆͰݕఆՄೳ ɾγϣʔτεύΠΫ͕ 
 ఆৗͱ൑ఆ͞ΕΔ ɾADFݕఆ͸ʮࠩ෼ܥྻʯ ͕ఆৗੑͷੑ࣭Λຬͨ͢ ͔ݕఆ εύΠΫ͸ฏۉ΁ͱ 
 ճؼͯ͠͠·͏ ߨԋͰ͸εΩοϓ
  17. 26 ҟৗݕ஌ख๏ͱͯ͠޿͘஌ΒΕ͍ͯΔख๏Λ࢖͏ ౷ܭతҟৗݕ஌ͷجຊͷεςοϓ ֬཰෼෍Ͱਖ਼ৗύλʔϯΛදݱͰ͖ΔͱԾఆ 1. ෼෍ͷਪఆ 2. ҟৗ౓ͷࢉग़ 3. ᮢ஋ͷઃఆ

    ҟৗ౓ʹର͢Δᮢ஋ͷઃఆʹΑΓҟৗΛ൑ఆ ະ஌ύϥϝʔλΛؚΉ֬཰෼෍Λਖ਼ৗϞσϧͱͯ͠Ծఆ σʔλ͔Βະ஌ύϥϝʔλΛਪఆ ਖ਼ৗ͔ΒͷͣΕͷ౓߹͍Λ༧Ίఆٛͯ͠ࢉग़ ࢀߟɿҪख߶, ೖ໳ ػցֶशʹΑΔҟৗ ݕ஌, 7.3અ ίϩφࣾ, 2015..
  18. 27 ࣌ܥྻͷ֤఺ͷ෼෍͕ਖ਼ن෼෍ʹै͏ͱԾఆ ֎Ε஋ݕग़ɿϗςϦϯάͷ ๏ T2 ࢀߟɿҪख߶, ೖ໳ ػցֶशʹΑΔҟৗݕ஌, 7.3અ ίϩφࣾ,

    2015.. ౓਺෼෍΁ 1. ෼෍ਪఆ f(x) = 1 2πσ2 exp ( − (x − μ)2 2σ2 ) ωοτϫʔΫૹ৴όΠτ਺ͷϝτϦΫε μ = 1 N N ∑ i=1 xi σ2 = 1 N N ∑ i=1 (xi − μ)2 2. ҟৗ౓ͷܭࢉ ඪຊฏۉ ඪຊ෼ࢄ a(x) = ( x − μ σ ) 2 3. ᮢ஋൑ఆ ҟৗ౓ͷ෼෍͸ࣗ༝౓1ͷΧΠೋ৐෼෍ʹै͏ ᮢ஋Λ֬཰ͰܾఆͰ͖Δʢ0.01, 0.05ͳͲʣ ϊΠζʹऑ͍ ະ஌ 
 ύϥϝʔλ
  19. 28 ɾࣗݾճؼϞσϧɿ͋Δ࣌ࠁ t ͷ஋Λɺ࣌ࠁ t Ҏલͷ஋Λ࢖ͬͯճؼ͢ΔϞσϧ ɾ௚؍ɿະདྷ͸աڈͷ஋͔Β༧ଌͰ͖ΔͱԾఆ ࣗݾճؼϞσϧʢARϞσϧʣʹΑΔҟৗݕ஌ ҟৗ౓ ࣮ଌ஋ʢ੨ʣ༧ଌ஋ʢᒵ৭ʣ

    yn = r ∑ t=1 at yn−t + vn ʢ༧ଌ஋ − ؍ଌ஋ = ༧ଌޡࠩʣ= ҟৗ౓ ࣌ܥྻ ͕༩͑ΒΕͨͱ͖ y1 , . . . yN : ϥά࣍਺ r : ࣗݾճؼ܎਺ at : ฏۉ0ɺ෼ࢄ ͷ 
 ਖ਼ن෼෍ʹै͏ϗϫΠτ ϊΠζ vn σ2 ɾ Λܾఆ͠ɺ࣮ଌ஋͔Β܎਺ ͱ෼ࢄ Λਪఆ͢Δ ɾ ͸੺஑৘ใྔج४ʢAICʣΑΓܾఆ͢Δͷ͕Ұൠత ɾ ͱ ͷۙࣅ஋Λ࠷খೋ৐๏ʹΑΓ࠷໬ਪఆ r at σ2 r at σ2 ࢀߟɿҪख߶, ೖ໳ ػցֶशʹΑΔҟৗݕ஌, 7.3અ ίϩφࣾ, 2015..
  20. 29 ະ஌ύϥϝʔλʢ܎਺ ʣΛ؍ଌσʔλ ͢΂ͯʹରͯ͠ਪఆ͢Δ at ࣗݾճؼϞσϧɿαϯϓϧ಺༧ଌͱαϯϓϧ֎༧ଌ αϯϓϧ಺༧ଌͷޡࠩ αϯϓϧ֎༧ଌͷޡࠩ ֶशظؒͱݕূظؒʹ෼ׂ͢ΔʢਤͰ͸1:1ʣ ̋ݦஶͳ

    
 ֎Ε஋ ະ஌ύϥϝʔλΛֶशσʔλ͔Βਪఆ ͠ɺݕূσʔλͱͷ༧ଌޡࠩΛࢉग़͢Δ ̋ϗϫΠτ 
 ϊΠζ ✗Ϩϕϧγϑτʹ ա৒ద߹ͯ͠ޡ ͕ࠩখ͍͞ ✗ϊΠζͷ൓ԠʹΑΔ 
 ޡݕ஌ ϗςϦϯάͱಉ͡…? ̋Ϩϕϧγϑτ
  21. ϑΣʔζ̍ ΦϑϥΠϯ ҟৗݕ஌ ϑΣʔζ2 ܗঢ় 
 ΫϥελϦϯά ҼՌάϥϑͷ 
 ੜ੒

    લॲཧ ݪҼ਍அ ҼՌάϥϑͷ 
 ߨԋͰ͸εΩοϓ ϑΣʔζ̎ ܗঢ়ΫϥελϦϯά ߨԋͰ͸εΩοϓ
  22. 33 ΫϥελϦϯάͷൣғ Proxy App DB Cluster 1 Cluster 2 ಉҰίϯϙʔωϯτ಺Ͱ

    
 ΫϥελϦϯά ɾίϯϙʔωϯτΛ·͍ͨͩϝτϦΫεΛΫϥελϦϯά͢Δͱ 
 ҼՌάϥϑʹඞཁͳϊʔυ͕ࣦΘΕΔ ɾͨ·ͨ·঱ঢ়ϝτϦΫεͱݪҼϝτϦΫεͷܗঢ়͕ྨࣅ͢ΔέʔεͳͲ ߨԋͰ͸εΩοϓ
  23. 34 ࣌ܥྻΫϥελϦϯάͷੈք [Paparrizos 15]: Paparrizos J, Gravano L. k-Shape: Ef

    fi cient and accurate clustering of time series. SIGMOD 2015. [Paparrizos 15] Fig. 1. Time-series clustering taxonomy. ΑΓసࡌ [Paparrizos 15] Fig. 2. The time-series clustering approaches. ΑΓసࡌ ࣌ܥྻશମͷ 
 ΫϥελϦϯά ܗঢ়ʹجͮ͘ 
 Ξϓϩʔν ਓ͕ؒ࣌ܥྻάϥϑΛΈͯࣅ͍ͯΔͱ 
 ൑அ͢Δͷ͸ɺܗঢ়ʹج͍͍ͮͯΔ͸ͣ ߨԋͰ͸εΩοϓ
  24. 35 ࣌ܥྻΫϥελϦϯάͷܗঢ়ྨࣅੑ Paparrizos J, Gravano L. k-Shape: Ef fi cient

    and accurate clustering of time series. SIGMOD 2015. [k-Shape 2015] Figure 2: Similarity computation ΑΓ 
 Ұ෦ൈਮͯ͠సࡌ ED: ϢʔΫϦουڑ཭ [k-Shape 2015] Figure1: ΑΓҰ෦ൈਮͯ͠సࡌ ॎ࣠ʹରͯ͠৳ॖͨ͠ͱ͖ʹࣅ͍ͯΔ͔ʁ ԣ࣠ʹରͯ͠γϑτͨ͠ͱ͖ʹࣅ͍ͯΔ͔ʁ z-scoreม׵ (ฏۉ0,ඪ४ภࠩ1ͱ͢Δม׵)Ͱୡ੒ ԣ࣠γϑτʹର͢Δߟྀ͕ͳ͍ Scalingෆมੑ Shifttingෆมੑ DTW: ಈతۭؒ৳ॖ๏ʢओྲྀʣ 2ຊͷ࣌ܥྻͷ֤఺ಉ࢜ͷڑ཭Λશͯܭࢉ͢Δͨ Ίɺܭࢉྔ͕େ͖͍ ߨԋͰ͸εΩοϓ
  25. 36 SBD (Shape-Based Distance) Paparrizos J, Gravano L. k-Shape: Ef

    fi cient and accurate clustering of time series. SIGMOD 2015. ૬ޓ૬ؔ ϕΫτϧͷ಺ੵɿϕΫτϧಉ͕࢜֯౓͕͍ۙͱ͖ʢ ͕0ʹ͍ۙͱ͖ʣେ͖ͳ஋ΛͱΔ θ x ⋅ y = |x||y|cos θ 1ຊͷ࣌ܥྻΛ1ݸͷϕΫτϧͱΈͳ͢ ϕΫτϧΛͣΒ͠ͳ͕Β಺ੵΛܭࢉ͍͖ͯ͠ɺ಺ੵ͕࠷େͱͳΔγϑτ Λൃݟ͢Δ w CCw (x, y) = Rw−m (x, y) Rk (x, y) = { ∑m−k l=1 xl+k ⋅ yl R−k (y, x) x = (x1 , . . . , xm ) y = (y1 , . . . , ym ) ճ ΛӈʹฏߦҠಈͤͨ͞ 
 ͱ͖ͷ ͱͷྨࣅ౓Λ ͱ͢Δ k = w − m y x CCw (x, y) Λ 
 ʹରͯ͠ܭࢉ CCw (x, y) w ∈ 1,2,...,2m − 1 ͷܭࢉྔΛ 
 ߴ଎ϑʔϦΤม׵Ͱ 
 ΁ O(m2) O(mlogm) ߨԋͰ͸εΩοϓ
  26. 37 ΫϥελԽͷखॱͷݕ౼ ΫϥελϦϯά ֊૚తΫϥελϦϯά ෼ׂ࠷దԽతΫϥελϦϯά ࢀߟɿਆቇ හ߂, ΫϥελϦϯάʢClusteringʣ https://www.kamishima.net/archive/clustering.pdf Ϋϥελͷྑ͞Λදؔ͢਺Λఆٛ͠ɼ

    
 ͦͷؔ਺Λ࠷దԽ͢ΔΑ͏ͳ 
 ΫϥελΛൃݟ ڽूܕ ෼ׂܕ σʔλҰ͕ͭݸʑͷΫϥελͷ 
 ঢ়ଶ͔Βɼॱ࣍ΫϥελΛซ߹ σʔλू߹શମ͕ҰͭͷΫϥελ ͷঢ়ଶ͔Βɼॱ࣍ΫϥελΛ෼ׂ ܭࢉྔ͕ଟ͍ ܭࢉྔ͕ଟ͍ ࣄલʹΫϥελ਺Λ 
 ܾΊͳ͚Ε͹ͳΒͳ͍ ༗໊ͳk-means͸ 
 ͪ͜Β ܭࢉྔ͕গͳ͍ ߨԋͰ͸εΩοϓ
  27. ϑΣʔζ̍ ΦϑϥΠϯ ҟৗݕ஌ ϑΣʔζ2 ܗঢ় 
 ΫϥελϦϯά લॲཧ ݪҼ਍அ ҼՌάϥϑͷ

    
 ੜ੒ͯ֬͠ೝ͍ͨ͠ ݪҼ਍அɿҼՌάϥϑੜ੒ ҼՌάϥϑͷ 
 ੜ੒ 
  28. 41 ɾMݸͷ֬཰ม਺ ɾ֬཰ม਺͝ͱʹNݸͷඪຊ ҼՌ୳ࡧΛϝτϦΫεͷݪҼ਍அʹԠ༻ DAGʢ༗޲ඇ८ճάϥϑʣ ग़ྗ front-end:latency user:latency ɾ 


    ɾ 
 ɾ Mݸͷ 
 ࣌ܥྻ Nݸͷඪຊ orders:latency user:cpu_usage user-db:cpu_usage orders:network_transmit_bytes ঱ঢ়ϝτϦΫε orders-db:network_receive_bytes
  29. 42 ౷ܭతҼՌ୳ࡧͷੈք ؍ଌσʔλ͔ΒҼՌؔ܎Λਪఆ͢Δ ੍໿ϕʔε είΞϕʔε Glymour C, et al., Review

    of causal discovery methods based on 
 graphical models. Frontiers in genetics. 2019 ߏ଄ํఔࣜϕʔε ϊϯύϥϝτϦοΫʢԾఆΛ͓͔ͳ͍ʣ ύϥϝτϦοΫɾ 
 ηϛύϥϝτϦοΫ PCΞϧΰϦζϜ (1991) ม਺ؒͷ৚݅෇͖ಠཱੑΛ੍໿ 
 ͱͯ͠ҼՌάϥϑΛߏங͢Δ FCIΞϧΰϦζϜ (1995) ະ؍ଌڞ௨ݪҼΛߟྀ ಉ͡৚݅෇͖ಠཱੑΛ༩ ͑ΔҼՌάϥϑͷू߹Ͱ ͋ΔϚϧίϑಉ஋ྨ͝ͱ ʹɼϞσϧͷΑ͞ΛධՁ GESΞϧΰϦζϜ (2003) LiNGAM (2006) ݪҼΛXΛ݁ՌΛYͱͯ͠ ߏ଄ΛํఔࣜͰදݱ ؔ਺ܥ΍ޡࠩม਺ͷ෼෍ ʹԾఆΛஔ͘
  30. 43 1. ॳظԽɿ Mݸͷ֬཰ม਺ͷ׬શ࿈݁άϥϑΛߏங͢Δ PCΞϧΰϦζϜ 2. Τοδ࡟আɿ ྡ઀͢Δ֤ม਺ʹ͍ͭͯɺ৚݅෇͖ಠཱੑ͕ଘࡏ͢Ε͹ɺ2 ม਺ؒͷΤοδΛ࡟আɻ 3.

    ϧʔϧϕʔεͷํ޲ܾఆʢv-structureɺΦϦΤϯςʔγϣϯϧʔϧʣ ಠཱੑɿม਺ؒͷ૬ؔΛΈΔͳͲͷख๏Ͱಠཱ͔Ͳ͏͔Λਪఆɻ ৚݅෇͖ಠཱੑɿ͋Δม਺Λ৚݅ͱͨ͠ͱ͖ͷଞͷ2ม਺ͷಠཱੑ
  31. 44 1. ॳظԽ PCΞϧΰϦζϜͷࣄલ஌ࣝʹΑΔಠ֦ࣗு 2. Τοδ࡟আ 3. ϧʔϧϕʔεͷํ޲ܾఆ ํ޲ܾఆ͸ؒҧ͍͑ͯΔ͜ͱ͕͋Δ ᶃ

    ௚઀ωοτϫʔΫ௨৴Λ͍ͯ͠ͳ͍ม਺ؒͷΤοδΛແ৚݅Ͱ࡟আ ᶄ ௨৴ͷऩूσʔλΛجʹɺํ޲Λܾఆɾमਖ਼͢Δ 
 ɹL4௨৴ɿTCP/UDPͷ઀ଓ։࢝ํ޲ɺύέοτͷϥά૬ؔ ɹL7௨৴ɿHTTPͷϦΫΤετͷ޲͖ … ׬શ࿈݁άϥϑͰ͸Τοδ͕ଟ͍ͨΊɺΤοδ࡟আͷॲཧ͕࣌ؒ௕͍ ࡟আͨ݁͠Ռ ߨԋͰ͸εΩοϓ
  32. 46 ݪҼͱ঱ঢ়ͷϝτϦΫεؒͷ 
 ܦ࿏͕੾ΒΕΔ PCΞϧΰϦζϜͷ৚݅෇͖ಠཱੑݕఆͷ՝୊ front-end:latency user:latency orders:latency user:memory_usage user-db:cpu_usage

    orders:network_receive_bytes ঱ঢ়ϝτϦΫε orders-db:network_transmit_bytes ݪҼ 
 ϝτϦΫε ௐࠪ ৚݅෇͖ಠཱੑݕఆͰ͸ɺ৚݅ม਺ͷӨڹΛআڈ͠ ্ͨͰɺ2ม਺ͷ૬ؔΛΈΔʢภ૬ؔʣ ো֐ൃੜ࣌ͷࣅͨมಈͷܥྻ͕ଟ͍ͱ͖ʹޡͬͯ Τοδ͕੾ΒΕ΍͍͢Α͏ʹΈ͑Δ front-end: 
 latency orders-db: 
 network_transmit 
 _bytes user: 
 memory_ 
 usage user: 
 latency ࣅͨมಈΛࣔ͢3ม਺ؒͷ Τοδ͕ͳ͔ͥ੾ΒΕ͍ͯΔ ✗