クラウドのシステム運用技術に機械学習を応用する研究 / CLOUD AI

第6回さくらインターネット研究会

Transcript

1. Ϋϥ΢υͷγεςϜӡ༻ٕज़ʹ
   ػցֶशΛԠ༻͢Δݚڀ
   ௶಺ ༎थ @yuuk1t
   ୈ6ճ͘͞ΒΠϯλʔωοτݚڀձ
   2020೥12݄9೔
   

   View Slide

2. 2
   ࣗݾ঺հ
   Yuuki Tsubouchi / Ώ͏͏͖ https://yuuk.io/
   ܦྺ
   גࣜձࣾ͸ͯͳ
   WebΦϖϨʔγϣϯΤϯδχΞɾSRE
   ͘͞ΒΠϯλʔωοτݚڀॴ ݚڀһ
   ژ౎େֶ৘ใֶݚڀՊ ത࢜ޙظ՝ఔ
   WebαʔϏεͷ
   ։ൃɾӡ༻
   SREͷݚڀ
   5೥
   ݱࡏ
   Site Reliability Engineering(SRE) Researcher
   @yuuk1t
   ৘ใॲཧֶձ Πϯλʔωοτͱӡ༻ٕज़ݚڀձ ӡӦҕһ
   

   View Slide

3. 3
   1. ਺೥୯Ґͷݚڀߏ૝
   2. ݚڀॴͰͷ࠷৽ͷݚڀ
   3. ·ͱΊͱ͘͞Β΁ͷߩݙՄೳੑ
   ࠓ೔͓࿩͢Δ͜ͱ
   Ϋϥ΢υΛߏ੒͢ΔγεςϜͷӡ༻ٕज़Λର৅ʹ
   

   View Slide

4. 1.
   ਺೥୯ҐͰͷݚڀߏ૝
   

   View Slide

5. 5
   ຊݚڀͷର৅γεςϜͷείʔϓ
   “Cloud computing“, Wikipedia ΑΓҾ༻
   ιϑτ΢ΣΞ
   ͷελοΫ
   

   View Slide

6. 6
   ɾίϯςφΦʔέετϨʔγϣϯ
   ɾϚΠΫϩαʔϏεΞʔΩςΫνϟ
   ɾαʔόʔϨεΞʔΩςΫνϟ
   ɾ෼ࢄσʔλϕʔε ʢNoSQLɺNewSQLʣ
   ɾαʔϏεϝογϡ
   ɾΤοδɾϑΥάίϯϐϡʔςΟϯά
   ෳࡶԽ͢ΔΫϥ΢υͷγεςϜΞʔΩςΫνϟ
   ιϑτ΢ΣΞʹΑΔಈత੍ޚٕज़͕ීٴ͍ͯ͠Δ
   

   View Slide

7. 7
   Ironies of Automation (1983)
   1. ࣗಈԽʹΑΓਓؒͷ࡞ۀෛ୲Λ௿ݮͰ͖Δ
   2. ͔͠͠ɼࣗಈԽ͢Ε͹͢Δ΄Ͳਓؒͷೝ஌ෛՙ͕ߴ·Δ
   3. ೝ஌ෛՙʹ଱͑ΒΕΔΑ͏ʹߴ౓ͳ܇࿅͕ඞཁͱͳΔ
   • L. Bainbridge, "Ironies of automation,” Analysis, design and evaluation of man–machine systems, pp.129-135 1983.
   • G. Baxter, et al. "The ironies of automation: still going strong at 30?,” the 30th European Conference on Cognitive Ergonomics 2012.
   • B. Strauch, "Ironies of automation: Still unresolved after all these years," IEEE Transactions on Human-Machine Systems, vol. 48, no. 5, pp. 419-433 2017.
   • J. Paul Reed, “When /bin/sh Attacks: Revisiting "Automate All the Things”,” USENIX LISA19 2019.
   • Tanner Lund, “Ironies of Automation: A Comedy in Three Parts,”, USENIX SREcon19 Asia/Pacific 2019.
   શͯΛࣗಈԽ͠Α͏ͱͯ͠΋͏·͍͔͘ͳ͍
   ෳࡶ͞Λࣺͯͯɺγϯϓϧʹͨ͠ͱͯ͠΋ɺͰ͖ͳ͍͜ͱ͸Ͱ͖ͳ͍··
   ෳࡶ͞Λड༰ͯ͠ɺ͍͔ʹਓؒͷೝ஌ෛՙΛ௿Լͤ͞Δ͔
   ࣗಈԽͷൽ೑
   

   View Slide

8. 8
   Ironies Of Automationʹର͢ΔେલఏΞϓϩʔν
   Ironies of Automationʹର͢ΔΞϓϩʔν
   ɾ͍͔ʹෳࡶͳγεςϜʹରͯ͠ɺೝ஌Λ֫ಘ͍͔ͯ͘͠
   ɾೝ஌ෆ଍ͷঢ়ଶ => ࣦഊ͕ා͍ͷͰγεςϜมߋͷͨΊʹௐࠪʹ࣌ؒ
   Λ͔͚Δ => ෳࡶ͕͞૿͍ͯ͘͠΄Ͳ͕͔͔࣌ؒΔ => ೝ஌͕௥͍͔ͭ
   ͳ͍
   Site Reliability Engineering (SRE)
   1. ࣦഊΛڐ༰͢ΔલఏͰӡ༻ΛઃܭΛ͢Δ
   2. ਓؒͷೝ஌ͦͷ΋ͷΛίϯϐϡʔλʹΦϑϩʔυ͢Δ
   AIٕज़
   ※௶಺༎थ, Ϋϥ΢υܥͷࠃࡍձٞIEEE CLOUD 2020ࢀՃ࿥, https://blog.yuuk.io/entry/2020/ieeecloud2020
   ※
   

   View Slide

9. ৴པੑͱ͸ γεςϜ͕ٻΊΒΕΔػೳΛɼఆΊΒΕͨ৚݅ͷԼͰఆ
   ΊΒΕͨظؒʹΘͨΓো֐Λى͜͢͜ͱͳ࣮͘ߦ͢Δ֬཰ ※2
   9
   ɾαʔϏεϨϕϧࢦඪʢService Level Indicator, SLIʣ
   ɾαʔϏεϨϕϧ໨ඪʢService Level Objective, SLOʣ
   SRE: 100%ͷ৴པੑΛ໨ࢦ͞ͳ͍
   *2 P. O’Connor, A. Kleyner. Practical Reliability Engineering, 5th edition, Wiley 2012.
   Ϋϥ΢υ্Ͱల։͍ͯ͠Δଟ਺ͷαʔϏεࣄۀऀ͕SLIɾSLOʹΑΓ
   ৴པੑΛఆྔతʹܭଌ͠ɼ݁ՌΛҙࢥܾఆʹར༻͍ͯ͠Δ
   ৴པੑͷࢦඪͱͦͷ໨ඪ஋Λܾఆ͠ɼܭଌظؒதʹ໨ඪ஋ΛԼճΒͳ
   ͍ݶΓɼαʔϏεࣄۀऀ͸ੵۃతʹγεςϜΛมߋͰ͖Δ
   ※SLAʢService Level Agreementʣ͸Ϗδωε্ͷܖ໿Ͱ͋Γɺ Ϣʔβʔͷෆຬʹର͢ΔิঈͳͲؚ͕·ΕΔ
   

   View Slide

10. 10
    1. ࣦഊͷڐ༰ => ҙਤతʹނোΛ஫ೖ
    SREΛલఏʹAIΛద༻͢Δ
    Chaos Engineering
    2. SLIɾSLOΛϕʔεʹAIͰো֐͔Βճ෮ΛࣗಈԽ
    ɾChaosͷఆٛʮγεςϜͷऑ఺Λ໌Β͔ʹ͢ΔͨΊͷ࣮ݧͷଅਐʯ
    ɾʮఆৗঢ়ଶʯΛܾΊͯɺԾઆݕূͷϑϨʔϜϫʔΫΛར༻
    ɾγεςϜͷऑ఺ʹର͢Δೝ஌Λଅਐͤ͞Δ
    ɾ࠷ۙɺऔΓ૊ΜͰ͍Δͷ͸ͪ͜Β
    

    View Slide

11. 11
    SLIɾSLOΛத৺ʹਾ͑ͨγεςϜҟৗ΁ͷΞϓϩʔν
    ɾ༧ଌɹաڈʹSLI͕௿Լͨ͠௚લͷৼΔ෣͍ͱྨࣅ͢ΔৼΔ෣͍Λൃݟ
    ɾҟৗՕॴͷಛఆɹSLOҧ൓࣌ʹҟৗՕॴΛ୳ࡧ
    ɾݪҼڀ໌ɹʢ޻ࣄதʣ
    ɾճ෮ɹSLIΛλʔήοτͱ͢ΔϑΟʔυόοΫ੍ޚ
    ɾSLIͷ୅ସɹࠓ͋ΔϝτϦοΫΛ૊Έ߹ΘͤͯSLIͷ୅ସࢦඪΛ࡞੒
    ༧ଌ ҟৗՕॴͷಛఆ ݪҼڀ໌ ճ෮
    ݕ஌
    SLOʹجͮ͘Ξϥʔτ
    ௚ۙͷڵຯର৅
    

    View Slide

12. 12
    ɾػցֶश͸ɺσʔλΛར༻ֶͯ͠श͢Δ
    ɾ͔͠͠ɺҟৗൃੜ࣌ͷσʔλ͸ݱ࣮ʹ͸͋·ΓೖखͰ͖ͳ͍…
    ɾChaos EngineeringʹΑΓɺҙਤతʹҟৗΛൃੜͤ͞ɺֶशʹར༻
    ՄೳͳσʔλΛੜ੒ͤ͞Δ
    ͞ΒͳΔల๬: Chaos Engineering x AI
    ɾ੍ޚཧ࿦Ͱ͸ಈతγεςϜϞσϧͷύϥϝʔλΛܾఆ͢ΔͨΊʹɺ
    ࣮ݧʹಈ࡞ͤ͞ΔϑΣʔζ͕͋ΔʢγεςϜಉఆʣ
    ɾΫϥ΢υͷγεςϜͰ੍ޚϞσϧΛઃఆͯ͠ɺγεςϜಉఆ͸
    Chaos EngineneringͰߦ͏
    

    View Slide

13. 2.
    ࠷৽ͷݚڀͷ࿩
    

    View Slide

14. TSifter: ϚΠΫϩαʔϏεʹ͓͚Δੑೳҟৗͷ
    ਝ଎ͳ਍அʹ޲͍ͨ࣌ܥྻσʔλͷ࣍ݩ࡟ݮख๏
    ௶಺ ༎थʢ͘͞ΒΠϯλʔωοτɺژ౎େֶʣ
    ௽ా തจʢ͘͞ΒΠϯλʔωοτʣ
    ݹ઒ խେʢ͸ͯͳʣ
    https://speakerdeck.com/yuukit/tsifter-in-proceedings-of-iots2020
    

    View Slide

15. 1.
    ݚڀͷഎܠͱ໨త
    

    View Slide

16. 16
    ϚΠΫϩαʔϏεߏ੒ͷීٴ
    ϞϊϦε ϚΠΫϩαʔϏε
    ػೳผͷ෼ࢄߏ੒΁
    มભ
    WebαʔϏεͷιϑτ΢ΣΞن໛͕૿େ͠ɺ։ൃऀ͕ιϑτ΢ΣΞΛ
    มߋ͢Δ͜ͱ͕೉͘͠ͳ͍ͬͯΔ
    

    View Slide

17. 17
    ؂ࢹσʔλྔ
    ͷ૿େ
    ϚΠΫϩαʔϏεΛӡ༻͢Δࡍͷ໰୊ҙࣝ
    ґଘؔ܎ͷෳࡶੑ
    ιϑτ΢ΣΞͷ
    มߋස౓޲্
    γεςϜͷೝ஌ෛՙ͕ߴ·Δ
    ੑೳҟৗͷݪҼΛ਍அ͢ΔͨΊͷ࣌ؒΛཁ͢ΔΑ͏ʹͳΔ
    

    View Slide

18. 18
    ੑೳҟৗΛ਍அ͢ΔͨΊͷطଘͷΞϓϩʔν
    ϝτϦοΫ
    ςΩετϩά
    ࣮ߦτϨʔε
    ๛෋ͳ৘ใΛ΋͕ͭϩάʹग़ྗ͞Εͳ͍΋ͷ΋
    ͋Δ
    ॲཧܦ࿏ͷล୯Ґͷεϧʔϓοτ΍࣮ߦ࣌ؒΛ
    ೺ѲͰ͖ΔɻΞϓϦέʔγϣϯʹܭଌॲཧΛઃ
    ఆ͢Δख͕ؒ͋Δ
    ݸʑͷ৘ใྔ͸গͳ͍͕ऩूɺอଘɺՄࢹԽ͠
    ΍͍͢ɻ
    ࣮؀ڥ΁ͷద༻ੑΛ౿·͑ͯɺʮϝτϦοΫʯʹண໨
    

    View Slide

19. 19
    ϝτϦοΫϕʔεΞϓϩʔν
    ֤αʔϏεͷܥྻάϥϑ͔Β૬ؔΛൃݟͰ͖Δ͕ɺݪҼՕॴ͕ෆ໌
    ʮ౷ܭతҼՌ୳ࡧʯΛԠ༻ͨ͠Ξϓϩʔν͕௚ۙ਺೥ͰఏҊ͞Ε͍ͯΔ
    Service A
    response time
    Service D
    response time
    Service E
    response time
    Service A
    response time
    Service D
    response time
    Service E
    response time
    Service F
    response time
    Service C
    response time
    ᶃܥྻؒͷҼՌ఻ൖάϥϑͷߏங ᶄҼՌͷܦ࿏ͷਪ࿦
    Ma, M.,et al., AutoMAP: Diagnose Your Microservice-based Web Applications Automatically, WWW2020.
    Qiu, J.,et al., A Causality Mining and Knowledge Graph Based Method of Root Cause Diagnosis for Performance Anomaly in Cloud Applications, Applied Sciences, 2020.
    Lin, J.,et al., Microscope: Pinpoint Performance Issues with Causal Graphs in Micro-Service Environments, ICSOC2018.
    Service A
    response time
    Service F
    response time
    Service C
    response time
    Top-1
    Top-2
    Service B
    response time
    

    View Slide

20. ɾ਍அʹར༻͢ΔϝτϦοΫͷछྨͷ૊߹ͤ͸ݻఆ ʢ1ʙ7ݸఔ౓ʣ
    ɾྫʣԠ౴஗ԆͷΈɺ{Ԡ౴஗Ԇ, CPUར༻཰, ϝϞϦ࢖༻ྔ,…} ͳͲ
    ɾΑΓݪҼʹ͍ۙϝτϦοΫ͕݁Ռ͔Βআ֎͞ΕΔՄೳੑ͕͋Δ
    20
    ϝτϦοΫϕʔεΞϓϩʔνͷ՝୊
    Ͱ͖ΔݶΓଟ͘ͷϝτϦοΫͷܥྻΛ୳ࡧ͢Δඞཁ͕͋Δ
    TCPͷ࠶ૹΤϥʔ͕ൃੜ͍ͯ͠Δ͕ɺ
    ωοτϫʔΫଳҬͷมԽྔ͕খ͍͞ͳͲ
    

    View Slide

21. 21
    ੑೳҟৗʹର͢ΔϝτϦοΫͷܥྻͷ࣍ݩ࡟ݮͷఏҊ
    ໨త: ϚΠΫϩαʔϏεʹͯҟৗͷ఻೻ܦ࿏ΛࣗಈͰਪ࿦͢ΔͨΊͷج൫
    ఏҊ: ҟৗͷݕ஌ʹ൓Ԡͯ͠ɺʮҰ࣌తʹʯ਍அʹ༗༻ͳܥྻΛશܥྻ͔
    Βߴ଎ʹநग़͢Δ࣍ݩ࡟ݮख๏ “TSifter” (Time series Sifter)
    ɾᶃਖ਼֬ੑ :਍அʹ༗༻ͳܥྻ͕࡟ݮ͞Ε͍ͯͳ͍
    ɾᶄ࣍ݩ࡟ݮ཰: ແ༻ͳܥྻΛͳΔ΂͘ଟ͘࡟ݮ͍ͨ͠
    ɾᶅߴ଎ੑ : ਝ଎ʹݪҼΛΈ͚͍ͭͨ (ཧ૝͸1෼ఔ౓)
    ܥྻ਺ʢ=࣍ݩ਺ʣ͕૿Ճ͢ΔͱҼՌ఻ൖάϥϑ͕ڊେԽ͢Δ
    3ͭͷཁ݅
    

    View Slide

22. 22
    ࠷ऴతʹ࣮ݱ͍ͨ͠ݪҼ਍அγεςϜͷશମ૾
    શܥྻ
    औಘ
    ܥྻͷ
    ࣍ݩ࡟ݮ
    ݪҼ਍அ
    ࣌ܥྻ
    σʔλϕʔε
    ܥྻͷऩू
    ҟৗݕ஌
    ఏҊख๏ͷείʔϓ
    YES
    Service A/
    req_errors
    Service D/
    connections
    Service E/
    ܥྻؒͷҼՌͷܦ࿏
    ᶃ
    ᶄ
    ᶅ ᶆ ᶇ
    ʢҼՌάϥϑߏஙʣ
    αʔϏε୯ҐͰ
    ࣍ݩ࡟ݮ
    

    View Slide

23. 2.
    ੑೳҟৗͷݪҼ਍அʹ޲͍ͨ
    ϝτϦοΫͷ࣍ݩ࡟ݮख๏
    

    View Slide

24. 24
    ఏҊख๏ TSifter ͷཁ݅ͱղܾ
    ᶃਖ਼֬ੑ
    ᶄ࣍ݩ࡟ݮ཰
    ᶅߴ଎ੑ
    ಎ࡯1
    ಎ࡯2
    ҟৗൃੜલޙͰ࣌ܥྻͷ܏޲͕มԽ͠ͳ͍
    ܥྻ͸਍அ࣌ʹෆཁ
    → ࣌ܥྻσʔλͷఆৗੑΛ΋ͭܥྻΛআ֎
    ࣌ܥྻάϥϑͷܗঢ়͕ࣅ͍ͯΔܥྻ܈͸ҟ
    ৗͷ਍அ࣌ʹ৑௕
    → αʔϏε୯ҐͰ࣌ܥྻͷΫϥελϦϯά
    ܥྻ਺nʹରͯ͠ΫϥελϦϯάॲཧ͸ , ...
    → ಎ࡯1ͷআ֎ॲཧ Λઌʹ࣮ߦ͢Δ
    O(kn) O(n2)
    O(n)
    

    View Slide

25. 25
    TSifter: 2ஈ֊ͷ࣍ݩ࡟ݮख๏
    ɾɾɾ
    ɾɾɾ
    ɾɾɾ
    εςοϓ1
    ఆৗੑΛ΋ͭ
    ܥྻΛআڈ
    ੜͷܥྻ ඇఆৗͳܥྻ ΫϥελԽ͞Εͨܥྻ
    ୅දܥྻ
    ҟৗظؒ
    ΫϥελϦϯάޙʹΫϥελ
    ͷ୅දܥྻΛબ୒
    εςοϓ2
    ྨࣅͷܗঢ়Λ
    ͱΔܥྻΛ
    ΫϥελϦϯά
    ҟৗൃੜલn෼ͷ
    ݻఆ௕ͷ΢Οϯυ΢෯
    

    View Slide

26. 26
    ɾఆৗੑ: σʔλͷਫ४΍͹Β͖ͭɺࣗݾ૬ؔͷؔ܎͕࣌఺ʹΑΒͣҰఆ
    ɾ࣌ܥྻσʔλͷఆৗੑݕఆʹ޿͘ར༻͞ΕΔADFݕఆΛར༻
    ɾશͯͷܥྻΛ1ͭͣͭݕఆ͠ɺఆৗੑΛ΋ͭܥྻΛআڈ
    εςοϓ1: ݸʑͷܥྻͷఆৗੑʹண໨
    ࢒ཹ͢Δඇఆৗͳܥྻͷྫ
    

    View Slide

27. 27
    εςοϓ2: ܥྻؒͷܗঢ়ྨࣅੑʹண໨
    αʔϏε಺ͷܥྻ܈
    ܗঢ়ͷྨࣅੑΛද͢ڑ཭ई౓ shape-based distance (SBD) Λ࠾༻
    ʢ࣌ؒ࣠ํ޲ʹγϑτɺॎ࣠ʹ৳ॖ͍ͯͯ͠΋ྨࣅͱΈͳ͢ʣ
    Paparrizos, J. and Gravano, L., k-Shape: Efficient and Accurate Clustering of Time Series,(SIGMOD2015)
    ߴ଎ԽͷͨΊɺ1ճͷॲཧͰΫϥελ਺ΛܾఆՄೳͳ֊૚తΫϥελϦϯάΛ࠾༻
    ʢ ͕ͩɺ1αʔϏε͋ͨΓͷܥྻ਺͕খ͍ͨ͞Ί໰୊ʹͳΒͳ͍ʣ
    O(n2)
    αʔϏεͷ୅දܥྻ܈
    Ϋϥελ
    ୅දܥྻͷબ୒
    ଞͷܥྻͱͷڑ཭ͷ૯࿨͕࠷খͷܥྻ
    

    View Slide

28. 3.
    ࣮ݧͱධՁ
    

    View Slide

29. 29
    ࣮ݧ؀ڥ
    ੍ޚαʔό
    Locust
    Kubernetes
    CPUෛՙ஫ೖ ωοτϫʔΫ஗Ԇ஫ೖ
    ϚΠΫϩαʔϏεΫϥελ
    Front-End
    Catalogue Orders
    Payment
    Shipping
    User
    Carts
    ղੳαʔό
    Prometheus
    ֎෦ෛՙͷ
    ੜ੒
    ܥྻऔಘϞδϡʔϧ
    stress-ng tc
    ղੳϞδϡʔϧ
    ܥྻऩूִؒ: 5ඵ
    ܥྻͷ΢Οϯυ΢෯: 30෼
    Intel Xeon 3.10GHz, 8core,32GB
    ܥྻͷऩूɾอଘ
    Sock Shop
    

    View Slide

30. 30
    ϕʔεϥΠϯख๏: Sieve
    ɾεςοϓ1: ෼ࢄ஋ͷখ͍͞ϝτϦοΫΛऔΓআ͘
    ɾεςοϓ2: k-ShapeʹΑΔΫϥελϦϯά
    Thalheim, J., et al., Sieve: Actionable Insights from Monitored Metrics in Distributed Systems, (Middleware 2017)
    ߃ৗతʹར༻ՄೳͳγεςϜͷಛ௃Λநग़͢Δ͜ͱ͕໨తͰ͋Γɺຊ
    ݚڀͱ͸໨త͕ҟͳΔ͕ɺҟͳΔ໨తʹ΋Ԡ༻Ͱ͖ΔՄೳੑ͕͋Δ
    ࣌ܥྻσʔλͷ࣍ݩ࡟ݮख๏
    Paparrizos, J. and Gravano, L., k-Shape: Efficient and Accurate Clustering of Time Series,(SIGMOD2015)
    

    View Slide

31. 31
    ᶃਖ਼֬ੑ: ҟৗ͝ͱͷݪҼͱͳΔܥྻͷਖ਼ޡ
    TSifter͸શͯͷέʔεʹରͯ͠ਖ਼͘͠ݪҼͱͳΔܥྻΛநग़
    ϕʔεϥΠϯख๏͸shippingαʔϏεͷCPUաෛՙͷέʔεͷΈෆਖ਼ղ
    

    View Slide

32. 32
    ᶄ࣍ݩ࡟ݮ཰ͷධՁ: ҟৗ4έʔε
    ɾ͍ͣΕͷέʔεʹ͓͍ͯ΋ɺ91%Ҏ্ͷ࣍ݩ࡟ݮ཰Ͱ͋Γɺ1/10Ҏ
    ԼʹߜΓࠐΊ͍ͯΔ
    ɾϕʔεϥΠϯख๏ͷ΄͏͕࣍ݩ࡟ݮ཰͸Θ͔ͣʹߴ͍
    ɾTSifter͸εςοϓ1ͰΑΓଟ͘ͷϝτϦοΫΛ࡟ݮͰ͖͍ͯΔ
    

    View Slide

33. 33
    ᶅߴ଎ੑͷධՁ: ֤ॲཧεςοϓͷ࣮ߦ࣌ؒ
    ɾCPUίΞ਺4ɺϝτϦοΫ਺100kͷ؀ڥ
    ɾTSifter͸ϕʔεϥΠϯʹରͯ͠ɺ311ഒߴ଎ͱͳͬͨ
    ɾʢޙड़ͷ௥Ճ࣮ݧͰ͸ɺ࠷௿Ͱ΋270ഒߴ଎ʣ
    εςοϓ1 (sec)
    ࣄલআڈ
    εςοϓ2 (sec)
    ΫϥελϦϯά
    ߹ܭ࣮ߦ࣌ؒ (sec)
    TSifter 54.41 8.68 63.09
    ϕʔεϥΠϯ 32.33 19590.83 19623.16
    

    View Slide

34. 34
    ɾ྆ख๏ͱ΋ʹɺCPUίΞ਺·ͨ͸ܥྻ਺ʹରͯ͠ɺઢܗʹεέʔϧ
    ᶅߴ଎ੑͷධՁ: εέʔϥϏϦςΟ
    TSifter ϕʔεϥΠϯ
    0
    20
    40
    60
    20000 40000 60000 80000 100000
    Execution time (sec)
    Number of metrics
    Clustering
    1.21
    2.43
    3.81
    5.72
    8.68
    Filtering
    10.24
    20.28
    31.05
    42.14
    54.41
    Total
    11.45
    22.71
    34.86
    47.86
    63.09
    0
    5000
    10000
    15000
    20000
    20000 40000 60000 80000 100000
    Execution time (sec)
    Number of metrics
    Clustering
    3908.10
    7773.00
    11710.26
    15670.81
    19590.83
    Filtering
    2.88 7.63 13.54 22.91 32.33
    Total
    3910.98
    7780.63
    11723.80
    15693.72
    19623.16
    0
    200
    400
    600
    800
    1000
    1200
    1400
    1 2 3 4
    Execution time (sec)
    Number of CPU cores
    Clustering
    1224.87
    613.31
    416.55
    317.65
    Filtering
    0.17 0.17 0.17 0.17
    Total
    1225.04
    613.48
    416.72
    317.82
    0
    1
    2
    3
    4
    1 2 3 4
    Execution time (sec)
    Number of CPU cores
    Clustering
    0.37
    0.21
    0.20
    0.15
    Filtering
    3.57
    1.81
    1.26
    0.99
    Total
    3.93
    2.02
    1.46
    1.14
    TSifter ϕʔεϥΠϯ
    

    View Slide

35. 35
    ࣮ߦ࣌ؒ͸1෼Ҏ಺͕ཧ૝Ͱ͋ΓɺϕʔεϥΠϯख๏ͷ࣮ߦ
    ࣌ؒ͸1225ඵʢ20෼ʣͰ͋Γɺݱ৔Ͱͷཁ݅Λຬͨͤͳ͍
    ֤ཁ݅ʹର͢ΔධՁͷ·ͱΊ
    ᶃਖ਼֬ੑ
    ᶄ࣍ݩ
    ࡟ݮ཰
    ᶅߴ଎ੑ
    ࣮ݧͰ͸ɺαʔϏεͷछྨ΍ނোέʔε͕ݶఆతͳͨΊɺ
    ௥ՃͷධՁ͕ඞཁ
    ࣍ݩ࡟ݮ཰͸ϕʔεϥΠϯख๏͕Θ͔ͣʹ্ճΔ
    ࠷ऴతʹཁٻ͞ΕΔ࣍ݩ࡟ݮ཰ͷఔ౓͸ࠓޙͷ՝୊
    CPUίΞ਺ͱܥྻ਺͕มԽͯ͠΋ɺ྆ख๏ͷ࣮ߦ࣌ؒൺ͸ಉ
    ౳
    

    View Slide

36. 36
    ͳͥϕʔεϥΠϯख๏ʹରͯ͠ߴ଎ͳͷ͔ʁ
    ϕʔεϥΠϯ TSifter
    ࠷దͳΫϥελ਺Λܾఆ͢ΔͨΊʹ
    ܁Γฦ࣮͠ߦ
    ΫϥελϦϯά࣮ߦճ਺͸310ճ
    ֊૚తΫϥελϦϯά
    ΫϥελϦϯά
    ࣮ߦճ਺͸7ճ
    ڑ཭ͷᮢ஋Λઃఆͯ͠
    Ϋϥελ਺Λܾఆ
    

    View Slide

37. 4.
    ·ͱΊͱࠓޙͷల๬
    

    View Slide

38. 38
    ɾҟৗͷݕ஌ʹ൓Ԡͯ͠ɺେྔͷϝτϦοΫ͔ΒʮҰ࣌తʹʯ਍அʹ༗༻ͳ
    ϝτϦοΫΛߴ଎ʹநग़͢ΔͨΊͷ࣍ݩ࡟ݮख๏ΛఏҊ
    ɾ࣮ݧͷൣғ಺Ͱ͸ɺϕʔεϥΠϯʹରͯ͠ɺ࠷௿Ͱ΋270ഒͷߴ଎ԽΛୡ੒
    ɾਖ਼֬ੑɺ࣍ݩ࡟ݮ཰ɺεέʔϥϏϦςΟͰ͸ಉ౳ఔ౓
    ɾ10 ສϝτϦοΫʹରͯ͠1෼ఔ౓ͷ࣌ؒͰ࣮ߦՄೳ
    ·ͱΊͱࠓޙͷల๬
    ɾࠓޙͷల๬
    ɾఏҊͷྑ͕͞ΑΓ໌֬ͱͳΔධՁͷ௥ՃʢΑΓదͨ͠ϕʔεϥΠϯͷબ୒
    ͳͲʣ
    ɾTSifterΛ૊ΈࠐΜͩݪҼ਍அγεςϜͷ࣮ݱ
    

    View Slide

39. 3.
    ·ͱΊ
    

    View Slide

40. 40
    ɾෳࡶԽ͢ΔγεςϜʹରͯ͠ɺIronies of Automationͷڭ܇Λൽ੾
    Γʹɺೝ஌ෛՙΛ௿ݮͤ͞Δ͜ͱ͕ॏཁͱͳΔ
    ɾΤϥʔΛڐ༰ => SRE => Chaos Engineering
    ɾೝ஌ͷࣗಈԽ => AI => ҟৗൃੜ͔Βճ෮·ͰΛAIͰղܾ
    ɾ࠷ॳͷऔΓ૊Έͱͯ͠ɺ࠷৽ͷݚڀ੒ՌͰ͋Δ࣌ܥྻσʔλͷ࣍ݩ
    ࡟ݮख๏Λ঺հͨ͠
    ·ͱΊ
    

    View Slide

41. 41
    ɾAIٕज़Λద༻͠Α͏ͱ͢Δͱɺࠓ·Ͱʹͳ͍ύλʔϯͷෛՙ͕ൃੜ
    ͢ΔՄೳੑ͕͋Δ
    ɾͦͷෛՙΛड͚ࢭΊΔγεςϜιϑτ΢ΣΞͷݚڀ͕ඞཁͱͳΔ
    ɾྫ͑͹ɺશܥྻΛऔಘ͢Δෛՙʹ࠷దԽͨ࣌͠ܥྻσʔλϕʔε
    ɾ͜Ε·ͰʹγεςϜιϑτ΢ΣΞͷݚڀ΋ߦ͖ͬͯͨܦݧΛ׆༻
    ɾҟछ෼ࢄKVSؒͷࣗಈ֊૚ԽʹΑΔߴੑೳͳ࣌ܥྻσʔλϕʔε
    ɾ෼ࢄͨ͠OSϓϩηεؒͷωοτϫʔΫґଘؔ܎ͷ௥੻
    γεςϜιϑτ΢ΣΞͷݚڀ
    

    View Slide

42. 42
    ͘͞Β΁ͷߩݙՄೳੑ
    ɾର֎తͳϓϨθϯεͷൃش
    ɾ࠷ઌ୺ٕज़ʹؔ͢Δ஌ࣝͷఏڙ
    ؒ઀తߩݙ
    ௚઀తߩݙ
    ɾࣾ಺ͷγεςϜӡ༻΁ͷ૊ΈࠐΈʢӡ༻ΤϯδχΞ΁ͷߩݙʣ
    ɾαʔϏε։ൃ΁ͷద༻ʢ͓٬༷΁ͷߩݙʣ
    ݚڀॴ͕ओମͰ΍Δͷ͸೉͍͠
    ࢀߟʮΠϯϑϥͷاۀݚڀͷՁ஋ͱ͜Ε͔Βʯhttps://speakerdeck.com/matsumoto_r/inhurafalseqi-ye-yan-jiu-falsejia-zhi-tokorekaraɹ
    ·ͣ͸Α͍ؔ܎ੑΛங͍͍ͯ͘͜ͱ͔Β
    ΋͏Ͱ͖͍ͯΔ
    

    View Slide