分散システムの性能異常に対する機械学習の解釈性に基づく原因診断手法 / A Method for Diagnosing the Causes of Performance Issues in Distributed Systems Based on the Interpretability of Machine Learning

Transcript

1. ͘͞ΒΠϯλʔωοτגࣜձࣾ
   (C) Copyright 1996-2019 SAKURA Internet Inc
   ͘͞ΒΠϯλʔωοτݚڀॴ
   ෼ࢄγεςϜͷੑೳҟৗʹର͢Δ
   ػցֶशͷղऍੑʹجͮ͘ݪҼ਍அख๏
   2019/12/06
   ୈ14ճ Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ (IOTS 2021)
   ɹ௽ా തจɼ௶಺ ༎थ
   ͘͞ΒΠϯλʔωοτגࣜձࣾ ͘͞ΒΠϯλʔωοτݚڀॴ
   

   View Slide

2. 2
   1. എܠͱ໨త
   2. ػցֶशͷղऍੑ
   3. ఏҊ͢ΔݪҼ਍அख๏
   4. ධՁͱߟ࡯
   5. ·ͱΊͱࠓޙͷల๬
   ໨࣍
   

   View Slide

3. 1.
   എܠͱ໨త
   

   View Slide

4. 4
   γεςϜͷෳࡶԽͱӡ༻ͷ՝୊
   γεςϜͷෳࡶԽ
   ӡ༻ͷ՝୊
   γεςϜ؅ཧऀͷೝ஌ෛՙ͕ߴ·Δ
   ϝτϦοΫͷܥྻ਺ͷ૿େ
   ϝτϦοΫͷܥྻ਺ͷมԽ
   γεςϜ؅ཧऀ͸ɼ֤ߏ੒ཁૉͷੑೳΛఆྔతʹࣔ͢ࢦඪ (Ҏ߱ɼϝτϦοΫ) Λऩू͠ɼ
   γεςϜͷঢ়ଶͷ؂ࢹʹ׆༻͍ͯ͠Δɽ
   ߏ੒ཁૉ਺ͷมԽ
   ߏ੒ཁૉ਺ͷ૿େ
   • Ϋϥ΢υ্ͷγεςϜ͸ɼෳ਺ͷҟͳΔߏ੒ཁૉ (αʔό΍ίϯςφͳͲ) ͕૬ޓʹωοτϫʔΫ௨৴
   ͢Δ෼ࢄγεςϜͱͯ͠ߏ੒͞ΕΔɽ
   • ͜ΕΒͷγεςϜ͸ɼར༻ऀ͔Βͷଟ༷ͳཁٻʹԠ͑ΔͨΊʹҎԼͷ܏޲͕ݟΒΕΔɽ
   • γεςϜߏ੒͕ෳࡶԽ͠ɼߏ੒ཁૉ਺΍छྨ͕૿େ͍ͯ͠Δɽ
   • γεςϜ΁ͷมߋස౓͕ߴ͘ͳΓɼߏ੒ཁૉ਺΍ෛՙಛੑͷมԽ͕଎͘ͳ͍ͬͯΔɽ
   γεςϜʹҟৗ͕ൃੜͨ͠ࡍʹɼͦͷݪҼಛఆʹཁ͢Δ͕࣌ؒ௕͘ͳΓɼαʔϏεͷఀࢭ࣌ؒͷ૿େ
   ΍ػձଛࣦͳͲʹܨ͕ΔͨΊɼҟৗͷݪҼΛࣗಈత͔ͭਝ଎ʹ਍அͰ͖Δख๏͕ඞཁͰ͋Δɽ
   

   View Slide

5. 5
   ઌߦख๏ɿػցֶशΛ༻͍ͨΞϓϩʔν
   • طଘख๏Ͱ͸ɼֶशʹ௕͍࣌ؒΛཁ͢Δਂ૚ֶशϕʔεͷϞσϧ͕༻͍ΒΕ͍ͯΔͨΊɼࣄલʹϞσϧ
   Λֶश͓ͯ͘͠ඞཁ͕͋Δɽ
   • Ϟσϧͷֶशͷࡍͷೖྗಛ௃ྔͷ਺͸ݻఆͰ͋ΔͨΊɼϝτϦοΫͷܥྻ਺͕૿ݮͨ͠৔߹͸ɼ࠶౓ɼ
   ৽ͨͳϞσϧΛֶश͠ͳ͚Ε͹ͳΒͳ͍ɽ
   ΑͬͯɼγεςϜͷมߋʹਝ଎ʹ௥ैͨ͠ݪҼ਍அΛߦ͏͜ͱ͕೉͍͠ɽ
   ※3 L. Wu et al, Performance Diagnosis in Cloud Microservices using Deep Learning, International Conference on Service-Oriented Computing, 2020.
   ※4 H. Zhao et al, Multivariate Time-series Anomaly Detection via Graph Attention Network, arXiv:2009.02040 2020.
   MTAD-GATͷΞʔΩςΫνϟ※3,4
   2. ҟৗ౓ͷࢉग़
   3. ߩݙ౓ͷࢉग़
   ࣮ߦͷྲྀΕ
   2. ҟৗ౓ͷࢉग़
   ֶशޙͷϞσϧͷ༧ଌ஋ͱ࣮ଌ஋ͱͷޡࠩΛҟৗ౓
   ͱͯ͠ࢉग़
   3. ߩݙ౓ͷࢉग़
   ҟৗ౓ʹର͢Δೖྗಛ௃ྔ (ϝτϦοΫ)ͷߩݙ౓Λࢉग़
   1. Ϟσϧͷֶश
   ਖ਼ৗ࣌ͷϝτϦοΫͷܥྻΛػցֶशϞσϧΛ༻͍ͯ
   ֶश
   ػցֶशΛ༻͍ͯҟৗͷݪҼͱͳΔϝτϦοΫΛ਍அ͢Δख๏͕੝Μʹݚڀ͞Ε͓ͯΓɼߴ͍਍அਫ਼౓
   Λୡ੒͍ͯ͠Δ※3,4ɽ
   

   View Slide

6. 6
   ݚڀͷ໨త
   ຊൃදͰ͸ɼҟৗͷݕ஌ޙʹߴ଎ʹֶशՄೳͰ͋ΔܰྔͳػցֶशϞσϧͱɼػցֶशͷղऍख๏
   ͱͯ͠஫໨͞Ε͍ͯΔγϟʔϓϨΠ஋Λ༻͍ͯɼγεςϜߏ੒ͷมߋʹର͢Δ௥ैੑͱߴ͍਍அਫ਼
   ౓Λཱ྆ͨ͠ݪҼ਍அख๏ΛఏҊ͢Δɽ
   γεςϜߏ੒ͷมߋʹର͢Δ௥ैੑͱߴ͍਍அਫ਼౓Λཱ྆ͨ͠ݪҼ਍அख๏ͷఏҊΛ໨తͱ͢Δɽ
   • ߴ଎ʹֶश͕ՄೳͰ͋ΔܰྔͳػցֶशϞσϧΛ
   ༻͍ͯɼҟৗͷݕ஌ޙʹϞσϧͷֶशΛߦ͏ɽ
   • γεςϜͷมߋʹ௥ैͯ͠ϞσϧΛֶश͠௚͢ඞ
   ཁ͕ͳ͘ɼҟৗൃੜ࣌ͷγεςϜͷߏ੒Λ൓ө͠
   ͨݪҼ਍அ͕Ͱ͖Δ
   • ܰྔͳػցֶशϞσϧ͸ਂ૚ֶशΛ༻͍ͨϞσϧ
   ΑΓ΋දݱྗ͕௿͍ͨΊɼݪҼ਍அͷਫ਼౓͕௿͘
   ͳΔՄೳੑ͕͋Δɽ
   • ػցֶशͷղऍੑͷ෼໺Ͱ༗༻ੑ͕ࣔ͞Ε͍ͯΔ
   γϟʔϓϨΠ஋Λ༻͍ͯ਍அਫ਼౓ΛߴΊΒΕΔ͔
   ݕ౼͢Δɽ
   ఏҊͷ֓ཁ
   

   View Slide

7. 2.
   ػցֶशͷղऍੑ
   

   View Slide

8. 8
   ػցֶशͷղऍੑ
   • ਂ૚ֶशΛ͸͡Ίͱ͢ΔػցֶशϞσϧ͸ͦͷܭࢉաఔ͕ෳࡶͰ͋ΔͨΊɼϞσϧͷ༧ଌ΍൑அ
   ͷࠜڌΛਓ͕ؒཧղ͢Δ͜ͱ͕Ͱ͖ͳ͍ϒϥοΫϘοΫεͱͳΔ͜ͱ͕໰୊ࢹ͞Ε͓ͯΓɼػց
   ֶशϞσϧͷղऍੑʹ͍ͭͯͷݚڀ͕஫໨͞Ε͍ͯΔ※5ɽ
   • AIར׆༻ݪଇҊ (૯຿লɼ2018೥)
   • ಁ໌ੑͷݪଇɼΞΧ΢ϯλϏϦςΟ(આ໌੹೚)
   ͷݪଇ
   • DARPA (ถࠃ๷ߴ౳ݚڀܭըہ)
   • Explainable Artificial Intelligence (XAI)
   ϓϩδΣΫτ
   ※5 A. Adadi and M. Berrada, Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI), IEEE Access, 2018.
   ػցֶशͷղऍੑɾઆ໌ੑʹؔ͢Δ࿦จ਺ͷਪҠ※5
   • ಛʹɼػցֶशϞσϧͷ༧ଌʹର͢Δೖྗಛ௃ྔͷ
   ߩݙ౓Λఏࣔ͢Δख๏͕਺ଟ͘ఏҊ͞Ε͍ͯΔɽ
   

   View Slide

9. 9
   γϟʔϓϨΠ஋Λ༻͍ͨղऍ
   ※6 S. Lundberg and S. I. Lee, A Unified Approach to Interpreting Model Predictions, Advances in Neural Information Processing Systems 30(NIPS 2017), 2017.
   ※ https://github.com/slundberg/shap
   ڠྗήʔϜͱػցֶशͷରൺ
   ήʔϜ
   ϓϨΠϠʔ རಘ ಛ௃ྔ ༧ଌ஋
   Ϟσϧ
   ػցֶश
   ϓϨʔϠʔΛಛ௃ྔɼརಘΛ༧ଌ஋ʹஔ͖׵͑
   ը૾ͷղऍͷྫ※
   • ୅දతͳղऍख๏ͱͯ͠ɼڠྗήʔϜཧ࿦ͷγϟʔϓϨΠ஋Λ༻͍ͨղऍख๏͕͋Δ※6ɽ
   • γϟʔϓϨΠ஋ͱ͸ɼڠྗήʔϜʹ͓͍ͯෳ਺ϓϨΠϠʔͷڠྗʹΑͬͯಘΒΕͨརಘΛ֤ϓϨΠϠʔͷ
   ߩݙ౓ʹԠͯ͡ެਖ਼ʹ෼഑͢ΔͨΊͷखஈͷҰͭ
   • γϟʔϓϨΠ஋Ͱ͸ɼ͋ΒΏΔϓϨΠϠʔͷ૊Έ߹ΘͤͰಘΒΕΔརಘΛݩʹɼଞͷϓϨΠϠʔͱͷڠྗ
   ʹΑΓൃੜ͢ΔརಘͷӨڹΛഉআͨ͠ಛఆͷϓϨΠϠʔͷߩݙ౓Λࢉग़
   

   View Slide

10. 10
    ղऍੑΛ༻͍ͨҟৗͷݪҼ਍அ
    • Ϟσϧͱͯ͠ɼೖྗσʔλʹର͢Δҟৗ౓Λग़ྗ͢ΔϞσϧΛ༻͍ͨ৔߹ɼγϟʔϓϨΠ஋Λ༻͍ͯɼ
    ҟৗ౓ʹର͢Δ֤ಛ௃ྔͷߩݙ౓ΛٻΊΒΕΔͨΊɼҟৗͷݪҼಛఆʹ׆༻Ͱ͖Δɽ
    • γϟʔϓϨΠ஋Λ༻͍ͨղऍख๏͸ɼҟৗͷݪҼ਍அʹ༗༻Ͱ͋Δ͜ͱΛࣔ݁͢Ռ͕ใࠂ͞Ε͍ͯΔ※7,8ɽ
    ಛ௃ྔ ༧ଌ஋
    Ϟσϧ
    ػցֶश
    ϝτϦοΫ (ex. CPU usage)
    ҟৗ౓
    • γϟʔϓϨΠ஋͸ɼಛ௃ྔͷ਺͕૿͑ΔʹͭΕͯܭࢉྔ͕๲େʹͳΔɽ
    • ଈ࣌ੑ͕ٻΊΒΕΔҟৗͷݪҼ਍அʹ͓͍ͯɼγϟʔϓϨΠ஋ͷܭࢉ͕
    ࣮༻తͳ࣌ؒ಺ʹܭࢉՄೳͰ͋Δ͔ͷٞ࿦͸ߦΘΕ͍ͯͳ͍ɽ
    γϟʔϓϨΠ஋ΛݪҼ਍அʹ༻͍Δࡍͷ՝୊
    ※7 L. Antwarg et al., Explaining Anomalies Detected by Autoencoders Using SHAP, arXiv:1903.02407, 2019.
    ※8 N. Takeishi and Y. Kawahara, On Anomaly Interpretation via Shapley Values, arXiv:2004.04464, 2020.
    

    View Slide

11. 3.
    ఏҊ͢ΔݪҼ਍அख๏
    

    View Slide

12. 12
    ΞʔΩςΫνϟ֓ཁ
    ఏҊ͢ΔݪҼ਍அख๏ͷ֓ཁਤ
    4. γϟʔϓϨΠ஋ʹΑΔ਍அ
    ࣮ߦͷྲྀΕ
    3. ҟৗ౓ͷࢉग़
    2. ػցֶशϞσϧͷֶश
    1. ϝτϦοΫͷऩूͱ෼ׂ
    0. ҟৗͷݕ஌
    

    View Slide

13. 13
    ΞʔΩςΫνϟ֓ཁ
    ఏҊ͢ΔݪҼ਍அख๏ͷ֓ཁਤ
    4. γϟʔϓϨΠ஋ʹΑΔ਍அ
    ࣮ߦͷྲྀΕ
    3. ҟৗ౓ͷࢉग़
    2. ػցֶशϞσϧͷֶश
    1. ϝτϦοΫͷऩूͱ෼ׂ
    0. ҟৗͷݕ஌
    • ػցֶशϞσϧ͸ɼݱࡏҟৗݕ஌ʹ޿͘ར༻͞Ε͍ͯΔ͕ɼఏҊख๏Ͱ͸ɼػցֶशϞσϧΛ༻͍ͯ
    ҟৗݕ஌ΛߦΘͣɼҟৗ͕ݕ஌͞ΕͨޙͷݪҼ਍அΛߦ͏͜ͱʹয఺Λ౰͍ͯͯΔɽ
    • ҟৗݕ஌ͷͨΊʹ͸ɼαʔϏεͷ৴པੑʹؔ͢Δ໨ඪ஋Ͱ͋ΔSLOʢService Level Objectiveʣ΍
    ϝτϦοΫͷܥྻ͝ͱʹઃఆͨ͠ᮢ஋ͳͲΛ༻͍Δɽ
    

    View Slide

14. 14
    ΞʔΩςΫνϟ֓ཁ
    ఏҊ͢ΔݪҼ਍அख๏ͷ֓ཁਤ
    4. γϟʔϓϨΠ஋ʹΑΔ਍அ
    ࣮ߦͷྲྀΕ
    3. ҟৗ౓ͷࢉग़
    2. ػցֶशϞσϧͷֶश
    1. ϝτϦοΫͷऩूͱ෼ׂ
    0. ҟৗͷݕ஌
    • ҟৗ͕ݕ஌͞Εͨ৔߹ɼγεςϜͷ֤ߏ੒ཁૉ͔ΒಘΒΕΔϝτϦοΫΛݕ஌ͷ͔࣌ؒΒҰఆظؒ
    Ḫͬͯऔಘ͢Δɽ
    • औಘͨ͠ෳ਺ͷϝτϦοΫͷܥྻʹ͸ɼҟৗ࣌ͷϝτϦοΫͷܥྻͱɼͦͷ௚ۙͷਖ਼ৗ࣌ͷϝτ
    ϦοΫͷܥྻؚ͕·Ε͍ͯΔͨΊɼ্ਤͷΑ͏ʹϝτϦοΫͷܥྻΛ࣌ؒͷ࣠ʹରͯ͠ਖ਼ৗ࣌ͱҟ
    ৗ࣌ʹ෼ׂ͢Δɽ
    

    View Slide

15. 15
    ΞʔΩςΫνϟ֓ཁ
    ఏҊ͢ΔݪҼ਍அख๏ͷ֓ཁਤ
    4. γϟʔϓϨΠ஋ʹΑΔ਍அ
    ࣮ߦͷྲྀΕ
    3. ҟৗ౓ͷࢉग़
    2. ػցֶशϞσϧͷֶश
    1. ϝτϦοΫͷऩूͱ෼ׂ
    0. ҟৗͷݕ஌
    • ෼ׂͨ͠σʔλͷ͏ͪɼਖ਼ৗ࣌ͷϝτϦοΫͷΈΛֶशσʔλͱͯ͠ɼػցֶशϞσϧͷڭࢣͳ͠
    ֶशΛߦ͏ɽ
    

    View Slide

16. 16
    ΞʔΩςΫνϟ֓ཁ
    ఏҊ͢ΔݪҼ਍அख๏ͷ֓ཁਤ
    4. γϟʔϓϨΠ஋ʹΑΔ਍அ
    ࣮ߦͷྲྀΕ
    3. ҟৗ౓ͷࢉग़
    2. ػցֶशϞσϧͷֶश
    1. ϝτϦοΫͷऩूͱ෼ׂ
    0. ҟৗͷݕ஌
    • ҟৗ࣌ͷϝτϦοΫΛֶशࡁΈͷϞσϧʹೖྗ͠ɼϞσϧ͔Βͷग़ྗͱͯ͠ҟৗ౓ΛಘΔɽ
    

    View Slide

17. 17
    ΞʔΩςΫνϟ֓ཁ
    ఏҊ͢ΔݪҼ਍அख๏ͷ֓ཁਤ
    4. γϟʔϓϨΠ஋ʹΑΔ਍அ
    ࣮ߦͷྲྀΕ
    3. ҟৗ౓ͷࢉग़
    2. ػցֶशϞσϧͷֶश
    1. ϝτϦοΫͷऩूͱ෼ׂ
    0. ҟৗͷݕ஌
    • ҟৗ౓ʹର͢Δ֤ϝτϦοΫͷߩݙ౓ΛγϟʔϓϨΠ஋Λ༻͍ͯࢉग़͠ɼࢉग़ͨ͠ߩݙ౓Λ߱ॱʹ
    ੔ྻ͢Δ͜ͱͰɼҟৗͷݪҼͱͳΔϝτϦοΫͷީิΛϥϯΫ෇͚͢Δɽ
    • ϥϯΫ෇͚ͨ͠ϝτϦοΫΛ਍அ݁Ռͱͯ͠ɼγεςϜ؅ཧऀʹఏࣔ͢Δɽ
    

    View Slide

18. 18
    ΞʔΩςΫνϟ֓ཁ
    ఏҊ͢ΔݪҼ਍அख๏ͷ֓ཁਤ
    4. γϟʔϓϨΠ஋ʹΑΔ਍அ
    ࣮ߦͷྲྀΕ
    3. ҟৗ౓ͷࢉग़
    2. ػցֶशϞσϧͷֶश
    1. ϝτϦοΫͷऩूͱ෼ׂ
    0. ҟৗͷݕ஌
    2ͭͷεςοϓʹ͍ͭͯ
    ΑΓৄࡉʹઆ໌͢Δ
    

    View Slide

19. 19
    ػցֶशϞσϧͷֶश
    • ਝ଎ʹ਍அ݁ՌΛఏࣔ͢ΔͨΊʹɼϞσϧͷֶशΛߴ଎ʹ࣮ߦՄೳͰ͋Δ͜ͱɽ
    • Ϟσϧ͸ೖྗσʔλʹରͯ͠ɼҟৗ౓Λฦ͢Α͏ͳϞσϧͰ͋Δ͜ͱɽ
    ఏҊख๏Ͱ͸ɼγεςϜߏ੒ͷมߋʹର͢Δ௥ैੑͷཁ݅Λຬͨͨ͢ΊʹɼػցֶशϞσϧͷֶश
    ΛࣄલʹߦΘͣʹҟৗݕ஌ޙʹߦ͏ΞʔΩςΫνϟΛ࠾༻͢Δɽ
    Ϟσϧʹର͢Δཁ݅
    Ϟσϧͷީิ
    • ্هͷཁ݅Λຬͨ͢Ϟσϧͱͯ͠ɼओ੒෼෼ੳ (PCA) ϕʔεͷϞσϧ͕ڍ͛ΒΕΔɽ
    • PCA͸ɼߴ࣍ݩͷσʔλΛΑΓ௿࣍ݩͷσʔλʹѹॖ͢Δ࣍ݩ
    ѹॖख๏ͷҰͭͰ͋Δɽ
    • ਖ਼ৗ࣌ͷσʔλͷΈΛ༻͍ͯɼ௿࣍ݩ΁ͷѹॖํ๏Λֶश͠ɼ
    ֶशޙʹ༩͑ΒΕΔಛఆͷσʔλʹରͯ͠ɼҰ౓࣍ݩΛѹॖ͠
    ͨޙʹ࠶౓ݩͷ࣍ݩʹσʔλΛ෮ݩͨ͠ࡍͷޡࠩ (࠶ߏ੒ޡࠩ)
    Λҟৗ౓ͱͯ͠ղऍͰ͖Δɽ
    ѹॖ ෮ݩ
    ೖྗ
    ϝτϦοΫ
    m1
    m2
    mN
    ⋅ ⋅ ⋅
    m′
    1
    m′
    2
    m′
    N
    ⋅ ⋅ ⋅
    e1
    = m′
    1
    − m1
    ࠶ߏ੒ޡࠩ
    e2
    eN
    

    View Slide

20. 20
    γϟʔϓϨΠ஋ʹΑΔ਍அ
    • ਝ଎ʹ਍அ݁ՌΛఏࣔ͢ΔͨΊʹɼγϟʔϓϨΠ஋ͷܭࢉΛߴ଎ʹߦ͏͜ͱɽ
    • γϟʔϓϨΠ஋͸ɼಛ௃ྔͷ਺͕૿େ͢ΔʹͭΕͯݫີͳܭࢉ͕ෆՄೳͰ͋ΔͨΊɼγϟʔϓϨΠ஋
    Λߴ଎͔ͭਖ਼֬ʹۙࣅ͢Δख๏͕ඞཁɽ
    γϟʔϓϨΠ஋ͷܭࢉʹٻΊΒΕΔཁ݅
    γϟʔϓϨΠ஋ͷۙࣅख๏
    • ্هͷཁ݅Λຬͨ͢γϟʔϓϨΠ஋ͷۙࣅख๏ͱͯ͠ɼSHAP͕ڍ͛ΒΕΔɽ
    • SHAPʹ͸ɼ͍͔ͭ͘ͷۙࣅΞϧΰϦζϜ͕ଘࡏ͢Δ͕ɼͦͷதͰ΋͋ΒΏΔػցֶशϞσϧʹ
    ద༻ՄೳͳKernel SHAP※6͕༗ޮͰ͋Δɽ
    • Kernel SHAP͸ɼॏΈ෇͖ͷ࠷খೋ৐ઢܗճؼʹΑΓγϟʔϓϨΠ஋ΛۙࣅͰ͖Δɽ
    ఏҊख๏Ͱ͸ɼߴ͍਍அਫ਼౓Λୡ੒͢ΔͨΊʹɼҟৗ౓ʹର͢Δ֤ϝτϦοΫͷߩݙ౓ΛγϟʔϓϨΠ
    ஋Λ༻͍ͯࢉग़͠ɼҟৗͷݪҼͱͳΔϝτϦοΫΛಛఆ͢ΔΞʔΩςΫνϟΛ࠾༻͢Δ
    ※6 S. Lundberg and S. I. Lee, A Unified Approach to Interpreting Model Predictions, Advances in Neural Information Processing Systems 30(NIPS 2017), 2017.
    

    View Slide

21. 4.
    ධՁͱߟ࡯
    

    View Slide

22. ࣮ݧͷ؀ڥ
    22
    ※ ௶಺༎थ, ੨ࢁਅ໵, MeltriaɿϚΠΫϩαʔϏεʹ͓͚Δҟৗݕ஌ɾݪҼ෼ੳͷͨΊͷσʔληοτͷಈతੜ੒γεςϜ, Πϯλʔωοτͱӡ༻ٕज़γϯϙδ΢Ϝ࿦จू,
    2021, 63-70 (2021-11-18), 2021೥11݄
    Fron-end
    Catalogue Orders Carts
    User
    Payment
    Shipping
    Sock Shop
    Locust
    Prometheus
    ϚΠΫϩαʔϏεΫϥελ ੍ޚαʔό
    ֎෦ෛՙͷੜ੒
    ϝτϦοΫͷ
    ऩूɾอଘ
    ղੳαʔό
    ϝτϦοΫ
    औಘϞδϡʔϧ
    ղੳϞδϡʔϧ
    4core, 24GB
    • KubernetesΫϥελͷࣗಈ؅ཧαʔϏεͰ͋ΔGKE (Google Kubernetes Engine)্ʹϚΠΫϩαʔϏεͷ
    ϕϯνϚʔΫΞϓϦέʔγϣϯͰ͋ΔSock ShopΛߏஙͨ͠ɽ
    • Sock ShopΛߏ੒͢Δίϯςφ͔ΒcAdvisorΛ༻͍ͯCPU࢖༻཰ͳͲͷϝτϦοΫΛ15ඵִؒͰऩूͨ͠ɽ
    • ੑೳҟৗΛٖࣅతʹੜ੒͢ΔͨΊʹɼCPUաෛՙ͓ΑͼϝϞϦϦʔΫΛಛఆͷίϯςφ্Ͱ࠶ݱͨ͠ɽ
    ࠶ݱํ๏ͷৄࡉ͸ɼIOTS2021ͰൃදͷMeltriaͷ࿦จ※Λࢀর
    

    View Slide

23. ࣮ݧͷઃఆ
    23
    ධՁ಺༰ͱࢦඪ
    1. ݪҼͷ਍அਫ਼౓
    2. ਍அʹཁ͢Δ࣌ؒ (Ҏ߱ɼ਍அ࣌ؒ)
    Top-kਫ਼౓ɿݪҼͱͳΔϝτϦοΫͷܥྻ্͕Ґk൪໨·Ͱʹ͋Δ֬཰ʢk=1͓Αͼk=3ʣ
    ਍அʹ༻͍ΔϝτϦοΫ
    ਍அʹ༻͍ΔίϯςφͷϝτϦοΫ
    ֤ίϯςφ͔Βӈදʹڍ͛ͨ6ݸͷϝτϦοΫΛऔಘ
    ҟৗΛ࠶ݱޙɼ5෼ܦաͨ͠ͷͪʹ௚ۙ30෼ؒͷίϯςφ
    ͷϝτϦοΫΛऔಘ
    ݪҼ਍அख๏
    ఏҊख๏ɿػցֶशϞσϧʹPCAΛɼγϟʔϓϨΠ஋ͷܭࢉʹKernel SHAPΛ༻͍ͨ (PCA+SHAP)
    1. Gaussian Based Thresholding (GBT)Λ༻͍ͨख๏ (GBT)
    2. PCAͷ࠶ߏ੒ޡࠩΛ༻͍ͨख๏ (PCA+࠶ߏ੒ޡࠩ)
    ϝτϦοΫͷܥྻ͝ͱͷਖ਼ৗ࣌ͷฏۉ஋ͱҟৗ࣌ͷฏۉ஋ͷࠩ෼Λҟৗ
    ΁ͷߩݙ౓ͱͯ͠ݪҼ਍அΛߦ͏ɽ
    PCA͕ग़ྗ͢ΔϝτϦοΫ͝ͱͷ༧ଌ஋ͱ࣮ଌ஋ͱͷޡࠩͰ͋Δ
    ࠶ߏ੒ޡࠩΛҟৗ΁ͷߩݙ౓ͱͯ͠ݪҼ਍அΛߦ͏ɽ
    ϕʔεϥΠϯͱ͢Δख๏ɿ
    ѹॖ ෮ݩ
    ೖྗ
    ϝτϦοΫ
    ૯࿨
    m1
    m2
    mN
    ⋅ ⋅ ⋅
    m′
    1
    m′
    2
    m′
    N
    ⋅ ⋅ ⋅
    N
    ∑
    i
    ei
    SHAP
    e1
    = m′
    1
    − m1
    ࠶ߏ੒ޡࠩ
    e2
    eN
    

    View Slide

24. 24
    ਍அਫ਼౓ͷධՁ
    ݁Ռͱߟ࡯ɿ਍அਫ਼౓ͷൺֱ
    ධՁ݁Ռ
    શͯͷ਍அظؒʹ͓͍ͯ਍அਫ਼౓͸ɼPCA +SHAP (ఏҊख๏) > PCA+࠶ߏ੒ޡࠩ > GBT
    Top-1ਫ਼౓ Top-3ਫ਼౓ ਍அظؒͷఆٛ
    ϝτϦοΫͷऔಘظؒͷ͏ͪɼҟৗ࣌ͷϝτϦοΫͷ
    ࣌ؒൣғʢҎ߱ɼ਍அظؒʣΛมԽͤͯ͞਍அਫ਼౓Λ
    ධՁͨ͠ɽ
    

    View Slide

25. 25
    ਍அਫ਼౓ͷධՁ
    ݁Ռͱߟ࡯ɿ਍அਫ਼౓ͷൺֱ
    ධՁ݁Ռ
    શͯͷ਍அظؒʹ͓͍ͯ਍அਫ਼౓͸ɼPCA +SHAP (ఏҊख๏) > PCA+࠶ߏ੒ޡࠩ > GBT
    • GBTͰ͸ɼϝτϦοΫ͝ͱʹฏۉ஋ͷมԽΛݟΔख๏Ͱ͋ΓɼϝτϦοΫͷܥྻؒͷ૬ؔΛऔΓѻΘͳ͍ɽ
    • Ұํɼίϯςφಉ࢜͸ɼ૬ޓͷωοτϫʔΫ௨৴΍ɼϗετͱͳΔϊʔυͷڞ༗ͷͨΊɼϝτϦοΫͷܥྻؒ
    ʹ૬ؔΛ࣋ͭɽ
    • PCAΛ༻͍Δ͜ͱͰɼϝτϦοΫͷܥྻؒͷ૬ؔΛߟྀͨ͠਍அ͕ՄೳͱͳΔɽ
    Top-1ਫ਼౓ Top-3ਫ਼౓ ਍அظؒͷఆٛ
    ϝτϦοΫͷऔಘظؒͷ͏ͪɼҟৗ࣌ͷϝτϦοΫͷ
    ࣌ؒൣғʢҎ߱ɼ਍அظؒʣΛมԽͤͯ͞਍அਫ਼౓Λ
    ධՁͨ͠ɽ
    

    View Slide

26. 26
    ਍அਫ਼౓ͷධՁ
    ݁Ռͱߟ࡯ɿ਍அਫ਼౓ͷൺֱ
    ධՁ݁Ռ
    શͯͷ਍அظؒʹ͓͍ͯ਍அਫ਼౓͸ɼPCA +SHAP (ఏҊख๏) > PCA+࠶ߏ੒ޡࠩ > GBT
    • PCA+࠶ߏ੒ޡࠩͰ͸ɼҟৗͷݪҼͱͳΔϝτϦοΫͷܥྻ͕มಈͨ͠৔߹ɼਖ਼ৗ࣌ʹ૬͕ؔߴ͔ͬͨϝτ
    ϦοΫͷ࠶ߏ੒ޡࠩ΋େ͖͘ͳΓɼҟৗ΁ͷߩݙ౓͕ߴ͘ݟੵ΋ΒΕΔ͜ͱ͕͋Δɽ
    • SHAPΛ༻͍ͨख๏Ͱ͸ɼγϟʔϓϨΠ஋ͷੑ࣭ʹΑΓɼਖ਼ৗ࣌ͷ૬ؔͷߴ͞ʹΑΔӨڹΛഉআͯ͠ɼݸʑͷ
    ϝτϦοΫͷߩݙ౓ΛܭࢉͰ͖Δɽ
    Top-1ਫ਼౓ Top-3ਫ਼౓
    ϝτϦοΫͷऔಘظؒͷ͏ͪɼҟৗ࣌ͷϝτϦοΫͷ
    ࣌ؒൣғʢҎ߱ɼ਍அظؒʣΛมԽͤͯ͞਍அਫ਼౓Λ
    ධՁͨ͠ɽ
    ਍அظؒͷఆٛ
    

    View Slide

27. 27
    ਍அਫ਼౓ͷධՁ
    ݁Ռͱߟ࡯ɿ਍அظؒͷӨڹ
    ධՁ݁Ռ
    • ਍அظؒͷઃఆ͕਍அͷਫ਼౓ʹେ͖͘ӨڹΛ༩͑Δ͜ͱ͕Θ͔ͬͨɽ
    • ਍அظؒ < 5෼ɿػցֶशϞσϧʹ༩͑Δֶशσʔλʹҟৗ࣌ͷσʔλ͕
    ؚ·ΕΔͨΊɼϞσϧͷֶश͕͏·͘ߦ͑ͣʹਫ਼౓͕௿Լ͢Δɽ
    • ਍அظؒ > 5෼ɿϞσϧ͸ਖ਼ৗ࣌ͷσʔλͷΈͰֶशͰ͖Δ΋ͷͷɼҟৗ
    ΁ͷߩݙ౓ͷܭࢉʹਖ਼ৗ࣌ͷσʔλؚ͕·ΕΔ͜ͱʹΑΓਫ਼౓͕௿Լ͢Δ
    ਍அظؒΛ5෼ͱͨ͠৔߹͕ɼ֤ܥྻΛ࠷΋Α͘ਖ਼ৗ
    ࣌ͱҟৗ࣌ʹ෼ׂͰ͖Δ
    Top-1ਫ਼౓ Top-3ਫ਼౓ ਍அظؒͷఆٛ
    औಘͨ͠ϝτϦοΫ͔ΒదԠతʹ
    ਍அظؒΛઃఆ͢Δख๏ͷݕ౼͕
    ࠓޙͷ՝୊
    ϝτϦοΫͷऔಘظؒͷ͏ͪɼҟৗ࣌ͷϝτϦοΫͷ
    ࣌ؒൣғʢҎ߱ɼ਍அظؒʣΛมԽͤͯ͞਍அਫ਼౓Λ
    ධՁͨ͠ɽ
    

    View Slide

28. 28
    ਍அ࣌ؒͷධՁɿCPUίΞ਺
    ධՁ݁Ռ
    • CPUίΞ਺͕૿Ճ͢ΔʹͭΕͯఏҊख๏ͷ਍அ͕࣌ؒ୹͘ͳͬͨɽ
    • PCAͷֶश͸2 msͰ͋Γɼ਍அ࣌ؒʹରͯ͠ͷӨڹ͕ແࢹͰ͖Δ΄Ͳখ͍͞ɽ͕ͨͬͯ͠ɼఏҊख๏ͷ਍அ࣌ؒ
    ͸ɼSHAPʹΑΔߩݙ౓ͷܭࢉʹཁ͢Δ࣌ؒͱ΄΅ಉ౳Ͱ͋Δɽ
    ਍அظؒɿ5෼
    ϝτϦοΫͷܥྻ਺ɿ66 (11ίϯςφ×6ϝτϦοΫ)
    Kernel SHAPͷܭࢉɿϚϧνίΞͰฒྻܭࢉ͕Մೳͳ
    ShapPack※Λར༻
    CPUίΞ਺ʹର͢Δ਍அ࣌ؒͷมԽ
    ਍அ࣌ؒͷ಺༁ (CPUίΞ਺ɿ4)
    ࣮ݧͷઃఆ
    ※ https://github.com/tsurubee/shappack
    ਍அ࣌ؒɿҟৗΛݕ஌͔ͯ͠ΒɼγεςϜ؅ཧऀʹ
    ɹɹɹɹɹ਍அ݁ՌΛఏࣔ͢Δ·Ͱͷ࣌ؒ
    

    View Slide

29. 29
    ਍அ࣌ؒͷධՁɿϝτϦοΫͷܥྻ਺
    ධՁ݁Ռ
    • ϝτϦοΫͷܥྻ਺ͷ૿Ճʹ൐͍ɼ਍அ͕࣌ؒ૿Ճͨ͠ɽ
    • ίϯςφ਺250ͷγεςϜͷ৔߹ɼ਍அ࣌ؒ͸3෼17ඵͰ͋Δɽ
    • ίϯςφ਺300ͷγεςϜͷ৔߹ɼ਍அ࣌ؒ͸5෼Λ௒͑Δ͜ͱ͕Θ͔ͬͨɽ
    ϝτϦοΫͷܥྻ਺ʹର͢Δ਍அ࣌ؒͷมԽ
    ਍அظؒɿ5෼
    CPUίΞ਺ɿ4
    ࣮ݧͷઃఆ
    • γεςϜ؅ཧऀ͕ҟৗݕ஌ͷ௨஌ʹؾ͖ͮɼखಈͰ෮چͷ࡞ۀ
    Λߦ͏࣌ؒΛߟྀ͢Δͱɼ਍அ࣌ؒΛඵ୯ҐͰ׬ྃͤ͞Δඞཁ
    ͸ͳ͍ɽ
    • Google CloudͷSLA (Service Level Agreement)※ʹΑΔͱɼ5෼
    Ҏ಺Ͱ͋Ε͹αʔϏεͷఀࢭظؒͱΈͳ͍ͯ͠ͳ͍
    ཧ૝తͳ਍அ࣌ؒΛ5෼Ҏ಺ʹઃఆ
    ※ https://cloud.google.com/compute/sla-20151016/
    ίϯςφ਺
    0 83 166 250 333
    ϝτϦοΫͷܥྻ਺ͷ૿Ճ͸γεςϜΛߏ੒͢Δίϯςφ਺
    ͷ૿ՃʹରԠ͢Δɽ
    SHAPʹΑΔߩݙ౓ͷܭࢉ
    ͷߴ଎Խ͸ࠓޙͷ՝୊
    

    View Slide

30. 5.
    ·ͱΊͱࠓޙͷల๬
    

    View Slide

31. 31
    ·ͱΊͱࠓޙͷల๬
    • ຊൃදͰ͸ɼҟৗͷݕ஌Λى఺ʹܰྔͳػցֶशϞσϧͷֶशΛߦ͍ɼγϟʔϓϨΠ஋Λ༻͍ͯϞ
    σϧ͔ΒಘΒΕΔҟৗ౓ʹର͢Δ֤ϝτϦοΫͷߩݙ౓Λࢉग़͢Δ͜ͱͰɼҟৗͷݪҼ਍அΛߦ͏
    ख๏ΛఏҊͨ͠ɽ
    • ϚΠΫϩαʔϏεͷςετϕου؀ڥʹͯҟৗΛ࠶ݱ͢Δ࣮ݧͷ݁Ռɼ਍அظؒͷઃఆ͕ద੾ʹߦ
    ͑ͨ৔߹ɼఏҊख๏͸ҟৗͷࠜຊݪҼͷϝτϦοΫͷܥྻΛ44.8%ͷਫ਼౓Ͱ্Ґ1Ґʹɼ82.3%ͷ
    ਫ਼౓Ͱ্Ґ3ҐҎ಺ʹಛఆͰ͖Δ͜ͱΛ֬ೝͨ͠ɽ
    • ఏҊख๏ͷ਍அਫ਼౓͸ɼ਍அظؒͷઃఆʹେ͖͘ґଘ͢Δ͜ͱ͕࣮ݧ͔Β໌Β͔ʹͳͬͨͨΊɼ
    ҟৗݕ஌ޙʹऔಘͨ͠ϝτϦοΫ͔ΒదԠతʹ਍அظؒΛઃఆ͢Δख๏Λݕ౼͢Δɽ
    • ఏҊख๏ΛΑΓߏ੒ཁૉ਺͕େ͖͍ن໛ͷγεςϜʹରԠͤ͞ΔͨΊʹɼSHAPʹΑΔߩݙ౓ͷܭ
    ࢉΛߴ଎Խ͢Δख๏Λݕ౼͢Δɽ
    ·ͱΊ
    ࠓޙͷల๬
    

    View Slide