correlation-for-timeseries

hbstyle/2016/07/14

Transcript

1. Correla'on(similarity) for 'meseries
   hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 1
   

   View Slide

2. Correla'on
   hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 2
   

   View Slide

3. ʦ໊ʧ(εϧ)ೋͭͷ΋ͷ͕ີ઀ʹ͔
   ͔ΘΓ߹͍ͬͯΔ͜ͱɻ
   — σδλϧେࣙઘ
   hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 3
   

   View Slide

4. Why Correla+on for ops engineer?
   hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 4
   

   View Slide

5. hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 5
   

   View Slide

6. !
   hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 6
   

   View Slide

7. Why Correla+on for ops engineer?
   • everything is a ϝτϦΫε
   • Ξϥʔτ͕ग़Δͨͼʹ݂؟ʹͳͬͯ૬ؔੑΛ୳͍ͯ͠Δ
   • ex.) ͜͜Ͱ Load average ্͕͕ͬͨͷ͸ʁCPU Sys ্͕͕͍ͬͯΔ͔Β...ʁͦΕͱ΋...ʁ
   • => ফ໣ => ࣗಈͰ΍Γ͍ͨ
   • ϝτϦΫε͸࣌ܥྻσʔλͷू߹
   • ࣌ܥྻσʔλʹରͯ͠૬ؔੑΛఆٛग़དྷͳ͍͔ʁ
   • ԿΛ࣋ͬͯ૬ؔͱ͢Δ͔ʁ
   • ಉཻ͡౓ͷϝτϦΫε͹͔ΓͰ͸ͳ͍͚Ͳ૬ؔੑΛఆ͍ٛͨ͠ʁ
   • ࣌ܥྻσʔλͷڑ཭Λఆٛͯ͠ྨࣅ౓ΛଌͬͯΈΔ
   hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 7
   

   View Slide

8. Distance
   1 h$p:/
   /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applicameasures-and-dimensionality-reduchbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 8
   

   View Slide

9. Methods examples
   • ࣌ܥྻσʔλͷڑ཭Λܭࢉ͢Δ୅දతͳख๏
   • ACF(ࣗݾ૬ؔؔ਺)
   • COR(ϐΞιϯ૬ؔ)
   • EUCL(ϢʔΫϦουڑ཭)
   • DWTWARP(DTWڑ཭)
   • ...
   hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 9
   

   View Slide

10. Methods examples
    • ࣌ܥྻσʔλͷڑ཭Λܭࢉ͢Δ୅දతͳख๏
    • ACF(ࣗݾ૬ؔؔ਺)
    • COR(ϐΞιϯ૬ؔ)
    • EUCL(ϢʔΫϦουڑ཭)
    • DWTWARP(DTWڑ཭)
    • ...
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 10
    

    View Slide

11. EUCL(ϢʔΫϦουڑ཭)
    • , ͱ͍͏࣌ܥྻσʔλΛൺֱ͢Δͱ͢Δ(࣌ܥྻͷ௕͞͸ͦΕͧΕ ͱ͢Δ)
    • ൪໨ͷσʔλΛͦΕͧΕ , ͱ͢Δ
    • ͜ͷ࣌ϢʔΫϦουڑ཭͸ ͱఆٛͰ͖Δ
    • ܭࢉྔ͸
    • ൺֱ͢Δ 2 ͭͷ࣌ܥྻσʔλͷཻ౓͕ἧ͍ͬͯΔඞཁ͕͋Δ
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 11
    

    View Slide

12. 1 h$p:/
    /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applicameasures-and-dimensionality-reduchbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 12
    

    View Slide

13. DTWARP(DTWڑ཭)
    • , ͱ͍͏࣌ܥྻσʔλΛൺֱ͢Δͱ͢Δ (࣌ܥྻͷ௕͞͸ͦΕͧΕ , ͱ͢Δ)
    • ൪໨ͷσʔλΛͦΕͧΕ , ͱ͢Δ
    • ্هͷఆٛͷ࣌ɺ ͷ஋͕ DTW ڑ཭ͱఆٛͰ͖Δ
    • ܭࢉྔ͸
    • ࣌ܥྻσʔλͷྨࣅ౓ݕग़ਫ਼౓͸ߴ͍
    • ϊΠζʹऑ͍ɾܭࢉྔ͕ଟ͍ͱ͍͏໰୊΋͋Δ
    • FastDTW ౳ͷվྑ͞ΕͨΞϧΰϦζϜ΋ଘࡏ͢Δ
    • ॳظͷԻ੠ೝࣝͰΑ͘࢖ΘΕͨ(ӅΕϚϧίϑϞσϧ͕ҰൠԽ͞Εͯ࢖ΘΕͳ͘ͳͬͨ)
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 13
    

    View Slide

14. 1 h$p:/
    /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applicameasures-and-dimensionality-reduchbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 14
    

    View Slide

15. 2 h$p:/
    /sinhrks.hatenablog.com/entry/2014/11/14/232603
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 15
    

    View Slide

16. 2 h$p:/
    /sinhrks.hatenablog.com/entry/2014/11/14/232603
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 16
    

    View Slide

17. Real world example
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 17
    

    View Slide

18. etsy/oculus example
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 18
    

    View Slide

19. !
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 19
    

    View Slide

20. ·ͱΊ
    • ࣌ܥྻσʔλͷ૬ؔ(ݫີʹ͸ྨࣅ౓)ʹؔͯ͠આ໌
    • ͳͥΠϯϑϥΤϯδχΞ͕࣌ܥྻσʔλͷ૬ؔΛ஌Δඞཁ͕͋Δͷ͔આ໌
    • ࣌ܥྻσʔλͷྨࣅ౓ΛਤΔࢦඪΛ͍͔ͭ͘঺հ
    • ϢʔΫϦουڑ཭ʹ͍ͭͯৄ͘͠આ໌
    • DTW ڑ཭ʹ͍ͭͯৄ͘͠આ໌
    • ࣌ܥྻσʔλͷྨࣅ౓ΛϝτϦΫεʹద༻͍ͯ͠Δ OSS ͷྫΛઆ໌
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 20
    

    View Slide

21. ࢀߟจݙ
    • Package 'TSclust'
    • h1ps:/
    /cran.r-project.org/web/packages/TSclust/TSclust.pdf
    • Making Time-series ClassificaDon More Accurate Using Learned Constraints
    • h1p:/
    /www.cs.ucr.edu/~eamonn/RATANAMC.pdf
    • ಈత࣌ؒ৳ॖ๏ / DTW (Dynamic Time Warping) ΛՄࢹԽ͢Δ
    • h1p:/
    /sinhrks.hatenablog.com/entry/2014/11/14/232603
    • Similarity Measures and Dimensionality ReducDon Techniques for Time Series Data Mining
    • h1p:/
    /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applicaDons/similarity-measures-and-dimensionality-
    reducDon-techniques-for-Dme-series-data-mining
    • {TSclust} Ͱ͸͡ΊΔ࣌ܥྻΫϥελϦϯά
    • h1p:/
    /sinhrks.hatenablog.com/entry/2014/11/16/194012
    • etsy/oculus
    • h1ps:/
    /github.com/etsy/oculus
    hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 21
    

    View Slide