correlation-for-timeseries

hbstyle/2016/07/14

Transcript

1. Correla'on(similarity) for 'meseries hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 1
2. Correla'on hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 2
3. ʦ໊ʧ(εϧ)ೋͭͷ΋ͷ͕ີ઀ʹ͔ ͔ΘΓ߹͍ͬͯΔ͜ͱɻ — σδλϧେࣙઘ hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 3
4. Why Correla+on for ops engineer? hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 4
5. hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 5
6. ! hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 6
7. Why Correla+on for ops engineer? • everything is a ϝτϦΫε • Ξϥʔτ͕ग़Δͨͼʹ݂؟ʹͳͬͯ૬ؔੑΛ୳͍ͯ͠Δ • ex.) ͜͜Ͱ Load average ্͕͕ͬͨͷ͸ʁCPU Sys ্͕͕͍ͬͯΔ͔Β...ʁͦΕͱ΋...ʁ • => ফ໣ => ࣗಈͰ΍Γ͍ͨ • ϝτϦΫε͸࣌ܥྻσʔλͷू߹ • ࣌ܥྻσʔλʹରͯ͠૬ؔੑΛఆٛग़དྷͳ͍͔ʁ • ԿΛ࣋ͬͯ૬ؔͱ͢Δ͔ʁ • ಉཻ͡౓ͷϝτϦΫε͹͔ΓͰ͸ͳ͍͚Ͳ૬ؔੑΛఆ͍ٛͨ͠ʁ • ࣌ܥྻσʔλͷڑ཭Λఆٛͯ͠ྨࣅ౓ΛଌͬͯΈΔ hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 7
8. Distance 1 h$p:/ /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applica<ons/similarity- measures-and-dimensionality-reduc<on-techniques-for-<me-series-data-mining hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 8
9. Methods examples • ࣌ܥྻσʔλͷڑ཭Λܭࢉ͢Δ୅දతͳख๏ • ACF(ࣗݾ૬ؔؔ਺) • COR(ϐΞιϯ૬ؔ) • EUCL(ϢʔΫϦουڑ཭) • DWTWARP(DTWڑ཭) • ... hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 9
10. Methods examples • ࣌ܥྻσʔλͷڑ཭Λܭࢉ͢Δ୅දతͳख๏ • ACF(ࣗݾ૬ؔؔ਺) • COR(ϐΞιϯ૬ؔ) • EUCL(ϢʔΫϦουڑ཭) • DWTWARP(DTWڑ཭) • ... hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 10
11. EUCL(ϢʔΫϦουڑ཭) • , ͱ͍͏࣌ܥྻσʔλΛൺֱ͢Δͱ͢Δ(࣌ܥྻͷ௕͞͸ͦΕͧΕ ͱ͢Δ) • ൪໨ͷσʔλΛͦΕͧΕ , ͱ͢Δ • ͜ͷ࣌ϢʔΫϦουڑ཭͸ ͱఆٛͰ͖Δ • ܭࢉྔ͸ • ൺֱ͢Δ 2 ͭͷ࣌ܥྻσʔλͷཻ౓͕ἧ͍ͬͯΔඞཁ͕͋Δ hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 11
12. 1 h$p:/ /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applica<ons/similarity- measures-and-dimensionality-reduc<on-techniques-for-<me-series-data-mining hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 12
13. DTWARP(DTWڑ཭) • , ͱ͍͏࣌ܥྻσʔλΛൺֱ͢Δͱ͢Δ (࣌ܥྻͷ௕͞͸ͦΕͧΕ , ͱ͢Δ) • ൪໨ͷσʔλΛͦΕͧΕ , ͱ͢Δ • ্هͷఆٛͷ࣌ɺ ͷ஋͕ DTW ڑ཭ͱఆٛͰ͖Δ • ܭࢉྔ͸ • ࣌ܥྻσʔλͷྨࣅ౓ݕग़ਫ਼౓͸ߴ͍ • ϊΠζʹऑ͍ɾܭࢉྔ͕ଟ͍ͱ͍͏໰୊΋͋Δ • FastDTW ౳ͷվྑ͞ΕͨΞϧΰϦζϜ΋ଘࡏ͢Δ • ॳظͷԻ੠ೝࣝͰΑ͘࢖ΘΕͨ(ӅΕϚϧίϑϞσϧ͕ҰൠԽ͞Εͯ࢖ΘΕͳ͘ͳͬͨ) hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 13
14. 1 h$p:/ /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applica<ons/similarity- measures-and-dimensionality-reduc<on-techniques-for-<me-series-data-mining hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 14
15. 2 h$p:/ /sinhrks.hatenablog.com/entry/2014/11/14/232603 hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 15
16. 2 h$p:/ /sinhrks.hatenablog.com/entry/2014/11/14/232603 hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 16
17. Real world example hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 17
18. etsy/oculus example hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 18
19. ! hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 19
20. ·ͱΊ • ࣌ܥྻσʔλͷ૬ؔ(ݫີʹ͸ྨࣅ౓)ʹؔͯ͠આ໌ • ͳͥΠϯϑϥΤϯδχΞ͕࣌ܥྻσʔλͷ૬ؔΛ஌Δඞཁ͕͋Δͷ͔આ໌ • ࣌ܥྻσʔλͷྨࣅ౓ΛਤΔࢦඪΛ͍͔ͭ͘঺հ • ϢʔΫϦουڑ཭ʹ͍ͭͯৄ͘͠આ໌ • DTW ڑ཭ʹ͍ͭͯৄ͘͠આ໌ • ࣌ܥྻσʔλͷྨࣅ౓ΛϝτϦΫεʹద༻͍ͯ͠Δ OSS ͷྫΛઆ໌ hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 20
21. ࢀߟจݙ • Package 'TSclust' • h1ps:/ /cran.r-project.org/web/packages/TSclust/TSclust.pdf • Making Time-series ClassificaDon More Accurate Using Learned Constraints • h1p:/ /www.cs.ucr.edu/~eamonn/RATANAMC.pdf • ಈత࣌ؒ৳ॖ๏ / DTW (Dynamic Time Warping) ΛՄࢹԽ͢Δ • h1p:/ /sinhrks.hatenablog.com/entry/2014/11/14/232603 • Similarity Measures and Dimensionality ReducDon Techniques for Time Series Data Mining • h1p:/ /www.intechopen.com/books/advances-in-data-mining-knowledge-discovery-and-applicaDons/similarity-measures-and-dimensionality- reducDon-techniques-for-Dme-series-data-mining • {TSclust} Ͱ͸͡ΊΔ࣌ܥྻΫϥελϦϯά • h1p:/ /sinhrks.hatenablog.com/entry/2014/11/16/194012 • etsy/oculus • h1ps:/ /github.com/etsy/oculus hbstyle 2016/07/14 - Yoshikawa Ryota ( @rrreeeyyy ) 21