Anomaly Detection with the Elastic Stack

Prelert: Elastic Stackを利用した異常検知
Elastic{ON} Tour Tokyo 2016

Transcript

1. Prelert: Elastic StackΛར༻ͨ͠ҟৗݕ஌ େྠ ߂ৄ | Kosho Owa Solutions Architect, Elastic
2. ೉໰
   
   *5ΦϖϨʔγϣϯ • ࣗ෼ͷγεςϜ͸ਖ਼ৗʹՔಇ͍ͯ͠Δ? • ͲͷΑ͏ʹᮢ஋Λ൑அ͢Δ? • ໰୊͕ൃੜͨ࣌͠ʹɺͲͷΑ͏ʹݪҼΛݟ͚ͭΔ? 2
3. ೉໰
   
   *5ηΩϡϦςΟ • Ϛϧ΢ΣΞʹ৵ೖ͞Ε͍ͯΔγεςϜ͸ແ͍͔? • Ϛϧ΢ΣΞ͕ͲͷΑ͏ʹײછΛ޿͔͛ͨ? • જࡏతʹڴҖͱͳΔ૊৫಺෦ͷϢʔβʔ͸୭͔? 3
4. ೉໰
   
   ͦͷଞ • ͲͷΑ͏ʹɺଟ͘ͷछྨͷ࣌ܥྻσʔλͱ޲͖߹͏͔? • ޻৔͸ਖ਼ৗʹՔಇ͍ͯ͠Δ? • Ͳͷަ௨ࣄނ͕࠷΋ौ଺ΛҾ͖ى͍ͯ͜͠Δ͔? 4
5. σʔλ͔Β༗ҙٛͳ৘ใΛݟ͚ͭΔํ๏ 5 Search Aggregations Visualization Machine Learning
6. t_900 - Dashboard New Add Save Open Share Options ! ~ 3 years ago to ~ 3 years ago 900 - Actual 2013-09-18 00:00 2013-09-21 00:00 2013-09-24 00:00 2013-09-27 00:00 2013-09-30 00:00 2013-10-03 00:00 2013-10-06 00:00 2013-10-09 00:00 2013-10-12 00:00 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 Actual 900 - Moving Average 6000000 7000000 8000000 Moving Average Actual Anomaly " ҟৗݕ஌΁ͷνϟϨϯδ 6 1 3 2 4 2 2 2 2 week
7. ҠಈฏۉʹΑΔҟৗݕ஌ 7 t_900 - Dashboard New Add Save Open Share Options ! ~ 3 years ago to ~ 3 years ago 900 - Moving Average 2013-09-18 00:00 2013-09-21 00:00 2013-09-24 00:00 2013-09-27 00:00 2013-09-30 00:00 2013-10-03 00:00 2013-10-06 00:00 2013-10-09 00:00 2013-10-12 00:00 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 Moving Average Actual Anomaly 900 - Holt-Winters 8000000 9000000 HoltWinters Actual Anomaly "
8. )PMU8JOUFSTʹΑΔҟৗݕ஌ 8 _900 - Dashboard New Add Save Open Share Options ! ~ 3 years ago to ~ 3 years ago 900 - Holt-Winters 2013-09-18 00:00 2013-09-21 00:00 2013-09-24 00:00 2013-09-27 00:00 2013-09-30 00:00 2013-10-03 00:00 2013-10-06 00:00 2013-10-09 00:00 2013-10-12 00:00 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 HoltWinters Actual Anomaly ly timeline : detector Interval: Auto Sep 17 2013 Sep 19 2013 Sep 21 2013 Sep 23 2013 Sep 25 2013 Sep 27 2013 Sep 29 2013 Oct 1 2013 Oct 3 2013 Oct 5 2013 Oct 7 2013 Oct 9 2013 Oct 11 2013 non_zero_count "
9. 1SFMFSUʹΑΔҟৗݕ஌ 9 rer Jobs Summary view Explorer Connections Support ! " # $ September 15th 2013, 00:00:00.000 to October 13th All jobs * debug 900 Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 0 500000 1000000 1500000 2000000 Infl y timeline : detector Interval: Auto Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 non_zero_count All jobs * debug 900 Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 0 500000 1000000 1500000 2000000 Influ ly timeline : detector Interval: Auto Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 non_zero_count lies 1 3 2 4
10. ୈिͷΫϩʔζΞοϓ 10 orer Jobs Summary view Explorer Connections Support ! " # $ September 22nd 2013, 00:00:00.000 to September 29th 2 All jobs * l debug _900 Sep 22 09:00 Sep 22 21:00 Sep 23 09:00 Sep 23 21:00 Sep 24 09:00 Sep 24 21:00 Sep 25 09:00 Sep 25 21:00 Sep 26 09:00 Sep 26 21:00 Sep 27 09:00 Sep 27 21:00 Sep 28 09:00 Sep 28 21:00 0 500000 1000000 1500000 2000000 Influ aly timeline by: detector Interval: Auto Sep 22 09:00 Sep 22 21:00 Sep 23 09:00 Sep 23 21:00 Sep 24 09:00 Sep 24 21:00 Sep 25 09:00 Sep 25 21:00 Sep 26 09:00 Sep 26 21:00 Sep 27 09:00 Sep 27 21:00 Sep 28 09:00 Sep 28 21:00 non_zero_count 3 2
11. 1SFMFSUͷςΫϊϩδʔ 11 σʔλʹજΉߦಈϞσϧΛ
 ࣗಈతʹڭࢣͳֶ͠श ݱࡏͷߦಈ͕༧ଌϞσϧͱ
 ݦஶʹҟͳΔ৔߹ʹ௨஌
12. Demo
13. %FNP
    *5ΦϖϨʔγϣϯ
    
14. %FNP
    *5ΦϖϨʔγϣϯ
    
15. %FNP
    *5ΦϖϨʔγϣϯ
    
16. %FNP
    *5ηΩϡϦςΟ
17. ϩʔυϚοϓ • ϕʔλ൛Λఏڙத (prelert.com) • Elastic StackͱͷڧݻͳΠϯςάϨʔγϣϯ͕ਐߦத • 2017೥্൒ظͷϦϦʔεΛ໨ඪ 17