Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Anomaly Detection with the Elastic Stack

34cbde72de5f384380d5489543294dc5?s=47 Kosho Owa
December 15, 2016

Anomaly Detection with the Elastic Stack

Prelert: Elastic Stackを利用した異常検知
Elastic{ON} Tour Tokyo 2016

34cbde72de5f384380d5489543294dc5?s=128

Kosho Owa

December 15, 2016
Tweet

Transcript

  1. Prelert: Elastic StackΛར༻ͨ͠ҟৗݕ஌ େྠ ߂ৄ | Kosho Owa Solutions Architect,

    Elastic
  2. ೉໰*5ΦϖϨʔγϣϯ • ࣗ෼ͷγεςϜ͸ਖ਼ৗʹՔಇ͍ͯ͠Δ? • ͲͷΑ͏ʹᮢ஋Λ൑அ͢Δ? • ໰୊͕ൃੜͨ࣌͠ʹɺͲͷΑ͏ʹݪҼΛݟ͚ͭΔ? 2

  3. ೉໰*5ηΩϡϦςΟ • Ϛϧ΢ΣΞʹ৵ೖ͞Ε͍ͯΔγεςϜ͸ແ͍͔? • Ϛϧ΢ΣΞ͕ͲͷΑ͏ʹײછΛ޿͔͛ͨ? • જࡏతʹڴҖͱͳΔ૊৫಺෦ͷϢʔβʔ͸୭͔? 3

  4. ೉໰ͦͷଞ • ͲͷΑ͏ʹɺଟ͘ͷछྨͷ࣌ܥྻσʔλͱ޲͖߹͏͔? • ޻৔͸ਖ਼ৗʹՔಇ͍ͯ͠Δ? • Ͳͷަ௨ࣄނ͕࠷΋ौ଺ΛҾ͖ى͍ͯ͜͠Δ͔? 4

  5. σʔλ͔Β༗ҙٛͳ৘ใΛݟ͚ͭΔํ๏ 5 Search Aggregations Visualization Machine Learning

  6. t_900 - Dashboard New Add Save Open Share Options !

    ~ 3 years ago to ~ 3 years ago 900 - Actual 2013-09-18 00:00 2013-09-21 00:00 2013-09-24 00:00 2013-09-27 00:00 2013-09-30 00:00 2013-10-03 00:00 2013-10-06 00:00 2013-10-09 00:00 2013-10-12 00:00 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 Actual 900 - Moving Average 6000000 7000000 8000000 Moving Average Actual Anomaly " ҟৗݕ஌΁ͷνϟϨϯδ 6 1 3 2 4 2 2 2 2 week
  7. ҠಈฏۉʹΑΔҟৗݕ஌ 7 t_900 - Dashboard New Add Save Open Share

    Options ! ~ 3 years ago to ~ 3 years ago 900 - Moving Average 2013-09-18 00:00 2013-09-21 00:00 2013-09-24 00:00 2013-09-27 00:00 2013-09-30 00:00 2013-10-03 00:00 2013-10-06 00:00 2013-10-09 00:00 2013-10-12 00:00 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 Moving Average Actual Anomaly 900 - Holt-Winters 8000000 9000000 HoltWinters Actual Anomaly "
  8. )PMU8JOUFSTʹΑΔҟৗݕ஌ 8 _900 - Dashboard New Add Save Open Share

    Options ! ~ 3 years ago to ~ 3 years ago 900 - Holt-Winters 2013-09-18 00:00 2013-09-21 00:00 2013-09-24 00:00 2013-09-27 00:00 2013-09-30 00:00 2013-10-03 00:00 2013-10-06 00:00 2013-10-09 00:00 2013-10-12 00:00 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 HoltWinters Actual Anomaly ly timeline : detector Interval: Auto Sep 17 2013 Sep 19 2013 Sep 21 2013 Sep 23 2013 Sep 25 2013 Sep 27 2013 Sep 29 2013 Oct 1 2013 Oct 3 2013 Oct 5 2013 Oct 7 2013 Oct 9 2013 Oct 11 2013 non_zero_count "
  9. 1SFMFSUʹΑΔҟৗݕ஌ 9 rer Jobs Summary view Explorer Connections Support !

    " # $ September 15th 2013, 00:00:00.000 to October 13th All jobs * debug 900 Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 0 500000 1000000 1500000 2000000 Infl y timeline : detector Interval: Auto Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 non_zero_count All jobs * debug 900 Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 0 500000 1000000 1500000 2000000 Influ ly timeline : detector Interval: Auto Sep 17 2013 Sep 20 2013 Sep 23 2013 Sep 26 2013 Sep 29 2013 Oct 2 2013 Oct 5 2013 Oct 8 2013 Oct 11 2013 non_zero_count lies 1 3 2 4
  10. ୈिͷΫϩʔζΞοϓ 10 orer Jobs Summary view Explorer Connections Support !

    " # $ September 22nd 2013, 00:00:00.000 to September 29th 2 All jobs * l debug _900 Sep 22 09:00 Sep 22 21:00 Sep 23 09:00 Sep 23 21:00 Sep 24 09:00 Sep 24 21:00 Sep 25 09:00 Sep 25 21:00 Sep 26 09:00 Sep 26 21:00 Sep 27 09:00 Sep 27 21:00 Sep 28 09:00 Sep 28 21:00 0 500000 1000000 1500000 2000000 Influ aly timeline by: detector Interval: Auto Sep 22 09:00 Sep 22 21:00 Sep 23 09:00 Sep 23 21:00 Sep 24 09:00 Sep 24 21:00 Sep 25 09:00 Sep 25 21:00 Sep 26 09:00 Sep 26 21:00 Sep 27 09:00 Sep 27 21:00 Sep 28 09:00 Sep 28 21:00 non_zero_count 3 2
  11. 1SFMFSUͷςΫϊϩδʔ 11 σʔλʹજΉߦಈϞσϧΛ
 ࣗಈతʹڭࢣͳֶ͠श ݱࡏͷߦಈ͕༧ଌϞσϧͱ
 ݦஶʹҟͳΔ৔߹ʹ௨஌

  12. Demo

  13. %FNP*5ΦϖϨʔγϣϯ

  14. %FNP*5ΦϖϨʔγϣϯ

  15. %FNP*5ΦϖϨʔγϣϯ

  16. %FNP*5ηΩϡϦςΟ

  17. ϩʔυϚοϓ • ϕʔλ൛Λఏڙத (prelert.com) • Elastic StackͱͷڧݻͳΠϯςάϨʔγϣϯ͕ਐߦத • 2017೥্൒ظͷϦϦʔεΛ໨ඪ 17