Save 37% off PRO during our Black Friday Sale! »

Elastic{ON}Tour Tokyo 2017 - Machine Learning Deep Dive

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
December 14, 2017

Elastic{ON}Tour Tokyo 2017 - Machine Learning Deep Dive

5.5でリリースされたMachine Learningは時系列データの異常検知を可能にしました。本セッションでは、NGINXログの投入から、情報付加、可視化、分析、そして異常の予測まで、Elastic Stackを利用した手順をご覧いただきます。

Sophie Chang | Machine Learning Team Lead | Elastic
Kosho Owa| Solution Architect | Elastic

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

December 14, 2017
Tweet

Transcript

  1. Sophie Chang, Team Lead, Machine Learning Kosho Owa, Solution Architect

    Machine Learning Deep Dive
  2. None
  3. 3 Anomaly Detection in Time Series Data

  4. 4 Anomalies == Trouble 2017-02-27 9:37am

  5. 5

  6. 6 IT Operational Analytics

  7. 7 DNS Are there signs of data exfiltration? packetbeat Traffic

    Is one of my users an insider threat? metricbeat Auth Logs Is a brute- force attack underway? filebeat Security Analytics
  8. 8 Unusual spike in user latency Server woes or regional

    outage Rare event from sensor Failing device Metrics
  9. 9 Dashboards aren’t enough

  10. 10 Rules Don’t Scale • Where do you set the

    threshold? • Who updates the rules? • False positives are costly
  11. 11 It All Begins with Data Discovering information in NGINX

    logs 68.75.44.178, 172.68.146.54, 127.0.0.1 - - [15/May/ 2017:12:16:27 +0200] "GET /sites/default/files/styles/ company_profile_cover_crop/public/1500x500_1_10.jpg? itok=RUgim2UQ&sc=297009042628d7de3f0eb50e807d29e4 HTTP/1.1" 200 92763 "https://www.startus.cc/company/finleap" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36”
  12. 12 Ingest, Enrich, Visualize, Analyze, Alert Elasticsearch X-pack Master Nodes

    (3) Ingest Nodes (X) Data Nodes - Hot (X) Data Nodes - Warm (X) Beats Log Files Metrics Wire Data your(beat) Filebeat Module NGINX Kibana X-pack Instances (X)
  13. Demo

  14. 14

  15. 15

  16. 16

  17. 17

  18. Thank You