Elastic{ON}Tour Tokyo 2017 - Machine Learning Deep Dive

5.5でリリースされたMachine Learningは時系列データの異常検知を可能にしました。本セッションでは、NGINXログの投入から、情報付加、可視化、分析、そして異常の予測まで、Elastic Stackを利用した手順をご覧いただきます。

Sophie Chang | Machine Learning Team Lead | Elastic
Kosho Owa| Solution Architect | Elastic

Transcript

1. Sophie Chang, Team Lead, Machine Learning Kosho Owa, Solution Architect Machine Learning Deep Dive
3. 3 Anomaly Detection in Time Series Data
4. 4 Anomalies == Trouble 2017-02-27 9:37am
6. 6 IT Operational Analytics
7. 7 DNS Are there signs of data exfiltration? packetbeat Traffic Is one of my users an insider threat? metricbeat Auth Logs Is a brute- force attack underway? filebeat Security Analytics
8. 8 Unusual spike in user latency Server woes or regional outage Rare event from sensor Failing device Metrics
9. 9 Dashboards aren’t enough
10. 10 Rules Don’t Scale • Where do you set the threshold? • Who updates the rules? • False positives are costly
11. 11 It All Begins with Data Discovering information in NGINX logs 68.75.44.178, 172.68.146.54, 127.0.0.1 - - [15/May/ 2017:12:16:27 +0200] "GET /sites/default/files/styles/ company_profile_cover_crop/public/1500x500_1_10.jpg? itok=RUgim2UQ&sc=297009042628d7de3f0eb50e807d29e4 HTTP/1.1" 200 92763 "https://www.startus.cc/company/finleap" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36”
12. 12 Ingest, Enrich, Visualize, Analyze, Alert Elasticsearch X-pack Master Nodes (3) Ingest Nodes (X) Data Nodes - Hot (X) Data Nodes - Warm (X) Beats Log Files Metrics Wire Data your(beat) Filebeat Module NGINX Kibana X-pack Instances (X)
13. Demo
18. Thank You