Machine Learning Deep Dive

GA'ed in 5.5, machine learning can uncover insights from your data automatically. This session walks you through using the Elastic Stack to ingest, enrich, visualize, analyze, and alert on NGINX logs to detect and eventually predict anamolies in your data.

5.5 버전부터 Machine Learning을 통해 데이터에 대한 인사이트를 자동적으로 얻을 수 있습니다. 본 세션에서는 Elastic Stack을 사용하여 NGINX 로그를 수집, 강화, 시각화, 분석하고 이에 대한 알람을 설정하여 이상 징후를 예측하는 방법에 대해 배우실 수 있습니다.

Sophie Chang | Machine Learning Team Lead | Elastic

Transcript

1. Sophie Chang Team Lead, Machine Learning Machine Learning Deep Dive
3. 3 Anomaly Detection in Time Series Data
4. 4 Anomalies == Trouble 2017-02-27 9:37am
6. 6 IT Operational Analytics
7. 7 DNS Are there signs of data exfiltration? packetbeat Traffic Is one of my users an insider threat? metricbeat Auth Logs Is a brute- force attack underway? filebeat Security Analytics
8. 8 Unusual spike in user latency Server woes or regional outage Rare event from sensor Failing device Metrics
9. 9 Dashboards aren’t enough
10. 10 Rules Don’t Scale • Where do you set the threshold? • Who updates the rules? • False positives are costly
11. 11 It All Begins with Data Discovering information in NGINX logs 68.75.44.178, 172.68.146.54, 127.0.0.1 - - [15/May/ 2017:12:16:27 +0200] "GET /sites/default/files/styles/ company_profile_cover_crop/public/1500x500_1_10.jpg? itok=RUgim2UQ&sc=297009042628d7de3f0eb50e807d29e4 HTTP/1.1" 200 92763 "https://www.startus.cc/company/finleap" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36”
12. 12 Ingest, Enrich, Visualize, Analyze, Alert Elasticsearch X-pack Master Nodes (3) Ingest Nodes (X) Data Nodes - Hot (X) Data Nodes - Warm (X) Beats Log Files Metrics Wire Data your(beat) Filebeat Module NGINX Kibana X-pack Instances (X)
13. Demo
14. Thank You