Save 37% off PRO during our Black Friday Sale! »

Machine Learning Deep Dive

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
December 12, 2017

Machine Learning Deep Dive

GA'ed in 5.5, machine learning can uncover insights from your data automatically. This session walks you through using the Elastic Stack to ingest, enrich, visualize, analyze, and alert on NGINX logs to detect and eventually predict anamolies in your data.

5.5 버전부터 Machine Learning을 통해 데이터에 대한 인사이트를 자동적으로 얻을 수 있습니다. 본 세션에서는 Elastic Stack을 사용하여 NGINX 로그를 수집, 강화, 시각화, 분석하고 이에 대한 알람을 설정하여 이상 징후를 예측하는 방법에 대해 배우실 수 있습니다.

Sophie Chang | Machine Learning Team Lead | Elastic

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

December 12, 2017
Tweet

Transcript

  1. Sophie Chang Team Lead, Machine Learning Machine Learning Deep Dive

  2. None
  3. 3 Anomaly Detection in Time Series Data

  4. 4 Anomalies == Trouble 2017-02-27 9:37am

  5. 5

  6. 6 IT Operational Analytics

  7. 7 DNS Are there signs of data exfiltration? packetbeat Traffic

    Is one of my users an insider threat? metricbeat Auth Logs Is a brute- force attack underway? filebeat Security Analytics
  8. 8 Unusual spike in user latency Server woes or regional

    outage Rare event from sensor Failing device Metrics
  9. 9 Dashboards aren’t enough

  10. 10 Rules Don’t Scale • Where do you set the

    threshold? • Who updates the rules? • False positives are costly
  11. 11 It All Begins with Data Discovering information in NGINX

    logs 68.75.44.178, 172.68.146.54, 127.0.0.1 - - [15/May/ 2017:12:16:27 +0200] "GET /sites/default/files/styles/ company_profile_cover_crop/public/1500x500_1_10.jpg? itok=RUgim2UQ&sc=297009042628d7de3f0eb50e807d29e4 HTTP/1.1" 200 92763 "https://www.startus.cc/company/finleap" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36”
  12. 12 Ingest, Enrich, Visualize, Analyze, Alert Elasticsearch X-pack Master Nodes

    (3) Ingest Nodes (X) Data Nodes - Hot (X) Data Nodes - Warm (X) Beats Log Files Metrics Wire Data your(beat) Filebeat Module NGINX Kibana X-pack Instances (X)
  13. Demo

  14. Thank You