Elastic for Time Series Data and Predictive Analytics

098332e9d988080a9057816f84d668f7?s=47 Elasticsearch Inc
January 12, 2016
2.6k

Elastic for Time Series Data and Predictive Analytics

098332e9d988080a9057816f84d668f7?s=128

Elasticsearch Inc

January 12, 2016
Tweet

Transcript

  1. 1 ChristophWurm, Solutions Architect January 2016 ELASTIC FOR TIME SERIES

    DATA PREDICTIVE ANALYTICS
  2. 2 The Elastic Community 40,000 Community members 35,000 Commits against

    Elastic stack to-date
  3. 3 Viral Adoption Mar’15 Oct’12 Apr’13 Apr’14 Oct’13 20. Millions

    of Downloads 10. 40+ Million Downloads Cumulative across Elastic products to date Nov’15 40. Sept’14
  4. 4 What is Elastic? Platformaround a distributed data store By

    developers for developers Massive amounts of structured and unstructured data Real-time at scale
  5. 5 Elastic stack Logstash Collect, parse and enrich data Elasticsearch

    Store, search, analyze Hadoop Ecosystem Hadoop connector Beats Tap into your wire data Shield Security Watcher Scheduler Marvel - Monitoring Found Scale in the cloud Kibana Visualize and explore data Training Professional Services Support Subscriptions BUILT FOR TODAY’S SCALABLE, DISTRIBUTED SYSTEMS
  6. 6 What is Time Series Data? Has a timestamp Older

    and newer data Older data is less important Very old data will be deleted Random variation Trends and predictions
  7. 7 Time Series Architecture Filebeat Log files Packetbeat Packet sniffing

    Topbeat Server metrics Execbeat Arbitrary commands logstash-input-* JDBC, Twitter, *MQ, etc. Roll your own! Java, .NET, Python, etc. Logstash ES ES ES Kibana Timelion Custom
  8. 8 Demo #1 TIMELION

  9. 9 New in Elasticsearch2.0 Pipeline Aggregations “Aggregations on top of

    other aggregations” Derivatives Moving average Holt-Winters (prediction / anomaly detection) Custom
  10. 10 Moving Average

  11. 11 Linear Trend

  12. 12 Cyclic Trends (Holt-Winters)

  13. 13 In-Depth MOVING AVERAGE

  14. 14 { model: simple window: 180 } Simple, unweighted moving

    average (basically the mean)
  15. 15 { model: simple window: 720 } Simple, unweighted moving

    average (basically the mean)
  16. 16 Simple, unweighted moving average (basically the mean) { model:

    simple window: 10 }
  17. 17 Simple, unweighted moving average (basically the mean) { model:

    simple window: 100 }
  18. 18 { model: linear window: 180 } Linear weighted moving

    average
  19. 19 { model: ewma window: 180 } Exponential weighted moving

    average (Overfitting?)
  20. 20 { model: holt window: 180 } Holt-Linear double exponential

    weighted moving average Trend
  21. 21 { model: holt_winters window: 360 predict: 120 settings: {

    type: mult period: 120 } } Holt-Winters triple exponential weighted moving average Prediction
  22. 22 Demo #2 PREDICTIVE ANALYTICS

  23. 23 MADRID, Spain January 19 - 21 BERLIN,Germany January 25

    - 28 COPENHAGEN, Denmark January 26 - 29 PARIS, France February 1 - 4 LONDON, United Kingdom February 3 - 5 AMSTERDAM, Netherlands February 8 - 11 training.elastic.co
  24. 24

  25. 25 Q&A ASK ME ANYTHING