Elastic for Time Series Data and Predictive Analytics

Transcript

1. 1 ChristophWurm, Solutions Architect January 2016 ELASTIC FOR TIME SERIES DATA PREDICTIVE ANALYTICS
2. 2 The Elastic Community 40,000 Community members 35,000 Commits against Elastic stack to-date
3. 3 Viral Adoption Mar’15 Oct’12 Apr’13 Apr’14 Oct’13 20. Millions of Downloads 10. 40+ Million Downloads Cumulative across Elastic products to date Nov’15 40. Sept’14
4. 4 What is Elastic? Platformaround a distributed data store By developers for developers Massive amounts of structured and unstructured data Real-time at scale
5. 5 Elastic stack Logstash Collect, parse and enrich data Elasticsearch Store, search, analyze Hadoop Ecosystem Hadoop connector Beats Tap into your wire data Shield Security Watcher Scheduler Marvel - Monitoring Found Scale in the cloud Kibana Visualize and explore data Training Professional Services Support Subscriptions BUILT FOR TODAY’S SCALABLE, DISTRIBUTED SYSTEMS
6. 6 What is Time Series Data? Has a timestamp Older and newer data Older data is less important Very old data will be deleted Random variation Trends and predictions
7. 7 Time Series Architecture Filebeat Log files Packetbeat Packet sniffing Topbeat Server metrics Execbeat Arbitrary commands logstash-input-* JDBC, Twitter, *MQ, etc. Roll your own! Java, .NET, Python, etc. Logstash ES ES ES Kibana Timelion Custom
8. 8 Demo #1 TIMELION
9. 9 New in Elasticsearch2.0 Pipeline Aggregations “Aggregations on top of other aggregations” Derivatives Moving average Holt-Winters (prediction / anomaly detection) Custom
10. 10 Moving Average
11. 11 Linear Trend
12. 12 Cyclic Trends (Holt-Winters)
13. 13 In-Depth MOVING AVERAGE
14. 14 { model: simple window: 180 } Simple, unweighted moving average (basically the mean)
15. 15 { model: simple window: 720 } Simple, unweighted moving average (basically the mean)
16. 16 Simple, unweighted moving average (basically the mean) { model: simple window: 10 }
17. 17 Simple, unweighted moving average (basically the mean) { model: simple window: 100 }
18. 18 { model: linear window: 180 } Linear weighted moving average
19. 19 { model: ewma window: 180 } Exponential weighted moving average (Overfitting?)
20. 20 { model: holt window: 180 } Holt-Linear double exponential weighted moving average Trend
21. 21 { model: holt_winters window: 360 predict: 120 settings: { type: mult period: 120 } } Holt-Winters triple exponential weighted moving average Prediction
22. 22 Demo #2 PREDICTIVE ANALYTICS
23. 23 MADRID, Spain January 19 - 21 BERLIN,Germany January 25 - 28 COPENHAGEN, Denmark January 26 - 29 PARIS, France February 1 - 4 LONDON, United Kingdom February 3 - 5 AMSTERDAM, Netherlands February 8 - 11 training.elastic.co
25. 25 Q&A ASK ME ANYTHING