Elastic for Time Series Data and Predictive Analytics

Transcript

1. 1 ChristophWurm, Solutions Architect January 2016 ELASTIC FOR TIME SERIES

   DATA PREDICTIVE ANALYTICS

2. 2 The Elastic Community 40,000 Community members 35,000 Commits against

   Elastic stack to-date

3. 3 Viral Adoption Mar’15 Oct’12 Apr’13 Apr’14 Oct’13 20. Millions

   of Downloads 10. 40+ Million Downloads Cumulative across Elastic products to date Nov’15 40. Sept’14

4. 4 What is Elastic? Platformaround a distributed data store By

   developers for developers Massive amounts of structured and unstructured data Real-time at scale

5. 5 Elastic stack Logstash Collect, parse and enrich data Elasticsearch

   Store, search, analyze Hadoop Ecosystem Hadoop connector Beats Tap into your wire data Shield Security Watcher Scheduler Marvel - Monitoring Found Scale in the cloud Kibana Visualize and explore data Training Professional Services Support Subscriptions BUILT FOR TODAY’S SCALABLE, DISTRIBUTED SYSTEMS

6. 6 What is Time Series Data? Has a timestamp Older

   and newer data Older data is less important Very old data will be deleted Random variation Trends and predictions

7. 7 Time Series Architecture Filebeat Log files Packetbeat Packet sniffing

   Topbeat Server metrics Execbeat Arbitrary commands logstash-input-* JDBC, Twitter, *MQ, etc. Roll your own! Java, .NET, Python, etc. Logstash ES ES ES Kibana Timelion Custom

8. 8 Demo #1 TIMELION

9. 9 New in Elasticsearch2.0 Pipeline Aggregations “Aggregations on top of

   other aggregations” Derivatives Moving average Holt-Winters (prediction / anomaly detection) Custom

10. 10 Moving Average

11. 11 Linear Trend

12. 12 Cyclic Trends (Holt-Winters)

13. 13 In-Depth MOVING AVERAGE

14. 14 { model: simple window: 180 } Simple, unweighted moving

    average (basically the mean)

15. 15 { model: simple window: 720 } Simple, unweighted moving

    average (basically the mean)

16. 16 Simple, unweighted moving average (basically the mean) { model:

    simple window: 10 }

17. 17 Simple, unweighted moving average (basically the mean) { model:

    simple window: 100 }

18. 18 { model: linear window: 180 } Linear weighted moving

    average

19. 19 { model: ewma window: 180 } Exponential weighted moving

    average (Overfitting?)

20. 20 { model: holt window: 180 } Holt-Linear double exponential

    weighted moving average Trend

21. 21 { model: holt_winters window: 360 predict: 120 settings: {

    type: mult period: 120 } } Holt-Winters triple exponential weighted moving average Prediction

22. 22 Demo #2 PREDICTIVE ANALYTICS

23. 23 MADRID, Spain January 19 - 21 BERLIN,Germany January 25

    - 28 COPENHAGEN, Denmark January 26 - 29 PARIS, France February 1 - 4 LONDON, United Kingdom February 3 - 5 AMSTERDAM, Netherlands February 8 - 11 training.elastic.co

24. 24

25. 25 Q&A ASK ME ANYTHING