PyTN 2019

Transcript

1. Python & InfluxDB PyTennessee 2019
2. A time series is a series of data points indexed (or listed or graphed) in time order.
4. 10:15
5. 12 10:15
6. 12 10:15 10:17
7. 12 16 10:15 10:17
8. 12 16 10:15 10:17 10:19
9. 12 16 15 10:15 10:17 10:19
10. 12 16 15 10:15 10:17 10:19 10:21
11. 12 16 15 20 10:15 10:17 10:19 10:21
12. https://en.wikipedia.org/wiki/Time_series
13. https://finance.yahoo.com/quote/%5EDJI?p=^DJI
14. https://finance.yahoo.com/quote/GOOG?p=GOOG
15. https://finance.yahoo.com/quote/GOOG?p=GOOG
16. https://darksky.net/details/40.7295,-73.9936/2018-3-7/us12/en
17. https://darksky.net/details/40.7295,-73.9936/2018-3-7/us12/en
18. http://www.imreference.com/cardiology/ekg
19. https://victorops.com/blog/the-future-of-monitoring-and-alerting-victorops-and-grafana
20. https://help.papertrailapp.com/kb/how-it-works/event-viewer/
21. https://developer.lightbend.com/docs/cinnamon/latest/extensions/opentracing.html
22. http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html
23. Time series data relies on measurements
24. https://www.riverfronttimes.com/newsblog/2010/05/17/how-to-identify-a-census-worker
25. https://www.hunker.com/13408232/mercury-thermometer-vs-alcohol-thermometer
26. https://www.hunker.com/13408232/mercury-thermometer-vs-alcohol-thermometer
27. http://steinn.org/post/flask-statsd/
28. http://werkzeug.pocoo.org/docs/0.14/middlewares/
29. http://steinn.org/post/flask-statsd/
30. http://erikboertjes.com/?page_id=670
31. Measurements over time model change
32. http://www.harnois.com/en/
33. https://www.medicalnewstoday.com/articles/319725.php
34. https://www.relumination.com/parking-lot-lighting-dont-leave-customers-dark/
35. https://victorops.com/blog/the-future-of-monitoring-and-alerting-victorops-and-grafana
36. http://theregister.co.nz/news/2015/11/graph-tracking-almost-decade-retail-sales-new-zealand
37. https://www.slideshare.net/g9yuayon/streaming-analytics-in-uber
38. Algorithms
39. Simple Moving Average
40. 12 16 15 20 7:00 7:02 7:04 7:06
41. 12 16 15 20 7:00 7:02 7:04 7:06 15.75
42. 12 16 15 20 7:00 7:02 7:04 7:06
43. 12 16 15 20 7:00 7:02 7:04 7:06 10 6:58
44. 12 16 15 20 7:00 7:02 7:04 7:06 10 6:58 25 7:08
45. 12 16 15 20 7:00 7:02 7:04 7:06 10 6:58 13.25 25 7:08
46. 12 16 15 20 7:00 7:02 7:04 7:06 15.75 10 6:58 13.25 25 7:08
47. 12 16 15 20 7:00 7:02 7:04 7:06 15.75 10 6:58 13.25 25 7:08 19.00
48. https://finance.yahoo.com/quote/^DJI/chart
49. Seasonality
50. http://shawndubravac.com/2011/11/why-dont-retail-sales-show-more-seasonality/
51. https://blog.algorithmia.com/introduction-to-time-series/ Multiplicative Additive
52. Multiplicative Time series = t (trend) * s (seasonality) * n (noise)
53. https://blog.algorithmia.com/introduction-to-time-series/ Multiplicative Additive
54. Additive Time series = t (trend) + s (seasonality) + n (noise)
55. https://blog.algorithmia.com/introduction-to-time-series/ Multiplicative Additive
56. Decomposition
57. https://blog.algorithmia.com/introduction-to-time-series/
58. Further Reading
59. Intro to Time Series Stephanie Kim https://blog.algorithmia.com/introduction-to-time-series/
60. • Cumulative Moving Average, Exponential Moving Average • Forecasting: Autoregressive Integrated Moving Average • Anomaly Detection, Neural Nets in Time Series
61. Python Software for Time Series
62. Telegraf Collect InfluxDB Store Kapacitor Process Chronograf Visualize
63. InfluxDB • Architected for Time Series Data • Scalable & Reliable • High Rate of Ingest • Efficient compression • Built-in functionality for working with time
64. Client Libraries Go, Python, Ruby, Java, C#, JavaScript
65. Telegraf • Plugin-Driven Architecture: Inputs, Processors, Aggregators, Outputs • Easily extensible, high rate of adoption by community • Over 150 plugins • Generic plugins • Application-specific plugins
66. Telegraf: Generic Plugins • Socket Listener (TCP / UDP / Unix Socket) • HTTP • File • exec
67. Chronograf & Kapacitor • Chronograf is our visualization tool and has some administrative functions • Kapacitor is a processing engine • Stream or batch query data from database • Perform any kind of processing you’d like • Downsampling, Anomaly Detection, Alerting, Etc.
68. influxdb-python Library • Community Maintained • Key functions for working with InfluxDB: • InfluxDBClient • DataFrameClient • Series Helper & ResultSet github.com/influxdata/influxdb-python
69. community.influxdata.com Feedback appreciated! Twitter, GitHub: @noahcrowley Email: noah@influxdata.com
70. Demo code available at github.com/noahcrowley/pytn-2019/