Kibana + Timelion: Time series with the Elastic Stack

Timelion is an extension to Kibana that is dedicated to time series processing and vizualisation based on a powerful expression language.

We start by a overview of the Elastic Stack 5.0 release, do a quick overview of Kibana before diving into Timelion

Talk given at Capitole du Libre in Toulouse, FR
https://2016.capitoledulibre.org/

Transcript

1. 1 Sylvain Wallez @bluxte Kibana + Timelion: Time series with the Elastic Stack
2. 2 Agenda From ELK to Elastic Stack 5.0 Kibana Timelion Conclusion 1 2 3 4
4. 4 The Elastic Stack Elastic Cloud Security Monitoring Aler0ng Graph X-Pack Kibana User Interface Elasticsearch Store, Index, & Analyze Ingest Logstash Beats +
5. 5 What’s new in Elastic Stack 5.0 ? ‒  Dimensional fields – speed up and better compression of numerical data ‒  Ingest node – avoids using Logstash for simple setups ‒  Rollover & Shrink API – better handling of non-hot data ‒  Painless – new fast & secure scripting langage ‒  Complete UI redesign ‒  Control center: config, monitoring, dev tools… and dashboards ‒  Timelion: for time series Unified release, same version number for all products
6. 6 What’s new in Elastic Stack 5.0 ? •  ‒  Up to 50% faster ‒  Integration with Kafka ‒  New filters •  ‒  Libbeat – framework pour construire des agents ‒  MetricBeat, PacketBeat, FileBeat ‒  {Community}Beats, lots of them ‒  Immediate availability of new versions ‒  Automated upgrades Unified release, same version number for all products
7. 7 Agenda From ELK to Elastic Stack 5.0 Kibana Timelion Conclusion 1 2 3 4
8. 8 Kibana evolution Data Visualization Management
9. 9 Kibana evolution: 4.x Data Visualization Management Discover Dashboard Visualize
10. 10 Kibana evolution: 4.x Data Visualization Management Discover Dashboard Visualize Monitoring
11. 11 Graph Kibana evolution: 4.x Data Visualization Management Discover Dashboard Visualize Monitoring
12. 12 Graph Timelion Sense Kibana evolution: 4.x Data Visualization Management Discover Dashboard Visualize Monitoring
13. 13 Discover Dashboard Graph DevTools Timelion Console Monitoring Visualize Kibana evolution: 5.0 Data Visualization Management
14. 14 Discover Dashboard Graph Settings Users DevTools Timelion Monitoring Visualize Console Kibana evolution: 5.0 Data Visualization Management
15. 15 Creating a Kibana dashboard 1 2 3 4 Configure Select indices Discover Explore & filter Visualize Create charts Dashboard Layout charts 5 It’s alive! Live update & filtering
17. 17 Creating a Kibana dashboard 1 2 3 4 Configure Select indices Discover Explore & filter Visualize Create charts Dashboard Layout charts 5 It’s alive! Live update & filtering
19. 19 Creating a Kibana dashboard 1 2 3 4 Configure Select indices Discover Explore & filter Visualize Create charts Dashboard Layout charts 5 It’s alive! Live update & filtering
21. 21 Creating a Kibana dashboard 1 2 3 4 Configure Select indices Discover Explore & filter Visualize Create charts Dashboard Layout charts 5 It’s alive! Live update & filtering
23. 23 Creating a Kibana dashboard 1 2 3 4 Configure Select indices Discover Explore & filter Visualize Create charts Dashboard Layout charts 5 It’s alive! Live update & filtering
25. 25 25 Demo!
26. 26 Agenda From ELK to Elastic Stack 5.0 Kibana Timelion Conclusion 1 2 3 4
27. 27 Why Timelion? • Elasticsearch queries ‒  Select lots of items (lightning fast) ‒  Aggregations make them understandable • Need more than that ‒  Correlation, calculation, filtering ‒  Assemble multiple data sources • Timelion started as an experiment ‒  Escape the UI and use the flexibility of a language Because there’s more than search
28. 28 Time series resampling A common time reference to allow calculations Value Value Time Time Bucket
29. 29 Timelion expression language A transformation & aggregation pipeline ES World Bank Graphite - Sampling - Aggregation - Calculation - Graphic attributes Data sources Transformations Time series - Data - Metadata - Graphic attributes
30. 30 The Timelion expression language • Functions ‒  abs, cusum, derivative, fit, holt, log, min, max, movingaverage, movingstd, sum, trend • Combinations ‒  add, divide, multiply, plus, substract • Filtering ‒  condition, if • Graphic attributes ‒  bars, color, hide, label, legend, lines, points, range, title, yaxis • Data sources ‒  elastic, graphite, wbi, quantl A rich function library
31. 31 The Timelion expression language
32. 32 32 Demo!
33. 33 Agenda From ELK to Elastic Stack 5.0 Kibana Timelion Conclusion 1 2 3 4
34. 34 Questions? Answers!