Data Science Master. ElasticSearch and Kibana. Session 3: Kibana

Introduction to Kibana as a way to do some basic analytics and visualizations.

Transcript

1. Kibana Daniel Izquierdo Cortázar dizquierdo@bitergia.com / @dizquierdo Bitergia
2. Outline Introduction Kibana Terminology Basic and Advanced Use Creating your own widgets Timelion Painless (if there’s time!) Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 2
3. Introduction Visualize ElasticSearch data Basic visualizations: bars, pies, line, maps and others Create, share and export your dashboards Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 3
4. End User TimeFrame selection Drill Down Sharing/Embedding Data export (CSV…) Validation of data Search box Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 4
5. Config Tabs Discover: create Searches, JSON docs, mapping... Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 5
6. Config Tabs Visualize: create your own widgets Some terms: filters, ranges, Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 6
7. Config Tabs Dashboard: create your own dashboards Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 7
8. Config Tabs Settings: manage index patterns, create your own fields You need to actively import index patterns Eg: Import our index in ElasticSearch Eg: Index pattern restart to update mappings Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 8
9. Create a full dashboard First Design your dashboard What’s the goal of this? Some hints: try to be specific, have a goal, not too many widgets, understandable metrics… Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 9
10. Create a full dashboard Searches Widgets/Panels Dashboard Let’s create your first dashboard! Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 10
11. Timelion Focus on time series Useful when mixing different data sources Lots of time series functions not available in Kibana Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 11
12. Timelion Specification of index and timefield You can use by default parameters logstash-* @timestamp Or change those (Mng->Adv.Sett.->timelion) Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 12
13. Timelion Everything starts with .es(*) Functions start with “.” Specify our index: .es(index=git_opnfv, timefield=author_date) Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 13
14. Timelion Everything starts with .es(*) [for ElasticSearch] Functions start with “.” Specify our index: .es(index=git_opnfv, timefield=author_date) Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 14
15. Timelion Add a new Timelion chart with the Add option Multiple charts allow to have the hover-over cursor mirrored in all of them Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 15
16. Timelion Data Sources: ElasticSearch: .es() Graphite: .graphite() Quandl: .quandl() World Bank: .worldbank_indicators() and .worldbank() https://github.com/elastic/timelion/blob/master/FUNCTIONS.md Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 16
17. Timelion Functions: .abs(), .cusum(), .derivative(), .log(), .max(), .min(), .movingaverage(), .subtract(), .sum(), .trend(), etc... https://github.com/elastic/timelion/blob/master/FUNCTIONS.md Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 17
18. Timelion Example: Population of Spain and France: .worldbank_indicators('ES').worldbank_indicators('FR') https://github.com/elastic/timelion/blob/master/FUNCTIONS.md Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 18
19. Timelion Soft. dev. Analytics example: Evolution of commits: .es(timefield='author_date', index=git_opnfv) Cumulative: .es(timefield='author_date', index=git_opnfv).cusum() Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 19
20. Timelion Soft. dev. Analytics example: Evolution of commits and unique authors .es(timefield='author_date', index=git_opnfv, metric='cardinality:author_name').label('unique authors') .es(timefield='author_date', index=git_opnfv).label('commits') Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 20
21. Timelion Soft. dev. Analytics example: Looking for trend changes Basic trends: .es(timefield='author_date', index=git_opnfv).label('commits').trend().label('commits trend') Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 21
22. Timelion Soft. dev. Analytics example: Looking for trend changes Moving averages: .es(timefield='author_date', index=git_opnfv) .es(timefield='author_date', index=git_opnfv).movingaverage(window=3, position=center) .es(timefield='author_date', index=git_opnfv).movingaverage(window=15) Daniel Izquierdo Cortázar Máster en Data Science. ETSII. 22