GrimoireCon 2017 Brussels - Building visualizations (training)

Slides for GrimoireCon 2017 Brussels tutorial/training about building Grimoire Lab visualizations and dashboards

Transcript

1. Bitergia Analytics Dashboard & Platform: Creating visualizations & dashboards dizquierdo@bitergia.com jgb@bitergia.com jsmanrique@bitergia.com http://bitergia.com
2. Outline Bitergia Analytics Dashboard The Kibana editor Visualizations and Searches Modifying and creating dashboards Dashboard: https://xxx.biterg.io Editor: https://xxx.biterg.io/edit ElasticSearch: https://xxx.biterg.io/data
3. Bitergia Analytics Dashboard
4. Overview
5. Architecture Original Data Sources Mining Tools Perceval @ GrimoireLab Info Enrich. GrimoireELK @ GrimoireLab Viz ElasticSearch + Kibiter (Kibana4)
6. Some Features Drill down Time frame selection Sharing / embedding Data export (CSV…) Query API (ElasticSearch) Allowed users can create widgets and panels Easy validation Links to real artifacts (commits, tickets, etc.) Search box
7. The Kibana Editor
8. Kibana editor for the dashboard Same data, with interface for editing - Edit & create visualizations - Tweak searches - Build your own panels
9. Main components Index patterns (mapping to ES indexes) Searches (may include filters) Visualizations (on top of searches or indexes) Dashboards (panels)
10. Index patterns
11. Index patterns Mapping in Kibana of one or more Elasticsearch indexes Fields of different types (number, string, date) Unfolding items, field contents are shown… ...and they can be used as filters
12. Visualizations & Searches
13. Searches
14. Searches Views for index patterns May include a filter May include only some fields Can be included in a dashboard Can be saved / recovered
15. Visualizations
16. Visualizations
17. Visualizations Based on an index pattern, or in a search Different kinds Data organized in metrics and buckets Can be saved / retrieved / edited
18. Visualizations
19. Modifying & Creating Dashboards
20. Dashboards (panels)
21. Dashboards (panels) Visualizations can be added, removed, moved, edited Dashboards can be saved, retrieved Filters, date filters can be saved with the dashboard
22. Final remarks
23. How to play You can create your own Start by saving panels with a new name Edit visualizations (save with different names) If needed, modify the searches
24. ElasticSearch All indexes available from ElasticSearch instance You can query live REST HTTP interface Modules for Java, JavaScript, Python, and many more languages
25. ElasticSearch (just checking) $ curl -XGET 'https://user:XXX@project.biterg.io/data' { "name" : "Morpheus", "cluster_name" : "elasticsearch", "version" : { "number" : "2.2.2", "build_hash" : "fcc01dd81f4de6b2852888450ce5a56436fd5852", "build_timestamp" : "2016-03-29T08:49:35Z", "build_snapshot" : false, "lucene_version" : "5.4.1" }, "tagline" : "You Know, for Search" }
26. ElasticSearch (index description) $ curl -XGET 'https://user:XXX@project.biterg.io/data/git?pretty=true' { "git_project_160620_enriched_160913" : { "aliases" : { "git" : { } }, "mappings" : { "items" : { "dynamic" : "true", "properties" : { "Author" : { "type" : "string", "index" : "not_analyzed" }, "Committer" : { ...
27. ElasticSearch (query for data) $ curl --silent -XPOST \ 'https://user:XXX@project.biterg.io/data/git/_search?q=lines_changed:0' ... { "_index": "git_project_160620_enriched_160913", "_type": "items", "_id": “...", "_source": { "is_git_commit": 1, "commit_date": "2013-09-10T21:29:28", "files": 1, "hash_short": "6bfe1f", "author_domain": "ifup.org", "message_analyzed": "Merge pull request #213 from...", ...
28. Documentation A lot of documentation & videos are available: - GrimoireLab Training https://jgbarah.gitbooks.io/grimoirelab-training/ - ElasticSearch: The Definitive Guide https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html - Kibana User Manual https://www.elastic.co/guide/en/kibana/current/index.html
29. Come and play Feel free to play with the dashboard Warn us if you break something! (We can recover from scratch) Create your own visualizations, panels We're creating documentation for fields in indexes