The Open Documentation Dashboard - FOSDEM 2017

Slides for The ODF Dashboard talk at FOSDEM 2017

Transcript

1. The Open Document Foundation Dashobard Jesus M. Gonzalez-Barahona (URJC) @jgbarah jgb at bitergia dot com https://speakerdeck.com/bitergia FOSDEM 2017, Mozilla devroom Brussels (Belgium) February 4th 2017
2. Outline Some context The dashboard The software doing the magic Your turn
3. Some context
4. /me Like five years ago I was having coffees with the gang of Bitergia founders Involved in the company since then http://bitergia.com I work at Universidad Rey Juan Carlos... ...researching about software development http://gsyc.es/~jgb My two hats:
5. The dashboard
6. Bitergia Analytics Development Dashboard https://dashboard.documentfoundation.org
7. Bitergia Analytics Development Dashboard Git panel https://dashboard.documentfoundation.org
8. Bitergia Analytics Development Dashboard Gerrit panel https://dashboard.documentfoundation.org
9. Bitergia Analytics Development Dashboard Gerrit Timing panel https://dashboard.documentfoundation.org
10. Bitergia Analytics Development Dashboard Gerrit Backlog panel https://dashboard.documentfoundation.org
11. Bitergia Analytics Development Dashboard Bugzilla Issues panel https://dashboard.documentfoundation.org
12. Bitergia Analytics Development Dashboard Bugzilla Issues Backlog panel https://dashboard.documentfoundation.org
13. You can filter You can drill down You can share You can follow links https://dashboard.documentfoundation.org
14. The software doing the magic
15. Architecture Original Data Sources Retrieval Perceval Enrichment GrimoireELK Visualization Kibiter (Kibana4/5) ElasticSearch GrimoireLab component
16. GrimoireLab http://grimoirelab.github.io
17. GrimoireLab http://grimoirelab.github.io
18. GrimoireLab http://grimoirelab.github.io
19. GrimoireLab https://jgbarah.gitbooks.io/grimoirelab-training/
20. Your turn
21. Your turn You can play with the dashboard You can play with the ElasticSearch data You can produce scripts, link the data to programs You can improve GrimoireLab You can report bugs, ask for features
22. Working with data Download data from the dashboard UI (CSV)
23. Working with data Query ElasticSearch API with curl $ curl -XGET "http://elasticsearch_url/git/_search/?size=1&pretty" {... "hits" : { "total" : 407, "hits" : [ { "_index" : "commits", "_type" : "summary", "_id" : "AVfPp9Po5xUyv5saVPKU", "_score" : 1.0, "_source" : { "hash" : "d1253dd9876bb76e938a861acaceaae95241b46d", "commit" : "Santiago Dueñas <sduenas@bitergia.com>", "author" : "Santiago Dueñas <sduenas@bitergia.com>", "author_date" : "Wed Nov 18 10:59:52 2015 +0100", "files_no" : 3, "commit_date" : "Wed Nov 18 14:41:21 2015 +0100" }} ] } }
24. Your turn from elasticsearch import Elasticsearch from elasticsearch_dsl import Search es = Elasticsearch(["http://xx:yy@dashboard.documentfoundation.org/data”] s = Search(using=es, index=’git’) s = s.filter('range', files={'gt':0}) s = s.filter('range', author_date={'gt': datetime(2016, 7, 1)}) s.aggs.metric('commits', 'cardinality', field='hash') s.aggs.bucket('histogram', 'date_histogram', field='author_date', interval='quarter') by_q = s.execute() for quarter in by_q.to_dict()['aggregations']['histogram']['buckets']: print("Unique commits for quarter starting on ", Quarter['key_as_string'], ": ", quarter['doc_count']) Python scripting
25. The Cauldron http://cauldron.io
26. Your turn Enjoy! http://grimoirelab.github.io https://dashboard.documentfoundation.org