The Open Documentation Dashboard - FOSDEM 2017

Slides for The ODF Dashboard talk at FOSDEM 2017

Transcript

1. The Open Document
   Foundation Dashobard
   Jesus M. Gonzalez-Barahona (URJC)
   @jgbarah
   jgb at bitergia dot com
   https://speakerdeck.com/bitergia
   FOSDEM 2017, Mozilla devroom
   Brussels (Belgium)
   February 4th 2017
   

   View Slide

2. Outline
   Some context
   The dashboard
   The software doing the magic
   Your turn
   

   View Slide

3. Some context
   

   View Slide

4. /me
   Like five years ago I
   was having coffees
   with the gang of
   Bitergia founders
   Involved in the
   company since then
   http://bitergia.com
   I work at
   Universidad Rey
   Juan Carlos...
   ...researching about
   software
   development
   http://gsyc.es/~jgb
   My two hats:
   

   View Slide

5. The dashboard
   

   View Slide

6. Bitergia
   Analytics
   Development
   Dashboard
   https://dashboard.documentfoundation.org
   

   View Slide

7. Bitergia
   Analytics
   Development
   Dashboard
   Git panel
   https://dashboard.documentfoundation.org
   

   View Slide

8. Bitergia
   Analytics
   Development
   Dashboard
   Gerrit panel
   https://dashboard.documentfoundation.org
   

   View Slide

9. Bitergia
   Analytics
   Development
   Dashboard
   Gerrit Timing
   panel
   https://dashboard.documentfoundation.org
   

   View Slide

10. Bitergia
    Analytics
    Development
    Dashboard
    Gerrit Backlog
    panel
    https://dashboard.documentfoundation.org
    

    View Slide

11. Bitergia
    Analytics
    Development
    Dashboard
    Bugzilla Issues
    panel
    https://dashboard.documentfoundation.org
    

    View Slide

12. Bitergia
    Analytics
    Development
    Dashboard
    Bugzilla Issues
    Backlog panel
    https://dashboard.documentfoundation.org
    

    View Slide

13. You can filter
    You can drill
    down
    You can share
    You can follow
    links
    https://dashboard.documentfoundation.org
    

    View Slide

14. The software
    doing the magic
    

    View Slide

15. Architecture
    Original
    Data Sources
    Retrieval
    Perceval
    Enrichment
    GrimoireELK
    Visualization
    Kibiter
    (Kibana4/5)
    ElasticSearch
    GrimoireLab
    component
    

    View Slide

16. GrimoireLab
    http://grimoirelab.github.io
    

    View Slide

17. GrimoireLab
    http://grimoirelab.github.io
    

    View Slide

18. GrimoireLab
    http://grimoirelab.github.io
    

    View Slide

19. GrimoireLab
    https://jgbarah.gitbooks.io/grimoirelab-training/
    

    View Slide

20. Your turn
    

    View Slide

21. Your turn
    You can play with the dashboard
    You can play with the ElasticSearch data
    You can produce scripts, link the data to
    programs
    You can improve GrimoireLab
    You can report bugs, ask for features
    

    View Slide

22. Working with
    data Download data from the dashboard UI (CSV)
    

    View Slide

23. Working with
    data
    Query ElasticSearch API with curl
    $ curl -XGET "http://elasticsearch_url/git/_search/?size=1&pretty"
    {...
    "hits" : {
    "total" : 407,
    "hits" : [ {
    "_index" : "commits",
    "_type" : "summary",
    "_id" : "AVfPp9Po5xUyv5saVPKU",
    "_score" : 1.0,
    "_source" : {
    "hash" : "d1253dd9876bb76e938a861acaceaae95241b46d",
    "commit" : "Santiago Dueñas ",
    "author" : "Santiago Dueñas ",
    "author_date" : "Wed Nov 18 10:59:52 2015 +0100",
    "files_no" : 3,
    "commit_date" : "Wed Nov 18 14:41:21 2015 +0100"
    }} ] } }
    

    View Slide

24. Your turn
    from elasticsearch import Elasticsearch
    from elasticsearch_dsl import Search
    es =
    Elasticsearch(["http://xx:yy@dashboard.documentfoundation.org/data”]
    s = Search(using=es, index=’git’)
    s = s.filter('range', files={'gt':0})
    s = s.filter('range', author_date={'gt': datetime(2016, 7, 1)})
    s.aggs.metric('commits', 'cardinality', field='hash')
    s.aggs.bucket('histogram', 'date_histogram',
    field='author_date', interval='quarter')
    by_q = s.execute()
    for quarter in by_q.to_dict()['aggregations']['histogram']['buckets']:
    print("Unique commits for quarter starting on ",
    Quarter['key_as_string'], ": ", quarter['doc_count'])
    Python scripting
    

    View Slide

25. The Cauldron
    http://cauldron.io
    

    View Slide

26. Your turn
    Enjoy!
    http://grimoirelab.github.io
    https://dashboard.documentfoundation.org
    

    View Slide