Meet GrimoireLab: Making your life easier

Meet GrimoireLab: Making your life easier

Presentation at ICSME'18 (Madrid, Sept 25th 2018), tailored mainly to researchers. GrimoireLab is a Python toolset for software development analytics. It can deal with data from 20+ kinds of data sources (from git, GitHub and GitLab to Slack, mailing lists, Jenkins, Bugzilla, Jira or Gerrit) with a uniform Python API. It provides tools for identity management, data enrichment, visualization, and massive data retrieval.

B7081d0131ad47821467b8e81434cf7a?s=128

Jesus M. Gonzalez-Barahona

September 24, 2018
Tweet

Transcript

  1. 1.
  2. 2.

    chaoss.community /me Like six years ago I was having coffees

    with the gang of Bitergia founders Involved in the company since then bitergia.com I work at Universidad Rey Juan Carlos... ...researching about software development gsyc.es/~jgb My two hats:
  3. 7.

    chaoss.community GrimoireLab Software development analytics with free, open source software

    (a CHAOSS project) chaoss.github.io/grimoirelab chaoss.github.io/grimoirelab-tutorial
  4. 10.

    chaoss.community Installing from pip $ python3 -m venv /tmp/gl $

    source /tmp/gl/bin/activate (gl) $ pip install --upgrade pip setuptools wheel (gl) $ pip install grimoirelab (gl) $ grimoirelab -v chaoss.github.io/grimoirelab-tutorial/basics/install.html
  5. 11.

    chaoss.community Running from Docker images docker run -p 5601:5601 -p

    9200:9200 -v $(pwd)/credentials.cfg:/override.cfg -t grimoirelab/full credentials.cfg: [github] api-token = XXX
  6. 13.

    chaoss.community Example: Perceval (gl) $ pip install perceval (gl) $

    perceval git https://github.com/grimoirelab/perceval (gl) $ perceval github grimoirelab perceval --sleep-for-rate -t XXXXX ...
  7. 14.

    chaoss.community Example: Perceval from perceval.backends.core.git import Git repo_url = 'http://github.com/grimoirelab/perceval'

    repo_dir = '/tmp/perceval' repo = Git(uri=repo_url, gitpath=repo_dir) for commit in repo.fetch(): print(commit['data']['commit'])
  8. 15.

    chaoss.community Producing a dashboard (assumes Elasticsearch at http://localhost:9200, Kibiter at

    http://localhost:5601 ) Prepare: mordred.cfg Prepare: projects.json Maybe: identities.yaml Maybe: menu.yaml (es) mordred -c mordred.cfg chaoss.github.io/grimoirelab-tutorial/sirmordred/dashboard.html
  9. 16.

    chaoss.community Using Perceval to compute metrics • Get items from

    the data source • Store them in a data structure • Compute metrics • You can also use Pandas, of course github.com/chaoss/wg-gmd/blob/master/examples/Code_ Commits.ipynb
  10. 19.

    chaoss.community Computing metrics from enriched indexes • Connect to Elasticsearch

    indexes (produced with p2o, mordred) • Query them to get items of interest • Compute metrics • You can also use Pandas, of course chaoss.github.io/grimoirelab-tutorial/python/indexes.html
  11. 21.

    chaoss.community Example: producing a dashboard (es) $ pip install grimoire-mordred

    (es) $ mordred -c mordred.cfg (assumes Elasticsearch at http://localhost:9200, Kibiter at http://localhost:5601 ) Needs: mordred.cfg, projects.json, identities.yaml, menu.yaml chaoss.github.io/grimoirelab-tutorial/sirmordred/dashboard.html
  12. 22.

    chaoss.community GrimoireLab Software development analytics with free, open source software

    (a CHAOSS project) chaoss.github.io/grimoirelab chaoss.github.io/grimoirelab-tutorial
  13. 23.

    chaoss.community License & Attributions “Pain Relief”, by Nick Youngson, Alpha

    Stock Images, CC by-sa 3.0 “Broken Glass Sadness Disappointment Can't Sad”, from Max Pixel, CC0 Slides licensed under Creative Commons ShareAlike 4.0 (CC by-sa 4.0) Copyright CHAOSS & Jesus M. Gonzalez-Barahona 2018