$30 off During Our Annual Pro Sale. View Details »

The MetricsGrimoire toolkit

The MetricsGrimoire toolkit

Presentation at FLOSS Community Metrics Meeting EU 2015. Brussels, Belgium, January 30th 2015.

Jesus M. Gonzalez-Barahona

January 29, 2015
Tweet

More Decks by Jesus M. Gonzalez-Barahona

Other Decks in Technology

Transcript

  1. The MetricsGrimoire toolkit
    Jesus M. Gonzalez-Barahona
    [email protected]
    http://twitter.com/jgbarah
    Bitergia
    GSyC/LibreSoft (Universidad Rey Juan Carlos)
    FLOSS Community Metrics Meeting,
    Brussles, Belgium, January 30th 2015
    http://bit.ly/fcm2eu-grimoire
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 1 / 26

    View Slide

  2. c 2012-2015 Bitergia
    Some rights reserved. This presentation is distributed under the
    “Attribution-ShareAlike 3.0” license, by Creative Commons, available at
    http://creativecommons.org/licenses/by-sa/3.0/
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 2 / 26

    View Slide

  3. Structure of the presentation
    1 Me, myself, my circumstances
    2 Open source and beyond
    3 Grimoire
    4 MetricsGrimoire
    5 Why open development analytics?
    6 Drawing conclusions
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 3 / 26

    View Slide

  4. Me, myself, my circumstances
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 4 / 26

    View Slide

  5. My two hats: University, company
    Uni Rey Juan Carlos:
    LibreSoft research team
    Understanding free, open source software development
    ...since 2000
    Data analytics approach
    Bitergia:
    From research to practice
    The software development analytics company
    Dashboards, reports, consultancy...
    http://gsyc.es/~jgb http://bitergia.com
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 5 / 26

    View Slide

  6. Bitergia: analytics for your peace of mind
    Started operations in July 2012
    Builds on the experience of LibreSoft R&D group
    Professional products and services, focused on:
    Grimoire Dashboard: software development analytics
    (including community metrics)
    Specific studies and reports
    (based on metrics and facts collection)
    http://bitergia.com/projects.html
    http://blog.bitergia.com
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 6 / 26

    View Slide

  7. Open source and beyond
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 7 / 26

    View Slide

  8. A new dimension of openness
    When we develop in the open
    we produce a great deal of data
    about how we develop
    “Show me the development data”
    as a step beyond
    “show me the code”
    Let’s do it!
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 8 / 26

    View Slide

  9. Grimoire
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 9 / 26

    View Slide

  10. Data lives in many different repositories
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 10 / 26

    View Slide

  11. Data has to be extracted, mined...
    Repositories don’t release all their data easily:
    tools are needed to retrieve and extract it
    Data includes many complexities and details
    tools are needed to filter, organize it
    MetricsGrimoire
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 11 / 26

    View Slide

  12. Data has to be analyzed, visualized...
    Casual observation is not enough:
    tools are needed
    for statistical and other kinds of analysis
    Analysis is not enough:
    tools are needed for interactive visualization
    vizGrimoire
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 12 / 26

    View Slide

  13. The Grimoire landscape
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 13 / 26

    View Slide

  14. Grimoire: tools for retrieval, analysis, visualization
    MetricsGrimoire vizGrimoire Grimoire
    Dashboard
    Data retrieval Analysis, User
    from visualization interaction,
    repositories panels, views
    Free software
    Commercial support available
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 14 / 26

    View Slide

  15. The Grimoire Development Dashboard
    http://dashboard.eclipse.com
    http://demo.bitergia.com
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 15 / 26

    View Slide

  16. MetricsGrimoire
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 16 / 26

    View Slide

  17. MetricsGrimoire tools (1)
    Retrieving information repositories:
    CVSAnalY: source code management
    CVS, Subversion, git... (Mercurial, others through git)
    Bicho: issue tracking systems
    Bugzilla, Jira, GitHub, Allura, Launchpad,
    Storyboard...
    & code review systems (Gerrit)
    MLStats: mailing lists
    mbox files, Mailman & GMane archives...
    Sybyl: Q&A websites
    Askbot, Stackoverflow, Discourse
    http://metricsgrimoire.github.io
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 17 / 26

    View Slide

  18. MetricsGrimoire tools (1)
    pullpo: Pull request analysis
    GitHub pull requests
    MediaWikiAnalysis: wikis
    IRCAnalysis: IRC logs
    Several log formats
    Octopus: crawl forges
    Puppet forge...
    sortinghat: manage identities
    Most MetricsGrimoire databases
    Automator: automating Grimoire toolchains
    https://github.com/MetricsGrimoire
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 18 / 26

    View Slide

  19. Why open development analytics?
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 19 / 26

    View Slide

  20. From open development to open development analytics
    Information about code, community, development
    for open development projects
    can be retrieved, organized, analyzed
    Let’s publish analytics results & data
    Open Development Analytics:
    A new standard for transparency
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 20 / 26

    View Slide

  21. Open development analytics
    Who is interested?
    Developers
    Project managers
    Community managers
    Evaluators
    ...
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 21 / 26

    View Slide

  22. Open development analytics
    Why?
    Free software produced
    with open development models
    is more and more important
    for IT users, producers, integrators
    It is different & complex,
    yet transparent,
    many details are public,
    and it can be improved
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 22 / 26

    View Slide

  23. Some areas of interest
    Performance (understanding activity)
    Company participation (beyond copyright
    notices)
    Transparency (available information)
    Auditing (certify participation, experience, etc.)
    Profiling (key people, companies)
    Neutrality (fair treatment)
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 23 / 26

    View Slide

  24. Drawing conclusions
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 24 / 26

    View Slide

  25. In summary...
    Let’s analyze free software with free software,
    and let’s publish the results!
    MetricsGrimoire extracts data from repositories
    vizGrimoire analyzes it and produces dashboards
    They’re simple to use, and becoming simpler
    We can incrementally develop a powerful platform
    Make your pet project
    open-development-data ready!
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 25 / 26

    View Slide

  26. Thanks go to...
    The *Grimoire community
    LibreSoft developers who developed a sizable part of MetricsGrimoire
    The community maintaining MetricsGrimoire
    Bitergia developers producing vizGrimoire
    The (future) community maintaining vizGrimoire
    The many free software developers that produced all the software on
    which these tools rely
    The many free software developers that produced all the software that
    gives us projects to analyze
    http://metricsgrimoire.github.io
    http://bitergia.com
    Slides: http://bit.ly/fcm2eu-grimoire
    Jesus Gonzalez-Barahona (Bitergia) The MetricsGrimoire toolkit FCM2 EU 2015 26 / 26

    View Slide