Evaluating Free / Open Source Software Projects Jesus M. Gonzalez-Barahona [email protected] http://twitter.com/jgbarah Bitergia GSyC/LibreSoft (Universidad Rey Juan Carlos) FLOSS Community Metrics Meeting, Portland, USA, July 19th 2015 http://bit.ly/fcm2-evaluation Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 1 / 30 

c 2012-2015 Bitergia Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 3.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/3.0/ Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 2 / 30 

Structure of the presentation 1 Some context 2 The origins 3 The book 4 Contents 5 Bonus track 6 The future Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 3 / 30 

Some context Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 4 / 30 

I am I, and my circumstance Co-founder of Bitergia: The software development analytics company Dashboards, reports, consultancy... http://bitergia.com Associate Professor at URJC: Understanding free, open source software development Empirical, data analytics approach http://gsyc.es/~jgb Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 5 / 30 

Bitergia: analytics for your peace of mind Started operations in July 2012 Builds on the experience of LibreSoft R&D group Professional products and services, focused on: Grimoire Dashboard: software development analytics (including community metrics) Specific studies and reports (based on metrics and facts collection) http://bitergia.com/ projects.html http://blog.bitergia.com Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 6 / 30 

The origins Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 7 / 30 

Preparing the book (and not noticing) Gathering ideas and materials for a long time First: research context Then: education context Finally: the industry needs it During the last years, the field of FOSS evaluation has matured we can share our expertise in it FOSS projects (not products) evaluation!!! Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 8 / 30 

Preparing the book (and not noticing) Knowledge is maturing Tools are emerging Interest is raising It seems about the right time! Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 9 / 30 

The book Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 10 / 30 

First version released (yesterday night) http://jgbarah.gitbooks.io/evaluating-foss-projects/ Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 11 / 30 

Structure What is FOSS projects evaluation. How it can be done What can be expected from it Different aspects of FOSS project evaluation (community, processes, ...) Tools available including details of Grimoire (to be written) http://jgbarah.gitbooks.io/evaluating-foss-projects/ Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 12 / 30 

Contents Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 13 / 30 

Before performing evaluation Select evaluation targets: Can be defined in advance (eg: your project) Or you may need some research Objectives and evaluation criteria: Define the objectives of your evaluation Base the evaluation criteria on them You may use Goal-Question-Metric Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 14 / 30 

Performing evaluation Surveys (to users, to developers...) Interviews with experts Study of documentation Analysis of source code Analysis of software development repositories Study of project bylaws The last three: particular to FOSS Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 15 / 30 

Evaluation results Tags: binary valued, such as “mature” Scales: numbers or enumerations, such as “closed tickets: 67 %” or “maturity: high” Metadata: more detailed description Free text / factoids Quantitative vs. qualitative results Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 16 / 30 

Differences for FOSS Access to the product Availability of information Community Competing market Transparency (or lack of it) The specific case of open development Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 17 / 30 

Sources of information Source code management (git, ...) Issue tracking systems (Bugzilla, GitHub tickets, ...) Code review systems (Gerrit, GitHub pull requests, ...) Asynchronous communication systems (mailing lists, StackOverflow, ...) Synchronous communication systems (irc, Slack, ...) Testing & continuous integration Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 18 / 30 

Evaluation of FOSS communities Scopes: developers, contributors, users, ecosystem (different data sources, different traces) Activity (actions, persons) The community aging chart Time zones, time of activity Affiliation Bus factor, pony factor Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 19 / 30 

Evaluation of development process Performance (processes open, closed, ratios) Periods, snapshots (evolution, backlogs) Workflow patterns Diversity Neutrality Cases: tickets, code review, mailing lists Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 20 / 30 

Evaluation models Long history of proposed models Some specific cases: OpenBRR (Business Readiness Rating) QSOS (Qualification and Selection of Open Source) ... Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 21 / 30 

Evaluation dashboards (GitHub) Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 22 / 30 

Evaluation dashboards (OpenHub) Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 23 / 30 

Evaluation dashboards (Stackalytics) Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 24 / 30 

Evaluation dashboards (Grimoire) Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 25 / 30 

Evaluation dashboards (Grimoire-NG) Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 26 / 30 

Bonus track Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 27 / 30 

Two new toys Grimoire-NG: http://projects.bitergia.com/ previews/ng/ Cauldron (do it yourself): http://biterg.io Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 28 / 30 

The future Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 29 / 30 

Still a lot to do Improve, complete existing chapters A whole part on practical matters still missing To be used in second semester 2015-16 Master on free software UOC/URJC Using GitBook/GitHub, for easing contribution Using Markdown, for easing contribution You can contribute!!! http://jgbarah.gitbooks.io/evaluating-foss-projects/ Jesus Gonzalez-Barahona (Bitergia) Evaluating Free / Open Source Software Projects FCM2 US 2015 30 / 30