How open development data benefits your project and your community

Slides for the presentation given during LinuxTag'14 about measuring free / open source software development, the need of development analytics and their possible areas of interest and available tools.

Transcript

1. How open development data benefits your project and your community Jesus M. Gonzalez-Barahona jgb@bitergia.com @jgbarah Bitergia / LibreSoft (URJC) http://bit.ly/open-sw-analytics Linux Tag 2014 Berlin (Germany), May 8th 2014 Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 1 / 33
2. c 2012-2014 Bitergia Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 3.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/3.0/ Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 2 / 33
3. Structure of the presentation 1 Measuring free / open source software development 2 Why open development analytics? 3 Areas of interest 4 Tools Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 3 / 33
4. Measuring free / open source software development Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 4 / 33
5. A successful development model? Free (open source) software has shown to be a great success ...but there are many details to be understood ...and (a lot of) interest in understanding ...but there is room for improvement ...and (a lot of) interest in improving Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 5 / 33
6. A successful development model? (2) There are *lots* of development models Common characteristics for many of them: Community-based development Intensive use of tools, processes for coordination Open development models (as opposed to in-house, hidden models) Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 6 / 33
7. The importance of the community [Crowd at FOSDEM 2008, by Jes´ us Corrius, CC Attribution 2.0] http://www.flickr.com/photos/jcorrius/2302302707/ Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 7 / 33
8. The importance of the community (2) Persons (and organizations) with different interests common goals Need for coordination, common decision making Availability of data as a tool: Transparency to the community (fairness) Transparency to third parties (trust) Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 8 / 33
9. Diversity of tools, processes Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 9 / 33
10. Diversity of tools, processes (2) Despite diversity, a large fraction of projects: Use tools & services from a small set git / svn / hg Bugzilla / Jira / GitHub tickets Gerrit Mailman / Gmane ... use similar processes: bug fixing coordination using tickets pre-merge code review general discussion in mailing lists ... Collection and analysis of data is possible Publication of data makes sense Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 10 / 33
11. Why open development analytics? Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 11 / 33
12. From open development to open development analytics Information about code, community, development for open development projects can be retrieved, organized, analyzed Let’s publish analytics results & data Open Development Analytics: A new standard for transparency Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 12 / 33
13. Open development analytics Who is interested? Large & small free software communities ...and thousands of large & small companies, public administrations, foundations participating in them, depending on their software [Who can afford not to be interested? It is a key strategic need for many actors] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 13 / 33
14. Open development analytics Why? Free software produced with open development models is more and more important for IT users, producers, integrators It is different & complex, yet transparent, many details are public, and it can be improved Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 14 / 33
15. Areas of interest Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 15 / 33
16. Some areas of interest Performance (understanding activity) Company participation (beyond copyright notices) Transparency (available information) Auditing (certify participation, experience, etc.) Profiling (key people, companies) Neutrality (fair treatment) Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 16 / 33
17. Areas of interest: community management Issues Parameters Activity Raw volume, participants, ... Reliability Reaction times, pending issues, ... Sustainability Growth rate, structure, ... Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 17 / 33
18. Areas of interest: community management [Puppet committers community: Attraction / retention] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 18 / 33
19. Areas of interest: community management [Linux kernel: age of developers per cohort] http://blog.bitergia.com/2013/02/01/ demographics-of-linux-kernel-developers-how-old-are-they/ Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 19 / 33
20. Areas of interest: engineering Liferay: time-to-close tickets (quantiles) http://blog.bitergia.com/ 2012/10/25/preview-of-the-analysis-of-liferay/ Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 20 / 33
21. Areas of interest: engineering [MediaWiki community: tickets-related parameters] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 21 / 33
22. Areas of interest: company participation [Main companies in OpenStack Havana (partial view)] http://activity.openstack.org/dash/releases/ Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 22 / 33
23. Areas of interest: company participation [IBM participation in OpenStack Havana (partial view)] http://activity.openstack.org/dash/releases/company.html?company=IBM Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 23 / 33
24. Areas of interest: transparency Development communities: companies and developers working together Policies, procedures, tools, source code... and development data Do they really provide enough data to enable assessment? Analysis of all repositories (data sources)... ...and associated information (eg: affiliation) Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 24 / 33
25. Areas of interest: auditing [OpenStack top contributors (December 2013)] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 25 / 33
26. Areas of interest: profiling [oVirt developer profile] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 26 / 33
27. Areas of interest: neutrality q q q q q q q q 0 1 2 3 250 500 1000 2000 4000 Number of accepted reviews Iterations per accepted review (median) [WebKit code review data per company (2012)] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 27 / 33
28. Tools Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 28 / 33
29. Tools: Grimoire system MetricsGrimoire: Free software for retrieving data from repositories vizGrimoire: Free software for analyzing, visualizing data Grimoire Dashboard: Many panels, different views of the project (charts, summaries, statistic analysis) Commercially supported by Bitergia Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 29 / 33
30. Tools: Grimoire Dashboard [Dashboard for the GlusterFS project http://projects.bitergia.com/redhat-glusterfs-dashboard/] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 30 / 33
31. Summarizing Let’s go one step further: Open Development Analytics Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 31 / 33
32. Relationship with EU-funded R&D projects Markos: License analyzer New tools for software development analysis Production of linked open data PROSE: Software development analytics to track results of R&D projects Open Source Projects Europe forge: development analytics facilities http://www.markosproject.eu/ http://www.ict-prose.eu/ https://opensourceprojects.eu/ Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 32 / 33
33. This is the end [Questions, comments...] Jesus Gonzalez-Barahona (Bitergia) Open development data Linux Tag 2014 33 / 33