The numbers of the open cloud (OpenStack Summit edition)

Talk at OpenStack Summit, November 5th, Paris (France). [Video]

The talk will present a quantitative analysis of the projects producing the main free, open source software cloud platforms: OpenStack, Apache CloudStack, OpenNebula and Eucalyptus. The analysis will focus on the communities behind those projects, their main development parameters, and the trends that can be observed.

Understanding the inner life of free / open source software projects is of fundamental importance to developers, users and decision makers. This talk will help to understand what's happening behind the curtains in the most relevant open cloud platforms, by analyzing the rich data obtained from their developing repositories.

The talk will cover, for the four analyzed projects, the following topics:

* How open is their development model, and to which extent they provide access to their development data.

* Some parameters related to the overall activity and effort put into the development and maintenance of their products.

* The main characteristics of their development community: size, diversity, engagement, attraction, etc.

* The main parameters characterizing their performance, such as time-to-fix tickets or time-to-review patches.

* Some visualizations of the most relevant aspects.

All the tools used to produce the analysis presented in this talk are free, open source software, based on the Grimoire technology, http://vizgrimoire.github.io. The talk will also provide some hints on how to reproduce the study, on these or other projects.

Illustrative example of the kind of information that will be presented: OpenStack Grimoire Dashboard, http://activity.openstack.org/dash

This talk was an update and extension of the one I delivered in the Open Cloud Day, at OSCON 2014.

Transcript

1. The numbers of the Open Cloud Jesus M. Gonzalez-Barahona jgb@bitergia.com @jgbarah Bitergia / LibreSoft (URJC) http://bit.ly/openstack-opencloud OpenStack Summit 2014 Paris, France, November 5th 2014 Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 1 / 57
2. c 2012-2014 Bitergia Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 3.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/3.0/ Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 2 / 57
3. Structure of the presentation 1 The context 2 The study 3 What: the source code 4 How: the process 5 Who: the community 6 Bonus track 7 The end Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 3 / 57
4. The context Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 4 / 57
5. Me, myself, my circumstances Bitergia: From research to the real world The software development analytics company Dashboards, reports, consultancy... Uni Rey Juan Carlos: LibreSoft research team Understanding free, open source software development Data analytics approach http://bitergia.com http://gsyc.es/~jgb Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 5 / 57
6. Four usual suspects Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 6 / 57
7. ...in no particular order... Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 7 / 57
8. ...really! ;-) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 8 / 57
9. All of them claim to be similar Similar in functionality ...but not exactly the same, differences may be important Free / open source software ...but different licenses, business models, languages All are popular ...but different market shares, different targets Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 9 / 57
10. The study Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 10 / 57
11. What we did, what we didn’t Focus on how they are developed: Source code: what do they produce? Processes: how are they performing? Community: who is contributing? We didn’t analyze: Functionality Run-time performance Popularity We produced a dashboard for each of the projects Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 11 / 57
12. OpenNebula dashboard http://bit.ly/dashboard-opennebula Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 12 / 57
13. Eucalyptus dashboard http://bit.ly/dashboard-eucalyptus-2 Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 13 / 57
14. CloudStack dashboard http://bit.ly/dashboard-cloudstack Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 14 / 57
15. OpenStack dashboard http://bit.ly/dashboard-openstack Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 15 / 57
16. Transparency analysis Did we have data for all of them? Public source code management (git) and issue tracking systems All code seems to land in git at some point OpenStack and CloudStack: all tickets in public issue tracking system OpenNebula and Eucalyptus: maybe not all tickets in public issue tracking systems Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 16 / 57
17. What: the source code Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 17 / 57
18. Size Files Code Comments OpenStack 14,5 K 1,841 K 406 K CloudStack 7,7 K 1,544 K 394 K Eucalyptus 8,0 K 1,148 K 329 K OpenNebula 1,1 K 193 K 59 K Source code in git repositories, July 2014 Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 18 / 57
19. OpenNebula: Languages Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 19 / 57
20. Eucalyptus: Languages Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 20 / 57
21. CloudStack: Languages Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 21 / 57
22. OpenStack: Languages Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 22 / 57
23. How: the process Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 23 / 57
24. OpenNebula: Activity Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 24 / 57
25. Eucalyptus: Activity Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 25 / 57
26. CloudStack: Activity Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 26 / 57
27. OpenStack: Activity Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 27 / 57
28. OpenNebula: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 28 / 57
29. Eucalyptus: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 29 / 57
30. CloudStack: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 30 / 57
31. OpenStack: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 31 / 57
32. Who: the community Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 32 / 57
33. OpenNebula: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 1 2 3 4 5 5 4 5 5 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 33 / 57
34. Eucalyptus: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 5 10 15 20 16 19 15 13 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 34 / 57
35. CloudStack: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 5 10 15 20 25 30 19 25 30 29 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 35 / 57
36. OpenStack: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 50 100 150 200 250 300 97 153 218 260 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 36 / 57
37. OpenNebula: Aging (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 37 / 57
38. Eucalyptus: Aging (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 38 / 57
39. CloudStack: Aging (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 39 / 57
40. OpenStack: Aging (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 40 / 57
41. OpenNebula: Companies (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 41 / 57
42. Eucalyptus: Companies (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 42 / 57
43. CloudStack: Companies (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 43 / 57
44. OpenStack: Companies (Jul 2014) Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 44 / 57
45. OpenNebula: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 45 / 57
46. Eucalyptus: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 46 / 57
47. CloudStack: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 47 / 57
48. OpenStack: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 48 / 57
49. Bonus track Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 49 / 57
50. OpenStack: code review 12-Q4 13-Q1 13-Q2 13-Q3 13-Q4 14-Q1 14-Q2 14-Q3 0 5 10 15 20 25 30 4.520 6.040 7.550 9.790 12.000 16.570 17.820 22.070 1.190 1.830 2.460 4.080 4.290 6.220 6.080 6.780 Time to review (days): OpenStack Software mean median Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 50 / 57
51. OpenStack: code review (2) 12-Q4 13-Q1 13-Q2 13-Q3 13-Q4 14-Q1 14-Q2 14-Q3 0 1 2 3 4 5 6 7 2.760 3.340 3.690 4.250 4.860 5.120 5.490 5.290 2.000 2.000 2.000 2.000 2.000 3.000 3.000 3.000 Patchsets per Changeset: OpenStack Software mean median Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 51 / 57
52. OpenStack: code review (3) 12-Q4 13-Q1 13-Q2 13-Q3 13-Q4 14-Q1 14-Q2 14-Q3 0 2 4 6 8 3.180 3.510 4.090 6.660 6.210 7.180 7.560 6.270 0.180 0.310 0.450 0.520 0.780 0.960 0.910 0.760 Time waiting for the reviewer: OpenStack Software avg median 12-Q4 13-Q1 13-Q2 13-Q3 13-Q4 14-Q1 14-Q2 14-Q3 0 2 4 6 8 10 12 14 8.440 6.910 7.750 10.110 8.390 8.410 9.430 10.210 0.230 0.270 0.550 0.470 0.690 0.730 0.790 0.830 Time waiting for the submitter: OpenStack Software avg median [OpenStack core: waiting for reviewer / submitter] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 52 / 57
53. Openstack: time zone analysis 15 10 5 0 5 10 15 tz 0 10 20 30 40 50 authors 15 10 5 0 5 10 15 tz 0 50 100 150 200 250 300 350 400 450 authors [Authors in git per time zone (2010, 2014)] Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 53 / 57
54. The end Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 54 / 57
55. Final considerations There are huge differences in most of the metrics Openstack’s activity and community are clearly different But we cannot define good or bad: that depends on your target Look at the details... ...and draw your own conclusions The bottom line: the Open Cloud is really transparent Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 55 / 57
56. Disclamer and transparency package OpenStack Foundation and Citrix are Bitergia’s customers They fund the OpenStack and the CloudStack dashboards All the data has been checked, but could have some errors All database dumps and resulting JSON files are available from the corresponding dashboards. Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 56 / 57
57. Final note Show me the numbers! OpenNebula dashboard: http://bit.ly/dashboard-opennebula Eucalyptus dashboard: http://bit.ly/dashboard-eucalyptus CloudStack dashboard: http://bit.ly/dashboard-cloudstack OpenStack dashboard: http://bit.ly/dashboard-openstack This presentation: http://bit.ly/oscon-opencloud Jesus Gonzalez-Barahona (Bitergia) The numbers of the Open Cloud OpenStack Summit 2014 57 / 57