The quantitative state of the Open Cloud

Slides for my talk at OSCON, Portland, 2014. Video: https://vimeo.com/105213660

Understanding the inner life of free / open source software projects is of fundamental importance to developers, users and decision makers. This talk helps to understand what's happening behind the curtains in the most relevant open cloud platforms, by analyzing the rich data obtained from their developing repositories.

The talk covers, for the four analyzed projects, the following topics:

* How open is their development model, and to which extent they provide access to their development data.
* Some parameters related to the overall activity and effort put into the development and maintenance of their products.
* The main characteristics of their development community: size, diversity, engagement, attraction, etc.
* The main parameters characterizing their performance, such as time-to-fix tickets or time-to-review patches.
* Some visualizations of the most relevant aspects.

All the tools used to produce the analysis presented in this talk are free, open source software, based on the Grimoire technology http://vizgrimoire.github.io

Transcript

1. The quantitative state of the Open Cloud Jesus M. Gonzalez-Barahona jgb@bitergia.com @jgbarah Bitergia / LibreSoft (URJC) http://bit.ly/oscon-opencloud Open Cloud Day, OSCON 2014 Portland, OR (USA), July 21st 2014 Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 1 / 56
2. c 2012-2014 Bitergia Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 3.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/3.0/ Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 2 / 56
3. Structure of the presentation 1 The context 2 The study 3 What: the source code 4 How: the process 5 Who: the community 6 The end Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 3 / 56
4. The context Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 4 / 56
5. Me, myself, my circumstances Bitergia: From research to the real world The software development analytics company Dashboards, reports, consultancy... Uni Rey Juan Carlos: LibreSoft research team Understanding free, open source software development Data analytics approach http://bitergia.com http://gsyc.es/~jgb Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 5 / 56
6. Four usual suspects Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 6 / 56
7. ...in no particular order... Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 7 / 56
8. ...really! ;-) Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 8 / 56
9. All of them claim to be similar Similar in functionality ...but not exactly the same, differences may be important Free / open source software ...but different licenses, business models, languages All are popular ...but different market shares, different targets Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 9 / 56
10. The study Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 10 / 56
11. What we did, what we didn’t Focus on how they are developed: Source code: what do they produce? Processes: how are they performing? Community: who is contributing? We didn’t analyze: Functionality Run-time performance Popularity We produced a dashboard for each of the projects Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 11 / 56
12. OpenNebula dashboard http://bit.ly/dashboard-opennebula Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 12 / 56
13. Eucalyptus dashboard http://bit.ly/dashboard-eucalyptus Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 13 / 56
14. CloudStack dashboard http://bit.ly/dashboard-cloudstack Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 14 / 56
15. OpenStack dashboard http://bit.ly/dashboard-openstack Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 15 / 56
16. Transparency analysis Did we have data for all of them? Public source code management (git) and issue tracking systems All code seems to land in git at some point OpenStack and CloudStack: all tickets in public issue tracking system OpenNebula and Eucalyptus: maybe not all tickets in public issue tracking systems Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 16 / 56
17. What: the source code Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 17 / 56
18. Size Files Code Comments OpenStack 14,5 K 1,841 K 406 K CloudStack 7,7 K 1,544 K 394 K Eucalyptus 8,0 K 1,148 K 329 K OpenNebula 1,1 K 193 K 59 K Source code in git repositories, July 2014 Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 18 / 56
19. OpenNebula: Languages Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 19 / 56
20. Eucalyptus: Languages Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 20 / 56
21. CloudStack: Languages Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 21 / 56
22. OpenStack: Languages Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 22 / 56
23. How: the process Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 23 / 56
24. OpenNebula: Activity Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 24 / 56
25. Eucalyptus: Activity Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 25 / 56
26. CloudStack: Activity Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 26 / 56
27. OpenStack: Activity Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 27 / 56
28. OpenNebula: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 28 / 56
29. Eucalyptus: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 29 / 56
30. CloudStack: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 30 / 56
31. OpenStack: Tickets [Green: tickets opened per month / Blue: tickets closed per month] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 31 / 56
32. Who: the community Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 32 / 56
33. OpenNebula: Community Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 33 / 56
34. Eucalyptus: Community Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 34 / 56
35. CloudStack: Community Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 35 / 56
36. OpenStack: Community Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 36 / 56
37. OpenNebula: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 1 2 3 4 5 5 4 5 5 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 37 / 56
38. Eucalyptus: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 5 10 15 20 16 19 15 13 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 38 / 56
39. CloudStack: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 5 10 15 20 25 30 19 25 30 29 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 39 / 56
40. OpenStack: Evolution of core team 2012-S2 2013-S1 2013-S2 2014-S1 0 50 100 150 200 250 300 97 153 218 260 scm_core Core team contributes with 80 % commits Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 40 / 56
41. OpenNebula: Aging Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 41 / 56
42. Eucalyptus: Aging Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 42 / 56
43. CloudStack: Aging Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 43 / 56
44. OpenStack: Aging Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 44 / 56
45. OpenNebula: Companies Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 45 / 56
46. Eucalyptus: Companies Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 46 / 56
47. CloudStack: Companies Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 47 / 56
48. OpenStack: Companies Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 48 / 56
49. OpenNebula: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 49 / 56
50. Eucalyptus: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 50 / 56
51. CloudStack: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 51 / 56
52. OpenStack: Time zones [Commits (top) and authors (bottom) per time zone] Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 52 / 56
53. The end Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 53 / 56
54. Final considerations There are huge differences in most of the metrics But we cannot define good or bad: that depends on your target Look at the details... ...and draw your own conclusions The bottom line: the Open Cloud is really transparent you can drill down to any level of detail Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 54 / 56
55. Disclamer and transparency package OpenStack Foundation and Citrix are Bitergia’s customers They fund the OpenStack and the CloudStack dashboards All the data has been checked, but could have some errors All database dumps and resulting JSON files are available from the corresponding dashboards. Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 55 / 56
56. Final note Show me the numbers! OpenNebula dashboard: http://bit.ly/dashboard-opennebula Eucalyptus dashboard: http://bit.ly/dashboard-eucalyptus CloudStack dashboard: http://bit.ly/dashboard-cloudstack OpenStack dashboard: http://bit.ly/dashboard-openstack This presentation: http://bit.ly/oscon-opencloud Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2014 56 / 56