The Numbers of the Open Cloud (Tokyo Edition)

Talk at OpenStack Summit, Tokio 2015 (Oct. 29th) [video]. The talk presents a quantitative analysis of the projects producing the main free, open source software cloud platforms: OpenStack, Apache CloudStack, OpenNebula and Eucalyptus. The analysis will focus on the communities behind those projects, their main development parameters, and the trends that can be observed. A bonus track featuring some new Kibana--based dashboards for OpenStack is included.

Transcript

1. The quantitative state of the Open Cloud (Tokyo edition) Jesus M. Gonzalez-Barahona Daniel Izquierdo jgb,dizquierdo@bitergia.com @jgbarah,@dizquierdo Bitergia / URJC http://bit.ly/opencloud-tokyo Openstack Summit Tokyo 2015 Tokyo (Japan), October 29th 2015 Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 1 / 62
2. c 2012-2015 Bitergia Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 4.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/4.0/ Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 2 / 62
3. Structure of the presentation 1 A bit of context 2 Methodology 3 The projects, in numbers (July-October 2015) 4 Aging charts 5 Geographical origin and hourly patterns 6 Measuring corporate diversity 7 Companies 8 Bonus track: Kibana-based dashboards for OpenStack 9 The end Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 3 / 62
4. A bit of context Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 4 / 62
5. We, the speakers, and our company Co-founders of Bitergia: The software development analytics company Dashboards, reports, consultancy... Dani: data analyst Jesus: scientific advisor http://bitergia.com Jesus has also another life as Associate Professor at URJC http://gsyc.es/~jgb Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 5 / 62
6. The Quantitative State of the Open Cloud (previous eds) OSCON 2014, 2015 http://vimeo.com/105213660 OpenStack Summit Paris 2014 https: //youtube.com/watch?v=k73SIMHBHFc Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 6 / 62
7. Oct 2015: Tokyo Edition Bonus track Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 7 / 62
8. Methodology Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 8 / 62
9. From repositories to charts and numbers Transparency analysis MetricsGrimoire: retrieving data into a database GrimoireLib: querying, producing metrics vizGrimore: visualizing Preview: Grimoire NG & Kibana-based proof of concept (The whole system is free / open source software) Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 9 / 62
10. Facts from software development repositories Focus on how they are developed: Activity: how much contributions they get? Processes: how are they performing? Community: who is contributing? We didn’t analyze: Functionality Run-time performance Popularity We produced a dashboard for each of the projects Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 10 / 62
11. OpenNebula dashboard (per month) http://bit.ly/db-opennebula Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 11 / 62
12. Eucalyptus dashboard (per month) http://bit.ly/db-eucalyptus Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 12 / 62
13. CloudStack dashboard (per month) http://bit.ly/dashboard-cloudstack Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 13 / 62
14. OpenStack dashboard (per week) http://bit.ly/dashboard-openstack Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 14 / 62
15. Grimoire NG dashboards for all of them too! http://projects.bitergia.com/previews/ng/ Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 15 / 62
16. Transparency analysis Did we have data for all of them? Public source code management (git) and issue tracking systems All code seems to land in git at some point OpenStack, CloudStack, Eucalyptus: all tickets seem to be in public issue tracking system OpenNebula: maybe not all tickets in public issue tracking systems Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 16 / 62
17. The projects, in numbers (July-October 2015) Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 17 / 62
18. Activity, community OpenNebula Eucalyptus CloudStack OpenStack Commits 12,677 26,058 42,066 164,184 Devels 88 221 326 3,972 Core 7 27 34 337 Tickets 3,501 12,043 8,558 68,779 Fixers 9 63 154 1,078 Submitters 315 196 611 6,082 Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 18 / 62
19. Activity, community (last months) OpenNebula Eucalyptus CloudStack OpenStack Commits 220 140 330 2,500 Devels 12 12 35 500 Tickets closed 50 25 40 1,200 Closers 5 10 10 100 For the last months... (all numbers are approximate, per month) Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 19 / 62
20. Aging charts Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 20 / 62
21. The aging chart Find out the aging structure of a community. Attracted and retained developers per “generation” (usually, 6-months generations) How much “good old expertise” do you have? How much “new blood” do you have? How are you retaining each generation? http://radar.oreilly.com/2014/10/ measure-your-open-source-communitys-age-to-keep-it-healthy.html Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 21 / 62
22. CloudStack: Aging Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 22 / 62
23. OpenStack: Aging Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 23 / 62
24. Geographical origin and hourly patterns Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 24 / 62
25. OpenNebula: Time zones [Commits per time zone (July 2014 - June 2015)] Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 25 / 62
26. Eucalyptus: Time zones [Commits per time zone (July 2014 - June 2015)] Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 26 / 62
27. CloudStack: Time zones [Commits per time zone (July 2014 - June 2015)] Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 27 / 62
28. OpenStack: Time zones [Commits per time zone (July 2014 - June 2015)] Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 28 / 62
29. Hourly commit patterns [Commits per hour of the day (July 2014 - June 2015) OpenNebula (top), Eucalyptus (bottom)] Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 29 / 62
30. Hourly commit patterns [Commits per hour of the day (July 2014 - June 2015) CloudStack (top), OpenStack (bottom)] Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 30 / 62
31. Measuring corporate diversity Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 31 / 62
32. Companies Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 32 / 62
33. OpenNebula: Companies Commits per company (July 2014 - June 2015) Active companies per month Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 33 / 62
34. Eucalyptus: Companies Commits per company (July 2014 - June 2015) Active companies per month Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 34 / 62
35. CloudStack: Companies Commits per company (July 2014 - June 2015) Active companies per month Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 35 / 62
36. OpenStack: Companies Commits per company (July 2014 - June 2015) Active companies per week Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 36 / 62
37. Measuring diversity: Apache Pony Factor In words of Daniel Gruno: We [the ASF] created a term we have coined “Pony Factor” (because ASF is full of ponies, or people who think they are ponies). Pony Factor (PF) shows the diversity of a project in terms of the division of labor among committers in a project. Pony Factor is determined as: “The lowest number of committers whose total contribution constitutes the majority of the codebase” https://ke4qqq.wordpress.com/2015/02/08/pony-factor-math/ Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 37 / 62
38. Measuring diversity: Bitergia Elephant Factor Projects can benefit from powerful collaborations from companies (elephants). The elephant factor shows the diversity of a project in terms of the division of labor among companies (by mean of developers affiliated with them). Elephant factor is determined as: “The lowest number of companies whose total contribution (in commits by their employees) constitutes the majority of the commits” Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 38 / 62
39. Measuring diversity: some projects Pony Factor Elephant Factor Commits (excl bots) OpenNebula 4 1 12K Eucalyptus 5 1 25K CloudStack 14 1 42K OpenStack >100 6 126K CloudFoundry 41 1 60K OpenShift 10 1 15K Docker 15 1 18K Kubernetes 12 1 7K Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 39 / 62
40. Bonus track: Kibana-based dashboards for OpenStack Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 40 / 62
41. Kibana-based dashboards for OpenStack Go and play! Code review (Gerrit): http://wp.me/p2cQGW-kN Contributions (git): http://wp.me/p2cQGW-kA Still a proof of concept, but already usable Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 41 / 62
42. Kibana-based dashboards for OpenStack Contributions (git): http://wp.me/p2cQGW-kA Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 42 / 62
43. Kibana-based dashboards for OpenStack Code review (Gerrit): http://wp.me/p2cQGW-kN Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 43 / 62
44. Use case: Elephant Factor in Nova Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 44 / 62
45. Use case: Elephant Factor in Nova (last year) Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 45 / 62
46. Use case: Elephant Factor in Neutron (last year) Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 46 / 62
47. Use case: Elephant Factor in Heat (last year) Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 47 / 62
48. Use case: Elephant Factor in Cinder (last year) Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 48 / 62
49. Use case: Understanding reviews Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 49 / 62
50. Use case: Understanding reviews: patchsets / time open Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 50 / 62
51. Use case: Understanding reviews: abandoned Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 51 / 62
52. Use case: Understanding reviews: merged Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 52 / 62
53. Use case: Understanding reviews: current backlog Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 53 / 62
54. Use case: Understanding reviews: Nova Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 54 / 62
55. Use case: Understanding reviews: Neutron Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 55 / 62
56. Use case: Understanding reviews: Heat Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 56 / 62
57. Bitergia reports on OpenStack (by quarters) http://activity.openstack.org/dash/reports/ Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 57 / 62
58. The end Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 58 / 62
59. Final considerations There are huge differences in most of the metrics But we cannot define good or bad: that depends on your target Look at the details... ...and draw your own conclusions The bottom line: the Open Cloud is really transparent you can drill down to any level of detail Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 59 / 62
60. Final considerations (OpenStack OpenStack is large and complex But it offers a lot of data to understand it The data offers interesting, factual insight OpenStack is showing how to foster trust by transparency not by blind faith Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 60 / 62
61. Disclaimer and transparency package OpenStack Foundation and Citrix are Bitergia’s customers They fund the OpenStack and the CloudStack dashboards All the data has been checked, but could have some errors JSON files with the data used in this presentation are available from the corresponding dashboards. Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 61 / 62
62. Final note Show me the numbers! OpenNebula dashboard: http://bit.ly/db-opennebula Eucalyptus dashboard: http://bit.ly/db-eucalyptus CloudStack dashboard: http://bit.ly/dashboard-cloudstack OpenStack dashboard: http://bit.ly/dashboard-openstack GrimoireNG dashboards (preview): https://projects.bitergia.com/previews/ng/ This presentation: http://bit.ly/oscon-opencloud-15 Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 62 / 62