The Numbers of the Open Cloud (Tokyo Edition)

Talk at OpenStack Summit, Tokio 2015 (Oct. 29th) [video]. The talk presents a quantitative analysis of the projects producing the main free, open source software cloud platforms: OpenStack, Apache CloudStack, OpenNebula and Eucalyptus. The analysis will focus on the communities behind those projects, their main development parameters, and the trends that can be observed. A bonus track featuring some new Kibana--based dashboards for OpenStack is included.

Transcript

1. The quantitative state of the Open Cloud
   (Tokyo edition)
   Jesus M. Gonzalez-Barahona
   Daniel Izquierdo
   jgb,dizquierdo@bitergia.com @jgbarah,@dizquierdo
   Bitergia / URJC
   http://bit.ly/opencloud-tokyo
   Openstack Summit Tokyo 2015
   Tokyo (Japan), October 29th 2015
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 1 / 62
   

   View Slide

2. c 2012-2015 Bitergia
   Some rights reserved. This presentation is distributed under the
   “Attribution-ShareAlike 4.0” license, by Creative Commons, available at
   http://creativecommons.org/licenses/by-sa/4.0/
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 2 / 62
   

   View Slide

3. Structure of the presentation
   1 A bit of context
   2 Methodology
   3 The projects, in numbers (July-October 2015)
   4 Aging charts
   5 Geographical origin and hourly patterns
   6 Measuring corporate diversity
   7 Companies
   8 Bonus track: Kibana-based dashboards for OpenStack
   9 The end
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 3 / 62
   

   View Slide

4. A bit of context
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 4 / 62
   

   View Slide

5. We, the speakers, and our company
   Co-founders of Bitergia:
   The software development
   analytics company
   Dashboards, reports,
   consultancy...
   Dani: data analyst
   Jesus: scientific advisor
   http://bitergia.com
   Jesus has also another life
   as Associate Professor at URJC
   http://gsyc.es/~jgb
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 5 / 62
   

   View Slide

6. The Quantitative State of the Open Cloud (previous eds)
   OSCON 2014, 2015
   http://vimeo.com/105213660
   OpenStack Summit Paris 2014
   https:
   //youtube.com/watch?v=k73SIMHBHFc
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 6 / 62
   

   View Slide

7. Oct 2015: Tokyo Edition
   Bonus track
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 7 / 62
   

   View Slide

8. Methodology
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 8 / 62
   

   View Slide

9. From repositories to charts and numbers
   Transparency analysis
   MetricsGrimoire:
   retrieving data into a
   database
   GrimoireLib: querying,
   producing metrics
   vizGrimore: visualizing
   Preview: Grimoire NG & Kibana-based proof of
   concept
   (The whole system is free / open source software)
   Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 9 / 62
   

   View Slide

10. Facts from software development repositories
    Focus on how they are developed:
    Activity: how much contributions they get?
    Processes: how are they performing?
    Community: who is contributing?
    We didn’t analyze:
    Functionality
    Run-time performance
    Popularity
    We produced a dashboard for each of the projects
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 10 / 62
    

    View Slide

11. OpenNebula dashboard (per month)
    http://bit.ly/db-opennebula
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 11 / 62
    

    View Slide

12. Eucalyptus dashboard (per month)
    http://bit.ly/db-eucalyptus
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 12 / 62
    

    View Slide

13. CloudStack dashboard (per month)
    http://bit.ly/dashboard-cloudstack
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 13 / 62
    

    View Slide

14. OpenStack dashboard (per week)
    http://bit.ly/dashboard-openstack
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 14 / 62
    

    View Slide

15. Grimoire NG dashboards for all of them too!
    http://projects.bitergia.com/previews/ng/
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 15 / 62
    

    View Slide

16. Transparency analysis
    Did we have data for all of them?
    Public source code management (git)
    and issue tracking systems
    All code seems to land in git at some point
    OpenStack, CloudStack, Eucalyptus:
    all tickets seem to be in public issue tracking
    system
    OpenNebula:
    maybe not all tickets in public issue tracking
    systems
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 16 / 62
    

    View Slide

17. The projects, in numbers
    (July-October 2015)
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 17 / 62
    

    View Slide

18. Activity, community
    OpenNebula Eucalyptus CloudStack OpenStack
    Commits 12,677 26,058 42,066 164,184
    Devels 88 221 326 3,972
    Core 7 27 34 337
    Tickets 3,501 12,043 8,558 68,779
    Fixers 9 63 154 1,078
    Submitters 315 196 611 6,082
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 18 / 62
    

    View Slide

19. Activity, community (last months)
    OpenNebula Eucalyptus CloudStack OpenStack
    Commits 220 140 330 2,500
    Devels 12 12 35 500
    Tickets closed 50 25 40 1,200
    Closers 5 10 10 100
    For the last months...
    (all numbers are approximate, per month)
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 19 / 62
    

    View Slide

20. Aging charts
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 20 / 62
    

    View Slide

21. The aging chart
    Find out the aging structure of a community.
    Attracted and retained developers per “generation”
    (usually, 6-months generations)
    How much “good old expertise” do you have?
    How much “new blood” do you have?
    How are you retaining each generation?
    http://radar.oreilly.com/2014/10/
    measure-your-open-source-communitys-age-to-keep-it-healthy.html
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 21 / 62
    

    View Slide

22. CloudStack: Aging
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 22 / 62
    

    View Slide

23. OpenStack: Aging
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 23 / 62
    

    View Slide

24. Geographical origin and hourly
    patterns
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 24 / 62
    

    View Slide

25. OpenNebula: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 25 / 62
    

    View Slide

26. Eucalyptus: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 26 / 62
    

    View Slide

27. CloudStack: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 27 / 62
    

    View Slide

28. OpenStack: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 28 / 62
    

    View Slide

29. Hourly commit patterns
    [Commits per hour of the day (July 2014 - June 2015)
    OpenNebula (top), Eucalyptus (bottom)]
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 29 / 62
    

    View Slide

30. Hourly commit patterns
    [Commits per hour of the day (July 2014 - June 2015)
    CloudStack (top), OpenStack (bottom)]
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 30 / 62
    

    View Slide

31. Measuring corporate diversity
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 31 / 62
    

    View Slide

32. Companies
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 32 / 62
    

    View Slide

33. OpenNebula: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 33 / 62
    

    View Slide

34. Eucalyptus: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 34 / 62
    

    View Slide

35. CloudStack: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 35 / 62
    

    View Slide

36. OpenStack: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per week
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 36 / 62
    

    View Slide

37. Measuring diversity: Apache Pony Factor
    In words of Daniel Gruno:
    We [the ASF] created a term we have coined
    “Pony Factor” (because ASF is full of ponies, or
    people who think they are ponies). Pony Factor
    (PF) shows the diversity of a project in terms of
    the division of labor among committers in a
    project.
    Pony Factor is determined as:
    “The lowest number of committers whose
    total contribution constitutes the majority of
    the codebase”
    https://ke4qqq.wordpress.com/2015/02/08/pony-factor-math/
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 37 / 62
    

    View Slide

38. Measuring diversity: Bitergia Elephant Factor
    Projects can benefit from powerful collaborations
    from companies (elephants). The elephant factor
    shows the diversity of a project in terms of the
    division of labor among companies (by mean of
    developers affiliated with them).
    Elephant factor is determined as:
    “The lowest number of companies whose
    total contribution (in commits by their
    employees) constitutes the majority of the
    commits”
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 38 / 62
    

    View Slide

39. Measuring diversity: some projects
    Pony Factor Elephant Factor Commits (excl bots)
    OpenNebula 4 1 12K
    Eucalyptus 5 1 25K
    CloudStack 14 1 42K
    OpenStack >100 6 126K
    CloudFoundry 41 1 60K
    OpenShift 10 1 15K
    Docker 15 1 18K
    Kubernetes 12 1 7K
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 39 / 62
    

    View Slide

40. Bonus track: Kibana-based
    dashboards for OpenStack
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 40 / 62
    

    View Slide

41. Kibana-based dashboards for OpenStack
    Go and play!
    Code review (Gerrit): http://wp.me/p2cQGW-kN
    Contributions (git): http://wp.me/p2cQGW-kA
    Still a proof of concept, but already usable
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 41 / 62
    

    View Slide

42. Kibana-based dashboards for OpenStack
    Contributions (git): http://wp.me/p2cQGW-kA
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 42 / 62
    

    View Slide

43. Kibana-based dashboards for OpenStack
    Code review (Gerrit): http://wp.me/p2cQGW-kN
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 43 / 62
    

    View Slide

44. Use case: Elephant Factor in Nova
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 44 / 62
    

    View Slide

45. Use case: Elephant Factor in Nova (last year)
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 45 / 62
    

    View Slide

46. Use case: Elephant Factor in Neutron (last year)
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 46 / 62
    

    View Slide

47. Use case: Elephant Factor in Heat (last year)
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 47 / 62
    

    View Slide

48. Use case: Elephant Factor in Cinder (last year)
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 48 / 62
    

    View Slide

49. Use case: Understanding reviews
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 49 / 62
    

    View Slide

50. Use case: Understanding reviews: patchsets / time open
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 50 / 62
    

    View Slide

51. Use case: Understanding reviews: abandoned
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 51 / 62
    

    View Slide

52. Use case: Understanding reviews: merged
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 52 / 62
    

    View Slide

53. Use case: Understanding reviews: current backlog
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 53 / 62
    

    View Slide

54. Use case: Understanding reviews: Nova
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 54 / 62
    

    View Slide

55. Use case: Understanding reviews: Neutron
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 55 / 62
    

    View Slide

56. Use case: Understanding reviews: Heat
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 56 / 62
    

    View Slide

57. Bitergia reports on OpenStack (by quarters)
    http://activity.openstack.org/dash/reports/
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 57 / 62
    

    View Slide

58. The end
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 58 / 62
    

    View Slide

59. Final considerations
    There are huge differences
    in most of the metrics
    But we cannot define good or bad:
    that depends on your target
    Look at the details...
    ...and draw your own conclusions
    The bottom line:
    the Open Cloud is really transparent
    you can drill down to any level of detail
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 59 / 62
    

    View Slide

60. Final considerations (OpenStack
    OpenStack is large and complex
    But it offers a lot of data to understand it
    The data offers interesting, factual insight
    OpenStack is showing how to foster trust
    by transparency
    not by blind faith
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 60 / 62
    

    View Slide

61. Disclaimer and transparency package
    OpenStack Foundation and Citrix are Bitergia’s
    customers
    They fund the OpenStack and the CloudStack
    dashboards
    All the data has been checked,
    but could have some errors
    JSON files with the data used in this presentation
    are available from the corresponding dashboards.
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 61 / 62
    

    View Slide

62. Final note
    Show me the numbers!
    OpenNebula dashboard: http://bit.ly/db-opennebula
    Eucalyptus dashboard: http://bit.ly/db-eucalyptus
    CloudStack dashboard: http://bit.ly/dashboard-cloudstack
    OpenStack dashboard: http://bit.ly/dashboard-openstack
    GrimoireNG dashboards (preview):
    https://projects.bitergia.com/previews/ng/
    This presentation: http://bit.ly/oscon-opencloud-15
    Gonzalez-Barahona / Izquierdo (Bitergia) The Quantitative State of the Open Cloud Tokyo 2015 62 / 62
    

    View Slide