The quantitative state of the Open Cloud (2015 edition)

Slides for my talk at the Open Cloud Day, OSCON, The quantitative state of the Open Cloud (2015 edition), Portland, July 21th 2015.

Understanding the inner life of free / open source software projects is of fundamental importance to developers, users, and decision makers. This talk will help to understand what’s happening behind the curtain in the most relevant open cloud platforms, by analyzing the rich data obtained from their developing repositories.

The dashboards for the projects presented at the talk are:

GrimoireNG dashboard for all the projects (preview)
OpenStack Grimoire dashboard
CloudStack Grimoire dashboard
Eucalyptus Grimoire dashboard
OpenNebula Grimoire dashboard

Transcript

1. The quantitative state of the Open Cloud (2015 edition)
   Jesus M. Gonzalez-Barahona
   jgb@bitergia.com @jgbarah
   Bitergia / LibreSoft (URJC)
   http://bit.ly/oscon-opencloud-15
   Open Cloud Day, OSCON 2015
   Portland, OR (USA), July 21st 2015
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 1 / 51
   

   View Slide

2. c 2012-2015 Bitergia
   Some rights reserved. This presentation is distributed under the
   “Attribution-ShareAlike 3.0” license, by Creative Commons, available at
   http://creativecommons.org/licenses/by-sa/3.0/
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 2 / 51
   

   View Slide

3. Structure of the presentation
   1 A bit of context
   2 Methodology
   3 The projects, in numbers
   4 Opening and closing tickets
   5 Aging charts
   6 Companies
   7 Geographical origin
   8 Hourly patterns
   9 Bonus track
   10 The end
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 3 / 51
   

   View Slide

4. A bit of context
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 4 / 51
   

   View Slide

5. I am I, and my circumstance
   Co-founder of Bitergia:
   The software development
   analytics company
   Dashboards, reports,
   consultancy...
   http://bitergia.com
   Associate Professor at URJC:
   Understanding free, open source
   software development
   Empirical, data analytics approach
   http://gsyc.es/~jgb
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 5 / 51
   

   View Slide

6. July 2014: The Quantitative State of the Open Cloud
   http://vimeo.com/105213660
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 6 / 51
   

   View Slide

7. July 2015: The Quantitative State of the Open Cloud
   Bonus track
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 7 / 51
   

   View Slide

8. Methodology
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 8 / 51
   

   View Slide

9. From repositories to charts and numbers
   Transparency analysis
   MetricsGrimoire:
   retrieving data into a
   database
   GrimoireLib: querying,
   producing metrics
   vizGrimore: visualizing
   Preview: GrimoireNG Dashboard
   (The whole system is free / open source software)
   Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 9 / 51
   

   View Slide

10. Facts from software development repositories
    Focus on how they are developed:
    Activity: how much contributions they get?
    Processes: how are they performing?
    Community: who is contributing?
    We didn’t analyze:
    Functionality
    Run-time performance
    Popularity
    We produced a dashboard for each of the projects
    (in fact, three for each of them)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 10 / 51
    

    View Slide

11. OpenNebula dashboard (per month)
    http://bit.ly/db-opennebula
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 11 / 51
    

    View Slide

12. Eucalyptus dashboard (per month)
    http://bit.ly/db-eucalyptus
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 12 / 51
    

    View Slide

13. CloudStack dashboard (per month)
    http://bit.ly/dashboard-cloudstack
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 13 / 51
    

    View Slide

14. OpenStack dashboard (per week)
    http://bit.ly/dashboard-openstack
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 14 / 51
    

    View Slide

15. Grimoire NG dashboards for all of them too!
    http://projects.bitergia.com/previews/ng/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 15 / 51
    

    View Slide

16. Transparency analysis
    Did we have data for all of them?
    Public source code management (git)
    and issue tracking systems
    All code seems to land in git at some point
    OpenStack, CloudStack, Eucalyptus:
    all tickets seem to be in public issue tracking
    system
    OpenNebula:
    maybe not all tickets in public issue tracking
    systems
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 16 / 51
    

    View Slide

17. The projects, in numbers
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 17 / 51
    

    View Slide

18. Activity, community
    OpenNebula Eucalyptus CloudStack OpenStack
    Commits 12,677 26,058 42,066 164,184
    Devels 88 221 326 3,972
    Core 7 27 34 337
    Tickets 3,501 12,043 8,558 68,779
    Fixers 9 63 154 1,078
    Submitters 315 196 611 6,082
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 18 / 51
    

    View Slide

19. Activity, community (last months)
    OpenNebula Eucalyptus CloudStack OpenStack
    Commits 220 140 330 2,500
    Devels 12 12 35 500
    Tickets closed 50 25 40 1,200
    Closers 5 10 10 100
    For the last months...
    (all numbers are approximate, per month)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 19 / 51
    

    View Slide

20. Opening and closing tickets
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 20 / 51
    

    View Slide

21. OpenNebula: Tickets
    [Green: tickets opened per month / Blue: tickets closed per month]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 21 / 51
    

    View Slide

22. Eucalyptus: Tickets
    [Green: tickets opened per month / Blue: tickets closed per month]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 22 / 51
    

    View Slide

23. CloudStack: Tickets
    [Green: tickets opened per month / Blue: tickets closed per month]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 23 / 51
    

    View Slide

24. OpenStack: Tickets
    [Green: tickets opened per month / Blue: tickets closed per week]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 24 / 51
    

    View Slide

25. Aging charts
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 25 / 51
    

    View Slide

26. The aging chart
    Find out the aging structure of a community.
    Attracted and retained developers per “generation”
    (usually, 6-months generations)
    How much “good old expertise” do you have?
    How much “new blood” do you have?
    How are you retaining each generation?
    http://radar.oreilly.com/2014/10/
    measure-your-open-source-communitys-age-to-keep-it-healthy.html
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 26 / 51
    

    View Slide

27. CloudStack: Aging
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 27 / 51
    

    View Slide

28. OpenStack: Aging
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 28 / 51
    

    View Slide

29. Companies
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 29 / 51
    

    View Slide

30. OpenNebula: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 30 / 51
    

    View Slide

31. Eucalyptus: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 31 / 51
    

    View Slide

32. CloudStack: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 32 / 51
    

    View Slide

33. OpenStack: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per week
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 33 / 51
    

    View Slide

34. Geographical origin
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 34 / 51
    

    View Slide

35. OpenNebula: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 35 / 51
    

    View Slide

36. Eucalyptus: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 36 / 51
    

    View Slide

37. CloudStack: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 37 / 51
    

    View Slide

38. OpenStack: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 38 / 51
    

    View Slide

39. Hourly patterns
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 39 / 51
    

    View Slide

40. Hourly commit patterns
    [Commits per hour of the day (July 2014 - June 2015)
    OpenNebula (top), Eucalyptus (bottom)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 40 / 51
    

    View Slide

41. Hourly commit patterns
    [Commits per hour of the day (July 2014 - June 2015)
    CloudStack (top), OpenStack (bottom)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 41 / 51
    

    View Slide

42. Bonus track
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 42 / 51
    

    View Slide

43. Measuring diversity: Apache Pony Factor
    In words of Daniel Gruno:
    We [the ASF] created a term we have coined
    “Pony Factor” (because ASF is full of ponies, or
    people who think they are ponies). Pony Factor
    (PF) shows the diversity of a project in terms of
    the division of labor among committers in a
    project.
    Pony Factor is determined as:
    “The lowest number of committers whose
    total contribution constitutes the majority of
    the codebase”
    https://ke4qqq.wordpress.com/2015/02/08/pony-factor-math/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 43 / 51
    

    View Slide

44. Measuring diversity: Bitergia Elephant Factor
    Projects can benefit from powerful collaborations
    from companies (elephants). The elephant factor
    shows the diversity of a project in terms of the
    division of labor among companies (by mean of
    developers affiliated with them).
    Elephant factor is determined as:
    “The lowest number of companies whose
    total contribution (in commits by their
    employees) constitutes the majority of the
    commits”
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 44 / 51
    

    View Slide

45. Measuring diversity: some projects
    Pony Factor Elephant Factor Commits (excl bots)
    OpenNebula 4 1 12K
    Eucalyptus 5 1 25K
    CloudStack 14 1 42K
    OpenStack >100 6 126K
    CloudFoundry 41 1 60K
    OpenShift 10 1 15K
    Docker 15 1 18K
    Kubernetes 12 1 7K
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 45 / 51
    

    View Slide

46. The Grimoire NG Dashboard (preview)
    Go and play!
    https://projects.bitergia.com/previews/ng/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 46 / 51
    

    View Slide

47. The Grimoire NG Dashboard (preview for OpenStack)
    https://projects.bitergia.com/previews/ng/?db=openstack
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 47 / 51
    

    View Slide

48. The end
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 48 / 51
    

    View Slide

49. Final considerations
    There are huge differences
    in most of the metrics
    But we cannot define good or bad:
    that depends on your target
    Look at the details...
    ...and draw your own conclusions
    The bottom line:
    the Open Cloud is really transparent
    you can drill down to any level of detail
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 49 / 51
    

    View Slide

50. Disclaimer and transparency package
    OpenStack Foundation and Citrix are Bitergia’s
    customers
    They fund the OpenStack and the CloudStack
    dashboards
    All the data has been checked,
    but could have some errors
    JSON files with the data used in this presentation
    are available from the corresponding dashboards.
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 50 / 51
    

    View Slide

51. Final note
    Show me the numbers!
    OpenNebula dashboard: http://bit.ly/db-opennebula
    Eucalyptus dashboard: http://bit.ly/db-eucalyptus
    CloudStack dashboard: http://bit.ly/dashboard-cloudstack
    OpenStack dashboard: http://bit.ly/dashboard-openstack
    GrimoireNG dashboards (preview):
    https://projects.bitergia.com/previews/ng/
    This presentation: http://bit.ly/oscon-opencloud-15
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 51 / 51
    

    View Slide