$30 off During Our Annual Pro Sale. View Details »

The quantitative state of the Open Cloud (2015 edition)

The quantitative state of the Open Cloud (2015 edition)

Slides for my talk at the Open Cloud Day, OSCON, The quantitative state of the Open Cloud (2015 edition), Portland, July 21th 2015.

Understanding the inner life of free / open source software projects is of fundamental importance to developers, users, and decision makers. This talk will help to understand what’s happening behind the curtain in the most relevant open cloud platforms, by analyzing the rich data obtained from their developing repositories.

The dashboards for the projects presented at the talk are:

GrimoireNG dashboard for all the projects (preview)
OpenStack Grimoire dashboard
CloudStack Grimoire dashboard
Eucalyptus Grimoire dashboard
OpenNebula Grimoire dashboard

More Decks by Jesus M. Gonzalez-Barahona

Other Decks in Technology

Transcript

  1. The quantitative state of the Open Cloud (2015 edition)
    Jesus M. Gonzalez-Barahona
    [email protected] @jgbarah
    Bitergia / LibreSoft (URJC)
    http://bit.ly/oscon-opencloud-15
    Open Cloud Day, OSCON 2015
    Portland, OR (USA), July 21st 2015
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 1 / 51

    View Slide

  2. c 2012-2015 Bitergia
    Some rights reserved. This presentation is distributed under the
    “Attribution-ShareAlike 3.0” license, by Creative Commons, available at
    http://creativecommons.org/licenses/by-sa/3.0/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 2 / 51

    View Slide

  3. Structure of the presentation
    1 A bit of context
    2 Methodology
    3 The projects, in numbers
    4 Opening and closing tickets
    5 Aging charts
    6 Companies
    7 Geographical origin
    8 Hourly patterns
    9 Bonus track
    10 The end
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 3 / 51

    View Slide

  4. A bit of context
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 4 / 51

    View Slide

  5. I am I, and my circumstance
    Co-founder of Bitergia:
    The software development
    analytics company
    Dashboards, reports,
    consultancy...
    http://bitergia.com
    Associate Professor at URJC:
    Understanding free, open source
    software development
    Empirical, data analytics approach
    http://gsyc.es/~jgb
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 5 / 51

    View Slide

  6. July 2014: The Quantitative State of the Open Cloud
    http://vimeo.com/105213660
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 6 / 51

    View Slide

  7. July 2015: The Quantitative State of the Open Cloud
    Bonus track
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 7 / 51

    View Slide

  8. Methodology
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 8 / 51

    View Slide

  9. From repositories to charts and numbers
    Transparency analysis
    MetricsGrimoire:
    retrieving data into a
    database
    GrimoireLib: querying,
    producing metrics
    vizGrimore: visualizing
    Preview: GrimoireNG Dashboard
    (The whole system is free / open source software)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 9 / 51

    View Slide

  10. Facts from software development repositories
    Focus on how they are developed:
    Activity: how much contributions they get?
    Processes: how are they performing?
    Community: who is contributing?
    We didn’t analyze:
    Functionality
    Run-time performance
    Popularity
    We produced a dashboard for each of the projects
    (in fact, three for each of them)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 10 / 51

    View Slide

  11. OpenNebula dashboard (per month)
    http://bit.ly/db-opennebula
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 11 / 51

    View Slide

  12. Eucalyptus dashboard (per month)
    http://bit.ly/db-eucalyptus
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 12 / 51

    View Slide

  13. CloudStack dashboard (per month)
    http://bit.ly/dashboard-cloudstack
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 13 / 51

    View Slide

  14. OpenStack dashboard (per week)
    http://bit.ly/dashboard-openstack
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 14 / 51

    View Slide

  15. Grimoire NG dashboards for all of them too!
    http://projects.bitergia.com/previews/ng/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 15 / 51

    View Slide

  16. Transparency analysis
    Did we have data for all of them?
    Public source code management (git)
    and issue tracking systems
    All code seems to land in git at some point
    OpenStack, CloudStack, Eucalyptus:
    all tickets seem to be in public issue tracking
    system
    OpenNebula:
    maybe not all tickets in public issue tracking
    systems
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 16 / 51

    View Slide

  17. The projects, in numbers
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 17 / 51

    View Slide

  18. Activity, community
    OpenNebula Eucalyptus CloudStack OpenStack
    Commits 12,677 26,058 42,066 164,184
    Devels 88 221 326 3,972
    Core 7 27 34 337
    Tickets 3,501 12,043 8,558 68,779
    Fixers 9 63 154 1,078
    Submitters 315 196 611 6,082
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 18 / 51

    View Slide

  19. Activity, community (last months)
    OpenNebula Eucalyptus CloudStack OpenStack
    Commits 220 140 330 2,500
    Devels 12 12 35 500
    Tickets closed 50 25 40 1,200
    Closers 5 10 10 100
    For the last months...
    (all numbers are approximate, per month)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 19 / 51

    View Slide

  20. Opening and closing tickets
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 20 / 51

    View Slide

  21. OpenNebula: Tickets
    [Green: tickets opened per month / Blue: tickets closed per month]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 21 / 51

    View Slide

  22. Eucalyptus: Tickets
    [Green: tickets opened per month / Blue: tickets closed per month]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 22 / 51

    View Slide

  23. CloudStack: Tickets
    [Green: tickets opened per month / Blue: tickets closed per month]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 23 / 51

    View Slide

  24. OpenStack: Tickets
    [Green: tickets opened per month / Blue: tickets closed per week]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 24 / 51

    View Slide

  25. Aging charts
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 25 / 51

    View Slide

  26. The aging chart
    Find out the aging structure of a community.
    Attracted and retained developers per “generation”
    (usually, 6-months generations)
    How much “good old expertise” do you have?
    How much “new blood” do you have?
    How are you retaining each generation?
    http://radar.oreilly.com/2014/10/
    measure-your-open-source-communitys-age-to-keep-it-healthy.html
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 26 / 51

    View Slide

  27. CloudStack: Aging
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 27 / 51

    View Slide

  28. OpenStack: Aging
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 28 / 51

    View Slide

  29. Companies
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 29 / 51

    View Slide

  30. OpenNebula: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 30 / 51

    View Slide

  31. Eucalyptus: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 31 / 51

    View Slide

  32. CloudStack: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per month
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 32 / 51

    View Slide

  33. OpenStack: Companies
    Commits per company (July 2014 - June 2015)
    Active companies per week
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 33 / 51

    View Slide

  34. Geographical origin
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 34 / 51

    View Slide

  35. OpenNebula: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 35 / 51

    View Slide

  36. Eucalyptus: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 36 / 51

    View Slide

  37. CloudStack: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 37 / 51

    View Slide

  38. OpenStack: Time zones
    [Commits per time zone (July 2014 - June 2015)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 38 / 51

    View Slide

  39. Hourly patterns
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 39 / 51

    View Slide

  40. Hourly commit patterns
    [Commits per hour of the day (July 2014 - June 2015)
    OpenNebula (top), Eucalyptus (bottom)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 40 / 51

    View Slide

  41. Hourly commit patterns
    [Commits per hour of the day (July 2014 - June 2015)
    CloudStack (top), OpenStack (bottom)]
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 41 / 51

    View Slide

  42. Bonus track
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 42 / 51

    View Slide

  43. Measuring diversity: Apache Pony Factor
    In words of Daniel Gruno:
    We [the ASF] created a term we have coined
    “Pony Factor” (because ASF is full of ponies, or
    people who think they are ponies). Pony Factor
    (PF) shows the diversity of a project in terms of
    the division of labor among committers in a
    project.
    Pony Factor is determined as:
    “The lowest number of committers whose
    total contribution constitutes the majority of
    the codebase”
    https://ke4qqq.wordpress.com/2015/02/08/pony-factor-math/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 43 / 51

    View Slide

  44. Measuring diversity: Bitergia Elephant Factor
    Projects can benefit from powerful collaborations
    from companies (elephants). The elephant factor
    shows the diversity of a project in terms of the
    division of labor among companies (by mean of
    developers affiliated with them).
    Elephant factor is determined as:
    “The lowest number of companies whose
    total contribution (in commits by their
    employees) constitutes the majority of the
    commits”
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 44 / 51

    View Slide

  45. Measuring diversity: some projects
    Pony Factor Elephant Factor Commits (excl bots)
    OpenNebula 4 1 12K
    Eucalyptus 5 1 25K
    CloudStack 14 1 42K
    OpenStack >100 6 126K
    CloudFoundry 41 1 60K
    OpenShift 10 1 15K
    Docker 15 1 18K
    Kubernetes 12 1 7K
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 45 / 51

    View Slide

  46. The Grimoire NG Dashboard (preview)
    Go and play!
    https://projects.bitergia.com/previews/ng/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 46 / 51

    View Slide

  47. The Grimoire NG Dashboard (preview for OpenStack)
    https://projects.bitergia.com/previews/ng/?db=openstack
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 47 / 51

    View Slide

  48. The end
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 48 / 51

    View Slide

  49. Final considerations
    There are huge differences
    in most of the metrics
    But we cannot define good or bad:
    that depends on your target
    Look at the details...
    ...and draw your own conclusions
    The bottom line:
    the Open Cloud is really transparent
    you can drill down to any level of detail
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 49 / 51

    View Slide

  50. Disclaimer and transparency package
    OpenStack Foundation and Citrix are Bitergia’s
    customers
    They fund the OpenStack and the CloudStack
    dashboards
    All the data has been checked,
    but could have some errors
    JSON files with the data used in this presentation
    are available from the corresponding dashboards.
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 50 / 51

    View Slide

  51. Final note
    Show me the numbers!
    OpenNebula dashboard: http://bit.ly/db-opennebula
    Eucalyptus dashboard: http://bit.ly/db-eucalyptus
    CloudStack dashboard: http://bit.ly/dashboard-cloudstack
    OpenStack dashboard: http://bit.ly/dashboard-openstack
    GrimoireNG dashboards (preview):
    https://projects.bitergia.com/previews/ng/
    This presentation: http://bit.ly/oscon-opencloud-15
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of the Open Cloud OSCON 2015 51 / 51

    View Slide