Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Evolution of cloud-related free software projects

Evolution of cloud-related free software projects

Presentation at DevOps Open Source Azure Day, Madrid (Spain), November 12th 2015.

Jesus M. Gonzalez-Barahona

November 11, 2015
Tweet

More Decks by Jesus M. Gonzalez-Barahona

Other Decks in Technology

Transcript

  1. Evolution of cloud-related free software projects, and their importance for

    DevOps Jesus M. Gonzalez-Barahona [email protected] @jgbarah Bitergia / URJC http://bit.ly/opencloud-devops DevOps Open Source Azure Day Madrid (Spain), November 12th 2015 Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 1 / 35
  2. c 2012-2015 Bitergia Some rights reserved. This presentation is distributed

    under the “Attribution-ShareAlike 4.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/4.0/ Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 2 / 35
  3. Structure of the presentation 1 A bit of context 2

    Focusing on the openess 3 The Grimoire methodology 4 Characterizing developers 5 Measuring corporate diversity 6 Companies 7 The end Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 3 / 35
  4. Me, my circumstances Co-founder of Bitergia: The software development analytics

    company Dashboards, reports, consultancy... http://bitergia.com Associate Professor at URJC: Understanding free, open source software development Empirical, data analytics approach http://gsyc.es/~jgb Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 5 / 35
  5. Bitergia: analytics for your peace of mind Started operations in

    July 2012 Builds on the experience of LibreSoft R&D group Professional products and services, focused on: Grimoire Dashboard: software development analytics (including community metrics) Specific studies and reports (based on metrics and facts collection) http://bitergia.com/ projects.html http://blog.bitergia.com Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 6 / 35
  6. Free software is about... Free / open source software: Freedom:

    using, sharing, contributing, analyzing Openess: you can inspect all source code Open development models: Be transparent about how you develop Show problems, issues, decision making... Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 8 / 35
  7. From open development to open development analytics Information about code,

    community, development for open development projects can be retrieved, organized, analyzed Let’s publish analytics results & data Open Development Analytics: A new standard for transparency Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 9 / 35
  8. Open development analytics Who is interested? Developers Project managers Community

    managers Evaluators ... Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 10 / 35
  9. Open development analytics Why? Free software produced with open development

    models is more and more important for IT users, producers, integrators It is different & complex, yet transparent, many details are public, and it can be improved Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 11 / 35
  10. Some areas of interest Performance (understanding activity) Company participation (beyond

    copyright notices) Transparency (available information) Auditing (certify participation, experience, etc.) Profiling (key people, companies) Neutrality (fair treatment) Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 12 / 35
  11. From repositories to charts and numbers Transparency analysis MetricsGrimoire: retrieving

    data into a database GrimoireLib: querying, producing metrics vizGrimore: visualizing Preview: Grimoire NG & Kibana-based proof of concept (The whole system is free / open source software) Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 14 / 35
  12. Facts from software development repositories Focus on how they are

    developed: Activity: how much contributions they get? Processes: how are they performing? Community: who is contributing? Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 15 / 35
  13. Analyzing some projects OpenStack Docker Kubernetes Puppet Logstash / ElasticSearch

    / Kibana We produced some dashboards... Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 16 / 35
  14. Kibana-based dashboards for OpenStack Go and play! Code review (Gerrit):

    http://wp.me/p2cQGW-kN Contributions (git): http://wp.me/p2cQGW-kA Still a proof of concept, but already usable Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 19 / 35
  15. Kibana-based dashboards for devops-related projects Docker: http://s.bitergia.com/db-docker-git Kubernetes: http://s.bitergia.com/db-kubernetes-git ElasticSearch:

    http://s.bitergia.com/db-elasticsearch-git Puppet: http://s.bitergia.com/db-puppet-git Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 20 / 35
  16. Transparency analysis Did we have data for all of them?

    Public source code management (git) and issue tracking systems All code seems to land in git at some point Most of the tickets seem to be in public issue tracking system Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 21 / 35
  17. The aging chart Find out the aging structure of a

    community. Attracted and retained developers per “generation” (usually, 6-months generations) How much “good old expertise” do you have? How much “new blood” do you have? How are you retaining each generation? http://radar.oreilly.com/2014/10/ measure-your-open-source-communitys-age-to-keep-it-healthy.html Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 23 / 35
  18. OpenStack: Time zones [Commits per time zone (July 2014 -

    June 2015)] https: //opensource.com/business/15/11/git-mailing-list-time-zone-analysis Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 25 / 35
  19. Measuring diversity: Apache Pony Factor In words of Daniel Gruno:

    We [the ASF] created a term we have coined “Pony Factor” (because ASF is full of ponies, or people who think they are ponies). Pony Factor (PF) shows the diversity of a project in terms of the division of labor among committers in a project. Pony Factor is determined as: “The lowest number of committers whose total contribution constitutes the majority of the codebase” https://ke4qqq.wordpress.com/2015/02/08/pony-factor-math/ Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 30 / 35
  20. Measuring diversity: Bitergia Elephant Factor Projects can benefit from powerful

    collaborations from companies (elephants). The elephant factor shows the diversity of a project in terms of the division of labor among companies (by mean of developers affiliated with them). Elephant factor is determined as: “The lowest number of companies whose total contribution (in commits by their employees) constitutes the majority of the commits” Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 31 / 35
  21. Measuring diversity: some projects Pony Factor Elephant Factor Commits (excl

    bots) OpenNebula 4 1 12K Eucalyptus 5 1 25K CloudStack 14 1 42K OpenStack >100 6 126K CloudFoundry 41 1 60K OpenShift 10 1 15K Docker 15 1 18K Kubernetes 12 1 7K Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 32 / 35
  22. Final considerations For different projects There are huge differences in

    most of the metrics But we cannot define good or bad: that depends on your target Look at the details... ...and draw your own conclusions The bottom line: many free / open source softare projects are really transparent Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 34 / 35
  23. Disclaimer and transparency package OpenStack Foundation and Puppet Labs are

    Bitergia’s customers. All the data has been checked, but could have some errors JSON files with the data used in this presentation are available from the corresponding dashboards. This presentation: http://bit.ly/opencloud-devops Gonzalez-Barahona / Izquierdo (Bitergia) Cloud-related free software projects Madrid 2015 35 / 35