Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building and Supporting Open Source Communities through Metrics

Building and Supporting Open Source Communities through Metrics

Each community is different and therefore requires different metrics for data-driven decisions about building and supporting it. The CHAOSS Project was established to help empower open source communities through metrics. We have had many conversations with contributors and realized that everyone pays attention to different aspects of their communities.

This presentation shares what we have learned in the CHAOSS Project about having metrics for open source communities. Once the right metrics have been decided on, both technical and organizational challenges need to be overcome, which we discuss how to do. Finally, this presentation will describe real-world examples of how metrics have been used to help build and support open source communities.

Bitergia
PRO

October 05, 2022
Tweet

More Decks by Bitergia

Other Decks in Technology

Transcript

  1. Building and Supporting Open Source Communities Through Metrics Open Source

    Summit Europe ‘22 Georg Link and Emilio Galeano
  2. At the core of open source we care about using,

    sharing and collaborating
  3. 58% of software in Enterprises is open source 63% of

    companies want to increase their use Source: TODO Group Survey 2021
  4. • The heartbleed incident • The Struts/Equifax debacle • The

    Log4j security vulnerability
  5. The Mozilla Foundation released a report in 2019 on the

    different types of open source projects How can we solve these problems?
  6. using metrics when building and supporting open source projects

  7. • More than 15 years of experience • Maintains the

    open source GrimoireLab metrics tools • Official Metrics Partner of OpenInfra and NumFOCUS foundations • Co-founded the CHAOSS project in 2017
  8. Metrics in the Framework

  9. Metrics in the CHAOSS framework Common Metrics Diversity, Equity, &

    Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics https://chaoss.community/metrics/
  10. Metrics in the CHAOSS framework Common Metrics Diversity, Equity, &

    Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics https://chaoss.community/metrics/ • Understand what contributions from organizations and people are being made. • With focus areas such as Contributions, Time, People, Place • Type of Contributions: we can measure the types of contributions are being made
  11. • Identify the degree to which a project is valuable

    to researchers and academic institutions. • Academic, communal, individual and organizational value. • Project Velocity: with we can find What is the development speed for an organization? Metrics in the CHAOSS framework https://chaoss.community/metrics/ Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics
  12. • How the source code changes over time, mechanisms that

    the project has to perform and control changes. • Code Development Activity, Efficiency, Process Quality, Issue Resolution and Community Growth • New Contributors: How many contributors are making their first contribution to a given project and who are they? Metrics in the CHAOSS framework https://chaoss.community/metrics/ Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics
  13. Metrics in the CHAOSS framework https://chaoss.community/metrics/ • Identify the diversity,

    equity, and inclusion aspects at events. • Event diversity, Governance, leadership, and project and community. • Time Inclusion for Virtual Events: organizers be mindful of attendees and speakers in other time zones Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics
  14. Metrics in the CHAOSS framework https://chaoss.community/metrics/ • Understand how active

    a community exists around/to support a given software package. • Business risk, code quality, Dependency Risk Assessment, Licensing and Security. • Elephant Factor: with we can measure What is the distribution of work in the community? Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics
  15. Metrics: how to interpret them

  16. New Contributors & Contributions An increase or decline in new

    contributors and contributions can be an early indicator of community health.
  17. New Contributors https://www.mautic.org/blog/community/q1-2020-mautic-community-round

  18. New Contributors https://www.mautic.org/blog/community/q1-2020-mautic-community-round

  19. Growth in overall contributions https://www.mautic.org/blog/community/q1-2020-mautic-community-round

  20. Foundation for community growth https://www.mautic.org/blog/community/q1-2020-mautic-community-round Report Conclusion: Q1 2020 has

    been a busy one in the Mautic Community. A lot of positive steps have been taken to establish a solid foundation for growth. Teams are becoming more proactive and processes are being established which helps the community to function more effectively. We have seen, as a result, a significant increase in engagement and have welcomed new contributors to the project as a result.
  21. Organizational Diversity Organizational diversity expresses how many different organizations are

    involved in a project.
  22. Organizational Diversity https://dri.es/who-sponsors-drupal-development-2020

  23. Contributions by Organizations Who is making contributions? https://katacontainers.biterg.io/

  24. Change Requests Change requests are intended to be reviewed by

    other developers, who may suggest improvements. • Pull Request (GitHub) • Merge Request (GitLab) • Changeset (Gerrit)
  25. StarlingX Change Request Duration https://starlingx.biterg.io/

  26. StarlingX Change Request Duration https://starlingx.biterg.io/

  27. StarlingX Change Request Duration https://starlingx.biterg.io/

  28. Organizational and Technical Challenges

  29. image goes here Organizational Challenges ➔ Deciding on the right

    metrics ➔ Knowing what to do about metrics ➔ Knowing what is good and bad ➔ PII* concerns *Personally identifiable information (PII) uses data to confirm an individual's identity.
  30. image goes here Technical Challenges Where is the data? ➢

    Git ➢ GitHub ➢ GitLab ➢ BitBucket ➢ Jira ➢ Gerrit ➢ Confluence ➢ …
  31. image goes here Technical Challenges Where is the community? ➢

    Wiki ➢ Discourse ➢ Mailing List ➢ IRC ➢ Slack ➢ Meetup.com ➢ StackOverflow ➢ …
  32. image goes here How to get the data Raw data:

    • Get data from data sources Enriched data: • Unify data • Manage identities • Calculate metrics Useful data: • Visualize, report
  33. image goes here Raw data ➔ Easiest step ➔ Query

    logs ➔ Use APIs Challenge: Changing APIs and data formats
  34. image goes here Enriched data ➔ Unify data ➔ Manage

    identities ➔ Calculate metrics • Date formats • Level of detail • Metadata about different contributions • Convert everything into the desired database structure
  35. image goes here ➔ Unify data Enriched data ➔ Manage

    identities ➔ Calculate metrics • Who is who in the community • Who do contributors work for (now and before) • Different usernames and email • Assigning contributions to the correct person
  36. image goes here ➔ Calculate metrics Enriched data ➔ Unify

    data ➔ Manage identities • Primary metrics - summarizing original data • Secondary metrics ◦ Calculation from different data fields ◦ Combining data from different data sources ◦ Value judgements on data - e.g., quality models
  37. image goes here Useful data ➔ Who is the data

    user? ➔ How should the data be presented? ➔ What visualizations are most meaningful? ➔ What story does the data tell?
  38. image goes here Technical Solutions OSS Solutions • CHAOSS GrimoireLab

    ◦ Cauldron.io • CHAOSS Augur • Apache Kibble • CNCF Dev Analytics GrimoireLab cauldron.io
  39. Summary: Lessons learned

  40. Lessons learned Summary ➔ Use metrics ➔ Track metrics early

    ➔ Start with easy to get metrics ➔ Present metrics in context ➔ Be transparent with the community
  41. Georg Link Questions? Let’s Connect! Emilio Galeano [email protected] Georg Link

    @georglink [email protected] Emilio Galeano BOOTH LOCATION B24
  42. Thank You!