Building and Supporting Open Source Communities Through Metrics Open Source Summit Europe ‘22 Georg Link and Emilio Galeano

At the core of open source we care about using, sharing and collaborating

58% of software in Enterprises is open source 63% of companies want to increase their use Source: TODO Group Survey 2021

● The heartbleed incident ● The Struts/Equifax debacle ● The Log4j security vulnerability

The Mozilla Foundation released a report in 2019 on the different types of open source projects How can we solve these problems?

using metrics when building and supporting open source projects

● More than 15 years of experience ● Maintains the open source GrimoireLab metrics tools ● Official Metrics Partner of OpenInfra and NumFOCUS foundations ● Co-founded the CHAOSS project in 2017

Metrics in the Framework

Metrics in the CHAOSS framework Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics https://chaoss.community/metrics/

Metrics in the CHAOSS framework Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics https://chaoss.community/metrics/ ● Understand what contributions from organizations and people are being made. ● With focus areas such as Contributions, Time, People, Place ● Type of Contributions: we can measure the types of contributions are being made

● Identify the degree to which a project is valuable to researchers and academic institutions. ● Academic, communal, individual and organizational value. ● Project Velocity: with we can find What is the development speed for an organization? Metrics in the CHAOSS framework https://chaoss.community/metrics/ Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics

● How the source code changes over time, mechanisms that the project has to perform and control changes. ● Code Development Activity, Efficiency, Process Quality, Issue Resolution and Community Growth ● New Contributors: How many contributors are making their first contribution to a given project and who are they? Metrics in the CHAOSS framework https://chaoss.community/metrics/ Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics

Metrics in the CHAOSS framework https://chaoss.community/metrics/ ● Identify the diversity, equity, and inclusion aspects at events. ● Event diversity, Governance, leadership, and project and community. ● Time Inclusion for Virtual Events: organizers be mindful of attendees and speakers in other time zones Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics

Metrics in the CHAOSS framework https://chaoss.community/metrics/ ● Understand how active a community exists around/to support a given software package. ● Business risk, code quality, Dependency Risk Assessment, Licensing and Security. ● Elephant Factor: with we can measure What is the distribution of work in the community? Common Metrics Diversity, Equity, & Inclusion Metrics Value Metrics Evolution Metrics Risk Metrics

Metrics: how to interpret them

New Contributors & Contributions An increase or decline in new contributors and contributions can be an early indicator of community health.

New Contributors https://www.mautic.org/blog/community/q1-2020-mautic-community-round

New Contributors https://www.mautic.org/blog/community/q1-2020-mautic-community-round

Growth in overall contributions https://www.mautic.org/blog/community/q1-2020-mautic-community-round

Foundation for community growth https://www.mautic.org/blog/community/q1-2020-mautic-community-round Report Conclusion: Q1 2020 has been a busy one in the Mautic Community. A lot of positive steps have been taken to establish a solid foundation for growth. Teams are becoming more proactive and processes are being established which helps the community to function more effectively. We have seen, as a result, a significant increase in engagement and have welcomed new contributors to the project as a result.

Organizational Diversity Organizational diversity expresses how many different organizations are involved in a project.

Organizational Diversity https://dri.es/who-sponsors-drupal-development-2020

Contributions by Organizations Who is making contributions? https://katacontainers.biterg.io/

Change Requests Change requests are intended to be reviewed by other developers, who may suggest improvements. ● Pull Request (GitHub) ● Merge Request (GitLab) ● Changeset (Gerrit)

StarlingX Change Request Duration https://starlingx.biterg.io/

StarlingX Change Request Duration https://starlingx.biterg.io/

StarlingX Change Request Duration https://starlingx.biterg.io/

Organizational and Technical Challenges

image goes here Organizational Challenges ➔ Deciding on the right metrics ➔ Knowing what to do about metrics ➔ Knowing what is good and bad ➔ PII* concerns *Personally identifiable information (PII) uses data to confirm an individual's identity.

image goes here Technical Challenges Where is the data? ➢ Git ➢ GitHub ➢ GitLab ➢ BitBucket ➢ Jira ➢ Gerrit ➢ Confluence ➢ …

image goes here Technical Challenges Where is the community? ➢ Wiki ➢ Discourse ➢ Mailing List ➢ IRC ➢ Slack ➢ Meetup.com ➢ StackOverflow ➢ …

image goes here How to get the data Raw data: ● Get data from data sources Enriched data: ● Unify data ● Manage identities ● Calculate metrics Useful data: ● Visualize, report

image goes here Raw data ➔ Easiest step ➔ Query logs ➔ Use APIs Challenge: Changing APIs and data formats

image goes here Enriched data ➔ Unify data ➔ Manage identities ➔ Calculate metrics ● Date formats ● Level of detail ● Metadata about different contributions ● Convert everything into the desired database structure

image goes here ➔ Unify data Enriched data ➔ Manage identities ➔ Calculate metrics ● Who is who in the community ● Who do contributors work for (now and before) ● Different usernames and email ● Assigning contributions to the correct person

image goes here ➔ Calculate metrics Enriched data ➔ Unify data ➔ Manage identities ● Primary metrics - summarizing original data ● Secondary metrics ○ Calculation from different data fields ○ Combining data from different data sources ○ Value judgements on data - e.g., quality models

image goes here Useful data ➔ Who is the data user? ➔ How should the data be presented? ➔ What visualizations are most meaningful? ➔ What story does the data tell?

image goes here Technical Solutions OSS Solutions ● CHAOSS GrimoireLab ○ Cauldron.io ● CHAOSS Augur ● Apache Kibble ● CNCF Dev Analytics GrimoireLab cauldron.io

Summary: Lessons learned

Lessons learned Summary ➔ Use metrics ➔ Track metrics early ➔ Start with easy to get metrics ➔ Present metrics in context ➔ Be transparent with the community

Georg Link Questions? Let’s Connect! Emilio Galeano [email protected] Georg Link @georglink [email protected] Emilio Galeano BOOTH LOCATION B24

Thank You!