Metrics and KPIs for an InnerSource Office

InnerSource Commons Spring Summit 2019
Speakers: Ana Jiménez Santamaría, Bitergia and Daniel Izquierdo Cortázar, Bitergia.

Abstract: Metrics are an important part of the InnerSource journey. The InnerSource Commons has been a great place for discussion around the topic. Specifically the InnerSource Patterns community delivered a pattern focused on first steps with metrics [1] where no specific metrics are detailed, but this gives a starting point to discuss when, how and what to measure. This talk aims at detailing these set of metrics and KPIs for further discussion across the community. This is based on work in progress in the GrimoireLab/CHAOSS community, a Linux Foundation Project. The proposed set of metrics are split into three main areas: Activity, Community and Process.

[1]https://github.com/InnerSourceCommons/InnerSourcePatterns/blob/master/introducing-metrics-in-innersource.md

Transcript

1. InnerSource Commons Spring Summit 2019, Galway Metrics and KPIs for an InnerSource Office Daniel Izquierdo Cortázar, Ana Jiménez Santamaría @bitergia dizquierdo|anajs@bitergia.com
3. “To measure is to know” “If you can not measure it, you cannot improve it” Lord Kelvin
4. /motivation FIRST STEP InnerSource patterns about metrics strategy NEED Detail specific metrics based on InnerSource patterns FACT Metrics are part of the InnerSource journey Introducing set of metrics and KPIs
5. /context “The organization needs more objective information focused on development activities” “The organization does not have detailed KPIs to measure and where to start measuring” “Changes in the top level initiatives may affect the InnerSource program as they rely in the good will of some executive from the organization” “It’s difficult to justify that your InnerSource program is worthwhile without clear information.” https:/ /innersourcecommons.org/patterns
6. /why Awareness Motivational Actions Lead a Change
7. /method Metric #1 Question #1 GOAL #1 Question #2 Metric #2 Metric #3 Metric #1 Goal - Question - Metrics Approach
8. /strategy Act Check Do Plan & Policies ISPO Strategy
9. /areas PERFORMANCE Engagement: Contributions overview (total & evolution) Lifecycle: Active repositories (total & evolution) Activity time patterns (day & time) Geographical distribution Knowledge areas: Files activity ACTIVITY COMMUNITY
10. /areas PERFORMANCE ACTIVITY COMMUNITY Growth: Contributors overview (total & evolution) BUs Diversity (by activity or by contribs) Attraction & Retention(newcomers) Core/reg/casual contributors Collaboration networks
11. /areas PERFORMANCE ACTIVITY COMMUNITY Iterations to solve/close issues, pull/merge requests, code reviews, bottlenecks Efficiency: Times overview: to solve, open, to attend (total and evolution) BMI & REI, Backlog From idea to customer CI/CD
12. CHAOSS Community Linux foundation project focused on creating analytics and metrics to help define community health. /context Bitergia Analytics based on GrimoireLab one of CHAOSS Software founding projects focused on Data gathering, Data enrichment and Data visualization. 100% open source!
13. /Community Short Term Business Goal: Increase the adoption of the InnerSource projects within the organization. Question #1: How many people are contributing month by month? Question #2: Are our contributors base increasing? Question #3: What are the data sources with lower barriers to contribute? Question #4: How many months in a row have we had a growth in the number of contributors? Proposed panel: contributors growth.
14. /community
15. /Performance Short Term Business Goal: Reduce time to market of our products Question #1: How long does it take to close an issue and a PR? Question #2: Are we improving in our software development cycle once we have InnerSource in place? Question #3: What are the fastest data sources? (issues, PRs?) Question #4: What are the most efficient BUs? What can we learn from them? Proposed panel: efficiency panel.
17. /Activity Short Term Business Goal: Increase InnerSource projects engagement within the organization Question #1: Which projects contributors/BUs interacts most with? Question #2: What is the general trend my contributors follow across data sources? Question #3: What path are my contributors following when getting involved with InnerSource projects? Proposed panel: engagement panel Proposed panel: lifecycle
21. /lessons learned Metrics are another tool to use during the InnerSource journey (not a goal itself) Use and understand metrics take time, be patient! Don’t measure everything (but aggregate all of the data) Do not trust your developers! (they’re not following the process you think) Gamification may help at some point, but developers will cheat on metrics Work at the level of teams Involve several role levels in the metrics definition (from C-level to developers) Documentation!! (What do you mean by a commit? what do you mean by a contributor?)
22. InnerSource Spain
23. /innersource On the 21st of May, 2019. KubeCon co-located (Barcelona) Keynote - Creating InnerSource Communities that Drive Innovation By Diane Mueller, Director, Community Developer for Openshift. https:/ /www.meetup.com/es/InnerSource-Spain/events/260507248/
24. Ana Jiménez Santamaría Daniel Izquierdo Cortázar anajs|dizquierdo@bitergia.com Let’s go for questions!