The Metrics Journey

Title: The Metrics Journey
Speaker: Daniel Izquierdo Cortázar
Abstract: Adeo has been involved in the InnerSource journey during the last months. This is mainly a cultural change and software development process improvement based on top of the right infrastructure and tools. However, this is not easy and even more when we talk about culture.
How do we know if the initiative is currently progressing? Are we hitting the market earlier than before? Are several business units directly collaborating? What about breaking the internal silos within the company?
Metrics and KPIs are essential in this journey as they will help in three main areas: (a) awareness, to understand where we are, (b) lead a change to support decisions, and (c) motivate developers and middle managers by showing progress.
This talk will introduce several aspects to have into account when introducing, using and discussing metrics, and the lessons learned during the last years analyzing open source projects and in the InnerSource ecosystem.

Transcript

1. The Metrics Journey ADEO DevSummit, Lille 2019 Daniel Izquierdo Cortázar @dizquierdo
2. Ph.D on Free Software Engineering Bitergia co-founder Software Development Analytics for Open and Inner Source Projects 100% Open Source https://innersourcecommons.org Community Health Analytics for OSS Projects https://chaoss.community
3. What is InnerSource?
4. “InnerSource takes the lessons learned from developing open source software and applies them to the way companies develop software internally” https://innersourcecommons.org
5. Promises of InnerSource
6. ↑ Code Quality ↓ Time to market Innovation ↓Dev & maintenance costs ↑ Engagement
7. ↑ Skills (soft and hard) ↑ Dynamic environment Innovation ↑ Solve problems ↑ Empowerment
8. Why is this needed?
10. What is open source?
11. “Free software or libre software is computer software distributed under terms that allow users to run the software for any purpose as well as to study, change, and distribute it and any adapted versions” https://en.wikipedia.org/wiki/Free_software
12. What is actually open source?
14. fosdem.org
15. Culture Communication Transparency Collaboration Community Meritocracy
16. So we want to be open source, but internally Great!
17. Oh wait! Are we really working as an open source community?
18. Breaking silos Fostering collaboration Hitting market faster Improving the process Spreading the knowledge ...
19. How do you measure success?
20. Why do we need metrics?
21. “Without data, you are just another person with an opinion” W. Edwards Deming
22. “To measure is to know” “If you can not measure it, you can not improve it” Lord Kelvin
23. Awareness
24. Lead a Change
25. Motivational
26. What can we measure?
27. “Without data, you are just another person with an opinion” W. Edwards Deming Bitergia Analytics / GrimoireLab supported data sources https://chaoss.github.io/grimoirelab/
28. What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
29. We need to translate commits/events in any data source into metrics, vizs and KPIs
31. PERFORMANCE COMMUNITY ACTIVITY Engagement: Contributions overview (total & evolution) Lifecycle: Active repositories (total & evolution) Activity time patterns (day & time) Geographical distribution Knowledge areas: Files activity
32. PERFORMANCE ACTIVITY COMMUNITY Growth: Contributors overview (total & evolution) BUs Diversity (by activity or by contribs) Attraction & Retention(newcomers) Core/reg/casual contributors Collaboration networks
33. PERFORMANCE ACTIVITY COMMUNITY Iterations to solve/close issues, pull/merge requests, code reviews, bottlenecks Efficiency: Times overview: to solve, open, to attend (total and evolution) BMI & REI, Backlog From idea to customer CI/CD
34. And we need some method and strategy How can we measure? Where do we start?
35. Goal Question Metric Metric #1 Question #1 GOAL #1 Question #2 Metric #2 Metric #3 Metric #1
36. Act Check Do Plan Strategy
37. Increase the adoption of the InnerSource projects within the organization Question #1: How many people are contributing month by month? Question #2: Are our contributors base increasing? Question #3: What are the data sources with lower barriers to contribute? Question #4: How many months in a row have we had a growth in the number of contributors? Proposed panel: contributors growth
39. Reduce time to market of our products Question #1: How long does it take to close an issue and a PR? Question #2: Are we improving in our software development cycle once we have InnerSource in place? Question #3: What are the ones attended fastest? (issues, PRs?) Question #4: What are the most efficient BUs? What can we learn from them? Proposed panel: efficiency panel
41. Increase InnerSource projects engagement within the organization Question #1: Which projects contributors/BUs interacts most with? Question #2: What is the general trend my contributors follow across data sources? Question #3: What path are my contributors following when getting involved with InnerSource projects? Proposed panels: engagement panel, lifecycle
44. Lesson 1: Metrics are just another tool
45. Lesson 2: Use and understand metrics takes time, be patient!
46. Lesson 3: Don’t measure everything (but aggregate all of the data)
47. Lesson 4: Do not trust yourself! (you’re not following the process you think)
48. Lesson 5: Gamification may help at some point, but people will cheat on metrics
49. Lesson 6: Work at the level of teams
50. Lesson 7: Involve several role levels in the metrics definition (from C-level to developers)
51. Lesson 8: Documentation!! (What do you mean by a commit? what do you mean by a contributor?)
52. Interested in metrics? https://innersourcecommons.org Community Health Analytics for OSS Projects https://chaoss.community https://chaoss.github.io/grimoirelab/
53. “Without data, you are just another person with an opinion” W. Edwards Deming
54. “Without an opinion, you are just another person with data” https://www.forbes.com/sites/silberzahnjones/2016/03/15/without-an-opinion-youre-just-another-person-with-data
55. Daniel Izquierdo Cortázar dizquierdo@bitergia.com @dizquierdo
56. “By 2022, over half of the code in 75% of new apps will be from external sources. The need to improve developer productivity is driving the use of open source and commercial code repositories” “By the End of 2019, DX spending will reach $1.7 Trillion worldwide, a 42% increase from 2017” “By 2020, 30% of G2000 companies will have allocated capital budget equal to at least 10% of revenue to fuel their digital strategies” “By 2020, 25% of Global 2000 companies will have developed Digital Training programs and Digital Cooperatives to compete more effectively in talent wars”
57. So why an Open Source company? Why an InnerSource company?
58. “Paying employees to contribute to open source software boosts the company’s productivity from using the software by as much as 100 percent, when compared with free-riding competitors” “Being an open source publisher attracts talent and helps us get the kind of people who already know our coding style and technology focus areas” “We’re able to retain the world’s best engineers because they know they can open-source their work”
60. “To measure is to know” “If you can not measure it, you cannot improve it” Lord Kelvin
61. Bitergia Analytics info@bitergia.com @bitergia