Are you sure you're measuring what you want to measure?

Talk presented at the InnerSource Commons in Islandia, New York. Oct. 2018.

This presentation details the several ways of measuring any of the existing metrics and the importance of being sure about what each of us is measuring. The presentation focuses on commits, but this can be easily extended to any other metric.

Transcript

1. Are you sure you’re measuring what you want to measure? Daniel Izquierdo Cortázar @dizquierdo dizquierdo at bitergia.com https://speakerdeck.com/[bitergia|dicortazar] InnerSource Commons Fall Summit CA Technologies. Oct, 2018.
2. /context Measuring inner source Let’s assume that we already have: Methodology Infrastructure Strategy
3. /data_sources
4. /collaboration Everything is based on certain granularity We can measure collaboration at the level of: Repositories, commits, files? Pages in wikis, communication in threads? Pull requests comments, reviews or submissions? Issues, comments, participation IRC/Slack/Mattermost channels, threads
5. /viz
6. /viz
7. /motivation When we say {commit}, do we all mean the same?
8. /git_collaboration What is a commit? a commit adds … changes to … the repository, making these changes part of the head revision of the repository. … commits in version control systems are kept in the repository indefinitely. https://en.wikipedia.org/wiki/Commit_(version_control)
9. /git_collaboration What is a commit? a commit adds … changes to … the repository, making these changes part of the head revision of the repository. … commits in version control systems are kept in the repository indefinitely. https://en.wikipedia.org/wiki/Commit_(version_control)
10. /git_collaboration What is a commit? a commit adds … changes to … the repository, making these changes part of the head revision of the repository. … commits in version control systems are kept in the repository indefinitely. https://en.wikipedia.org/wiki/Commit_(version_control)
11. /git_collaboration What is a commit? a commit adds … changes to … the repository, https://en.wikipedia.org/wiki/Commit_(version_control)
12. /git_collaboration What is a commit? ¯\_(ツ)_/¯
13. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
14. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
15. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
16. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
17. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
18. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
19. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
20. /git_bots
21. /git_merge A---B---C branch / \ D---E---F---G---H master
22. /git_merge A---B---C branch / \ D---E---F---G---H master OpenStack vs InnerSource Patterns (1:1) vs (1:n)
23. /infra CVS vs SVN vs Git
24. /git_branches Branches vs Master A---B---C---I---J---Z branch / \ D---E---F---G---H master
25. /git_author Authors vs Committers Author: alpgarcia <alpgarcia@bitergia.com> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <dizquierdo@bitergia.com> CommitDate: Wed Dec 20 19:44:09 2017 +0100
26. /openstack_nova All commits
27. /openstack_nova All commits, but merges
28. /openstack_nova All commits, but merges and bots
29. /wrong_assumption Examples of wrong assumptions Automated commits but existing under some developer account. Example at OpenShift in 2013/2014. More than 7K commits.
30. /git_collaboration Examples of wrong assumptions Automated commits:
31. /wrong_assumption Brian Chang as the main developer by far in the project, even more than the bots https://github.com/liferay/liferay-portal/graphs/contributors
32. /summary
33. /keep_calm Define in a proper way each metric you’re using Involve developers and other roles into this Be able to reproduce the analysis Understand your infrastructure Dig into the data before assuming things Double check your final dataset
34. /viz
35. Software Development Analytics for your peace of mind dizquierdo@bitergia.com Chief Data Officer at Bitergia