Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Are you sure you’re measuring what you want to measure?

Bitergia
October 04, 2018

Are you sure you’re measuring what you want to measure?

Presentation by Daniel Izquierdo at the InnerSource Commons Fall Summit

Cheating on metrics is easy. And it happens that from time to time we realize that the metrics that we thought were the right ones, are in reality the wrong ones. This talk will analyze one of the first steps that everyone is measuring in the InnerSource initiative, and these are the contributions. There are of course several types of them, and not only source code: reviews, edits in a wiki, or conversations in asynchrounous channels. However, how can we sure that we all mean the same when we measure those contributions? Does a commit have the same meaning for everyone? Have we ever thought about automated actions by bots, merge commits, or people adding tabs to the source code? And even more, is better to measure commits than source code lines? And what about files?

Bitergia

October 04, 2018
Tweet

More Decks by Bitergia

Other Decks in Business

Transcript

  1. Are you sure you’re measuring what you want to measure?

    Daniel Izquierdo Cortázar @dizquierdo dizquierdo at bitergia.com https://speakerdeck.com/[bitergia|dicortazar] InnerSource Commons Fall Summit CA Technologies. Oct, 2018.
  2. /collaboration Everything is based on certain granularity We can measure

    collaboration at the level of: Repositories, commits, files? Pages in wikis, communication in threads? Pull requests comments, reviews or submissions? Issues, comments, participation IRC/Slack/Mattermost channels, threads
  3. /git_collaboration What is a commit? a commit adds … changes

    to … the repository, making these changes part of the head revision of the repository. … commits in version control systems are kept in the repository indefinitely. https://en.wikipedia.org/wiki/Commit_(version_control)
  4. /git_collaboration What is a commit? a commit adds … changes

    to … the repository, making these changes part of the head revision of the repository. … commits in version control systems are kept in the repository indefinitely. https://en.wikipedia.org/wiki/Commit_(version_control)
  5. /git_collaboration What is a commit? a commit adds … changes

    to … the repository, making these changes part of the head revision of the repository. … commits in version control systems are kept in the repository indefinitely. https://en.wikipedia.org/wiki/Commit_(version_control)
  6. /git_collaboration What is a commit? a commit adds … changes

    to … the repository, https://en.wikipedia.org/wiki/Commit_(version_control)
  7. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia

    <[email protected]> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
  8. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia

    <[email protected]> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
  9. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia

    <[email protected]> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
  10. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia

    <[email protected]> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
  11. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia

    <[email protected]> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
  12. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia

    <[email protected]> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
  13. /git_log What is a commit? commit 80d45e055f12c98b28d47cbff6a9d9b599b90181 2abd2ad00d6fee314762c32200f796ef0c36ad6a Author: alpgarcia

    <[email protected]> AuthorDate: Wed Dec 20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100 Fix error when enriching just the last Git commit [...] :100644 100644 7a84317... 7134d40... M enrich/enrich.py :100644 100644 400c5fd... 17de5ca... M tests/test_enrich.py 60 48 enrich/enrich.py 39 7 tests/test_enrich.py
  14. /git_author Authors vs Committers Author: alpgarcia <[email protected]> AuthorDate: Wed Dec

    20 18:35:11 2017 +0100 Commit: Daniel Izquierdo <[email protected]> CommitDate: Wed Dec 20 19:44:09 2017 +0100
  15. /wrong_assumption Examples of wrong assumptions Automated commits but existing under

    some developer account. Example at OpenShift in 2013/2014. More than 7K commits.
  16. /wrong_assumption Brian Chang as the main developer by far in

    the project, even more than the bots https://github.com/liferay/liferay-portal/graphs/contributors
  17. /keep_calm Define in a proper way each metric you’re using

    Involve developers and other roles into this Be able to reproduce the analysis Understand your infrastructure Dig into the data before assuming things Double check your final dataset