Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What you measure is what you get. Stories of metrics gone wrong

What you measure is what you get. Stories of metrics gone wrong

Situational awareness is vital in any situation. If you are unaware of an issue, then you are in a poor position to fix it. However, there is a natural human response to the publication of metrics, which is to optimise for what is measured. Here are some cautionary tales of how performance metrics inadvertently created incentives for behaviour that was bad for the system.

More Decks by FLOSS Community Metrics meeting

Other Decks in Technology


  1. This Summer, don't miss:
    Dave Neary
    [email protected]

    View Slide

  2. How things go bad

    Metrics should result in action

    Metrics create incentives

    Sometimes, metrics create destructive

    People are weird

    View Slide

  3. Setting the wrong incentives

    View Slide

  4. View Slide

  5. Measuring
    the wrong

    View Slide

  6. Myopia

    View Slide

  7. Drawing the wrong conclusions

    View Slide

  8. Conclusion

    Be careful what you measure – metrics create

    Focus on business and community success

    Use qualitative and quantitative analysis to
    interpret metrics

    View Slide