Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What you measure is what you get. Stories of metrics gone wrong

What you measure is what you get. Stories of metrics gone wrong

Situational awareness is vital in any situation. If you are unaware of an issue, then you are in a poor position to fix it. However, there is a natural human response to the publication of metrics, which is to optimise for what is measured. Here are some cautionary tales of how performance metrics inadvertently created incentives for behaviour that was bad for the system.

More Decks by FLOSS Community Metrics meeting

Other Decks in Technology


  1. This Summer, don't miss: Metrics Gone Bad Featuring: Dave Neary

  2. How things go bad • Metrics should result in action

    • Metrics create incentives • Sometimes, metrics create destructive incentives • People are weird
  3. Setting the wrong incentives

  4. None
  5. Measuring the wrong things

  6. Myopia

  7. Drawing the wrong conclusions

  8. Conclusion • Be careful what you measure – metrics create

    incentives • Focus on business and community success measurements • Use qualitative and quantitative analysis to interpret metrics