A few good development metrics

Presented at Agile India 2012 Bangalore
The problem with metrics is that there are so many to choose from. Lines of code, rule violations, dependency matrices, cyclomatic/npath complexity, code coverage, duplication - the list goes on. Tracking all of these results in too much data and too little insight. In this talk, we will see how to narrow down to a small set of useful metrics. We'll also see how aggregate metrics like toxicity help reduce the tracking footprint. Finally, we'll look at the difference between a measurement and a target and see why measurements should not be simply converted into targets.

Transcript

1. A few good development metrics Sriram Narayan ThoughtWorks
2. Common metrics • Velocity • Lines of code • Cyclomatic complexity • Unit test coverage
3. Slightly less common metrics • Cycle time • Commit Build time • % successful builds • Commits per day • Defect density • Duplication
4. Not quite metrics • Coding guidelines/checklist violations • Dependency matrix • Source code hotspots – from SCM history • NFR tests
5. Never exhaustive enough
6. Unidirectional Utility of Metrics Does 70% coverage indicate a good enough test suite?
7. What’s a good test?
8. Too much data, too little insight • Need to be selective • Very difficult to get an insight merely by looking at data. Need context as well. • Metrics should be just one of several mechanisms for understanding a given development effort • Think diagnostics
9. Aggregate metrics • We already use some – Velocity, build time, test coverage • Combine related metrics into meaningful aggregates • What if we could roll up a set of code quality metrics into one meaningful aggregate?
10. Toxicity x times threshold value contributes x units of toxicity Thresholds are defined at ‘toxic’ levels, not just bad
11. Suggested core • Toxicity balanced with duplication • Test Coverage balanced with test to code ratio • Commit Build time • Velocity
12. Measurement vs. target • Targets are optional • The more specific a measurement, the less reason to set a target for it • Targets influence behaviour and cause local optima • Targets encourage “hands off” red-green- amber supervision • Abolish the metrics cop
13. Enforcing thresholds • Fail the build • Ratchet • Bad idea if imposed on the team
14. Visualizing metrics • May be useful depending on the audience • It is possible for critical thinking to get clouded by slick visualization • A picture is not always worth a thousand words • Beware the dumbing-down effect
15. The use and abuse of metrics • Use them as one of the guiding inputs • The metrics are not in charge • Beware the green dashboard • Don’t use them as a reporting tool • Don’t use them as a means to separate decision-making/planning from execution
16. Questions, comments? Thank you