Mirror, Mirror on the way, what is the vainest metric of them all?

Transcript

1. Mirror Mirror on the wall, what’s the vainest metric of them all? Paul Stack @stack72
2. About Me Jobless Bum Infrastructure Enthusiast DevOps Extremist
3. What are vanity metrics?
4. The perfect example of what vanity metrics are
5. So all metrics can be bad?
6. 1. A company creates a new application 2. A PR email was created and sent out 3. 100,000 people arrived on the landing page for the application Is the measurement of 100,000 page views a real metric or a vanity metric and why?
7. What could we measure instead?
8. 1. A company creates a new application 2. A PR email was created and sent out 3. 100,000 people arrived on the landing page for the application
 
 
 4. The rate of conversion from page view to application download
9. A “conversion funnel”- https://www.truconversion.com/blog/conversion-rate-optimization/use-conversion-funnel-v3/
10. So how can I identify if a metric is measured in vain?
11. 1. Can I make a business decision based on this metric?
12. 2. Can I intentionally repeat this result?
13. 3. Is the data a real reflection of the truth?
14. We can apply these questions to any measurement and help us to decide if it’s important or not
15. Real life examples…. These are my personal opinions - you don’t have to agree
16. Twitter followers Blog pageviews Greens at the end of a conference talk Number of GitHub commits Number of GH issues / PRs Build Times Number of builds per day Number of GH stars / forks Number of Application Downloads
17. Number of “story points” achieved per sprint Estimation planning accuracy Meetings Cost of Infrastructure Cost of Travel Cost of Developer Productivity Code coverage!! Lines of code per day! Did I miss any?
18. The worst offender*
19. Cost of travel!
20. Honorable mentions…
21. Cost of infrastructure Cost of developer productivity
22. So tell me what are useful metrics to collect!!!!
23. WIP (work in progress) Cycle / lead time MTTR (mean time to resolution) MTBF (mean time between failures) Application crash rates Issue Open / Close rates Release cycle time Conversion rate (think of page view -> registration) A very small sample that are not targeted toward anything specific
24. Measure everything… right?
25. Summary There are 2 types of metrics Vanity Actionable Let’s care less about the vanity metrics and more about the actionable metrics! Continued use of vanity metrics is bad for both developers AND the health of an organisation!
26. Paul Stack @stack72