Mirror Mirror on the wall,
what’s the vainest metric of
them all?
Paul Stack
@stack72
Slide 2
Slide 2 text
About Me
Jobless Bum
Infrastructure Enthusiast
DevOps Extremist
Slide 3
Slide 3 text
What are vanity metrics?
Slide 4
Slide 4 text
The perfect example of what vanity metrics are
Slide 5
Slide 5 text
So all metrics can be bad?
Slide 6
Slide 6 text
1. A company creates a new application
2. A PR email was created and sent out
3. 100,000 people arrived on the landing page for the
application
Is the measurement of 100,000 page views a real
metric or a vanity metric and why?
Slide 7
Slide 7 text
What could we measure
instead?
Slide 8
Slide 8 text
1. A company creates a new application
2. A PR email was created and sent out
3. 100,000 people arrived on the landing page for the
application
4. The rate of conversion from page view to application
download
Slide 9
Slide 9 text
A “conversion funnel”- https://www.truconversion.com/blog/conversion-rate-optimization/use-conversion-funnel-v3/
Slide 10
Slide 10 text
So how can I identify if a
metric is measured in vain?
Slide 11
Slide 11 text
1. Can I make a business
decision based on this
metric?
Slide 12
Slide 12 text
2. Can I intentionally repeat
this result?
Slide 13
Slide 13 text
3. Is the data a real reflection
of the truth?
Slide 14
Slide 14 text
We can apply these
questions to any
measurement and help us to
decide if it’s important or not
Slide 15
Slide 15 text
Real life examples….
These are my personal opinions - you don’t have to
agree
Slide 16
Slide 16 text
Twitter followers
Blog pageviews
Greens at the end of a conference talk
Number of GitHub commits
Number of GH issues / PRs
Build Times
Number of builds per day
Number of GH stars / forks
Number of Application Downloads
Slide 17
Slide 17 text
Number of “story points” achieved per sprint
Estimation planning accuracy
Meetings
Cost of Infrastructure
Cost of Travel
Cost of Developer Productivity
Code coverage!!
Lines of code per day!
Did I miss any?
Slide 18
Slide 18 text
The worst offender*
Slide 19
Slide 19 text
Cost of travel!
Slide 20
Slide 20 text
Honorable mentions…
Slide 21
Slide 21 text
Cost of infrastructure
Cost of developer productivity
Slide 22
Slide 22 text
So tell me what are useful
metrics to collect!!!!
Slide 23
Slide 23 text
WIP (work in progress)
Cycle / lead time
MTTR (mean time to resolution)
MTBF (mean time between failures)
Application crash rates
Issue Open / Close rates
Release cycle time
Conversion rate (think of page view -> registration)
A very small sample that are not targeted toward anything specific
Slide 24
Slide 24 text
Measure everything… right?
Slide 25
Slide 25 text
Summary
There are 2 types of metrics
Vanity
Actionable
Let’s care less about the vanity metrics and more about
the actionable metrics!
Continued use of vanity metrics is bad for both
developers AND the health of an organisation!