@everydaykanban | @TeamLagom #devopscon18
Start with a clear
outcome in mind
Slide 9
Slide 9 text
Walk the path to discover your metrics
@everydaykanban | @TeamLagom #devopscon18
Slide 10
Slide 10 text
Walk the path to discover your metrics
@everydaykanban | @TeamLagom
Deliver value
more frequently
#devopscon18
Slide 11
Slide 11 text
Walk the path to discover your metrics
@everydaykanban | @TeamLagom
Deliver value
more frequently
Start more or
help finish?
#devopscon18
Slide 12
Slide 12 text
Walk the path to discover your metrics
@everydaykanban | @TeamLagom
Deliver value
more frequently
Start more or
help finish?
do WiP levels correlates
to delivery speed?
#devopscon18
Slide 13
Slide 13 text
Walk the path to discover your metrics
@everydaykanban | @TeamLagom
Deliver value
more frequently
Start more or
help finish?
do WiP levels correlates
to delivery speed?
Team’s current WiP levels
AND duration (cycle time)
#devopscon18
Slide 14
Slide 14 text
Know what each metric really tells you…
@everydaykanban | @TeamLagom #devopscon18
Fooling
yourself with
vanity
metrics
everydaykanban
I sure do
get a lot
done!
I add so
much
value!
Slide 15
Slide 15 text
Know what each metric really tells you…
@everydaykanban | @TeamLagom #devopscon18
Fooling
yourself with
vanity
metrics
everydaykanban
I sure do
get a lot
done!
I add so
much
value!
• lines of code written
• # of issues closed
• # of deploys per day
• # of points finished
Slide 16
Slide 16 text
Know what each metric really tells you…
@everydaykanban | @TeamLagom #devopscon18
Fooling
yourself with
vanity
metrics
everydaykanban
I sure do
get a lot
done!
I add so
much
value!
• lines of code written
• # of issues closed
• # of deploys per day
• # of points finished
• revenue generated
• revenue protected
• cost avoided
• cost saved
Slide 17
Slide 17 text
Put each metric to the
“So What?” test
#devopscon18
@everydaykanban | @TeamLagom
Slide 18
Slide 18 text
Put each metric to the
“So What?” test
#devopscon18
@everydaykanban | @TeamLagom
Does it:
1. matter to my customer?
2. help me make a decision or take action?
Slide 19
Slide 19 text
Choose metrics that compete with each other
@everydaykanban | @TeamLagom #devopscon18
Slide 20
Slide 20 text
@everydaykanban | @TeamLagom #devopscon18
Quality
How good?
Responsiveness
How fast?
Sustainability
How repeatable?
Productivity
How much?
Slide 21
Slide 21 text
Quality: Issues by Application / Service over time
@everydaykanban | @TeamLagom #devopscon18
0
25
50
75
100
Sys A Sys B Sys C Sys D
Q4 2017 Q1 2018 Q2 2018
Find your problem apps and focus
improvement efforts there…
Slide 22
Slide 22 text
Quality: Customer Satisfaction
@everydaykanban | @TeamLagom #devopscon18
0
25
50
75
100
Jan Feb Mar Apr May
Phone Website
Measure it however you can,
even if its qualitative…
Slide 23
Slide 23 text
Responsiveness: Duration (Cycle Time)
@everydaykanban | @TeamLagom #devopscon18
Can you see different patterns for
different types of work?
Slide 24
Slide 24 text
Responsiveness: Mean Time to Recovery (MTTR)
@everydaykanban | @TeamLagom #devopscon18
Not all outages are created equal
Slide 25
Slide 25 text
Productivity: Throughput
@everydaykanban | @TeamLagom #devopscon18
Gold star if you show how much
value the items generate!
Slide 26
Slide 26 text
Productivity: Deployment Frequency
@everydaykanban | @TeamLagom #devopscon18
Gold star if you show how much
value the deploys generate!
Slide 27
Slide 27 text
Sustainability: Open vs Closed
@everydaykanban | @TeamLagom #devopscon18
Are we keeping pace with
demand?
Slide 28
Slide 28 text
Sustainability: Team Satisfaction
@everydaykanban | @TeamLagom #devopscon18
Would you recommend people
to work here?
Slide 29
Slide 29 text
@everydaykanban | @TeamLagom #devopscon18
0
25
50
75
100
Sys A Sys B Sys C Sys D
Q4 2017 Q1 2018 Q2 2018
Incidents by System MTTR
Throughput Open vs Closed
Slide 30
Slide 30 text
Improvement: Flow Efficiency
@everydaykanban | @TeamLagom #devopscon18
The best metric you’re not using!
Lead Time
Wait Wait Wait
Work Work
Work
http://www.everydaykanban.com/2016/09/25/flow-efficiency/
Slide 31
Slide 31 text
@everydaykanban | @TeamLagom #devopscon18
It’s all 3 ways of DevOps
rolled up into one metric!
Improvement: Flow Efficiency
Slide 32
Slide 32 text
@everydaykanban | @TeamLagom #devopscon18
Slide 33
Slide 33 text
@everydaykanban | @TeamLagom #devopscon18
Better and with
company trend
Oops. Still good, but
trending adversely
Creeping up…
Slide 34
Slide 34 text
You don’t need perfect data, you just need more
information than you have right now. If you have
none, even just a little information is a big
improvement.
@everydaykanban | @TeamLagom #devopscon18
Remind yourself…
Slide 35
Slide 35 text
@everydaykanban | @TeamLagom #devopscon18
Avoid these
metric missteps
Slide 36
Slide 36 text
#devopscon18
Over-optimizing for a single metric
@everydaykanban | @TeamLagom
What happens when you
take it to extremes?
Slide 37
Slide 37 text
@everydaykanban | @TeamLagom #devopscon18
Slide 38
Slide 38 text
@everydaykanban | @TeamLagom #devopscon18
Slide 39
Slide 39 text
@everydaykanban | @TeamLagom #devopscon18
Slide 40
Slide 40 text
@everydaykanban | @TeamLagom #devopscon18
Created by Julia Wester 2018
More Info on https://lagom.solutions
Lagom Discovery Canvas Key
Decision
Hypothesis
Plan / Do Study Adjust
Lagom (Just Enough) Too Much (Extreme)
Too Little (Extreme)
What decision are you trying to make?
What steps do you think will get you closer to lagom?
What consequences occur when
you operate at this extreme…
positive and negative!
How will you test your hypothesis?
How will you know if you are closer
to lagom?
When will you review?
What do your success metrics tell you?
Are you closer to lagom, farther away or
the same distance?
Do you need to tweak the plan &
continue or make a new
hypothesis?
What consequences occur when
you operate at this extreme…
positive and negative!
What are the ideal outcomes when
you do this just right?
Natural Tendency
Check the box in the section below that
resembles your normal operating condition
Slide 41
Slide 41 text
@everydaykanban | @TeamLagom #devopscon18
Created by Julia Wester 2018
More Info on https://lagom.solutions
Lagom Discovery Canvas Key
Decision
Hypothesis
Plan / Do Study Adjust
Lagom (Just Enough) Too Much (Extreme)
Too Little (Extreme)
What decision are you trying to make?
What steps do you think will get you closer to lagom?
What consequences occur when
you operate at this extreme…
positive and negative!
How will you test your hypothesis?
How will you know if you are closer
to lagom?
When will you review?
What do your success metrics tell you?
Are you closer to lagom, farther away or
the same distance?
Do you need to tweak the plan &
continue or make a new
hypothesis?
What consequences occur when
you operate at this extreme…
positive and negative!
What are the ideal outcomes when
you do this just right?
Natural Tendency
Check the box in the section below that
resembles your normal operating condition
Slide 42
Slide 42 text
@everydaykanban | @TeamLagom #devopscon18
Created by Julia Wester 2018
More Info on https://lagom.solutions
Lagom Discovery Canvas Key
Decision
Hypothesis
Plan / Do Study Adjust
Lagom (Just Enough) Too Much (Extreme)
Too Little (Extreme)
What decision are you trying to make?
What steps do you think will get you closer to lagom?
What consequences occur when
you operate at this extreme…
positive and negative!
How will you test your hypothesis?
How will you know if you are closer
to lagom?
When will you review?
What do your success metrics tell you?
Are you closer to lagom, farther away or
the same distance?
Do you need to tweak the plan &
continue or make a new
hypothesis?
What consequences occur when
you operate at this extreme…
positive and negative!
What are the ideal outcomes when
you do this just right?
Natural Tendency
Check the box in the section below that
resembles your normal operating condition
Slide 43
Slide 43 text
@everydaykanban | @TeamLagom #devopscon18
• Fast delivery
• Keeping pace
• Team challenged, not overwhelmed
• Low # of bugs / High Customer
Satisfaction scores
• Fast delivery ✔
• Keeping pace ✔
• Bored People
• Mediocre quality
• Seen as overstaffed
• Slow delivery
• Not Keeping Pace
• Team Burnout
• Poor quality
• No time for improvement
How much work should we have in process at once?
❌
Just measuring these
gives a false +
measure a balanced set of
metrics to see negative
consequences
Slide 44
Slide 44 text
Measuring Individuals when you value Team outcomes
@everydaykanban | @TeamLagom #devopscon18
Slide 45
Slide 45 text
#devopscon18
Superstars aren’t always
the best team members
@everydaykanban | @TeamLagom
Slide 46
Slide 46 text
Using shame as an incentive
@everydaykanban | @TeamLagom #devopscon18
Courtesy of Troy Magennis
Slide 47
Slide 47 text
“Fear invites wrong
figures. Bearers of bad
news fare badly. To
keep his job, anyone
may present to his boss
only good news.”
–W. Edwards Deming
#devopscon18
@everydaykanban | @TeamLagom
–W. Edwards Deming
“If you give a manager a numerical target,
he’ll make it, even if he has to destroy the
organization in the process.”
@everydaykanban | @TeamLagom #devopscon18
Slide 50
Slide 50 text
To get ahead of these
issues, intentionally
explore the dark side of
metrics
@everydaykanban | @TeamLagom #devopscon18
I’m just misunderstood!
Slide 51
Slide 51 text
@everydaykanban | @TeamLagom #devopscon18
Remember, what you measure shows what you value…
Slide 52
Slide 52 text
–Eli Godratt
“Tell me how you’ll measure me and I’ll
tell you how I’ll behave. If you measure
me in an illogical way, do not complain
about illogical behavior.”
@everydaykanban | @TeamLagom #devopscon18
Slide 53
Slide 53 text
Know
when to
discard a
metric
@everydaykanban | @TeamLagom #devopscon18
Slide 54
Slide 54 text
#devopscon18
How can we
make this
safer?
What is the
impact?
How can you
game it? “How Can I Game It?”
Ask your team to play
and get a wider
perspective
@everydaykanban | @TeamLagom
Slide 55
Slide 55 text
@everydaykanban | @TeamLagom #devopscon18
Slide 56
Slide 56 text
@everydaykanban | @TeamLagom #devopscon18
What will you do differently
on Monday?
Slide 57
Slide 57 text
Do you have any metric tips to share?
Any questions?
@everydaykanban | @TeamLagom #devopscon18
Slide 58
Slide 58 text
@everydaykanban | @TeamLagom #devopscon18
[email protected]
@everydaykanban & @TeamLagom
https://lagom.solutions
http://everydaykanban.com
https://www.linkedin.com/in/juliawester/
Contact me to talk more