@everydaykanban | @55DegreesAB #agilecam
Getting started with
data-driven coaching
Julia Wester - Coach, Consultant, and Trainer - 55 Degrees AB
Slide 2
Slide 2 text
Meaning is in the eye of the beholder
@everydaykanban | @55DegreesAB 2 #agilecam
“What a slacker!”
“Wow, I bet they’re
working on other
priorities right now.”
“Makes sense. That
person’s code was part of
a deep exploratory
testing session.”
Slide 3
Slide 3 text
“Tell me how you’ll measure me
and I’ll tell you how I’ll behave.
–Eli Goldratt
@everydaykanban | @55DegreesAB 3 #agilecam
Slide 4
Slide 4 text
What do your metrics incentivize?
@everydaykanban | @55DegreesAB 4 #agilecam
Slide 5
Slide 5 text
What do your metrics incentivize?
@everydaykanban | @55DegreesAB 5 #agilecam
–W. Edwards Deming
“If you give a manager a numerical target,
he’ll make it, even if he has to destroy the
organization in the process.”
Slide 6
Slide 6 text
What if you were this developer?
@everydaykanban | @55DegreesAB 6 #agilecam
Slide 7
Slide 7 text
“Tell me how you’ll measure me
and I’ll tell you how I’ll behave.
If you measure me in illogical
ways, do not complain about
illogical behavior.”
–Eli Goldratt
@everydaykanban | @55DegreesAB 7 #agilecam
Slide 8
Slide 8 text
@everydaykanban | @55DegreesAB 8
What is Data-Driven Coaching?
#agilecam
Helping teams responsibly interpret data
to identify areas of improvement
Helping teams avoid over-emphasizing
any one measurement
1 2
Slide 9
Slide 9 text
Measure what
really matters
@everydaykanban | @55DegreesAB 9 #agilecam
not what’s easy or convenient
Slide 10
Slide 10 text
@everydaykanban | @55DegreesAB 10 #agilecam
Decide on outcomes then
determine your metrics
Reduced time /
cost for rework
Are we finding defects
early enough?
# of Escaped Defects
Should we increase time
spent on testing?
think of small,
day-to-day
decisions
Larry
Maccherone
Slide 11
Slide 11 text
Escaped Defects
@everydaykanban | @55DegreesAB 11 #agilecam
defects found in production
Question if the measure
really provides expected
insight!
Slide 12
Slide 12 text
@everydaykanban | @55DegreesAB 12 #agilecam
You might not get it right
the first time
Reduced time /
cost for rework
Are we finding defects
early enough?
Should we increase time
spent on testing?
# of Escaped Defects
# of defects found in each
environment
Slide 13
Slide 13 text
@everydaykanban | @55DegreesAB 13 #agilecam
Defects by
Environment
counts
Keep questioning until
you’re happy enough
Slide 14
Slide 14 text
@everydaykanban | @55DegreesAB 14 #agilecam
Inspect and adapt
Reduced time /
cost for rework
Are we finding defects
early enough?
Should we increase time
spent on testing?
# of Escaped Defects
# of defects found in each
environment
% of defects found in each
environment
Slide 15
Slide 15 text
@everydaykanban | @55DegreesAB 15 #agilecam
Defects by
Environment
percentages
Slide 16
Slide 16 text
@everydaykanban | @55DegreesAB 16 #agilecam
If a metric can’t be mapped
to an outcome, discard it!
Slide 17
Slide 17 text
@everydaykanban | @55DegreesAB 17 #agilecam
No longer need
the insight?
Discard the
metric!
Make these topics part of
your retrospectives!
Slide 18
Slide 18 text
Measure teams,
not individuals
@everydaykanban | @55DegreesAB 18 #agilecam
if you want people to act
like a team
Slide 19
Slide 19 text
What you measure tells the story of what you value
@everydaykanban | @55DegreesAB 19 #agilecam
Slide 20
Slide 20 text
Value vs. Cost - A true story
@everydaykanban | @55DegreesAB 20 #agilecam
Slide 21
Slide 21 text
The Superstar: A cautionary tale
@everydaykanban | @55DegreesAB 21 #agilecam
Slide 22
Slide 22 text
If you want stellar individuals,
put more focus on individual outcomes
If you want stellar teams,
put more focus on team outcomes.
@everydaykanban | @55DegreesAB 22 #agilecam
Slide 23
Slide 23 text
Don’t use metrics
to shame people
@everydaykanban | @55DegreesAB 23 #agilecam
if you want the data to
show reality
Slide 24
Slide 24 text
“Fear invites wrong
figures. Bearers of bad
news fare badly. To
keep his job, anyone
may present to his
boss only good news.”
-W. Edwards Deming
@everydaykanban | @55DegreesAB 24 #agilecam
Slide 25
Slide 25 text
What if this picture is misleading?
@everydaykanban | @55DegreesAB 25 #agilecam
What could these
people be doing to
stay behind the line?
Slide 26
Slide 26 text
Use multiple,
competing metrics
@everydaykanban | @55DegreesAB 26 #agilecam
to avoid unintended
consequences
https://john-lewis-artist.deviantart.com/
Slide 27
Slide 27 text
What will happen if you measure this, but not this?
@everydaykanban | @55DegreesAB 27 #agilecam
Slide 28
Slide 28 text
@everydaykanban | @55DegreesAB 28 #agilecam
Slide 29
Slide 29 text
@everydaykanban | @55DegreesAB 29 #agilecam
Slide 30
Slide 30 text
@everydaykanban | @55DegreesAB 30 #agilecam
A helpful template to guide you...
Metric for current
quadrant
Metric for opposing
quadrant
Troy Magennis
Focused Objective
Slide 31
Slide 31 text
@everydaykanban | @55DegreesAB 31 #agilecam
A helpful template to guide you...
Troy Magennis
Focused Objective
@everydaykanban | @55DegreesAB 33 #agilecam
A helpful template to guide you...
Troy Magennis
Focused Objective
Slide 34
Slide 34 text
@everydaykanban | @55DegreesAB 34 #agilecam
Give a cheat sheet!
• What is the chart
• Intended behavior
• How to game it
• When overdriven, what moves?
Download these tools:
http://bit.ly/SimResources
Troy Magennis
Focused Objective
Slide 35
Slide 35 text
@everydaykanban | @55DegreesAB 35 #agilecam
Reflect and Discuss
Are certain areas more
important in your context?
Slide 36
Slide 36 text
@everydaykanban | @55DegreesAB 36 #agilecam
Sometimes trade-offs
are ok, even necessary
Slide 37
Slide 37 text
Highlight trends,
not data points
@everydaykanban | @55DegreesAB 37 #agilecam
to avoid distraction from
insignificant variation
Slide 38
Slide 38 text
How to put a focus on trends
@everydaykanban | @55DegreesAB 38 #agilecam
No
Y-Axis
Tiny #s
When applicable, connect
data points with lines
Show Trend
Line
Project
Trend
Slide 39
Slide 39 text
Compare trends across teams to spot system-level opportunities
@everydaykanban | @55DegreesAB 39 #agilecam
Troy Magennis
Focused Objective
^in a safe way
Slide 40
Slide 40 text
Show me when
to worry
@everydaykanban | @55DegreesAB 40 #agilecam
don’t make me guess or
waste my time off
Slide 41
Slide 41 text
Not all variation is cause for alarm
@everydaykanban | @55DegreesAB 41 #agilecam
Slide 42
Slide 42 text
“First, do no harm.”
“If anyone adjusts a stable
process, the output that follows
will be worse than if (s)he had
left the process alone.”
-W. Edwards Deming
@everydaykanban | @55DegreesAB 42 #agilecam
Slide 43
Slide 43 text
@everydaykanban | @55DegreesAB 43
Find Leading Indicators
#agilecam
Slide 44
Slide 44 text
@everydaykanban | @55DegreesAB 44
When we do our job well, we help teams:
#agilecam
responsibly interpret data
to identify areas of improvement
avoid over-emphasizing any one measure
and create a healthy whole
1 2
Slide 45
Slide 45 text
@everydaykanban | @55DegreesAB 45 #agilecam
What’s your next step?
Slide 46
Slide 46 text
@everydaykanban | @55DegreesAB 46 #agilecam
[email protected]
@everydaykanban & @55DegreesAB
https://55degrees.se
http://everydaykanban.com
https://www.linkedin.com/in/juliawester/
I’d love to hear from you