Getting Started with Data-Driven Coaching

W. Edwards Deming said, “You can’t manage what you can’t measure.” But all too often, teams fall prey to misleading metrics, missing invaluable opportunities to grow and improve. Our job as coaches is to help shine the light down the ideal path. This session will show you how to visualise and use metrics in a team coaching practice. Come and learn how teams can see and balance a set of competing metrics and anticipate the impact of process changes.

Transcript

1. @everydaykanban | @55DegreesAB #agilecam
   Getting started with
   data-driven coaching
   Julia Wester - Coach, Consultant, and Trainer - 55 Degrees AB
   

   View Slide

2. Meaning is in the eye of the beholder
   @everydaykanban | @55DegreesAB 2 #agilecam
   “What a slacker!”
   “Wow, I bet they’re
   working on other
   priorities right now.”
   “Makes sense. That
   person’s code was part of
   a deep exploratory
   testing session.”
   

   View Slide

3. “Tell me how you’ll measure me
   and I’ll tell you how I’ll behave.
   –Eli Goldratt
   @everydaykanban | @55DegreesAB 3 #agilecam
   

   View Slide

4. What do your metrics incentivize?
   @everydaykanban | @55DegreesAB 4 #agilecam
   

   View Slide

5. What do your metrics incentivize?
   @everydaykanban | @55DegreesAB 5 #agilecam
   –W. Edwards Deming
   “If you give a manager a numerical target,
   he’ll make it, even if he has to destroy the
   organization in the process.”
   

   View Slide

6. What if you were this developer?
   @everydaykanban | @55DegreesAB 6 #agilecam
   

   View Slide

7. “Tell me how you’ll measure me
   and I’ll tell you how I’ll behave.
   If you measure me in illogical
   ways, do not complain about
   illogical behavior.”
   –Eli Goldratt
   @everydaykanban | @55DegreesAB 7 #agilecam
   

   View Slide

8. @everydaykanban | @55DegreesAB 8
   What is Data-Driven Coaching?
   #agilecam
   Helping teams responsibly interpret data
   to identify areas of improvement
   Helping teams avoid over-emphasizing
   any one measurement
   1 2
   

   View Slide

9. Measure what
   really matters
   @everydaykanban | @55DegreesAB 9 #agilecam
   not what’s easy or convenient
   

   View Slide

10. @everydaykanban | @55DegreesAB 10 #agilecam
    Decide on outcomes then
    determine your metrics
    Reduced time /
    cost for rework
    Are we finding defects
    early enough?
    # of Escaped Defects
    Should we increase time
    spent on testing?
    think of small,
    day-to-day
    decisions
    Larry
    Maccherone
    

    View Slide

11. Escaped Defects
    @everydaykanban | @55DegreesAB 11 #agilecam
    defects found in production
    Question if the measure
    really provides expected
    insight!
    

    View Slide

12. @everydaykanban | @55DegreesAB 12 #agilecam
    You might not get it right
    the first time
    Reduced time /
    cost for rework
    Are we finding defects
    early enough?
    Should we increase time
    spent on testing?
    # of Escaped Defects
    # of defects found in each
    environment
    

    View Slide

13. @everydaykanban | @55DegreesAB 13 #agilecam
    Defects by
    Environment
    counts
    Keep questioning until
    you’re happy enough
    

    View Slide

14. @everydaykanban | @55DegreesAB 14 #agilecam
    Inspect and adapt
    Reduced time /
    cost for rework
    Are we finding defects
    early enough?
    Should we increase time
    spent on testing?
    # of Escaped Defects
    # of defects found in each
    environment
    % of defects found in each
    environment
    

    View Slide

15. @everydaykanban | @55DegreesAB 15 #agilecam
    Defects by
    Environment
    percentages
    

    View Slide

16. @everydaykanban | @55DegreesAB 16 #agilecam
    If a metric can’t be mapped
    to an outcome, discard it!
    

    View Slide

17. @everydaykanban | @55DegreesAB 17 #agilecam
    No longer need
    the insight?
    Discard the
    metric!
    Make these topics part of
    your retrospectives!
    

    View Slide

18. Measure teams,
    not individuals
    @everydaykanban | @55DegreesAB 18 #agilecam
    if you want people to act
    like a team
    

    View Slide

19. What you measure tells the story of what you value
    @everydaykanban | @55DegreesAB 19 #agilecam
    

    View Slide

20. Value vs. Cost - A true story
    @everydaykanban | @55DegreesAB 20 #agilecam
    

    View Slide

21. The Superstar: A cautionary tale
    @everydaykanban | @55DegreesAB 21 #agilecam
    

    View Slide

22. If you want stellar individuals,
    put more focus on individual outcomes
    If you want stellar teams,
    put more focus on team outcomes.
    @everydaykanban | @55DegreesAB 22 #agilecam
    

    View Slide

23. Don’t use metrics
    to shame people
    @everydaykanban | @55DegreesAB 23 #agilecam
    if you want the data to
    show reality
    

    View Slide

24. “Fear invites wrong
    figures. Bearers of bad
    news fare badly. To
    keep his job, anyone
    may present to his
    boss only good news.”
    -W. Edwards Deming
    @everydaykanban | @55DegreesAB 24 #agilecam
    

    View Slide

25. What if this picture is misleading?
    @everydaykanban | @55DegreesAB 25 #agilecam
    What could these
    people be doing to
    stay behind the line?
    

    View Slide

26. Use multiple,
    competing metrics
    @everydaykanban | @55DegreesAB 26 #agilecam
    to avoid unintended
    consequences
    https://john-lewis-artist.deviantart.com/
    

    View Slide

27. What will happen if you measure this, but not this?
    @everydaykanban | @55DegreesAB 27 #agilecam
    

    View Slide

28. @everydaykanban | @55DegreesAB 28 #agilecam
    

    View Slide

29. @everydaykanban | @55DegreesAB 29 #agilecam
    

    View Slide

30. @everydaykanban | @55DegreesAB 30 #agilecam
    A helpful template to guide you...
    Metric for current
    quadrant
    Metric for opposing
    quadrant
    Troy Magennis
    Focused Objective
    

    View Slide

31. @everydaykanban | @55DegreesAB 31 #agilecam
    A helpful template to guide you...
    Troy Magennis
    Focused Objective
    

    View Slide

32. @everydaykanban | @55DegreesAB 32 #agilecam
    Behavioral
    Polarity
    Worksheet
    Troy Magennis
    Focused Objective
    

    View Slide

33. @everydaykanban | @55DegreesAB 33 #agilecam
    A helpful template to guide you...
    Troy Magennis
    Focused Objective
    

    View Slide

34. @everydaykanban | @55DegreesAB 34 #agilecam
    Give a cheat sheet!
    • What is the chart
    • Intended behavior
    • How to game it
    • When overdriven, what moves?
    Download these tools:
    http://bit.ly/SimResources
    Troy Magennis
    Focused Objective
    

    View Slide

35. @everydaykanban | @55DegreesAB 35 #agilecam
    Reflect and Discuss
    Are certain areas more
    important in your context?
    

    View Slide

36. @everydaykanban | @55DegreesAB 36 #agilecam
    Sometimes trade-offs
    are ok, even necessary
    

    View Slide

37. Highlight trends,
    not data points
    @everydaykanban | @55DegreesAB 37 #agilecam
    to avoid distraction from
    insignificant variation
    

    View Slide

38. How to put a focus on trends
    @everydaykanban | @55DegreesAB 38 #agilecam
    No
    Y-Axis
    Tiny #s
    When applicable, connect
    data points with lines
    Show Trend
    Line
    Project
    Trend
    

    View Slide

39. Compare trends across teams to spot system-level opportunities
    @everydaykanban | @55DegreesAB 39 #agilecam
    Troy Magennis
    Focused Objective
    ^in a safe way
    

    View Slide

40. Show me when
    to worry
    @everydaykanban | @55DegreesAB 40 #agilecam
    don’t make me guess or
    waste my time off
    

    View Slide

41. Not all variation is cause for alarm
    @everydaykanban | @55DegreesAB 41 #agilecam
    

    View Slide

42. “First, do no harm.”
    “If anyone adjusts a stable
    process, the output that follows
    will be worse than if (s)he had
    left the process alone.”
    -W. Edwards Deming
    @everydaykanban | @55DegreesAB 42 #agilecam
    

    View Slide

43. @everydaykanban | @55DegreesAB 43
    Find Leading Indicators
    #agilecam
    

    View Slide

44. @everydaykanban | @55DegreesAB 44
    When we do our job well, we help teams:
    #agilecam
    responsibly interpret data
    to identify areas of improvement
    avoid over-emphasizing any one measure
    and create a healthy whole
    1 2
    

    View Slide

45. @everydaykanban | @55DegreesAB 45 #agilecam
    What’s your next step?
    

    View Slide

46. @everydaykanban | @55DegreesAB 46 #agilecam
    julia@55degrees.se
    @everydaykanban & @55DegreesAB
    https://55degrees.se
    http://everydaykanban.com
    https://www.linkedin.com/in/juliawester/
    I’d love to hear from you
    

    View Slide