$30 off During Our Annual Pro Sale. View Details »

Data Driven Coaching - Agile 2018

Data Driven Coaching - Agile 2018

This session will show how to visualize and use metrics in a team coaching practice. By attending this session you will learn how teams can see and balance a set of competing metrics, and anticipate the impact of process changes.

Julia Wester
PRO

August 09, 2018
Tweet

More Decks by Julia Wester

Other Decks in Business

Transcript

  1. @everydaykanban | @bsktcase #agile2018
    Julia Wester & Cheryl Hammond
    Data Driven Coaching

    View Slide

  2. @everydaykanban | @bsktcase #agile2018
    Who are we?
    Julia Wester
    Co-Founder & Principal Consultant
    Lagom Solutions
    https://lagom.solutions
    @everydaykanban
    Cheryl Hammond
    Hire Me!
    Seattle, Washington, USA
    https://linkedin.com/in/bsktcase
    @bsktcase

    View Slide

  3. @everydaykanban | @bsktcase #agile2018

    View Slide

  4. @everydaykanban | @bsktcase #agile2018

    View Slide

  5. @everydaykanban | @bsktcase #agile2018
    Planning
    Stand-Up / Scrum
    Retrospective
    Time in Process
    Throughput
    WiP Levels
    WiP Age
    Escaped Defects
    Time in Process
    Throughput
    Net Flow
    Flow Efficiency
    Defects per Environment
    and more...

    View Slide

  6. @everydaykanban | @bsktcase #agile2018
    Meaning is in the eye of the beholder
    Courtesy of Troy Magennis (@t_magennis)
    Team member 1
    Team member 2
    Team member 3
    Team member 4
    Team member 5
    Team member 6
    Team member 7
    Team member 8
    Team member 9
    Team member 10
    Team member 11
    Unassigned
    Alias 1
    Alias 2
    “What a slacker!”
    “Wow, I bet they’re working
    on other priorities right now.”
    “Makes sense. That person’s
    code was part of a deep
    exploratory testing session.”

    View Slide

  7. @everydaykanban | @bsktcase #agile2018
    The two main goals of data driven coaching:
    help teams identify
    areas of improvement
    1 avoid over-emphasizing a
    single measure of success
    2

    View Slide

  8. @everydaykanban | @bsktcase #agile2018
    “Tell me how you’ll measure me and
    I’ll tell you how I’ll behave.
    If you measure me in illogical ways,
    do not complain about illogical
    behavior.”
    –Eli Goldratt

    View Slide

  9. @everydaykanban | @bsktcase #agile2018
    Best bets for measuring logically

    View Slide

  10. @everydaykanban | @bsktcase #agile2018
    Highlight trends,
    not data points
    to avoid distraction from
    insignificant variation

    View Slide

  11. @everydaykanban | @bsktcase #agile2018
    Trends show us:

    View Slide

  12. @everydaykanban | @bsktcase #agile2018
    How to put a focus on trends
    No Y-Axis
    Tiny #’s
    Trend Line
    Project
    Trend
    When applicable, connect data
    point with lines

    View Slide

  13. @everydaykanban | @bsktcase #agile2018
    Compare trends across teams
    Courtesy of Troy Magennis (@t_magennis)

    View Slide

  14. @everydaykanban | @bsktcase #agile2018
    Measure teams,
    not individuals
    if you want people
    to act like a team

    View Slide

  15. @everydaykanban | @bsktcase #agile2018
    What you measure tells the story of what you value...

    View Slide

  16. @everydaykanban | @bsktcase #agile2018
    “Fear invites wrong figures.
    Bearers of bad news fare badly.
    To keep his job, anyone may
    present to his boss only good
    news.”
    –W. Edwards Deming
    Quote from “New Economics” p. 94, Deming

    View Slide

  17. @everydaykanban | @bsktcase #agile2018
    Unexpected
    responses
    you are measuring safety
    whether you intend to or not

    View Slide

  18. @everydaykanban | @bsktcase #agile2018
    Use multiple
    competing metrics
    to create and maintain
    a stable system

    View Slide

  19. @everydaykanban | @bsktcase #agile2018
    The danger of over-optimizing for one thing...
    Measuring this,
    but not this.

    View Slide

  20. @everydaykanban | @bsktcase #agile2018
    Courtesy of Larry Maccherone (@LMaccherone)

    View Slide

  21. @everydaykanban | @bsktcase #agile2018
    QUALITY RESPONSIVENESS
    PRODUCTIVITY PREDICTABILITY
    How well? How fast?
    How much? Keeping pace? How repeatable?
    Competes with
    Competes with

    View Slide

  22. @everydaykanban | @bsktcase #agile2018
    Are certain quadrants more
    important for your team?

    View Slide

  23. @everydaykanban | @bsktcase #agile2018
    Define “health” for
    your context

    View Slide

  24. @everydaykanban | @bsktcase #agile2018
    ODIM
    https://medium.com/@lmaccherone/odim-12d80823222
    Courtesy of Larry Maccherone (@LMaccherone)

    View Slide

  25. @everydaykanban | @bsktcase #agile2018
    Decide on outcomes THEN
    Determine your metrics
    Reduced time /
    cost for rework
    Should we
    increase time
    spent on testing?
    Are we finding
    defects early
    enough in process?
    % defects found
    per environment
    https://medium.com/@lmaccherone/odim-12d80823222
    think of small,
    day-to-day decisions.
    Question if the measure
    really provides expected
    insight!
    Courtesy of Larry Maccherone (@LMaccherone)

    View Slide

  26. @everydaykanban | @bsktcase #agile2018
    ESCAPED DEFECTS
    Jan
    defects found in production
    Feb Mar Apr May Jun Jul Aug Sep
    4
    5
    8 8
    10
    6
    12
    8
    11
    50%

    View Slide

  27. @everydaykanban | @bsktcase #agile2018
    Defects By
    Environment
    # OF DEFECTS
    KEY
    counts
    Jan Feb Mar Apr May Jun Jul Aug Sep
    STAGE
    PROD
    DEV
    INT

    View Slide

  28. @everydaykanban | @bsktcase #agile2018
    % OF DEFECTS
    Jan Feb Mar Apr May Jun Jul Aug Sep
    Defects By
    Environment
    percents
    KEY
    STAGE
    PROD
    DEV
    INT

    View Slide

  29. @everydaykanban | @bsktcase #agile2018
    QUALITY RESPONSIVENESS
    PRODUCTIVITY PREDICTABILITY
    How well? How fast?
    How much? Keeping pace? How repeatable?
    pick best representative
    metric for each quadrant

    View Slide

  30. @everydaykanban | @bsktcase #agile2018
    A helpful template to guide you...
    QUADRANT
    QUADRANT
    Courtesy of Troy Magennis (@t_magennis)
    Metric for Opposing Quadrant
    Metric for Current Quadrant

    View Slide

  31. @everydaykanban | @bsktcase #agile2018
    A helpful template to guide you...
    RESPONSIVENESS
    RESPONSIVENESS
    Courtesy of Troy Magennis (@t_magennis)
    Time in
    Process Sprint
    ???
    (QUALITY)

    View Slide

  32. @everydaykanban | @bsktcase #agile2018
    Behavioral
    Polarity
    Worksheet
    Courtesy of Troy Magennis (@t_magennis)
    Troy Magennis
    Focused
    Objective

    View Slide

  33. @everydaykanban | @bsktcase #agile2018
    Lagom Discovery Canvas Get canvas and key at:
    https://lagom.solutions/resources

    View Slide

  34. @everydaykanban | @bsktcase #agile2018
    A helpful template to guide you...
    RESPONSIVENESS
    RESPONSIVENESS
    Courtesy of Troy Magennis (@t_magennis)
    Time in
    Process Sprint
    Escaped Defects
    (QUALITY)

    View Slide

  35. @everydaykanban | @bsktcase #agile2018
    Give a cheat sheet!
    ● What is the chart
    ● Intended behavior
    ● How to game it
    ● When overdriven,
    what moves?
    http://bit.ly/SimResources
    Courtesy of Troy Magennis (@t_magennis)

    View Slide

  36. @everydaykanban | @bsktcase #agile2018
    Sometimes it’s ok to sacrifice a
    bit in one area to bring the
    whole into harmony

    View Slide

  37. @everydaykanban | @bsktcase #agile2018
    Compare only
    in context
    to ensure you highlight
    meaningful differences

    View Slide

  38. @everydaykanban | @bsktcase #agile2018
    Bad Comparisons
    lead to
    Bad Insights
    which lead to
    Bad Decisions

    View Slide

  39. @everydaykanban | @bsktcase #agile2018
    Show me
    when to worry
    don’t make me guess or
    waste my time off

    View Slide

  40. @everydaykanban | @bsktcase #agile2018
    “First, do no harm.”
    “If anyone adjusts a stable process,
    the output that follows will be worse
    than if (s)he had left the process
    alone.”
    –W. Edward Deming

    View Slide

  41. @everydaykanban | @bsktcase #agile2018
    Not all variation is cause for alarm
    OLDEST
    NEWEST
    Wk 1 Wk 2 Wk 3 Wk 4 Wk 5 Wk 6 Wk 7 Wk 8
    Courtesy of Cat Swetel (@catswetel)
    WiP
    Aging

    View Slide

  42. @everydaykanban | @bsktcase #agile2018
    Use markers
    to show
    what’s
    concerning
    Courtesy of Cat Swetel (@catswetel)

    View Slide

  43. @everydaykanban | @bsktcase #agile2018
    Find Leading Indicators
    WiP Report % Defects Per Env

    View Slide

  44. @everydaykanban | @bsktcase #agile2018
    Recap: The two main goals of data driven coaching:
    help teams identify
    areas of improvement
    1 avoid over-emphasizing a
    single measure of success
    2

    View Slide

  45. @everydaykanban | @bsktcase #agile2018
    Recap: Key Tips to Remember

    View Slide

  46. @everydaykanban | @bsktcase #agile2018
    What
    steps will
    you take
    next?

    View Slide

  47. @everydaykanban | @bsktcase #agile2018
    Resources to keep learning
    Balanced Metrics Dashboard
    http://focusedobjective.com/team-metrics-right/
    by Troy Magennis, Focused Objective
    Lagom Discovery Canvas
    https://bit.ly/2vTaCXF
    by Julia Wester, Lagom Solutions
    One Metric to Rule Them All
    https://bit.ly/2M3gVCE
    by Cheryl Hammond, Secret Garden

    View Slide

  48. @everydaykanban | @bsktcase #agile2018

    View Slide

  49. @everydaykanban | @bsktcase #agile2018
    How You Can Contact Us
    Julia Wester
    Co-Founder & Principal Consultant
    Lagom Solutions
    https://lagom.solutions
    @everydaykanban
    Cheryl Hammond
    Hire Me!
    Seattle, Washington, USA
    https://linkedin.com/in/bsktcase
    @bsktcase

    View Slide

  50. @everydaykanban | @bsktcase #agile2018
    Appendix (previous slides)

    View Slide

  51. @everydaykanban | @bsktcase #agile2018
    Pro Tip:
    Pies are best
    for eating,
    less so for
    charting...
    http://www.thefunctionalart.com/2012/06/fun-note-on-pie-charts.html

    View Slide

  52. @everydaykanban | @bsktcase #agile2018
    Counterpoint!
    Why not both?

    View Slide

  53. @everydaykanban | @bsktcase #agile2018
    Don’t
    Just don’t

    View Slide

  54. @everydaykanban | @bsktcase #agile2018
    Correlation isn’t causation
    Does this fit in this section?

    View Slide

  55. @everydaykanban | @bsktcase #agile2018
    Avoid Confusing Activity with Progress
    Activity metrics
    ● lines of code
    ● # of issues closed
    ● # of deploys per day
    These aren’t “bad” metrics. We should:
    1. Understand what they really show
    2. Show them with competing metrics
    to reduce gaming
    J

    View Slide

  56. @everydaykanban | @bsktcase #agile2018
    Know The
    Expiry
    Dates
    For Your
    Metrics
    J

    View Slide

  57. @everydaykanban | @bsktcase #agile2018
    Goodhart’s Law
    “When a measure becomes a target, it ceases to be a good measure.”

    View Slide

  58. @everydaykanban | @bsktcase #agile2018
    Exercise: Map outcomes to metrics with ODIM
    1. List one outcome you are trying
    to achieve.
    2. Determine a decision you need to
    make to achieve the outcome.
    3. List an insight that can help you
    make the decision.
    4. List one or more metrics that can
    provide the needed insight.
    5. Repeat until time runs out.
    10 minutes

    View Slide

  59. @everydaykanban | @bsktcase #agile2018

    View Slide