Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Avoiding Metrics Mistakes

Avoiding Metrics Mistakes

Metrics. We gather them. We make decisions based on them. We are judged by them. We don’t question them as often as we should. Despite our best intentions, we don’t always know how to decide what to measure. Even when we get it right we can end up doing more damage than good because of how we use them.

This deck was presented to the LeanAgile London Meetup group by Julia Wester, product manager for ActionableAgile, to help people learn about some common metrics mistakes, how to know if they're making them, and, most importantly, how to avoid them in the future.

Julia Wester

August 13, 2020
Tweet

More Decks by Julia Wester

Other Decks in Business

Transcript

  1. SIX
    COMMON
    METRICS
    MISTAKES
    AND HOW
    TO AVOID
    THEM!!

    View full-size slide

  2. JULIA
    WESTER
    Co-Founder, 55 Degrees AB
    Scrum.org PST
    ProKanban.org Founding Member
    Agendashift Partner
    @everydaykanban
    Hello! I am

    View full-size slide

  3. ACTIONABLE AGILE POWERED BY 55 DEGREES AB
    TM
    http://actionableagile.com
    I’m a product
    manager too!

    View full-size slide

  4. NOT HAVING
    A “WHY”
    1

    View full-size slide

  5. Does it really
    give expected
    insight?
    think of small, day-
    to-day decisions.
    Courtesy of Larry Maccherone (@LMaccherone)
    https://medium.com/@lmaccherone/odim-12d80823222
    Reduced time /
    cost for rework
    Should we increase
    testing time / effort?
    Are we finding
    defects early
    enough?
    Escaped defects
    ODIM

    View full-size slide

  6. defects found in production
    ESCAPED DEFECTS

    View full-size slide

  7. Count
    DEFECTS PER ENVIRONMENT

    View full-size slide

  8. Percentiles
    DEFECTS PER ENVIRONMENT

    View full-size slide

  9. Does it really
    give expected
    insight?
    think of small, day-
    to-day decisions.
    Courtesy of Larry Maccherone (@LMaccherone)
    https://medium.com/@lmaccherone/odim-12d80823222
    Reduced time /
    cost for rework
    Should we increase
    testing time / effort?
    Are we finding
    defects early
    enough?
    % of defects found per
    environment
    ODIM

    View full-size slide

  10. REFUSING TO
    ACKNOWLEDGE
    THE GAME
    2

    View full-size slide

  11. TRUTH:
    What you measure
    shows what you value

    View full-size slide

  12. USED WITH PERMISSION

    View full-size slide

  13. ELI GOLDRATT
    “Tell me how you’ll measure me
    and I’ll tell you how I’ll behave.
    If you measure me in an
    illogical way, do not complain
    about illogical behavior.”

    View full-size slide

  14. How can you
    game it?
    How can we
    make this safer?
    What is the
    impact?
    A GAME TO PLAY
    WITH YOUR TEAM

    View full-size slide

  15. SHAMING PEOPLE
    WITH METRICS
    3

    View full-size slide

  16. “Fear invites wrong
    figures. Bearers of bad
    news fare badly. To
    keep his job, anyone
    may present to his boss
    only good news.”
    W. EDWARDS DEMING
    “NEW ECONOMICS”

    View full-size slide

  17. COURTESY OF TROY MAGENNIS (@T_MAGENNIS)
    WHAT IF YOU WERE ON THE LIST?
    “What a slacker!”
    “Wow, I bet they’re working on
    other priorities right now.”
    “Makes sense. That person’s
    code was part of a deep
    exploratory testing session.”
    Group A
    Team Member 1
    Team Member 3
    Team Member 5
    Team Member 7
    Team Member 9
    Team Member 11
    Group B
    Team Member 2
    Team Member 4
    Team Member 6
    Team Member 8
    Team Member 10
    Team Member 12

    View full-size slide

  18. BY REMOVING THE CHOICE BETWEEN COMPANY AND SELF
    MINIMIZE THE DANGER
    • Focus on desired outcomes.
    • Measure individuals only when it makes sense.
    • Understand the audience for your metrics.
    • Have your team help you identify metrics that put
    them in a bad spot.

    View full-size slide

  19. Fooling
    yourself with
    vanity
    metrics
    @everydaykanban
    I sure do
    get a lot
    done!
    I add so
    much
    value!
    CONFUSING
    ACTIVITY WITH
    SUCCESS
    4

    View full-size slide

  20. KNOW WHAT EACH METRIC REALLY TELLS YOU
    • lines of code written
    • # of issues closed
    • # of deploys per day
    • # of points finished
    Fooling
    yourself with
    vanity
    metrics
    ydaykanban
    I sure do
    get a lot
    done!
    I add so
    much
    value!
    • revenue generated
    • revenue protected
    • cost avoided
    • cost saved

    View full-size slide

  21. Does it:
    1. matter to my customer?
    2. help me make a decision or take action?
    ASK “SO WHAT”?

    View full-size slide

  22. BLINDLY PURSUING
    TARGETS
    5

    View full-size slide

  23. W. EDWARDS DEMING
    “If you give a manager a
    numerical target, he’ll make it,
    even if he has to destroy the
    organization in the process.”

    View full-size slide

  24. Accelerate:
    State of
    DevOps 2019
    https://services.google.com/fh/files/
    misc/state-of-devops-2019.pdf
    Let’s get down to
    10 minutes max!

    View full-size slide



  25. WATCH FOR DECISION POINTS

    View full-size slide

  26. WITH A FEW KEY TIPS
    MAKE TARGETS SAFER
    • Set targets in terms of outcomes rather than
    tactics.
    • Treat tactics as experiments. It’s ok to
    change direction.
    • Look at the broader cost of your target, and
    what you could put at risk.
    • Know when to call it “good enough”

    View full-size slide

  27. FIX IT
    HERE,
    BREAK
    IT THERE
    6

    View full-size slide

  28. COURTESY OF LARRY MACCHERONE (@LMACCHERONE)

    View full-size slide

  29. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES
    A Helpful Template
    Time in Process 1 Sprint
    Escaped Defects
    QUADRANT
    QUADRANT
    RESPONSIVENESS
    RESPONSIVENESS
    (QUALITY)

    View full-size slide

  30. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES
    BEHAVIORAL
    POLARITY
    WORKSHEET
    Troy Magennis
    Focused Objective

    View full-size slide

  31. Spectrum Thinking Worksheet
    For a more detailed worksheet, visit
    https://github.com/55-Degrees/worksheets
    PDSA

    View full-size slide

  32. Free Resources
    • Blog Posts, Webinars, etc.
    Short Training Sessions
    • Introduction to ActionableAgile Video Sessions
    In-Depth Workshops
    • Professional Scrum with Kanban (PSK)
    • Advanced Agile Metrics, Forecasting and Predictability
    LEARN
    https://55degrees.se/events

    View full-size slide

  33. ENGAGE
    Subscribe to our newsletter
    https://55degrees.se/subscribe
    Follow on Twitter
    @55degreesAB
    Engage with us on LinkedIn
    https://www.linkedin.com/company/55degrees

    View full-size slide

  34. ACTIONABLE AGILE POWERED BY 55 DEGREES AB
    TM
    http://actionableagile.com
    2 months free on new monthly subscriptions
    20% off of a new yearly subscription
    Using promo code lal-2months
    Using promo code lal-20percent
    OR

    View full-size slide

  35. LET’S INTERACT ON MURAL
    CHECK THE ZOOM CHAT FOR THE LINK TO MY MURAL PAGE

    View full-size slide