Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Avoiding Metrics Mistakes

Avoiding Metrics Mistakes

Metrics. We gather them. We make decisions based on them. We are judged by them. We don’t question them as often as we should. Despite our best intentions, we don’t always know how to decide what to measure. Even when we get it right we can end up doing more damage than good because of how we use them.

This deck was presented to the LeanAgile London Meetup group by Julia Wester, product manager for ActionableAgile, to help people learn about some common metrics mistakes, how to know if they're making them, and, most importantly, how to avoid them in the future.

Julia Wester
PRO

August 13, 2020
Tweet

More Decks by Julia Wester

Other Decks in Business

Transcript

  1. SIX
    COMMON
    METRICS
    MISTAKES
    AND HOW
    TO AVOID
    THEM!!

    View Slide

  2. JULIA
    WESTER
    Co-Founder, 55 Degrees AB
    Scrum.org PST
    ProKanban.org Founding Member
    Agendashift Partner
    @everydaykanban
    Hello! I am

    View Slide

  3. ACTIONABLE AGILE POWERED BY 55 DEGREES AB
    TM
    http://actionableagile.com
    I’m a product
    manager too!

    View Slide

  4. NOT HAVING
    A “WHY”
    1

    View Slide

  5. View Slide

  6. View Slide

  7. Does it really
    give expected
    insight?
    think of small, day-
    to-day decisions.
    Courtesy of Larry Maccherone (@LMaccherone)
    https://medium.com/@lmaccherone/odim-12d80823222
    Reduced time /
    cost for rework
    Should we increase
    testing time / effort?
    Are we finding
    defects early
    enough?
    Escaped defects
    ODIM

    View Slide

  8. defects found in production
    ESCAPED DEFECTS

    View Slide

  9. Count
    DEFECTS PER ENVIRONMENT

    View Slide

  10. Percentiles
    DEFECTS PER ENVIRONMENT

    View Slide

  11. Does it really
    give expected
    insight?
    think of small, day-
    to-day decisions.
    Courtesy of Larry Maccherone (@LMaccherone)
    https://medium.com/@lmaccherone/odim-12d80823222
    Reduced time /
    cost for rework
    Should we increase
    testing time / effort?
    Are we finding
    defects early
    enough?
    % of defects found per
    environment
    ODIM

    View Slide

  12. REFUSING TO
    ACKNOWLEDGE
    THE GAME
    2

    View Slide

  13. TRUTH:
    What you measure
    shows what you value

    View Slide

  14. USED WITH PERMISSION

    View Slide

  15. ELI GOLDRATT
    “Tell me how you’ll measure me
    and I’ll tell you how I’ll behave.
    If you measure me in an
    illogical way, do not complain
    about illogical behavior.”

    View Slide

  16. View Slide

  17. How can you
    game it?
    How can we
    make this safer?
    What is the
    impact?
    A GAME TO PLAY
    WITH YOUR TEAM

    View Slide

  18. SHAMING PEOPLE
    WITH METRICS
    3

    View Slide

  19. “Fear invites wrong
    figures. Bearers of bad
    news fare badly. To
    keep his job, anyone
    may present to his boss
    only good news.”
    W. EDWARDS DEMING
    “NEW ECONOMICS”

    View Slide

  20. COURTESY OF TROY MAGENNIS (@T_MAGENNIS)
    WHAT IF YOU WERE ON THE LIST?
    “What a slacker!”
    “Wow, I bet they’re working on
    other priorities right now.”
    “Makes sense. That person’s
    code was part of a deep
    exploratory testing session.”
    Group A
    Team Member 1
    Team Member 3
    Team Member 5
    Team Member 7
    Team Member 9
    Team Member 11
    Group B
    Team Member 2
    Team Member 4
    Team Member 6
    Team Member 8
    Team Member 10
    Team Member 12

    View Slide

  21. BY REMOVING THE CHOICE BETWEEN COMPANY AND SELF
    MINIMIZE THE DANGER
    • Focus on desired outcomes.
    • Measure individuals only when it makes sense.
    • Understand the audience for your metrics.
    • Have your team help you identify metrics that put
    them in a bad spot.

    View Slide

  22. Fooling
    yourself with
    vanity
    metrics
    @everydaykanban
    I sure do
    get a lot
    done!
    I add so
    much
    value!
    CONFUSING
    ACTIVITY WITH
    SUCCESS
    4

    View Slide

  23. KNOW WHAT EACH METRIC REALLY TELLS YOU
    • lines of code written
    • # of issues closed
    • # of deploys per day
    • # of points finished
    Fooling
    yourself with
    vanity
    metrics
    ydaykanban
    I sure do
    get a lot
    done!
    I add so
    much
    value!
    • revenue generated
    • revenue protected
    • cost avoided
    • cost saved

    View Slide

  24. Does it:
    1. matter to my customer?
    2. help me make a decision or take action?
    ASK “SO WHAT”?

    View Slide

  25. BLINDLY PURSUING
    TARGETS
    5

    View Slide

  26. W. EDWARDS DEMING
    “If you give a manager a
    numerical target, he’ll make it,
    even if he has to destroy the
    organization in the process.”

    View Slide

  27. Accelerate:
    State of
    DevOps 2019
    https://services.google.com/fh/files/
    misc/state-of-devops-2019.pdf
    Let’s get down to
    10 minutes max!

    View Slide



  28. WATCH FOR DECISION POINTS

    View Slide

  29. WITH A FEW KEY TIPS
    MAKE TARGETS SAFER
    • Set targets in terms of outcomes rather than
    tactics.
    • Treat tactics as experiments. It’s ok to
    change direction.
    • Look at the broader cost of your target, and
    what you could put at risk.
    • Know when to call it “good enough”

    View Slide

  30. FIX IT
    HERE,
    BREAK
    IT THERE
    6

    View Slide

  31. View Slide

  32. COURTESY OF LARRY MACCHERONE (@LMACCHERONE)

    View Slide

  33. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES
    A Helpful Template
    Time in Process 1 Sprint
    Escaped Defects
    QUADRANT
    QUADRANT
    RESPONSIVENESS
    RESPONSIVENESS
    (QUALITY)

    View Slide

  34. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES
    BEHAVIORAL
    POLARITY
    WORKSHEET
    Troy Magennis
    Focused Objective

    View Slide

  35. Spectrum Thinking Worksheet
    For a more detailed worksheet, visit
    https://github.com/55-Degrees/worksheets
    PDSA

    View Slide

  36. SUMMARY

    View Slide

  37. Free Resources
    • Blog Posts, Webinars, etc.
    Short Training Sessions
    • Introduction to ActionableAgile Video Sessions
    In-Depth Workshops
    • Professional Scrum with Kanban (PSK)
    • Advanced Agile Metrics, Forecasting and Predictability
    LEARN
    https://55degrees.se/events

    View Slide

  38. ENGAGE
    Subscribe to our newsletter
    https://55degrees.se/subscribe
    Follow on Twitter
    @55degreesAB
    Engage with us on LinkedIn
    https://www.linkedin.com/company/55degrees

    View Slide

  39. ACTIONABLE AGILE POWERED BY 55 DEGREES AB
    TM
    http://actionableagile.com
    2 months free on new monthly subscriptions
    20% off of a new yearly subscription
    Using promo code lal-2months
    Using promo code lal-20percent
    OR

    View Slide

  40. LET’S INTERACT ON MURAL
    CHECK THE ZOOM CHAT FOR THE LINK TO MY MURAL PAGE

    View Slide