Avoiding Metrics Mistakes

Avoiding Metrics Mistakes

Metrics. We gather them. We make decisions based on them. We are judged by them. We don’t question them as often as we should. Despite our best intentions, we don’t always know how to decide what to measure. Even when we get it right we can end up doing more damage than good because of how we use them.

This deck was presented to the LeanAgile London Meetup group by Julia Wester, product manager for ActionableAgile, to help people learn about some common metrics mistakes, how to know if they're making them, and, most importantly, how to avoid them in the future.

1100c19690e8eb006da7a35a6b17be97?s=128

Julia Wester

August 13, 2020
Tweet

Transcript

  1. SIX COMMON METRICS MISTAKES AND HOW TO AVOID THEM!!

  2. JULIA WESTER Co-Founder, 55 Degrees AB Scrum.org PST ProKanban.org Founding

    Member Agendashift Partner @everydaykanban Hello! I am
  3. ACTIONABLE AGILE POWERED BY 55 DEGREES AB TM http://actionableagile.com I’m

    a product manager too!
  4. NOT HAVING A “WHY” 1

  5. None
  6. None
  7. Does it really give expected insight? think of small, day-

    to-day decisions. Courtesy of Larry Maccherone (@LMaccherone) https://medium.com/@lmaccherone/odim-12d80823222 Reduced time / cost for rework Should we increase testing time / effort? Are we finding defects early enough? Escaped defects ODIM
  8. defects found in production ESCAPED DEFECTS

  9. Count DEFECTS PER ENVIRONMENT

  10. Percentiles DEFECTS PER ENVIRONMENT

  11. Does it really give expected insight? think of small, day-

    to-day decisions. Courtesy of Larry Maccherone (@LMaccherone) https://medium.com/@lmaccherone/odim-12d80823222 Reduced time / cost for rework Should we increase testing time / effort? Are we finding defects early enough? % of defects found per environment ODIM
  12. REFUSING TO ACKNOWLEDGE THE GAME 2

  13. TRUTH: What you measure shows what you value

  14. USED WITH PERMISSION

  15. ELI GOLDRATT “Tell me how you’ll measure me and I’ll

    tell you how I’ll behave. If you measure me in an illogical way, do not complain about illogical behavior.”
  16. None
  17. How can you game it? How can we make this

    safer? What is the impact? A GAME TO PLAY WITH YOUR TEAM
  18. SHAMING PEOPLE WITH METRICS 3

  19. “Fear invites wrong figures. Bearers of bad news fare badly.

    To keep his job, anyone may present to his boss only good news.” W. EDWARDS DEMING “NEW ECONOMICS”
  20. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) WHAT IF YOU WERE ON

    THE LIST? “What a slacker!” “Wow, I bet they’re working on other priorities right now.” “Makes sense. That person’s code was part of a deep exploratory testing session.” Group A Team Member 1 Team Member 3 Team Member 5 Team Member 7 Team Member 9 Team Member 11 Group B Team Member 2 Team Member 4 Team Member 6 Team Member 8 Team Member 10 Team Member 12
  21. BY REMOVING THE CHOICE BETWEEN COMPANY AND SELF MINIMIZE THE

    DANGER • Focus on desired outcomes. • Measure individuals only when it makes sense. • Understand the audience for your metrics. • Have your team help you identify metrics that put them in a bad spot.
  22. Fooling yourself with vanity metrics @everydaykanban I sure do get

    a lot done! I add so much value! CONFUSING ACTIVITY WITH SUCCESS 4
  23. KNOW WHAT EACH METRIC REALLY TELLS YOU • lines of

    code written • # of issues closed • # of deploys per day • # of points finished Fooling yourself with vanity metrics ydaykanban I sure do get a lot done! I add so much value! • revenue generated • revenue protected • cost avoided • cost saved
  24. Does it: 1. matter to my customer? 2. help me

    make a decision or take action? ASK “SO WHAT”?
  25. BLINDLY PURSUING TARGETS 5

  26. W. EDWARDS DEMING “If you give a manager a numerical

    target, he’ll make it, even if he has to destroy the organization in the process.”
  27. Accelerate: State of DevOps 2019 https://services.google.com/fh/files/ misc/state-of-devops-2019.pdf Let’s get down

    to 10 minutes max!
  28. WATCH FOR DECISION POINTS

  29. WITH A FEW KEY TIPS MAKE TARGETS SAFER • Set

    targets in terms of outcomes rather than tactics. • Treat tactics as experiments. It’s ok to change direction. • Look at the broader cost of your target, and what you could put at risk. • Know when to call it “good enough”
  30. FIX IT HERE, BREAK IT THERE 6

  31. None
  32. COURTESY OF LARRY MACCHERONE (@LMACCHERONE)

  33. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES A Helpful Template

    Time in Process 1 Sprint Escaped Defects QUADRANT QUADRANT RESPONSIVENESS RESPONSIVENESS (QUALITY)
  34. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES BEHAVIORAL POLARITY WORKSHEET

    Troy Magennis Focused Objective
  35. Spectrum Thinking Worksheet For a more detailed worksheet, visit https://github.com/55-Degrees/worksheets

    PDSA
  36. SUMMARY

  37. Free Resources • Blog Posts, Webinars, etc. Short Training Sessions

    • Introduction to ActionableAgile Video Sessions In-Depth Workshops • Professional Scrum with Kanban (PSK) • Advanced Agile Metrics, Forecasting and Predictability LEARN https://55degrees.se/events
  38. ENGAGE Subscribe to our newsletter https://55degrees.se/subscribe Follow on Twitter @55degreesAB

    Engage with us on LinkedIn https://www.linkedin.com/company/55degrees
  39. ACTIONABLE AGILE POWERED BY 55 DEGREES AB TM http://actionableagile.com 2

    months free on new monthly subscriptions 20% off of a new yearly subscription Using promo code lal-2months Using promo code lal-20percent OR
  40. LET’S INTERACT ON MURAL CHECK THE ZOOM CHAT FOR THE

    LINK TO MY MURAL PAGE