Avoiding Metrics Mistakes

Metrics. We gather them. We make decisions based on them. We are judged by them. We don’t question them as often as we should. Despite our best intentions, we don’t always know how to decide what to measure. Even when we get it right we can end up doing more damage than good because of how we use them.

This deck was presented to the LeanAgile London Meetup group by Julia Wester, product manager for ActionableAgile, to help people learn about some common metrics mistakes, how to know if they're making them, and, most importantly, how to avoid them in the future.

Transcript

1. SIX COMMON METRICS MISTAKES AND HOW TO AVOID THEM!!
2. JULIA WESTER Co-Founder, 55 Degrees AB Scrum.org PST ProKanban.org Founding Member Agendashift Partner @everydaykanban Hello! I am
3. ACTIONABLE AGILE POWERED BY 55 DEGREES AB TM http://actionableagile.com I’m a product manager too!
4. NOT HAVING A “WHY” 1
7. Does it really give expected insight? think of small, day- to-day decisions. Courtesy of Larry Maccherone (@LMaccherone) https://medium.com/@lmaccherone/odim-12d80823222 Reduced time / cost for rework Should we increase testing time / effort? Are we finding defects early enough? Escaped defects ODIM
8. defects found in production ESCAPED DEFECTS
9. Count DEFECTS PER ENVIRONMENT
10. Percentiles DEFECTS PER ENVIRONMENT
11. Does it really give expected insight? think of small, day- to-day decisions. Courtesy of Larry Maccherone (@LMaccherone) https://medium.com/@lmaccherone/odim-12d80823222 Reduced time / cost for rework Should we increase testing time / effort? Are we finding defects early enough? % of defects found per environment ODIM
12. REFUSING TO ACKNOWLEDGE THE GAME 2
13. TRUTH: What you measure shows what you value
14. USED WITH PERMISSION
15. ELI GOLDRATT “Tell me how you’ll measure me and I’ll tell you how I’ll behave. If you measure me in an illogical way, do not complain about illogical behavior.”
17. How can you game it? How can we make this safer? What is the impact? A GAME TO PLAY WITH YOUR TEAM
18. SHAMING PEOPLE WITH METRICS 3
19. “Fear invites wrong figures. Bearers of bad news fare badly. To keep his job, anyone may present to his boss only good news.” W. EDWARDS DEMING “NEW ECONOMICS”
20. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) WHAT IF YOU WERE ON THE LIST? “What a slacker!” “Wow, I bet they’re working on other priorities right now.” “Makes sense. That person’s code was part of a deep exploratory testing session.” Group A Team Member 1 Team Member 3 Team Member 5 Team Member 7 Team Member 9 Team Member 11 Group B Team Member 2 Team Member 4 Team Member 6 Team Member 8 Team Member 10 Team Member 12
21. BY REMOVING THE CHOICE BETWEEN COMPANY AND SELF MINIMIZE THE DANGER • Focus on desired outcomes. • Measure individuals only when it makes sense. • Understand the audience for your metrics. • Have your team help you identify metrics that put them in a bad spot.
22. Fooling yourself with vanity metrics @everydaykanban I sure do get a lot done! I add so much value! CONFUSING ACTIVITY WITH SUCCESS 4
23. KNOW WHAT EACH METRIC REALLY TELLS YOU • lines of code written • # of issues closed • # of deploys per day • # of points finished Fooling yourself with vanity metrics ydaykanban I sure do get a lot done! I add so much value! • revenue generated • revenue protected • cost avoided • cost saved
24. Does it: 1. matter to my customer? 2. help me make a decision or take action? ASK “SO WHAT”?
25. BLINDLY PURSUING TARGETS 5
26. W. EDWARDS DEMING “If you give a manager a numerical target, he’ll make it, even if he has to destroy the organization in the process.”
27. Accelerate: State of DevOps 2019 https://services.google.com/fh/files/ misc/state-of-devops-2019.pdf Let’s get down to 10 minutes max!
28. WATCH FOR DECISION POINTS
29. WITH A FEW KEY TIPS MAKE TARGETS SAFER • Set targets in terms of outcomes rather than tactics. • Treat tactics as experiments. It’s ok to change direction. • Look at the broader cost of your target, and what you could put at risk. • Know when to call it “good enough”
30. FIX IT HERE, BREAK IT THERE 6
32. COURTESY OF LARRY MACCHERONE (@LMACCHERONE)
33. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES A Helpful Template Time in Process 1 Sprint Escaped Defects QUADRANT QUADRANT RESPONSIVENESS RESPONSIVENESS (QUALITY)
34. COURTESY OF TROY MAGENNIS (@T_MAGENNIS) - HTTPS://BIT.LY/SIMRESOURCES BEHAVIORAL POLARITY WORKSHEET Troy Magennis Focused Objective
35. Spectrum Thinking Worksheet For a more detailed worksheet, visit https://github.com/55-Degrees/worksheets PDSA
36. SUMMARY
37. Free Resources • Blog Posts, Webinars, etc. Short Training Sessions • Introduction to ActionableAgile Video Sessions In-Depth Workshops • Professional Scrum with Kanban (PSK) • Advanced Agile Metrics, Forecasting and Predictability LEARN https://55degrees.se/events
38. ENGAGE Subscribe to our newsletter https://55degrees.se/subscribe Follow on Twitter @55degreesAB Engage with us on LinkedIn https://www.linkedin.com/company/55degrees
39. ACTIONABLE AGILE POWERED BY 55 DEGREES AB TM http://actionableagile.com 2 months free on new monthly subscriptions 20% off of a new yearly subscription Using promo code lal-2months Using promo code lal-20percent OR
40. LET’S INTERACT ON MURAL CHECK THE ZOOM CHAT FOR THE LINK TO MY MURAL PAGE