Dev and Test Metrics 101 - Agile 2018

Cat Swetel, accompanied by Julia Wester, will outline the most common metrics used for understanding flow and impediments in software development and test functions. This session should provide a clear picture of the types of metrics that help understand Agile development, and how to get started in capturing and using these for improving Agile adoption, improvement, and management.

Transcript

1. Development and Test Metrics 101 @catswetel | @everydaykanban
2. @catswetel | @everydaykanban
3. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
4. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY Software Development Performance Index (SDPI) Larry Maccherone
5. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
6. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY What’s missing?
7. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
8. “The RIGHTER we do the WRONG thing, the WRONGER we become.” -- Dr Russell Ackoff @catswetel | @everydaykanban
9. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY QUALITY RESPONSIVENESS @catswetel | @everydaykanban
10. @catswetel | @everydaykanban
11. Time in Process Units of time per unit of work @catswetel | @everydaykanban
12. TIME IN PROCESS DATE DELIVERED TIME IN PROCESS SCATTER PLOT
13. TIME IN PROCESS DATE DELIVERED When can we expect X? 90% 50%
14. TIME IN PROCESS DATE DELIVERED How likely by X? 30d: 77% 21d: 50%
15. TIME IN PROCESS DATE DELIVERED What’s the story?
16. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
17. FREQUENCY TIME IN PROCESS TIME IN PROCESS DISTRIBUTION Groups are easier to see This one happens more often
18. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
19. @catswetel | @everydaykanban
20. WEIBULL DISTRIBUTION @catswetel | @everydaykanban
21. WEIBULL DISTRIBUTION ...maybe @catswetel | @everydaykanban
22. FREQUENCY TIME IN PROCESS NOT Normal
23. FREQUENCY TIME IN PROCESS NOT Normal @catswetel | @everydaykanban
24. TIME IN PROCESS DATE DELIVERED Notice anything else?
25. TIME IN PROCESS DATE DELIVERED Notice anything else?
26. TIME IN PROCESS DATE DELIVERED 90% AVERAGE
27. MULTI MODAL @catswetel | @everydaykanban
28. TIME IN PROCESS DATE DELIVERED
29. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
30. TIME IN PROCESS DATE DELIVERED What else can we learn?
31. TIME IN PROCESS DATE DELIVERED THROUGHPUT Units of work per unit of time
32. TIME IN PROCESS DATE DELIVERED Boxes show variability of Time in Process, NOT Throughput
33. TIME IN PROCESS DATE DELIVERED Does it always make sense to count all the dots?
34. TIME IN PROCESS DATE DELIVERED 9 9 11 6 4 6 4 10 13
35. TIME IN PROCESS DATE DELIVERED 9 9 11 6 4 6 4 10 13 1 2 4 1 0 3 1 1 1
36. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
37. THROUGHPUT
38. 85% 50%
39. “We are forced to consider knowledge as something that changes as new evidence is provided by more data, or as new predictions are made from the same data by new theories.” Statistical Method from the Viewpoint of Quality Control by Walter A Shewhart @catswetel | @everydaykanban
40. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
41. Lagging quality @catswetel | @everydaykanban
42. ESCAPED DEFECTS Jan defects found in production Feb Mar Apr May Jun Jul Aug Sep 4 5 8 8 10 6 12 8 11 @catswetel | @everydaykanban type
43. ESCAPED DEFECTS Jan defects found in production Feb Mar Apr May Jun Jul Aug Sep 4 5 8 8 10 6 12 8 11 @catswetel | @everydaykanban type How can we tell if we’re getting better at finding defects earlier?
44. % OF DEFECTS DEFECTS BY ENV percentage Key: Jan Feb Mar Apr May Jun Jul Aug Sep @catswetel | @everydaykanban STAGE PROD DEV INT env
45. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
46. @catswetel | @everydaykanban
48. Predicting the movements of the one bee is nearly impossible without the context of the swarm of bees. Complex Adaptive Systems: An Introduction to Computational Models of Social Life by John H. Miller, Scott E. Page
49. @catswetel | @everydaykanban
50. MORE FINISHED THAN STARTED MORE STARTED THAN FINISHED @catswetel | @everydaykanban
51. @catswetel | @everydaykanban
52. @catswetel | @everydaykanban
53. Total WIP @catswetel | @everydaykanban
54. OLDEST NEWEST @catswetel | @everydaykanban
55. Average WIP @catswetel | @everydaykanban
56. @catswetel | @everydaykanban
57. @catswetel | @everydaykanban
58. @catswetel | @everydaykanban
59. LITTLE’S LAW @catswetel | @everydaykanban
60. It’s ok to make necessary trade-offs
61. Where should we focus improvement efforts? @catswetel | @everydaykanban
62. @catswetel | @everydaykanban FLOW EFFICIENCY Work Wait Total Duration
63. ...per state (incl. waiting states) @catswetel | @everydaykanban
64. @catswetel | @everydaykanban You can do any chart for any portion of the process!
65. Are we there yet? @catswetel | @everydaykanban
66. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY
67. #NOESTIMATES @catswetel | @everydaykanban
68. “Close examination reveals that every meaningful interpretation involves a prediction.” Statistical Method from the Viewpoint of Quality Control Walter A Shewhart @catswetel | @everydaykanban
69. 3 Components of Knowledge: 1. Evidence 2. Prediction 3. Degree of Belief @catswetel | @everydaykanban
70. EVIDENCE Agile Quantified (Measuring the impact of Agility) Larry Maccherone Wed 3:45 PREDICTION Forecasting and Estimation Hands-On Ozzie Yuce Thu 10:45 DEGREE OF BELIEF Data Driven Coaching Julia Wester & Cheryl Hammond Thu 9:00 @catswetel | @everydaykanban
71. Julia Wester Co-Founder & Principal Consultant Lagom Solutions julia@lagom.solutions @everydaykanban Cat Swetel Engineering Manager Ticketmaster cat@catswetel.com @catswetel
72. Pablo Picasso “Computers are useless. They can only give you answers.”
74. “STATISTICS are useless. They can only give you answers.”
77. LITTLE’S LAW @catswetel at #qconnyc Are your decisions making this more or less true? FEEL FREE TO CUT
78. Some joke about defining quality
79. VARIABILITY FEEL FREE TO CUT
80. TIME IN PROCESS DATE DELIVERED VARIABILITY FEEL FREE TO CUT
81. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY VARIABILITY VARIABILITY VARIABILITY SHEWHART: the cause of v in an expected attribute. Re JUDGEMENT to know which “important quality characte (causes of undesirable varia within the bounds of reason expected economic results. FEEL FREE TO CUT
83. DONE REVIEW DOING TO DO @catswetel at #qconnyc
84. CUMULATIVE FLOW DIAGRAM @catswetel at #qconnyc
85. CUMULATIVE FLOW DIAGRAM AVERAGE ARRIVAL RATE @catswetel at #qconnyc
86. CUMULATIVE FLOW DIAGRAM AVERAGE DEPARTURE RATE @catswetel at #qconnyc
87. CUMULATIVE FLOW DIAGRAM AVERAGE WORK IN PROCESS @catswetel at #qconnyc
88. CUMULATIVE FLOW DIAGRAM AVERAGE TIME IN PROCESS @catswetel at #qconnyc