Dev and Test Metrics 101 - Agile 2018

Dev and Test Metrics 101 - Agile 2018

Cat Swetel, accompanied by Julia Wester, will outline the most common metrics used for understanding flow and impediments in software development and test functions. This session should provide a clear picture of the types of metrics that help understand Agile development, and how to get started in capturing and using these for improving Agile adoption, improvement, and management.

1100c19690e8eb006da7a35a6b17be97?s=128

Julia Wester

August 08, 2018
Tweet

Transcript

  1. Development and Test Metrics 101 @catswetel | @everydaykanban

  2. @catswetel | @everydaykanban

  3. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  4. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY Software Development Performance Index (SDPI) Larry

    Maccherone
  5. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  6. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY What’s missing?

  7. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  8. “The RIGHTER we do the WRONG thing, the WRONGER we

    become.” -- Dr Russell Ackoff @catswetel | @everydaykanban
  9. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY QUALITY RESPONSIVENESS @catswetel | @everydaykanban

  10. @catswetel | @everydaykanban

  11. Time in Process Units of time per unit of work

    @catswetel | @everydaykanban
  12. TIME IN PROCESS DATE DELIVERED TIME IN PROCESS SCATTER PLOT

  13. TIME IN PROCESS DATE DELIVERED When can we expect X?

    90% 50%
  14. TIME IN PROCESS DATE DELIVERED How likely by X? 30d:

    77% 21d: 50%
  15. TIME IN PROCESS DATE DELIVERED What’s the story?

  16. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  17. FREQUENCY TIME IN PROCESS TIME IN PROCESS DISTRIBUTION Groups are

    easier to see This one happens more often
  18. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  19. @catswetel | @everydaykanban

  20. WEIBULL DISTRIBUTION @catswetel | @everydaykanban

  21. WEIBULL DISTRIBUTION ...maybe @catswetel | @everydaykanban

  22. FREQUENCY TIME IN PROCESS NOT Normal

  23. FREQUENCY TIME IN PROCESS NOT Normal @catswetel | @everydaykanban

  24. TIME IN PROCESS DATE DELIVERED Notice anything else?

  25. TIME IN PROCESS DATE DELIVERED Notice anything else?

  26. TIME IN PROCESS DATE DELIVERED 90% AVERAGE

  27. MULTI MODAL @catswetel | @everydaykanban

  28. TIME IN PROCESS DATE DELIVERED

  29. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  30. TIME IN PROCESS DATE DELIVERED What else can we learn?

  31. TIME IN PROCESS DATE DELIVERED THROUGHPUT Units of work per

    unit of time
  32. TIME IN PROCESS DATE DELIVERED Boxes show variability of Time

    in Process, NOT Throughput
  33. TIME IN PROCESS DATE DELIVERED Does it always make sense

    to count all the dots?
  34. TIME IN PROCESS DATE DELIVERED 9 9 11 6 4

    6 4 10 13
  35. TIME IN PROCESS DATE DELIVERED 9 9 11 6 4

    6 4 10 13 1 2 4 1 0 3 1 1 1
  36. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  37. THROUGHPUT

  38. 85% 50%

  39. “We are forced to consider knowledge as something that changes

    as new evidence is provided by more data, or as new predictions are made from the same data by new theories.” Statistical Method from the Viewpoint of Quality Control by Walter A Shewhart @catswetel | @everydaykanban
  40. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  41. Lagging quality @catswetel | @everydaykanban

  42. ESCAPED DEFECTS Jan defects found in production Feb Mar Apr

    May Jun Jul Aug Sep 4 5 8 8 10 6 12 8 11 @catswetel | @everydaykanban type
  43. ESCAPED DEFECTS Jan defects found in production Feb Mar Apr

    May Jun Jul Aug Sep 4 5 8 8 10 6 12 8 11 @catswetel | @everydaykanban type How can we tell if we’re getting better at finding defects earlier?
  44. % OF DEFECTS DEFECTS BY ENV percentage Key: Jan Feb

    Mar Apr May Jun Jul Aug Sep @catswetel | @everydaykanban STAGE PROD DEV INT env
  45. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  46. @catswetel | @everydaykanban

  47. None
  48. Predicting the movements of the one bee is nearly impossible

    without the context of the swarm of bees. Complex Adaptive Systems: An Introduction to Computational Models of Social Life by John H. Miller, Scott E. Page
  49. @catswetel | @everydaykanban

  50. MORE FINISHED THAN STARTED MORE STARTED THAN FINISHED @catswetel |

    @everydaykanban
  51. @catswetel | @everydaykanban

  52. @catswetel | @everydaykanban

  53. Total WIP @catswetel | @everydaykanban

  54. OLDEST NEWEST @catswetel | @everydaykanban

  55. Average WIP @catswetel | @everydaykanban

  56. @catswetel | @everydaykanban

  57. @catswetel | @everydaykanban

  58. @catswetel | @everydaykanban

  59. LITTLE’S LAW @catswetel | @everydaykanban

  60. It’s ok to make necessary trade-offs

  61. Where should we focus improvement efforts? @catswetel | @everydaykanban

  62. @catswetel | @everydaykanban FLOW EFFICIENCY Work Wait Total Duration

  63. ...per state (incl. waiting states) @catswetel | @everydaykanban

  64. @catswetel | @everydaykanban You can do any chart for any

    portion of the process!
  65. Are we there yet? @catswetel | @everydaykanban

  66. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY

  67. #NOESTIMATES @catswetel | @everydaykanban

  68. “Close examination reveals that every meaningful interpretation involves a prediction.”

    Statistical Method from the Viewpoint of Quality Control Walter A Shewhart @catswetel | @everydaykanban
  69. 3 Components of Knowledge: 1. Evidence 2. Prediction 3. Degree

    of Belief @catswetel | @everydaykanban
  70. EVIDENCE Agile Quantified (Measuring the impact of Agility) Larry Maccherone

    Wed 3:45 PREDICTION Forecasting and Estimation Hands-On Ozzie Yuce Thu 10:45 DEGREE OF BELIEF Data Driven Coaching Julia Wester & Cheryl Hammond Thu 9:00 @catswetel | @everydaykanban
  71. Julia Wester Co-Founder & Principal Consultant Lagom Solutions julia@lagom.solutions @everydaykanban

    Cat Swetel Engineering Manager Ticketmaster cat@catswetel.com @catswetel
  72. Pablo Picasso “Computers are useless. They can only give you

    answers.”
  73. None
  74. “STATISTICS are useless. They can only give you answers.”

  75. None
  76. None
  77. LITTLE’S LAW @catswetel at #qconnyc Are your decisions making this

    more or less true? FEEL FREE TO CUT
  78. Some joke about defining quality

  79. VARIABILITY FEEL FREE TO CUT

  80. TIME IN PROCESS DATE DELIVERED VARIABILITY FEEL FREE TO CUT

  81. QUALITY RESPONSIVENESS PRODUCTIVITY PREDICTABILITY VARIABILITY VARIABILITY VARIABILITY SHEWHART: the cause

    of v in an expected attribute. Re JUDGEMENT to know which “important quality characte (causes of undesirable varia within the bounds of reason expected economic results. FEEL FREE TO CUT
  82. None
  83. DONE REVIEW DOING TO DO @catswetel at #qconnyc

  84. CUMULATIVE FLOW DIAGRAM @catswetel at #qconnyc

  85. CUMULATIVE FLOW DIAGRAM AVERAGE ARRIVAL RATE @catswetel at #qconnyc

  86. CUMULATIVE FLOW DIAGRAM AVERAGE DEPARTURE RATE @catswetel at #qconnyc

  87. CUMULATIVE FLOW DIAGRAM AVERAGE WORK IN PROCESS @catswetel at #qconnyc

  88. CUMULATIVE FLOW DIAGRAM AVERAGE TIME IN PROCESS @catswetel at #qconnyc