Cincy Deliver 2022 Flow Metrics

Wouldn’t it be nice if you could use an agile approach and have good data for forecasting when something was going to be done, could shorten cycle time, optimize predictability and delivery speed? I’ll bet it would! This presentation addresses these questions by using a Kanban with Scrum approach, focusing on metrics around a flow-driven work effort for predictability. By the end of the session you’ll have a strategy for collecting and understanding data that can be applied to any of the above questions. Prerequisites: The attendees should know what the agile mindset is and have heard of Kanban. Information for Program Team: This session will be based on Scrum.org’s Professional Scrum with Kanban class, a well established class presented by numerous PSTs. It will take some of the flow metrics, WIP, cycle time, SLAs and show how these can be used to manage flow and set up a certain level of predictability. Frankly it has changed the way I approach scrum at the team level. Objectives: Student will describe the difference between throughput and cycle time. Students will be able to explain the importance of flow to optimizing throughput and cycle time. Student will understand the effect of WIP on throughput and cycle time. Students will be able to explain how throughput and cycle time can be used to enhance predictability. Students will be exposed to Monte Carlo simulation as a forecasting tool.

Transcript

1. Using Concrete Data for a More Predictable Flow of Work @ScrumDotOrg 1 Excerpts from:
2. This is me PST Developer Strategist Consultant Trainer https://www.scrum.org/classes?uid=128
3. Thank them! 3 2022 CincyDeliver Sponsors © 2022 - Agile Conferences, Inc. Diamond Platinum Gold Coffee Break Community
4. • Budget $ • Plan people’s allocation • Plan for delivery • Foresee and plan dependencies • Manage risks 4 The big question Why on earth do you estimate and forecast? Is sticking to a date more important than other risks?
5. Which of these is better? 5 There is a huge pot of gold behind the door! Or There may be a ninja standing behind the door with a sword who will strike you when you open the door. Nobody is there, trust me (but I’ve been wrong) Based on history, There is a 10% probability that the ninja is there Aren’t risks put in the form of probability? or
6. Story Points 6 Purpose: Group discussion about complexity and effort
7. Story points to forecast work 7 PRODUCT BACKLOG Size: 13 Size: 2 Size: 13 Size: 8 Size: 5 Size: 3 Size: 5 Size: 1 Size: 13 Size: 8 Size: 2 ? PRODUCT BACKLOG Size: 13 Size: 21 Size: 1 Size: 3 Size: 5 Size: 8 Size: 3 Size: 21 Size: 13 Size: 89 Size: 13 A Average Team Velocity = 33 Sprint Length = 2 weeks How long to get to item ”A” How much will be done in 8 weeks?
8. Story points have a poor relation to time on task 8 “On average, the relationship between story points and cycle time, or between story points and allocation, is quite weak” – Benjamin Kotrc (Data Scientist with Jellyfish) Higher is better < .4 = weak association .4-.6 moderate association >.6 = strong association https://jellyfish.co/blog/do-story-points-work/
9. A. Cat B. Ball C. Feather D. Trumpet E. Pizza F. Jaguar G. Sheet of paper H. Stone I. Bat 9 Estimation example Sequence these by effort to throw these items at least 6 ft most effort to least A. Cat – good luck cat B. Ball – 18 lb bowling C. Feather D. Trumpet – picture on a card E. Pizza – frozen in a box F. Jaguar – the car G. Sheet of paper – folded to airplane H. Stone – Stone Henge I. Bat - vampire
10. Story points to discuss effort, complexity, risk, and unknown 10 Story Points Risk Complexity Effort Unknown Planning Poker Relative Estimation T Shirt Size Velocity Will it fit? Did we iron out some details? Do we need to break them apart?
11. Keep using story points 11 Have conversations Use story points to refine work Don’t use to forecast releases
12. Cycle Time 12 Purpose: When will one item be complete?
13. Next PBI in Sprint Discovery Build Ready Building Deploy Ready Validate Release Ready Stage Production Doing Complete Doing Complete Kanban Cycle Time 13 Cycle Time Per Item
14. Cycle Time Scatterplot Chart 14 Cycle Time (Days) Completion Date 14 Calendar time – X axis marks when a specific item was completed Y axis marks Elapsed Time for a specific item
15. Cycle Time Scatterplot Chart 15 Cycle Time (Days) Completion Date
16. Cycle Time Scatterplot Chart 16 Cycle Time (Days) Completion Date
17. Cycle Time Scatterplot Chart 17 Cycle Time (Days) Completion Date
18. Cycle Time Scatterplot Chart 18 Cycle Time (Days) Completion Date How do we make sense of this “randomness”?
19. Cycle Time Scatterplot Chart – Confidence levels 19 Completion Date 7 50% 16 85% 20 95% Cycle Time (Days)
20. Cycle Time Scatterplot Chart – Confidence levels 20 Completion Date 7 50% 16 85% 20 95% Cycle Time (Days) Are these the same “size”? SLE 20 days or less 95% of the time 16 days or less 85% of the time 7 days or less 50% of the time
21. Cycle Time Scatterplot Chart – Confidence levels 21 Completion Date 50% 85% 95% Cycle Time (Days)
22. Use SLE and cycle time 22 Project when the “next” item may get done Use a % and a cycle time SLE = expectation, not a commitment
23. Throughput 23 Purpose: How many items over a given time
24. Next PBI in Sprint Discovery Build Ready Building Deploy Ready Validate Release Ready Stage Production Doing Complete Doing Complete Kanban Throuput 24 Throughput Count over time
25. Common ways to express throughput 25 15 items complete in 20 days 1 items complete in 1.34 days (20/15) .75 items complete in 1 day (15/20) Hey, that’s average cycle time!
26. Trend lines to smooth out throughput 26
27. Is throughput accurate? 27 This is poor flow
28. Use throughput and variability 28 Project how many items in the next time frame Keep flow consistent Use average throughput or trendlines
29. Monte Carlo Simulation 29 Purpose: Forecast groups of items
30. • Collect data • Create a percentile frequency table • Run simulation • Use simulation results to create an SLE 30 Basic flow (more complicated than cycle time and throughput)
31. 68 73 67 65 72 75 71 68 66 71 72 65 66 63 68 75 69 76 69 76 71 76 71 66 63 76 72 68 68 69 69 76 70 69 68 69 71 65 70 68 69 71 70 75 74 68 71 68 71 75 68 76 68 69 70 69 67 75 66 63 70 69 69 66 63 68 75 72 71 70 61 69 68 69 67 73 70 72 67 69 70 70 70 69 65 70 64 68 70 65 69 69 75 72 71 69 67 71 66 71 73 70 75 76 71 70 75 70 74 65 70 61 71 71 69 69 75 68 61 71 69 69 70 68 75 69 71 69 70 71 69 69 69 70 66 63 76 66 63 68 72 65 68 67 71 72 72 68 67 68 65 75 69 69 69 69 70 69 70 66 71 69 69 70 72 76 68 75 66 74 68 68 68 70 75 70 71 68 69 69 64 71 75 69 68 66 70 69 75 75 76 71 66 63 73 68 71 73 70 72 67 65 69 75 69 69 70 69 69 70 67 68 65 71 68 68 65 71 66 63 71 73 70 67 31 Collecting Data 262 Data Points People’s height in inches ..etc
32. 32 Create Percentile Frequency Table Height Count Percentage Frequency Percentile 61 5 2% 2% 62 8 3% 5% 63 3 1% 6% 64 3 1% 7% 65 11 4% 12% 66 8 3% 15% 67 19 7% 22% 68 32 12% 34% 69 38 15% 49% 70 32 12% 61% 71 32 12% 73% 72 22 8% 81% 73 14 5% 87% 74 8 3% 90% 75 19 7% 97% 76 8 3% 100% It’s not a normal distribution
33. 33 Our quest Possibility of average height in a room of 20 people
34. Sim Count Random Nbr Sim Pct Height 1 0.260143172 26% 68 2 0.420379851 42% 69 3 0.629653198 63% 71 4 0.616964709 62% 71 5 0.570671312 57% 70 6 0.238565477 24% 68 7 0.068043943 7% 63 8 0.599259149 60% 70 9 0.590470504 59% 70 10 0.890353632 89% 74 11 0.656214637 66% 71 12 0.280419716 28% 68 13 0.201394913 20% 67 14 0.197261558 20% 67 15 0.751368477 75% 72 16 0.596258241 60% 70 17 0.916890831 92% 75 18 0.785325701 79% 72 19 0.647894407 65% 71 20 0.616272812 62% 71 34 Run Simulation Frequency Percentile Low Bounds Height for Lookup 2% 0% 61 5% 3% 62 6% 6% 63 7% 7% 64 12% 8% 65 15% 13% 66 22% 16% 67 34% 23% 68 49% 35% 69 61% 50% 70 73% 62% 71 81% 74% 72 87% 82% 73 90% 88% 74 97% 91% 75 100% 98% 76 Input Return This average (20 items) = 69.9
35. Run 10,000 Simulations 35 Plot Average for 20 items I know this is not 10,000 runs
36. Make an SLE 36 SLE for 20 items Rounding of inches down 50% is 68 or less 85% is 71 or less 95% is 72 or less 50% 1280.5 85% 2176.85 95% 2432.95
37. Height = days to complete a PBI (throughput) Room size (20 people) = 20 PBIs Average height = how long it took to complete a group of 20 PBIs 37 Relate this to work
38. Two types of questions 38 How long to complete 40 items? Run the simulation for many days day When you reach 40 cumulative items stop That # of days is 1 simulation result How many items in 20 days? Run the simulation for 20 days Stop and add up the # of items The count is 1 simulation result
39. Example of a tool (how many) 39 How many items in 20 days? Run the simulation for 20 days Stop and add up the # of items The count is 1 simulation result
40. Example of a tool (when) 40 How long to complete 40 items? Run the simulation for many days day When you reach 40 cumulative items stop That # of days is 1 simulation result
41. Example dashboard 41
42. Use monte carlo for a better prediction 42 Provide forecasts with a probability Get a tool or hire a spreadsheet guru Study up on how this works
43. Miscellaneous 43 Purpose: Explain things that didn’t neatly fit
44. Little’s Law of Queuing Theory 44 Average Cycle Time = Average Work in Progress Average Throughput Where: Average Cycle Time = how long it takes one item to go through the process Average Work in Progress = how many items are in the process at any time Average Throughput = how many items are produced per unit of time John Little - Little’s Law Image:https://en.wikipedia.org/wiki/Little%27s_law
45. CFD: Cycle Time, WIP, and Throughput 45 Time Cumulative Quantity Average Completion Rate (Throughput) Average Arrival Rate WIP Approx. Avg. Cycle Time Avg. Cycle Time = Avg. Work in Progress Avg. Throughput
46. CFD: Cycle Time, WIP, and Throughput 46 Time Cumulative Quantity Average Arrival Rate WIP Approx. Avg. Cycle Time Average Completion Rate (Throughput) Avg. Cycle Time = Avg. Work in Progress Avg. Throughput Instability Isn’t good
47. A stable system is more predictable 47 WIP increases will do the opposite of what you hoped Flow = stability of arrival and completion Instability makes for bad predictability Backflow is disruptive
48. Cumulative Flow Diagram (CFD) 48 Purpose: visualize flow
49. Next PBI in Sprint Discovery Build Ready Building Deploy Ready Validate Release Ready Stage Production Doing Complete Doing Complete Kanban Cumulative Flow Metrics 49 5 4 3 4 2 2 Cycle Time Throughput Work in Progress
50. Cumulative Flow 50 Work Items Calendar Days Done Testing Developing (Done) Developing (Active) Specifying (Done) Specifying (Active) Legend Calendar Time Cumulative Work Item Count Process States
51. Cumulative Flow 51 Work Items Calendar Days Done Testing Developing (Done) Developing (Active) Specifying (Done) Specifying (Active) Legend Total Work in Progress (WIP)
52. Cumulative Flow 52 Work Items Calendar Days Approximate Average Cycle Time Done Testing Developing (Done) Developing (Active) Specifying (Done) Specifying (Active) Legend
53. Cumulative Flow 53 Work Items Calendar Days Done Testing Developing (Done) Developing (Active) Specifying (Done) Specifying (Active) Legend Slope of Top Line = Avg. Arrival Rate Slope of Bottom Line = Avg. Throughput
54. Role up 54 Use story points for discussions Use statistics for forecasts All you need to capture is start and end dates Forecast with probability Don’t put certainty into the uncertain
55. Thank you! Charles Suscheck Chuck@Suscheck.com www.Suscheck.com 55 https://www.scrum.org/classes?uid=128
56. You’re still here? 56 Chuck@Suscheck.com
57. Using Concrete Data for a More Predictable Flow of Work @ScrumDotOrg 57 Excerpts from: