What are the odds? Forecasting with Probabilities

This talk was given at DevOpsCon Berlin in June of 2022. It is a version of the PM72 talk I also have listed here but has less product placement.

See the abstract:
Do you feel like you spend too much time estimating and not enough time doing the work? Has anyone calculated how much that time costs the organization? How often do those expensive estimates turn out to be wrong despite your investment? What if I told you that there’s a way that you can save money and stress by quickly generating forecasts that are as reliable, or better than those you currently create? That way is called probabilistic forecasting and you don’t need to be a math or data wiz to take advantage. In this session you’ll learn how and why to create probabilistic forecasts using your historical data. We’ll also talk about underlying assumptions and misconceptions you might have about the data you’d need to use your historical data for forecasting. Leaving this session you’ll have solid tactics to answer "How much can we complete by X date?" and "How long will this take?" using the level of risk that your organization is willing to accept.

Transcript

1. Julia Wester @everydaykanban Co-Founder, 55 Degrees AB What are the odds? Forecasting with probabilities
2. Planning.
3. Benjamin Franklin November 1789 “…IN THIS WORLD, NOTHING IS CERTAIN EXCEPT DEATH AND TAXES.”
4. Often forecasts are interpreted as certainties because we don’t say otherwise CC BY-SA 3.0, https://en.wikipedia.org/w/index.php?curid=3422341
5. The 3 Gaps Outcomes Plans Actions ! Effects Gap Difference between what we expect our actions to achieve and what they actually achieve ! Knowledge Gap Difference between what we would like to know and what we actually know ! Alignment Gap Difference between what we want people to do and what they actually do
6. Used with permission from Jim Benson The Predictable Initial Response
7. The workplace version of ruling it harder… EE4: Building an SSV Timeline Total expected hours Total hours Sam Jansen Expected hours Total hours Alexander Meynen Expected hours Total hours Tinus Michiels Expected hours Total hours Jeroen van Aert Expected hours Total hours Jef Van Den Bergh Expected hours Total hours Bob Verschueren Expected hours Total hours 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Tasks Subtasks wk h h h h h h h h h h h h h h Engineering a. Case SSV 1 i. Motor/ Solar panel analysis 7 4 3 ii. Gear ratio 15 2,5 2,5 10 2,5 2,5 2,5 2,5 iii. Choice of materials 10 5 1 1 1 1 1 5 1 iv. Frame design 30 15 2 1 1 1 15 1 1 v. Optimalisation (bisection method) 25 4 15 10 4 vi. Sankey diagram 10 12 10 vii. Production parts 20 10 12 10 viii. Construction 60 10 10 10 10 10 10 b. Case Simulink i. Simulation solar panel 24 4 4 4 4 5 4 4 10 ii. Simulation DC-motor 7 4 3 iii. Simulation race 6 1 1 1 1 1 1 c. Case SSV 2 i. Slope run test 24 4 4 4 4 4 4 ii. Simulation drive shaft 7 3 4 iii. Technical drawings 40 10 30 iv. Collision analysis 15 10 5 d. Race day 30 5 5 5 5 5 5 Enterprising e. Wikipage i. Company information 3 3 1 ii. SSV information 5 5 1 f. Name & Logo i. Design 2 1 2 ii. Engraving 6 3 3 g. Documents i. Plan of approach 4 2 2 2 2 ii. Work breakdown structure 4 2 2 2 2 iii. Gantt chart 10 1 5 3 5 3 iv. Cooperation contract 2 2 2 v. Case 1 document 45 5 5 15 5 5 15 5 5 15 5 vi. Simulink document 15 5 10 5 10 vii. Case 2 document 40 15 5 10 10 h. Wikipage i. Upload documents 5 5 1 ii. Blog 5 5 2 2 iii. Design description 6 3 3 iv. Final process report 5 i. Evaluation i. Test 1 30 5,00 5 5 5 5 5 ii. Test 2 30 5,00 5 5 5 5 5 iii. Peer assessments 1 total 548 0 92 27 92 25 92 21 88 22 91 25 90 25 Legende milestone responsibillity Gantt chart 1 Educating Weeks CC BY-SA 3.0 - https://upload.wikimedia.org/wikipedia/commons/thumb/b/bf/AM4_Gantt.pdf
8. Deterministic Forecast IT WILL BE DONE ON X DATE •Communicates a single possible outcome •Provides no probability of that outcome •Appropriate only when certainty is assured
9. Probabilistic Forecast THERE’S AN 85% CHANCE IT WILL BE DONE ON OR BEFORE 
 X DATE •Two components: ‣ Range of outcomes ‣ Probability you’ll fall into that range •Appropriate whenever uncertainty is present
10. Single Item Forecasts Multiple Item Forecasts 📆 Fixed Date 📑 Fixed Scope 2 Types Let’s create these probabilistic forecasts
11. All we need are 3 data points 
 for each work item
12. How long will it take? Start Single Item Forecasts Cycle Time: The total elapsed time it takes a work item to travel from start to fi nish Finish
13. Finish Start (Finish Date - Start Date) + 1 
 You can substitute Date with a different unit of time Single Item Forecasts Cycle Time: The total elapsed time it takes a work item to travel from start to fi nish
14. 85% of work took 16 days or less 50% of work took 7 days or less Use a Cycle Time Scatterplot to calculate probabilities
15. Wrong as much as right… Contains too many outliers Just the right balance? Choose a probability 99% by understanding your risk tolerance Ask: What is the fallout if we fall outside of the probability?
16. 85% of the time we finish a 
 work item in 16 days or less 
 (from the day it starts) Deliver your forecast probabilistically
17. The underlying rule of thumb
18. Finish Start When will they all be done? Multiple Item Forecasts Throughput: The total count of items that pass the de fi ned fi nish point in a given unit of time
19. But you need a Monte Carlo Simulation to calculate the probabilities A Throughput Run Chart will show you the data
20. There’s an 85% chance that we will f inish these 45 items on or before May 25 Monte Carlo for Fixed Scope (When?)
21. There’s an 85% chance that we can f inish 67 or more items by then Monte Carlo for Fixed Date (How Many?)
22. Monte Carlo even works for your portfolio
23. • When more than one outcome is possible, probabilistic forecasts are necessary. • Probabilistic forecasts show risk in 2 ways: 
 a probability (%) and a range. • Choose a % that matches your risk tolerance • You can do everything manually, but tools exist to help you create these forecasts in minutes. Key Takeaways
24. https://55degrees.se Stay in touch /company/55degrees @55degreesAB @55degreesAB /55degreesAB /c/ActionableAgileAnalytics