What are the odds? Forecasting with Probabilities

This talk was given at DevOpsCon Berlin in June of 2022. It is a version of the PM72 talk I also have listed here but has less product placement.

See the abstract:
Do you feel like you spend too much time estimating and not enough time doing the work? Has anyone calculated how much that time costs the organization? How often do those expensive estimates turn out to be wrong despite your investment? What if I told you that there’s a way that you can save money and stress by quickly generating forecasts that are as reliable, or better than those you currently create? That way is called probabilistic forecasting and you don’t need to be a math or data wiz to take advantage. In this session you’ll learn how and why to create probabilistic forecasts using your historical data. We’ll also talk about underlying assumptions and misconceptions you might have about the data you’d need to use your historical data for forecasting. Leaving this session you’ll have solid tactics to answer "How much can we complete by X date?" and "How long will this take?" using the level of risk that your organization is willing to accept.

Transcript

1. Julia Wester
   @everydaykanban
   Co-Founder, 55 Degrees AB
   What are the odds?
   Forecasting with probabilities
   

   View Slide

2. Planning.
   
   
   

   View Slide

3. Benjamin Franklin
   
   
   November 1789
   “…IN THIS WORLD,
   NOTHING IS
   CERTAIN EXCEPT
   DEATH AND TAXES.”
   

   View Slide

4. Often forecasts are
   interpreted as certainties
   
   
   because we don’t say
   otherwise
   CC BY-SA 3.0, https://en.wikipedia.org/w/index.php?curid=3422341
   

   View Slide

5. The 3 Gaps
   Outcomes
   Plans
   Actions
   !
   Effects Gap
   Difference between what we
   expect our actions to achieve
   and what they actually achieve
   !
   Knowledge Gap
   Difference between what we
   would like to know and what we
   actually know
   !
   Alignment Gap
   Difference between what we want people
   to do and what they actually do
   

   View Slide

6. Used with permission from Jim Benson
   The Predictable
   Initial Response
   

   View Slide

7. The workplace version of ruling it harder…
   EE4: Building an SSV
   Timeline
   Total expected hours
   Total hours
   Sam Jansen
   Expected hours
   Total hours
   Alexander Meynen
   Expected hours
   Total hours
   Tinus Michiels
   Expected hours
   Total hours
   Jeroen van Aert
   Expected hours
   Total hours
   Jef Van Den Bergh
   Expected hours
   Total hours
   Bob Verschueren
   Expected hours
   Total hours
   1 2 3 4 5 6 7 8 9 10 11 12 13 14
   Tasks Subtasks wk h h h h h h h h h h h h h h
   Engineering
   a. Case SSV 1 i. Motor/ Solar panel analysis 7 4 3
   ii. Gear ratio 15 2,5 2,5 10 2,5 2,5 2,5 2,5
   iii. Choice of materials 10 5 1 1 1 1 1 5 1
   iv. Frame design 30 15 2 1 1 1 15 1 1
   v. Optimalisation (bisection method) 25 4 15 10 4
   vi. Sankey diagram 10 12 10
   vii. Production parts 20 10 12 10
   viii. Construction 60 10 10 10 10 10 10
   b. Case Simulink i. Simulation solar panel 24 4 4 4 4 5 4 4 10
   ii. Simulation DC-motor 7 4 3
   iii. Simulation race 6 1 1 1 1 1 1
   c. Case SSV 2 i. Slope run test 24 4 4 4 4 4 4
   ii. Simulation drive shaft 7 3 4
   iii. Technical drawings 40 10 30
   iv. Collision analysis 15 10 5
   d. Race day 30 5 5 5 5 5 5
   Enterprising
   e. Wikipage i. Company information 3 3 1
   ii. SSV information 5 5 1
   f. Name & Logo i. Design 2 1 2
   ii. Engraving 6 3 3
   g. Documents i. Plan of approach 4 2 2 2 2
   ii. Work breakdown structure 4 2 2 2 2
   iii. Gantt chart 10 1 5 3 5 3
   iv. Cooperation contract 2 2 2
   v. Case 1 document 45 5 5 15 5 5 15 5 5 15 5
   vi. Simulink document 15 5 10 5 10
   vii. Case 2 document 40 15 5 10 10
   h. Wikipage i. Upload documents 5 5 1
   ii. Blog 5 5 2 2
   iii. Design description 6 3 3
   iv. Final process report 5
   i. Evaluation i. Test 1 30 5,00 5 5 5 5 5
   ii. Test 2 30 5,00 5 5 5 5 5
   iii. Peer assessments 1
   total 548 0 92 27 92 25 92 21 88 22 91 25 90 25
   Legende milestone
   responsibillity
   Gantt chart 1
   Educating
   Weeks
   CC BY-SA 3.0 - https://upload.wikimedia.org/wikipedia/commons/thumb/b/bf/AM4_Gantt.pdf
   

   View Slide

8. Deterministic Forecast
   IT WILL BE
   
   
   DONE ON
   
   
   X DATE
   •Communicates a single
   possible outcome
   
   
   •Provides no probability of
   that outcome
   
   
   •Appropriate only when
   certainty is assured
   

   View Slide

9. Probabilistic Forecast
   THERE’S AN
   
   
   85% CHANCE
   
   
   IT WILL BE DONE
   
   
   ON OR BEFORE
   

   X DATE
   •Two components:
   
   
   ‣ Range of outcomes
   
   
   ‣ Probability you’ll fall into
   that range
   
   
   •Appropriate whenever
   uncertainty is present
   

   View Slide

10. Single Item Forecasts
    Multiple Item Forecasts
    📆
    
    
    Fixed Date
    📑
    
    
    Fixed Scope
    2 Types
    Let’s create these probabilistic forecasts
    

    View Slide

11. All we need are
    
    
    3 data points
    

    for each work item
    

    View Slide

12. How long will it take?
    Start
    Single Item Forecasts
    Cycle Time: The total elapsed time it takes a
    work item to travel from start to
    fi
    nish
    Finish
    

    View Slide

13. Finish
    Start
    (Finish Date - Start Date) + 1
    

    You can substitute Date with a different unit of time
    Single Item Forecasts
    Cycle Time: The total elapsed time it takes a
    work item to travel from start to
    fi
    nish
    

    View Slide

14. 85% of work took
    16 days or less
    50% of work took
    
    
    7 days or less
    Use a Cycle Time Scatterplot to calculate probabilities
    

    View Slide

15. Wrong as much
    as right…
    Contains too
    many outliers
    Just the right
    balance?
    Choose a probability
    99%
    by understanding your risk tolerance
    Ask: What is the fallout if we fall
    outside of the probability?
    

    View Slide

16. 85% of the time we finish a
    

    work item in 16 days or less
    

    (from the day it starts)
    Deliver your forecast probabilistically
    

    View Slide

17. The underlying rule of thumb
    

    View Slide

18. Finish
    Start
    When will
    they all be
    done?
    Multiple Item Forecasts
    Throughput: The total count of items that pass the
    de
    fi
    ned
    fi
    nish point in a given unit of time
    

    View Slide

19. But you need a Monte Carlo Simulation
    to calculate the probabilities
    A Throughput Run Chart will show you the data
    

    View Slide

20. There’s an 85%
    chance that we will
    f
    inish these 45 items
    on or before May 25
    Monte Carlo for Fixed Scope (When?)
    

    View Slide

21. There’s an
    85% chance that
    we can
    f
    inish
    
    
    67 or more items
    by then
    Monte Carlo for Fixed Date (How Many?)
    

    View Slide

22. Monte Carlo even works for your portfolio
    

    View Slide

23. • When more than one outcome is possible,
    probabilistic forecasts are necessary.
    
    
    • Probabilistic forecasts show risk in 2 ways:
    

    a probability (%) and a range.
    
    
    • Choose a % that matches your risk tolerance
    
    
    • You can do everything manually, but tools exist to
    help you create these forecasts in minutes.
    Key Takeaways
    

    View Slide

24. https://55degrees.se
    Stay in touch
    /company/55degrees
    @55degreesAB
    @55degreesAB
    /55degreesAB
    /c/ActionableAgileAnalytics
    

    View Slide