Software estimation is a delusion

It has been known since the 70s that developers tend to give very optimistic estimations. We prefer to have exact numbers, even if that means they are wrong most of the time. But maybe it isn’t really accuracy people are looking for. Maybe it is all about risk aversion.

Transcript

1. TAKE A WILD GUESS AND RUN WITH IT Ines Panker | April 2025 SOFTWARE ESTIMATION:
2. You will never be able to accurately estimate the time and effort it will take to build a piece of software ... for as long as you keep building new things.
3. It doesn't have to be accurate, just give me a number.
4. Predictions of future effort are always based on the facts of past effort.
6. Nature Na ture Nature
7. If you have done this project before, then you will be able to estimate accurately how much effort it will take to do it again.
8. As you do the same thing again and again, you get better and faster at it. But then the world changes and that makes you slow again.
9. Why are people even asking this question: “How long will it take to create this piece of software”? “What can you build for this much money”?
10. Let ’s build it iterativ ely CORE OPTIONAL CORE OPTIONAL CORE OPTIONAL CORE OPTIONAL
11. We don’t know how long this will take. It has not been done before. But here is a list of things we will gain and here is a list of risks we will take.
12. There is no point in giving estimations that you can’t stick to with a reasonable error margin.
13. Searching: a human or AI oracle
14. It has been known since the 70s that developers tend to give very optimistic estimations.
15. “We investigated more than 100 projects from 6 development organizations and found that about 30% of the projects had applied effort prediction intervals on the activity level. However, only 3 of these projects had logged the actual effort applying the same work break-down structure as used when estimating the effort. In fact, even these three projects had to split and combine some of the logged effort data to enable an analysis of the accuracy of the effort of PIs. (tn: PI = prediction interval, a minimum - maximum effort).” M. Jørgensen, K. Teigen, K. J. Moløkken-Østvold Better Sure Than Safe? Overconfidence in Judgment Based Software Development Effort Prediction Intervals
16. 200% 1100h 1500h 1550h 1339h 2251h
17. median % of 2400h Developers 660h 28% Project Managers 960h 40% Designers 1260h 53% Engagement Managers 1550h 65% M. Jørgensen, K. Teigen, K. J. Moløkken-Østvold Better Sure Than Safe? Overconfidence in Judgment Based Software Development Effort Prediction Intervals
18. “As a result, technical background did not lead to better effort PIs, only to more confidence in the estimates. This is in line with the distinction between an “inside” versus an “outside” view in predictions (Kahneman and Lovallo 1993).” M. Jørgensen, K. Teigen, K. J. Moløkken-Østvold Better Sure Than Safe? Overconfidence in Judgment Based Software Development Effort Prediction Intervals
19. “An inside view forecast is generated by focusing on the case at hand, by considering the plan and the obstacles to its completion, by constructing scenarios of future progress, and by extrapolating current trends. The outside view is the one that the curriculum expert was encouraged to adopt. It essentially ignores the details of the case at hand, and involves no attempt at detailed forecasting of the future history of the project. Instead, it focuses on the statistics of a class of cases chosen to be similar in relevant respects to the present one.” Kahneman and Lovallo Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking
20. What is a useless estim ate?
21. Developers admitted that they believe their managers will see them as less competent if they provide estimates with huge margins. Dev 2 Dev 1 Better to be precise than accurate Better to be precise than accurate
22. MATH neural networks case-based reasoning fuzzy logic modeling simulation based probability regression- based approaches classification and regression trees Bayesian statistics lexical analyses of requirement specifications
23. “The proportion of estimation studies where estimation methods are studied or evaluated in real-life situations is low. We could not, for example, find a single study on how software companies actually use formal estimation models. Our knowledge of the performance of formal estimation models is, therefore, limited to laboratory settings” M.Jørgensen and M.J.Shepperd A Systematic Review of Software Development Cost Estimation Studies
24. “Today, almost no model can estimate the cost of software with a high degree of accuracy. ... To improve the algorithmic models, there is a great need for the industry to collect project data on a wider scale.” Hareton Leung Software Cost Estimation
25. Estimation Variables
26. “For predictive measurement the model (tn: =metric) alone is not sufficient. Additionally, we need to define the procedures for a) determining model parameters and b) interpreting the results. The model, together with procedures (a) and (b), is called a prediction system. Using the same model will generally yield different results if we use different prediction procedures.“ N. Fenton Software Measurement: A Necessary Scientific Basis
27. “.. size is defined as the number of delivered source statements (tn: LOC), which is an attribute of the final implemented system. Since the prediction system is used at the specification phase, we have to predict the product attribute size in order to plug it into the model. This means that we are replacing one difficult prediction problem (effort prediction) with another prediction problem which may be no easier (size prediction)” N. Fenton Software Measurement: A Necessary Scientific Basis
28. “The important lesson to take from this paper is that no one method or model should be preferred over all others. The key to arriving at sound estimates is to use a variety of methods and tools and then investigating the reasons why the estimates provided by one might differ significantly from those provided by another.” B.Boehm and C. Abt Software Development Cost Estimation Approaches – A Survey
30. Maybe it is time we stop estimating tasks left and right and instead start managing the project’s risk and customer’s expectations. Ines Panker | April 2025