inevitable, even at JPL, so … – We need to plan for their discovery and repair – It is not enough to say “I need a maintenance budget.” Need to be able to jus?fy the budget and match predicHons with reality. • EssenHal to this are pragma?c models that can easily be implemented, calibrated and validated – AssumpHons need only be reasonably valid for our environment – Must be able to explain models to stakeholders. • Simple (if it works) is always beYer. – Conclusions need to fit our observaHons from over a decade of defect discovery. • Stakeholders won’t believe (and shouldn’t) believe a model unless it fits the data, is consistent their experience, and is grounded. – For this kind of engineering work: Good enough is perfect. • If theoreHcal nuances don’t maYer, they really don’t maYer.
At JPL we had the good fortune of having a “hydrogen atom” of soOware maintenance present itself. – Two systems under going repair only beginning in 2006 and 2008. • No new development, new development on an upgrade system – Complete defect histories tracked prior to the onset of repair only mode • The defect tracker bugzilla was adopted for all mission design and and navigaHon soOware in 2002 and is sHll used today. – Defects repaired quickly aOer their discovery (thanks to the adopHon of bugzilla and a rapid response process) – Uniform daily use • Based on uniform staff levels of users and system accounHng history. – The system was free of the perturbing forces of sporadic development that are part of most maintenance systems allowing for the development of a simple defect decay model.
rate of change in the expected number of defects in the system at Hme t, N(t), is proporHonal to the number of defects in the system. • Let T be any reference epoch, then N saHsfies • We cannot observe the iniHal number of defects N(t) We can observe A(t) — the accumulated number of defects at Hme t. • Use the model for N relaHve to the reference epoch T. – The number of accumulated defects is just the number that the total defect count has been reduced since the reference epoch N(T) – N(t). • Model depends only on Hme elapsed since any reference Hme. • The Hme it takes to reduce the defects by some factor depends only upon the esHmable decay constant α A(t) = N(T) 1− e−α (t −T ) ( ) N(t) N(t) = N (T)e−α (t −T).
together with the exponenHal decay model allows us to: (1) Tell projects what the range of defects they are likely to find in a new release (3) How long it will take to find them (4) Establish credibility • Our assumpHons have been challenged in the past and we have been able to win a gentlemen’s bet that we can predict what stakeholders will actually find. • Our models are now widely accepted by our stakeholders. No one will bet against our models.
models are: – Simple • It requires a minimal set of parameters – PracHcal • It is easy to implement and fit – PragmaHc • It are good enough when trying to answer reliability quesHons about our soOware. – Useful • Allows us to defend budgets • Allows us to characterize risk • Allows us to give good quanHtaHve answers to quesHons about the reliability of our soOware.
• The model can be used to understand defect accumulaHon in the new system with planned conHnued development – Defects repaired as soon as they are discovered – Uniform daily use – Uniform producHon of new code resulHng a uniform rate, D, of producing new defects. • The number of defects in the system will stabilize to the rate of producHon of defects D divided by the discovery rate α. • The defect accumulaHon will become linear over Hme with a rate of discovery equal to the rate of producHon.
examinaHon of our released defects shows that defects tend to be distributed widely across the code base are unrelated (recall defects are reported bugs, not code issues). • The previous chart implies a relaHvely constant defect density. • Under this assumpHon we assume that defects per release follow a Poisson distribuHon. – if the expected number defects is β, then – Under the exponenHal decay model, the distribuHon for the number of defects remains Poisson with the the expected number of defects decaying exponenHally Pr(n defects) = βn n! e−β = P β (n)
WAY, TOO SIMPLE! How can we validate this? – Solving for the best fit exponenHal decay constant and defect density, we can simulate the history of defect discoveries for the new system and compare this to the actual history 0" 500" 1000" 1500" 2000" 2500" 0" 500" 1000" 1500" 2000" 2500" 3000" 3500" Accumulated*Defects* Work*Days*Since*August*19,*2001* Theore>cal*Vs*Actual*Defects*Reported*for*Monte* Actual"Defect"History" Theory" R^2 = 0.9893
is just one possible history. Maybe we just got lucky and this is not the way new defects actually accumulate. • Compute distribuHon of all possible histories based on the model and see if the actual actual history lies within it. 0" 0.005" 0.01" 0.015" 0.02" 0.025" 0.03" 0.035" 0.04" 0" 0.2" 0.4" 0.6" 0.8" 1" 1.2" 0" "125.40834" "250.81668" "376.22502" " 501.6333599" " 627.0416999" " 752.4500399" " 877.8583799" " 1003.26672" " 1128.67506" "1254.0834" " 1379.49174" " 1504.90008" " 1630.30842" Empirical)Distribu/on)of)theoreical)histories) Actual" History" pdf" Indeed it does! And not even a rare one.
Your Podcast. Everywhere. Effortlessly.
dport96
aleyda
eileencodes
palkan
dugsong
jnunemaker
mkilby
.png){kind=avatar}
helenjbeal
lindahogenes
soudai
elsirapls
keithpitt
schlessera
