Towards Just-Enough Documentation for Agile Effort Estimation: What Information Should Be Documented?

Effort estimation is an integral part of activities
planning in Agile iterative development. An Agile team estimates
the effort of a task based on the available information which is
usually conveyed through documentation. However, as documentation has a lower priority in Agile, little is known about how
documentation effort can be optimized while achieving accurate
estimation. Hence, to help practitioners achieve just-enough
documentation for effort estimation, we investigated the different
types of documented information that practitioners considered
useful for effort estimation. We conducted a survey study with
121 Agile practitioners across 25 countries. Our survey results
showed that (1) despite the lower priority of documentation in
Agile practices, 98% of the respondents considered documented
information moderately to extremely important when estimating
effort, (2) 73% of them reported that they would re-estimate
a task when the documented information was changed, and
(3) functional requirements, user stories, definition of done, UI
wireframes, acceptance criteria, and task dependencies were
ranked as the most useful types of documented information for
effort estimation. Nevertheless, many respondents reported that
these useful types of documented information were occasionally
changing or missing. Based on our study results, we provide
recommendations for agile practitioners on how effort estimation
can be improved by focusing on just-enough documentation.

Transcript

1. Jirat
 Pasuksmit Towards Just-Enough Documentation 
 for Agile Effort Estimation What Information Should Be Documented? Patanamon (Pick)
 Thongtanunam Shanika
 Karunasekera @jiratpa ICSME 2021: Research track @pamon
2. Agile teams work in short sprints. Completed 
 Features + Planning Implementing Releasing Sprint 
 (2-4 weeks) Product Backlog Task Task Task Sprint Backlog Task Task Selecting tasks Estimate tasks Detail 
 analysis 1/14
3. Product Backlog Task Task Task Task Task Sprint Backlog Task Task Past Sprint Backlog Task Capacity Reliable effort estimation helps the team create a reliable sprint plan Next Sprint Backlog Task Task Task Sprint Backlog Task Task Next Sprint Backlog Task Task Task Time Inaccurate estimation can affect the sprint plan 2/14
4. (Kasauli et al., RE 2017, Andriyani et al., KSEM 2017, Gralha et al., ICSE 2018) Verbal communication Documentation An effort estimation is typically based on the available information, 
 which can be conveyed in various forms Example: estimation based on the available information = 2 days (Available 
 information) Implement a script to do […] that can be used as a server-side extension to […], on the machines used in […] . * Install necessary version of […]. 3/14
5. In Agile practices, documentation has a lower priority than working software Documented information is found with quality issues, e.g., incorrect, incomplete, outdated, or changed. (Aghajani et al., ICSE 2019, Kasauli et al., RE 2017, Zhi et al., JSS 2015) The Documented information may not be well maintained, resulting in inadequate information for effort estimation. Working 
 Software Comprehensive 
 Documentation Agile prioritize software delivery 
 over comprehensive documentation Implement a script to do […] that can be used as a server-side extension to […], on the machines used in […]. * Install necessary version of […].
 * Implement […] procedure.
 * Interface with […] via JSON inputs and outputs. The inputs include […]. = 2 days (Available 
 information) Implement a script to do […] that can be used as a server-side extension to […], on the machines used in […] . * Install necessary version of […]. = ? days (Actual 
 information) 4/14
6. RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? Investigating the importance and types of documented information that are 
 useful for effort estimation ? 2 days 5/14
7. Recruitment Process
 - 1,500 direct invitations were sent to
 software engineers through LinkedIn Survey respondents 121 respondents, used Agile and perform e ff ort estimation. AGILE had > 5 years of experience (72%) used scrum framework (78%) estimating e ff ort during sprint planning (80%) respondents were from Americas (36%), 
 Europe (20%), Asia (33%), and Oceania (10%) Pilot tests - with 5 software engineers with ~8 years
 of experiences Survey questions - importance of documented information (RQ1) - practices to re-estimate (RQ2) - useful types of documented information for 
 e ff ort estimation and their quality (RQ3) Conducting a survey study with 121 Agile practitioners 6/14
8. RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? ? 2 days Investigating the importance and types of documented information that are 
 useful for effort estimation 7/14
9. The respondents reported that documented information used, and considered to be important when estimating e ff ort. Documented information is moderately to extremely important for effort estimation 85% and 75% of the respondents reported that they update the document and estimated e ff ort most of the time. (Q4) Documented e ff ort
 updating frequency (Q3) Documentation
 updating frequency (Q2) Importance of 
 documented information
 for e ff ort estimation RQ1 - To what degree is documented information important for effort estimation? 8/14
10. RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? ? 2 days Documented information is moderately to extremely important for effort estimation Investigating the importance and types of documented information that are 
 useful for effort estimation 9/14
11. RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? ? 2 days Documented information is moderately to extremely important for effort estimation Investigating the importance and types of documented information that are 
 useful for effort estimation 9/14
12. The respondents tend to re-estimate a task when the documented information was changed more often than when the information from other sources was changed (Wilcoxon signed-rank test, p<0.05) Changes of documented information are more likely to influence re-estimation Do you re-estimate the task (Q5) when the documented 
 information was changed (Q6) when the information from 
 other sources was changed (before the work had started) (Q7) when the documented 
 information was changed (Q8) when the information from 
 other sources was changed (after the work had started) Do you re-estimate the task Reasons to re-estimate # - Re-estimate when change is signi fi cant or from a reliable source like documentation
 
 - Always re-estimate to correct the accuracy, velocity, or development plan 52
 
 
 
 26 Reasons to not re-estimate - The team has a policy to not re-estimate after the work has started
 
 - Absorb the change and/or roll the work over to the next sprint
 
 12
 
 
 9
 
 
 RQ2 - Is a change of documented information more likely to influence 
 re-estimation than the information from other sources? 10/14
13. RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? ? 2 days Investigating the importance and types of documented information that are 
 useful for effort estimation Changes of documented information are more likely to influence re-estimation 11/14 RQ1 - To what degree is documented information 
 important for e ff ort estimation? Documented information is moderately to extremely important for effort estimation
14. RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? ? 2 days Investigating the importance and types of documented information that are 
 useful for effort estimation Changes of documented information are more likely to influence re-estimation 11/14 RQ1 - To what degree is documented information 
 important for e ff ort estimation? Documented information is moderately to extremely important for effort estimation
15. The useful types of documented information have quality issues (changing / missing) The distribution of importance rank of documented information Functional requirements, user stories, de fi nition of done are the most useful documented information for e ff ort estimation 1 2 3 The useful types of documented information were found with quality issues, i.e., changing or missing. RQ3 - What types of documented information are useful for effort estimation? 12/14
16. Investigating the importance and types of documented information that are 
 useful for effort estimation 11/14 Functional requirements, User stories, and Definition of Done are the most useful information for effort estimation. 
 However, they have quality issues (changing / missing). RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? ? 2 days Changes of documented information are more likely to influence re-estimation Documented information is moderately to extremely important for effort estimation RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation?
17. The documented information is used and considered important for e ff ort estimation. The change of documented information can have an impact on the estimation accuracy. Prior to fi nalizing e ff ort estimates (e.g., at sprint planning), the team should ensure that the useful information for e ff ort estimation is documented. Future research should focus on developing tools and techniques to assess the quality of documented information for e ff ort estimation. Task Functional requirements / User stories De fi nition of Done UI wireframes Acceptance Criteria Task dependencies ? 2 days Summary and implication 14/14