Jirat
 Pasuksmit Towards Just-Enough Documentation 
 for Agile Effort Estimation What Information Should Be Documented? Patanamon (Pick)
 Thongtanunam Shanika
 Karunasekera @jiratpa ICSME 2021: Research track @pamon

Agile teams work in short sprints. Completed 
 Features + Planning Implementing Releasing Sprint 
 (2-4 weeks) Product Backlog Task Task Task Sprint Backlog Task Task Selecting tasks Estimate tasks Detail 
 analysis 1/14

Product Backlog Task Task Task Task Task Sprint Backlog Task Task Past Sprint Backlog Task Capacity Reliable effort estimation helps the team create a reliable sprint plan Next Sprint Backlog Task Task Task Sprint Backlog Task Task Next Sprint Backlog Task Task Task Time Inaccurate estimation can affect the sprint plan 2/14

(Kasauli et al., RE 2017, Andriyani et al., KSEM 2017, Gralha et al., ICSE 2018) Verbal communication Documentation An effort estimation is typically based on the available information, 
 which can be conveyed in various forms Example: estimation based on the available information = 2 days (Available 
 information) Implement a script to do […] that can be used as a server-side extension to […], on the machines used in […] . * Install necessary version of […]. 3/14

In Agile practices, documentation has a lower priority than working software Documented information is found with quality issues, e.g., incorrect, incomplete, outdated, or changed. (Aghajani et al., ICSE 2019, Kasauli et al., RE 2017, Zhi et al., JSS 2015) The Documented information may not be well maintained, resulting in inadequate information for effort estimation. Working 
 Software Comprehensive 
 Documentation Agile prioritize software delivery 
 over comprehensive documentation Implement a script to do […] that can be used as a server-side extension to […], on the machines used in […]. * Install necessary version of […].
 * Implement […] procedure.
 * Interface with […] via JSON inputs and outputs. The inputs include […]. = 2 days (Available 
 information) Implement a script to do […] that can be used as a server-side extension to […], on the machines used in […] . * Install necessary version of […]. = ? days (Actual 
 information) 4/14

RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? Investigating the importance and types of documented information that are 
 useful for effort estimation ? 2 days 5/14

Recruitment Process
 - 1,500 direct invitations were sent to
 software engineers through LinkedIn Survey respondents 121 respondents, used Agile and perform e ff ort estimation. AGILE had > 5 years of experience (72%) used scrum framework (78%) estimating e ff ort during sprint planning (80%) respondents were from Americas (36%), 
 Europe (20%), Asia (33%), and Oceania (10%) Pilot tests - with 5 software engineers with ~8 years
 of experiences Survey questions - importance of documented information (RQ1) - practices to re-estimate (RQ2) - useful types of documented information for 
 e ff ort estimation and their quality (RQ3) Conducting a survey study with 121 Agile practitioners 6/14

RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? ? 2 days Investigating the importance and types of documented information that are 
 useful for effort estimation 7/14

The respondents reported that documented information used, and considered to be important when estimating e ff ort. Documented information is moderately to extremely important for effort estimation 85% and 75% of the respondents reported that they update the document and estimated e ff ort most of the time. (Q4) Documented e ff ort
 updating frequency (Q3) Documentation
 updating frequency (Q2) Importance of 
 documented information
 for e ff ort estimation RQ1 - To what degree is documented information important for effort estimation? 8/14

RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? ? 2 days Documented information is moderately to extremely important for effort estimation Investigating the importance and types of documented information that are 
 useful for effort estimation 9/14

RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation? ? 2 days Documented information is moderately to extremely important for effort estimation Investigating the importance and types of documented information that are 
 useful for effort estimation 9/14

The respondents tend to re-estimate a task when the documented information was changed more often than when the information from other sources was changed (Wilcoxon signed-rank test, p<0.05) Changes of documented information are more likely to influence re-estimation Do you re-estimate the task (Q5) when the documented 
 information was changed (Q6) when the information from 
 other sources was changed (before the work had started) (Q7) when the documented 
 information was changed (Q8) when the information from 
 other sources was changed (after the work had started) Do you re-estimate the task Reasons to re-estimate # - Re-estimate when change is signi fi cant or from a reliable source like documentation
 
 - Always re-estimate to correct the accuracy, velocity, or development plan 52
 
 
 
 26 Reasons to not re-estimate - The team has a policy to not re-estimate after the work has started
 
 - Absorb the change and/or roll the work over to the next sprint
 
 12
 
 
 9
 
 
 RQ2 - Is a change of documented information more likely to influence 
 re-estimation than the information from other sources? 10/14

RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? ? 2 days Investigating the importance and types of documented information that are 
 useful for effort estimation Changes of documented information are more likely to influence re-estimation 11/14 RQ1 - To what degree is documented information 
 important for e ff ort estimation? Documented information is moderately to extremely important for effort estimation

RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? RQ3 - What types of documented information are 
 useful for e ff ort estimation? ? 2 days Investigating the importance and types of documented information that are 
 useful for effort estimation Changes of documented information are more likely to influence re-estimation 11/14 RQ1 - To what degree is documented information 
 important for e ff ort estimation? Documented information is moderately to extremely important for effort estimation

The useful types of documented information have quality issues (changing / missing) The distribution of importance rank of documented information Functional requirements, user stories, de fi nition of done are the most useful documented information for e ff ort estimation 1 2 3 The useful types of documented information were found with quality issues, i.e., changing or missing. RQ3 - What types of documented information are useful for effort estimation? 12/14

Investigating the importance and types of documented information that are 
 useful for effort estimation 11/14 Functional requirements, User stories, and Definition of Done are the most useful information for effort estimation. 
 However, they have quality issues (changing / missing). RQ2 - Is a change of documented information more
 likely to in fl uence re-estimation than the 
 information from other sources? ? 2 days Changes of documented information are more likely to influence re-estimation Documented information is moderately to extremely important for effort estimation RQ3 - What types of documented information are 
 useful for e ff ort estimation? RQ1 - To what degree is documented information 
 important for e ff ort estimation?

The documented information is used and considered important for e ff ort estimation. The change of documented information can have an impact on the estimation accuracy. Prior to fi nalizing e ff ort estimates (e.g., at sprint planning), the team should ensure that the useful information for e ff ort estimation is documented. Future research should focus on developing tools and techniques to assess the quality of documented information for e ff ort estimation. Task Functional requirements / User stories De fi nition of Done UI wireframes Acceptance Criteria Task dependencies ? 2 days Summary and implication 14/14