Feedback in Scrum: Data-Informed Retrospectives

Slides of the talk given at the Doctoral Symposium of ICSE 2019.
https://2019.icse-conferences.org/track/icse-2019-Doctoral-Symposium#program

1. Hasso Plattner Institute University of Potsdam, Germany christoph.matthies@hpi.de @chrisma0 Feedback in Scrum: Data-Informed Retrospectives Christoph Matthies Doctoral Symp., Canada, May ’19
2. Motivation 2 Software Engineering in General Software engineering must shed the folkloric advice [...], replace them with [...] empirical methods – Bertrand Meyer [Meyer, 2013] “ ” [Meyer, 2013] B. Meyer, H. Gall, M. Harman, and G. Succi, “Empirical Answers to Fundamental Software Engineering Problems (Panel),” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ser. ESEC/FSE 2013. New York, USA: ACM, 2013, pp. 14–18. Picture: https://commons.wikimedia.org/wiki/File:Bertrand_Meyer_recent.jpg
3. Motivation 3 The Role of Data in Scrum Scrum is founded on empirical process control theory [...]. Three pillars [...]: transparency, inspection, and adaptation. – The Scrum Guide [Schwaber, 2017] “ ” [Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” 2017, [online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf Picture: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland
4. Main Research Topic 4 Likely PhD Thesis Topic Supporting agile teams in their process adaptation efforts using transparency and inspection of their own project data
5. Related Work 5 [Svensson, 2019] [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
6. Unfulfilled Potential of DDDM 6 [Svensson, 2019] ▪ Survey of software practitioners ▪ How is data used in the company for making decisions? [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
7. Software Project Data 7 Mining Repositories of Teams [Kalliamvakou et al., 2016] ▪ Project data is continuously produced by development teams ▪ Holds insights into team processes code code analyses Project Data documentation Primary purpose: Communication Opportunity: Process Insights ... [Kalliamvakou et al., 2016] Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D. M., Damian, D. “An in-depth study of the promises and perils of mining GitHub”. Empirical Software Engineering, 21(5), pp. 2035–2071. 2016. https://doi.org/10.1007/s10664-015-9393-5
8. Agile Process Improvement 8 The Retrospective Meeting ▪ Scrum’s dedicated process improvement meeting ▪ Feedback on the product as well as the process
9. The Retrospective 9 Tracking Retrospective Action Items Did we improve what we planned? commits, reviews test runs tickets static analysis Retrospective Meeting Project Data Evidence of last iteration’s work
10. Current Research Hypothesis 10 Towards Data-Informed Process Improvement ▪ Development data is already created by Agile teams during regular development activities. ▪ It holds extensive information on how team members work and collaborate. ▪ Teams can use analyses of this data to inform and track their process improvement steps.
11. Related Work 11 Mining Software Repositories ▪ Draw from MSR techniques [Dyer et al., 2013] ▪ However, mostly focus on large amounts of code □ “What do README files look like?” [Prana et al., 2018] □ “most widely used open source license?” [Dyer et al., 2013] ▪ Little research: Few repositories, intricate knowledge of creators / users [Prana et al., 2018] Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. “Categorizing the Content of GitHub README Files”. Empirical Software Engineering. 2018. https://doi.org/10.1007/s10664-018-9660-3 [Dyer et al., 2013] Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. “Boa: A language and infrastructure for analyzing ultra-large-scale software repositories”. In Proceedings - International Conference on Software Engineering. pp. 422–431. 2013. IEEE.
12. Contributions So Far 12 ▪ Development data of student teams provided actionable insights □ into team processes [1,2] □ for exercise improvement [3] □ for improving teaching efforts [4,5] ▪ Measurements from course experience and from literature [1] Matthies, C., Kowark, T., Richly, K., Uflacker, M., & Plattner, H. “How Surveys, Tutors, and Software Help to Assess Scrum Adoption”. In Proceedings of the 38th International Conference on Software Engineering Companion - ICSE ’16. pp. 313–322 2016 [2] Matthies, C., Kowark, T., Uflacker, M., & Plattner, H. “Agile Metrics for a University Software Engineering Course”. In 2016 IEEE Frontiers in Education Conference (FIE). pp. 1–5. 2016. [3] Matthies, C., Treffer, A., & Uflacker, M. “Prof. CI: Employing Continuous Integration Services and GitHub Workflows to Teach Test-Driven Development”. In 2017 IEEE Frontiers in Education Conference (FIE). pp. 1–8. 2017 [4] Matthies, C. “Scrum2kanban: Integrating Kanban and Scrum in a University Software Engineering Capstone Course”. In Proceedings of the 2nd International Workshop on Software Engineering Education for Millennials - SEEM ’18. pp. 48–55. 2018 [5] Matthies, C., Teusner, R., & Hesse, G. “Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts”. In 2018 IEEE Frontiers in Education Conference (FIE). pp. 1–9. 2018
13. Next Steps 13 Application in Industry ▪ Learnings not directly transferable to industry □ Experienced professionals working full-time □ Custom development processes ▪ Study challenges of improving processes in industry □ How are Retrospectives implemented in industry? □ What are the outcomes of Retrospectives? □ Can / are action items tracked?
14. Current Industry Study 14 Interviews with Agile Facilitators ▪ Initial interviews in companies (Wikimedia, Signavio, Nokia HERE, SAP Teams) □ Project data usage: None to Jira with custom plugins □ Little usage of data for process improvement (except Kanban cycle time) □ No mentions of using data for tracking retro issues: “regression tests for processes” ▪ Interest in application of project data analysis for everything (also for management) ▪ Retrospectives not as mature as assumed
15. Next Steps in Industry 15 Interviews with Agile Facilitators ▪ Is project data being used or considered useful? ▪ Collect and organize the Retrospective outcomes in industry □ Action items which are directly related to data vs. those that are not, e.g. interpersonal issues. ▪ Form further hypotheses on how teams can be supported with tools for process improvement
16. Summary 16
17. Image Credits 17 In order of appearance ▪ retrospective meeting by Shocho from the Noun Project (CC BY 3.0 US) ▪ Mortar Board by Mike Chum from the Noun Project (CC BY 3.0 US) ▪ Target by Arthur Shlain from the Noun Project (CC BY 3.0 US) ▪ Paper By LUTFI GANI AL ACHMAD, ID the Noun Project (CC BY 3.0 US)