Assessing Technical Debt In Automated Tests Using CodeScene

Test automation promises several advantages such as shorter lead times, higher code quality, and an executable documentation of the system's behavior. However, test automation won't deliver on those promises unless the quality of the automated test code itself is maintained, and to manually inspect the evolution of thousands of tests that change on a daily basis is impractical at best. This paper investigates how CodeScene -- a tool for predictive analyses and visualizations – could be used to identify technical debt in automated test code.

Transcript

1. Assessing Technical Debt In Automated Tests Using CodeScene @AdamTornhill NEXTA 2018
2. Research Question How can CodeScene -- a tool for predictive analyses and visualizations – be used to identify technical debt in automated test code? @AdamTornhill CodeScene Demo: https://www.youtube.com/watch?v=SWFwPkgLcpo
3. Bad Code != Technical Debt [1] G.K. Gill, & C.F. Kemerer, “Cyclomatic complexity density and software maintenance productivity”, IEEE Transactions on Software Engineering (Volume: 17, Issue: 12), 1991 [2] B. Baker. “On Finding Duplication and Near-Duplication in Large Software Systems”, Proceedings of the Second Working Conference on Reverse Engineering, 1995 Static Analysis detects problematic code [1][2] …but problematic code is not necessarily technical debt. @AdamTornhill
4. There’s More to Code than Code [1] M. John, F. Maurer, & B. Tessem, “Human and social factors of software engineering: workshop summary”, ACM SIGSOFT Software Engineering Notes (Volume 30, Issue 4), 2005 Social factors have a strong impact on the project outcome [1] @AdamTornhill
5. View Static Code Properties through a Behavioural Lens Commit: b557ca5 Date: 2016-02-12 Author: Kevin Flynn Fix behavior of StartsWithPrefix 8 27 src/Mvc.Abstractions/ModelBinding/ModelStateDictionary.cs 1 10 src/Mvc.Core/ControllerBase.cs 1 1 src/Mvc.Core/Internal/ElementalValueProvider.cs 1 39 src/Mvc.Core/Internal/PrefixContainer.cs Commit: fd6d28d Date 2016-02-10 Author: Professor Falken Make AddController not overwrite existing IControllerTypeProvider 8 1 src/Core/Internal/ControllersAsServices.cs 48 0 test/Core.Test/Internal/ControllerAsServicesTest.cs 13 0 test/Mvc.FunctionalTests/ControllerFromServicesTests.cs Commit: 910f013 Date :2016-02-05 Author Lisbeth Salander Fixes #4050: Throw an exception when media types are empty. 20 1 src/Mvc.Core/Formatters/InputFormatter.cs Social Information A Time Dimension
6. Case Study: ASP.NET Core MVC An open source project from Microsoft: https://github.com/aspnet/Mvc 300,000 lines of code developed by 125 authors over three years Large investment in automated tests: • 68,000 lines of application code. • 155,000 lines of code for automated tests. @AdamTornhill
7. Hotspots Complicated Code that the Organisation has to work with @AdamTornhill
8. @AdamTornhill Prioritize Refactoring Candidates with Machine Learning Primary Targets Secondary Targets
10. M. D’Ambros, M. Lanza, and H Gall. Fractal Figures: Visualizing Development Effort for CVS Entities. Visualizing Software for Understanding and Analysis, 2005. VISSOFT 2005. 3rd IEEE International Workshop on. 1–6, 2005. Increasing Complexity
11. Complexity Calculation and Trends What we normally care about… A simpler view! A. Hindle, M.W. Godfrey, and R.C. Holt. Reading Beside the Lines: Indentation as a Proxy for Complexity Metrics. Program Comprehension, 2008. ICPC 2008. The 16th IEEE International Conference on. IEEE Computer Society Press, Washington, DC, 2008. @AdamTornhill
12. Inspecting the Refactoring Candidate A growing problem since April 2016 @AdamTornhill
13. Actionable Information with CodeScene’s X-Ray The Evolution of the CreateInvoker() method
14. Evaluation: Code Inspection of the Hotspot [1] V. Garousi, & B. Küçük, “Smells in software test code: A survey of knowledge in industry and academia”, Journal of Systems and Software (Volume 138), 2018. Code Smells: Mock Happy and Mocking Everything [1]
15. Evaluation: Automated Code Inspection of the Hotspot Inspired by @AdamTornhill Score: A to E where A is the best.
16. Summary and Conclusions Identify: The identified hotspot is a likely maintenance bottleneck. Prioritise: CodeScene prioritised 197 lines of code as an initial refactoring target, which allows iterative refinements. Communicate: The tool has the potential to improve the effectiveness of test automation by letting stakeholders share a view of what the code quality looks like. @AdamTornhill Try CodeScene at https://codescene.io/