Assessing Technical Debt In Automated Tests Using CodeScene @AdamTornhill NEXTA 2018

Research Question How can CodeScene -- a tool for predictive analyses and visualizations – be used to identify technical debt in automated test code? @AdamTornhill CodeScene Demo: https://www.youtube.com/watch?v=SWFwPkgLcpo

Bad Code != Technical Debt [1] G.K. Gill, & C.F. Kemerer, “Cyclomatic complexity density and software maintenance productivity”, IEEE Transactions on Software Engineering (Volume: 17, Issue: 12), 1991 [2] B. Baker. “On Finding Duplication and Near-Duplication in Large Software Systems”, Proceedings of the Second Working Conference on Reverse Engineering, 1995 Static Analysis detects problematic code [1][2] …but problematic code is not necessarily technical debt. @AdamTornhill

There’s More to Code than Code [1] M. John, F. Maurer, & B. Tessem, “Human and social factors of software engineering: workshop summary”, ACM SIGSOFT Software Engineering Notes (Volume 30, Issue 4), 2005 Social factors have a strong impact on the project outcome [1] @AdamTornhill

View Static Code Properties through a Behavioural Lens Commit: b557ca5 Date: 2016-02-12 Author: Kevin Flynn Fix behavior of StartsWithPrefix 8 27 src/Mvc.Abstractions/ModelBinding/ModelStateDictionary.cs 1 10 src/Mvc.Core/ControllerBase.cs 1 1 src/Mvc.Core/Internal/ElementalValueProvider.cs 1 39 src/Mvc.Core/Internal/PrefixContainer.cs Commit: fd6d28d Date 2016-02-10 Author: Professor Falken Make AddController not overwrite existing IControllerTypeProvider 8 1 src/Core/Internal/ControllersAsServices.cs 48 0 test/Core.Test/Internal/ControllerAsServicesTest.cs 13 0 test/Mvc.FunctionalTests/ControllerFromServicesTests.cs Commit: 910f013 Date :2016-02-05 Author Lisbeth Salander Fixes #4050: Throw an exception when media types are empty. 20 1 src/Mvc.Core/Formatters/InputFormatter.cs Social Information A Time Dimension

Case Study: ASP.NET Core MVC An open source project from Microsoft: https://github.com/aspnet/Mvc 300,000 lines of code developed by 125 authors over three years Large investment in automated tests: • 68,000 lines of application code. • 155,000 lines of code for automated tests. @AdamTornhill

Hotspots Complicated Code that the Organisation has to work with @AdamTornhill

@AdamTornhill Prioritize Refactoring Candidates with Machine Learning Primary Targets Secondary Targets

No content

M. D’Ambros, M. Lanza, and H Gall. Fractal Figures: Visualizing Development Effort for CVS Entities. Visualizing Software for Understanding and Analysis, 2005. VISSOFT 2005. 3rd IEEE International Workshop on. 1–6, 2005. Increasing Complexity

Complexity Calculation and Trends What we normally care about… A simpler view! A. Hindle, M.W. Godfrey, and R.C. Holt. Reading Beside the Lines: Indentation as a Proxy for Complexity Metrics. Program Comprehension, 2008. ICPC 2008. The 16th IEEE International Conference on. IEEE Computer Society Press, Washington, DC, 2008. @AdamTornhill

Inspecting the Refactoring Candidate A growing problem since April 2016 @AdamTornhill

Actionable Information with CodeScene’s X-Ray The Evolution of the CreateInvoker() method

Evaluation: Code Inspection of the Hotspot [1] V. Garousi, & B. Küçük, “Smells in software test code: A survey of knowledge in industry and academia”, Journal of Systems and Software (Volume 138), 2018. Code Smells: Mock Happy and Mocking Everything [1]

Evaluation: Automated Code Inspection of the Hotspot Inspired by @AdamTornhill Score: A to E where A is the best.

Summary and Conclusions Identify: The identified hotspot is a likely maintenance bottleneck. Prioritise: CodeScene prioritised 197 lines of code as an initial refactoring target, which allows iterative refinements. Communicate: The tool has the potential to improve the effectiveness of test automation by letting stakeholders share a view of what the code quality looks like. @AdamTornhill Try CodeScene at https://codescene.io/