Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Assessing Technical Debt In Automated Tests Using CodeScene

Assessing Technical Debt In Automated Tests Using CodeScene

Test automation promises several advantages such as shorter lead times, higher code quality, and an executable documentation of the system's behavior. However, test automation won't deliver on those promises unless the quality of the automated test code itself is maintained, and to manually inspect the evolution of thousands of tests that change on a daily basis is impractical at best. This paper investigates how CodeScene -- a tool for predictive analyses and visualizations – could be used to identify technical debt in automated test code.

Adam Tornhill

April 09, 2018
Tweet

More Decks by Adam Tornhill

Other Decks in Programming

Transcript

  1. Assessing
    Technical Debt In Automated Tests
    Using CodeScene
    @AdamTornhill NEXTA 2018

    View full-size slide

  2. Research Question
    How can CodeScene -- a tool for predictive analyses and visualizations –
    be used to identify technical debt in automated test code?
    @AdamTornhill
    CodeScene Demo: https://www.youtube.com/watch?v=SWFwPkgLcpo

    View full-size slide

  3. Bad Code != Technical Debt
    [1] G.K. Gill, & C.F. Kemerer, “Cyclomatic complexity density and software maintenance
    productivity”, IEEE Transactions on Software Engineering (Volume: 17, Issue: 12), 1991
    [2] B. Baker. “On Finding Duplication and Near-Duplication in Large Software Systems”,
    Proceedings of the Second Working Conference on Reverse Engineering, 1995
    Static Analysis detects problematic code [1][2]
    …but problematic code is not necessarily technical debt.
    @AdamTornhill

    View full-size slide

  4. There’s More to Code than Code
    [1] M. John, F. Maurer, & B. Tessem, “Human and social factors of software engineering:
    workshop summary”, ACM SIGSOFT Software Engineering Notes (Volume 30, Issue 4), 2005
    Social factors have a strong impact on the project outcome [1]
    @AdamTornhill

    View full-size slide

  5. View Static Code Properties through a Behavioural Lens
    Commit: b557ca5
    Date: 2016-02-12
    Author: Kevin Flynn
    Fix behavior of StartsWithPrefix
    8 27 src/Mvc.Abstractions/ModelBinding/ModelStateDictionary.cs
    1 10 src/Mvc.Core/ControllerBase.cs
    1 1 src/Mvc.Core/Internal/ElementalValueProvider.cs
    1 39 src/Mvc.Core/Internal/PrefixContainer.cs
    Commit: fd6d28d
    Date 2016-02-10
    Author: Professor Falken
    Make AddController not overwrite existing IControllerTypeProvider
    8 1 src/Core/Internal/ControllersAsServices.cs
    48 0 test/Core.Test/Internal/ControllerAsServicesTest.cs
    13 0 test/Mvc.FunctionalTests/ControllerFromServicesTests.cs
    Commit: 910f013
    Date :2016-02-05
    Author Lisbeth Salander
    Fixes #4050: Throw an exception when media types are empty.
    20 1 src/Mvc.Core/Formatters/InputFormatter.cs
    Social Information
    A Time Dimension

    View full-size slide

  6. Case Study: ASP.NET Core MVC
    An open source project from Microsoft: https://github.com/aspnet/Mvc
    300,000 lines of code developed by 125 authors over three years
    Large investment in automated tests:
    • 68,000 lines of application code.
    • 155,000 lines of code for automated tests.
    @AdamTornhill

    View full-size slide

  7. Hotspots
    Complicated Code that the Organisation has to work with
    @AdamTornhill

    View full-size slide

  8. @AdamTornhill
    Prioritize Refactoring Candidates with Machine Learning
    Primary Targets
    Secondary Targets

    View full-size slide

  9. M. D’Ambros, M. Lanza, and H Gall. Fractal Figures: Visualizing Development Effort for CVS
    Entities. Visualizing Software for Understanding and Analysis, 2005. VISSOFT 2005. 3rd IEEE
    International Workshop on. 1–6, 2005.
    Increasing Complexity

    View full-size slide

  10. Complexity Calculation and Trends
    What we normally
    care about…
    A simpler view!
    A. Hindle, M.W. Godfrey, and R.C. Holt. Reading Beside the Lines: Indentation as a Proxy for
    Complexity Metrics. Program Comprehension, 2008. ICPC 2008. The 16th IEEE International
    Conference on. IEEE Computer Society Press, Washington, DC, 2008.
    @AdamTornhill

    View full-size slide

  11. Inspecting the Refactoring Candidate
    A growing problem since April 2016
    @AdamTornhill

    View full-size slide

  12. Actionable Information with CodeScene’s X-Ray
    The Evolution of the CreateInvoker() method

    View full-size slide

  13. Evaluation: Code Inspection of the Hotspot
    [1] V. Garousi, & B. Küçük, “Smells in software test code: A survey of knowledge in
    industry and academia”, Journal of Systems and Software (Volume 138), 2018.
    Code Smells: Mock Happy and Mocking Everything [1]

    View full-size slide

  14. Evaluation: Automated Code Inspection of the Hotspot
    Inspired by
    @AdamTornhill
    Score: A to E where A is the best.

    View full-size slide

  15. Summary and Conclusions
    Identify: The identified hotspot is a likely maintenance bottleneck.
    Prioritise: CodeScene prioritised 197 lines of code as an initial refactoring
    target, which allows iterative refinements.
    Communicate: The tool has the potential to improve the effectiveness of test
    automation by letting stakeholders share a view of what the code quality looks
    like.
    @AdamTornhill
    Try CodeScene at https://codescene.io/

    View full-size slide