Mutation Testing - Backend Meetup @ STRV

Transcript

1. MUTATION TESTING
   Ruan Martinelli, Backend Developer at STRV
   

   View Slide

2. 2
   https://en.wikipedia.org/wiki/Category:Software_testing
   

   View Slide

3. 1. REPLACE "AND" WITH "OR"
   3
   

   View Slide

4. 2. REPLACE "+" WITH "-"
   4
   

   View Slide

5. 3. CODE COVERAGE
   5
   

   View Slide

6. 3. CODE COVERAGE
   6
   

   View Slide

7. 3. CODE COVERAGE
   7
   

   View Slide

8. “What if I told you there is a codebase
   that is not yours and it passes all your
   tests?
   What if told you there are 200?”
   8
   

   View Slide

9. Mutations are small "faults" or "bugs" inserted into your code.
   When a mutation is applied to the code, a mutant is created.
   Mutation testing:
   1. Change your code in small ways
   2. Run the test suite once for each changed version
   3. Analyze results
   - If one or more test breaks, it’s said that the mutant was killed
   - If all tests passes, it’s said that the mutant survived
   MUTATIONS, MUTANTS & MUTATION TESTING
   9
   

   View Slide

10. 4. LIST OF COMMON MUTATIONS
    10
    

    View Slide

11. List of Mutation Testing tools:
    ● Java - PIT
    ● Ruby - Mutant
    ● PHP - Infection
    ● C# - VisualMutator & Stryker.NET
    ● Python - MutPy
    ● JavaScript - Stryker
    TOOLING
    11
    

    View Slide

12. DEMO
    12
    

    View Slide

13. KOA.JS
    13
    ● Koa.js has 345 tests which take ~3s to run
    ● 4 files were mutated, 793 mutants were created
    ● Total execution time was 12m28s (8 cores)
    

    View Slide

14. KOA.JS
    14
    

    View Slide

15. KOA.JS
    15
    

    View Slide

16. KOA.JS
    16
    

    View Slide

17. KOA.JS
    17
    

    View Slide

18. KOA.JS
    18
    

    View Slide

19. KOA.JS
    19
    

    View Slide

20. KOA.JS
    20
    

    View Slide

21. Mutation testing is resource and time-consuming
    ● Run tests in the cloud periodically
    ● Parallelism and distributed computing
    ● Diff-based approach
    ● AI & Machine learning
    ● Run only on critical parts of your system
    GOING FURTHER
    21
    

    View Slide

22. THANK YOU!
    Ruan Martinelli / ruan.martinelli@strv.com / @ruanmartinelli
    22
    

    View Slide

23. QUESTIONS
    

    View Slide