Mutation Testing - Backend Meetup @ STRV

Transcript

1. MUTATION TESTING Ruan Martinelli, Backend Developer at STRV

2. 2 https://en.wikipedia.org/wiki/Category:Software_testing

3. 1. REPLACE "AND" WITH "OR" 3

4. 2. REPLACE "+" WITH "-" 4

5. 3. CODE COVERAGE 5

6. 3. CODE COVERAGE 6

7. 3. CODE COVERAGE 7

8. “What if I told you there is a codebase that

   is not yours and it passes all your tests? What if told you there are 200?” 8

9. Mutations are small "faults" or "bugs" inserted into your code.

   When a mutation is applied to the code, a mutant is created. Mutation testing: 1. Change your code in small ways 2. Run the test suite once for each changed version 3. Analyze results - If one or more test breaks, it’s said that the mutant was killed - If all tests passes, it’s said that the mutant survived MUTATIONS, MUTANTS & MUTATION TESTING 9

10. 4. LIST OF COMMON MUTATIONS 10

11. List of Mutation Testing tools: • Java - PIT •

    Ruby - Mutant • PHP - Infection • C# - VisualMutator & Stryker.NET • Python - MutPy • JavaScript - Stryker TOOLING 11

12. DEMO 12

13. KOA.JS 13 • Koa.js has 345 tests which take ~3s

    to run • 4 files were mutated, 793 mutants were created • Total execution time was 12m28s (8 cores)

14. KOA.JS 14

15. KOA.JS 15

16. KOA.JS 16

17. KOA.JS 17

18. KOA.JS 18

19. KOA.JS 19

20. KOA.JS 20

21. Mutation testing is resource and time-consuming • Run tests in

    the cloud periodically • Parallelism and distributed computing • Diff-based approach • AI & Machine learning • Run only on critical parts of your system GOING FURTHER 21

22. THANK YOU! Ruan Martinelli / ruan.martinelli@strv.com / @ruanmartinelli 22

23. QUESTIONS