Mutation Testing - Backend Meetup @ STRV

Mutation Testing - Backend Meetup @ STRV

76c84b310153282b760596d4abcf8168?s=128

Ruan Martinelli

April 09, 2020
Tweet

Transcript

  1. MUTATION TESTING Ruan Martinelli, Backend Developer at STRV

  2. 2 https://en.wikipedia.org/wiki/Category:Software_testing

  3. 1. REPLACE "AND" WITH "OR" 3

  4. 2. REPLACE "+" WITH "-" 4

  5. 3. CODE COVERAGE 5

  6. 3. CODE COVERAGE 6

  7. 3. CODE COVERAGE 7

  8. “What if I told you there is a codebase that

    is not yours and it passes all your tests? What if told you there are 200?” 8
  9. Mutations are small "faults" or "bugs" inserted into your code.

    When a mutation is applied to the code, a mutant is created. Mutation testing: 1. Change your code in small ways 2. Run the test suite once for each changed version 3. Analyze results - If one or more test breaks, it’s said that the mutant was killed - If all tests passes, it’s said that the mutant survived MUTATIONS, MUTANTS & MUTATION TESTING 9
  10. 4. LIST OF COMMON MUTATIONS 10

  11. List of Mutation Testing tools: • Java - PIT •

    Ruby - Mutant • PHP - Infection • C# - VisualMutator & Stryker.NET • Python - MutPy • JavaScript - Stryker TOOLING 11
  12. DEMO 12

  13. KOA.JS 13 • Koa.js has 345 tests which take ~3s

    to run • 4 files were mutated, 793 mutants were created • Total execution time was 12m28s (8 cores)
  14. KOA.JS 14

  15. KOA.JS 15

  16. KOA.JS 16

  17. KOA.JS 17

  18. KOA.JS 18

  19. KOA.JS 19

  20. KOA.JS 20

  21. Mutation testing is resource and time-consuming • Run tests in

    the cloud periodically • Parallelism and distributed computing • Diff-based approach • AI & Machine learning • Run only on critical parts of your system GOING FURTHER 21
  22. THANK YOU! Ruan Martinelli / ruan.martinelli@strv.com / @ruanmartinelli 22

  23. QUESTIONS