Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mutation Testing - Backend Meetup @ STRV

Mutation Testing - Backend Meetup @ STRV

Ruan Martinelli

April 09, 2020
Tweet

More Decks by Ruan Martinelli

Other Decks in Programming

Transcript

  1. MUTATION TESTING
    Ruan Martinelli, Backend Developer at STRV

    View Slide

  2. 2
    https://en.wikipedia.org/wiki/Category:Software_testing

    View Slide

  3. 1. REPLACE "AND" WITH "OR"
    3

    View Slide

  4. 2. REPLACE "+" WITH "-"
    4

    View Slide

  5. 3. CODE COVERAGE
    5

    View Slide

  6. 3. CODE COVERAGE
    6

    View Slide

  7. 3. CODE COVERAGE
    7

    View Slide

  8. “What if I told you there is a codebase
    that is not yours and it passes all your
    tests?
    What if told you there are 200?”
    8

    View Slide

  9. Mutations are small "faults" or "bugs" inserted into your code.
    When a mutation is applied to the code, a mutant is created.
    Mutation testing:
    1. Change your code in small ways
    2. Run the test suite once for each changed version
    3. Analyze results
    - If one or more test breaks, it’s said that the mutant was killed
    - If all tests passes, it’s said that the mutant survived
    MUTATIONS, MUTANTS & MUTATION TESTING
    9

    View Slide

  10. 4. LIST OF COMMON MUTATIONS
    10

    View Slide

  11. List of Mutation Testing tools:
    ● Java - PIT
    ● Ruby - Mutant
    ● PHP - Infection
    ● C# - VisualMutator & Stryker.NET
    ● Python - MutPy
    ● JavaScript - Stryker
    TOOLING
    11

    View Slide

  12. DEMO
    12

    View Slide

  13. KOA.JS
    13
    ● Koa.js has 345 tests which take ~3s to run
    ● 4 files were mutated, 793 mutants were created
    ● Total execution time was 12m28s (8 cores)

    View Slide

  14. KOA.JS
    14

    View Slide

  15. KOA.JS
    15

    View Slide

  16. KOA.JS
    16

    View Slide

  17. KOA.JS
    17

    View Slide

  18. KOA.JS
    18

    View Slide

  19. KOA.JS
    19

    View Slide

  20. KOA.JS
    20

    View Slide

  21. Mutation testing is resource and time-consuming
    ● Run tests in the cloud periodically
    ● Parallelism and distributed computing
    ● Diff-based approach
    ● AI & Machine learning
    ● Run only on critical parts of your system
    GOING FURTHER
    21

    View Slide

  22. THANK YOU!
    Ruan Martinelli / [email protected] / @ruanmartinelli
    22

    View Slide

  23. QUESTIONS

    View Slide