Upgrade to Pro — share decks privately, control downloads, hide ads and more …

2019-chicago-jug-mutation-testing.pdf

 2019-chicago-jug-mutation-testing.pdf

Jeanne Boyarsky

November 03, 2019
Tweet

More Decks by Jeanne Boyarsky

Other Decks in Programming

Transcript

  1. @jeanneboyarsky 1
    Intro to Mutation Testing
    Jeanne Boyarsky
    Monday, November 4, 2019
    Chicago Java User Group
    speakerdeck.com/boyarsky

    View Slide

  2. @jeanneboyarsky
    Shout out to the selection process!
    2

    View Slide

  3. @jeanneboyarsky
    Pause for a Commercial
    3

    View Slide

  4. @jeanneboyarsky
    Why write tests?
    4
    It used
    to
    work!

    View Slide

  5. @jeanneboyarsky
    How know if tests are good?
    5

    View Slide

  6. @jeanneboyarsky
    If test coverage low
    6
    Write more tests!
    Not ready
    for mutation
    testing of class

    View Slide

  7. @jeanneboyarsky
    If test coverage high
    7
    All done!
    Perfect
    code!

    View Slide

  8. @jeanneboyarsky
    Or is it?
    8
    Why is it
    saying
    nobody is
    registered?

    View Slide

  9. @jeanneboyarsky
    Code Review!
    9

    View Slide

  10. @jeanneboyarsky
    That’s a mutant!
    10

    View Slide

  11. @jeanneboyarsky
    Too many mutants!
    11
    They’re
    everywhere!

    View Slide

  12. @jeanneboyarsky
    PIT to the rescue!
    12

    View Slide

  13. @jeanneboyarsky
    What is PIT?
    • Originally Parallel Isolated Test
    • Now just PIT
    13

    View Slide

  14. @jeanneboyarsky
    Easy to run
    14

    View Slide

  15. @jeanneboyarsky
    Easy to run
    15

    View Slide

  16. @jeanneboyarsky
    The console flies by
    16

    View Slide

  17. @jeanneboyarsky
    We caught 5 mutants
    17

    View Slide

  18. @jeanneboyarsky
    More detail
    18

    View Slide

  19. @jeanneboyarsky
    The mutants live….
    19

    View Slide

  20. @jeanneboyarsky
    Level up
    20

    View Slide

  21. @jeanneboyarsky
    Now 32 survived
    21

    View Slide

  22. @jeanneboyarsky
    PIT
    22
    Mutant Generation Bytecode (ASM)
    Test Selection Coverage
    Mutant Insertion Instrumentation
    Mutant Detection Early Exit (fine)

    View Slide

  23. @jeanneboyarsky
    Mutant Generation Options
    1. Bytecode
    • Faster
    • May not map to line of code
    • Java 8 stream support was immediate
    2. Source Code
    • Slower
    • Clearer
    23
    Thank you to pitest.org for this material

    View Slide

  24. @jeanneboyarsky
    Test Selection Options
    1. Coverage
    • Fast - picks right test
    2. Naive
    • Slow - brute force
    3. Convention
    • Medium - uses naming conventions
    24
    Thank you to pitest.org for this material

    View Slide

  25. @jeanneboyarsky
    Mutant Insertion Options
    1. Instrumentation
    • Fast - picks right test
    2. Classloader
    • Uses a lot of memory
    3. Debugger Hotswap
    • Can be slower because debugger on
    25
    Thank you to pitest.org for this material

    View Slide

  26. @jeanneboyarsky
    Mutant Detection Options
    1. Early Exit (fine)
    • Stops when test finds a mutant
    2. Early Exit (coarse)
    • Stops when class finds a mutant
    3. Naive
    • Slow - brute force
    26
    Thank you to pitest.org for this material

    View Slide

  27. @jeanneboyarsky
    JUnit 5 Support
    • PitClipse assigned to next version on
    Oct 5
    27

    View Slide

  28. @jeanneboyarsky
    JUnit 5 Support
    • No date for IntelliJ
    28

    View Slide

  29. @jeanneboyarsky
    JUnit 5 Support
    • Supported by
    • Maven
    • Gradle
    • However…. I find IDE integration to be
    critical
    29

    View Slide

  30. @jeanneboyarsky
    Tips
    • JUnit tests must pass
    • Watch console to see if still running
    • Reports on whole project
    • Fixing one may take care of more
    30

    View Slide

  31. @jeanneboyarsky
    For More
    • Code Kata:
    https://github.com/vmzakharov/mutate-
    test-kata
    31

    View Slide

  32. @jeanneboyarsky
    Demo
    32

    View Slide