2019-chicago-jug-mutation-testing.pdf

@jeanneboyarsky 1 Intro to Mutation Testing Jeanne Boyarsky Monday, November
4, 2019 Chicago Java User Group speakerdeck.com/boyarsky

@jeanneboyarsky Shout out to the selection process! 2

@jeanneboyarsky Pause for a Commercial 3

@jeanneboyarsky Why write tests? 4 It used to work!

@jeanneboyarsky How know if tests are good? 5

@jeanneboyarsky If test coverage low 6 Write more tests! Not
ready for mutation testing of class

@jeanneboyarsky If test coverage high 7 All done! Perfect code!

@jeanneboyarsky Or is it? 8 Why is it saying nobody
is registered?

@jeanneboyarsky Code Review! 9

@jeanneboyarsky That’s a mutant! 10

@jeanneboyarsky Too many mutants! 11 They’re everywhere!

@jeanneboyarsky PIT to the rescue! 12

@jeanneboyarsky What is PIT? • Originally Parallel Isolated Test •
Now just PIT 13

@jeanneboyarsky Easy to run 14

@jeanneboyarsky Easy to run 15

@jeanneboyarsky The console flies by 16

@jeanneboyarsky We caught 5 mutants 17

@jeanneboyarsky More detail 18

@jeanneboyarsky The mutants live…. 19

@jeanneboyarsky Level up 20

@jeanneboyarsky Now 32 survived 21

@jeanneboyarsky PIT 22 Mutant Generation Bytecode (ASM) Test Selection Coverage
Mutant Insertion Instrumentation Mutant Detection Early Exit (fine)

@jeanneboyarsky Mutant Generation Options 1. Bytecode • Faster • May
not map to line of code • Java 8 stream support was immediate 2. Source Code • Slower • Clearer 23 Thank you to pitest.org for this material

@jeanneboyarsky Test Selection Options 1. Coverage • Fast - picks
right test 2. Naive • Slow - brute force 3. Convention • Medium - uses naming conventions 24 Thank you to pitest.org for this material

@jeanneboyarsky Mutant Insertion Options 1. Instrumentation • Fast - picks
right test 2. Classloader • Uses a lot of memory 3. Debugger Hotswap • Can be slower because debugger on 25 Thank you to pitest.org for this material

@jeanneboyarsky Mutant Detection Options 1. Early Exit (fine) • Stops
when test finds a mutant 2. Early Exit (coarse) • Stops when class finds a mutant 3. Naive • Slow - brute force 26 Thank you to pitest.org for this material

@jeanneboyarsky JUnit 5 Support • PitClipse assigned to next version
on Oct 5 27

@jeanneboyarsky JUnit 5 Support • No date for IntelliJ 28

@jeanneboyarsky JUnit 5 Support • Supported by • Maven •
Gradle • However…. I find IDE integration to be critical 29

@jeanneboyarsky Tips • JUnit tests must pass • Watch console
to see if still running • Reports on whole project • Fixing one may take care of more 30

@jeanneboyarsky For More • Code Kata: https://github.com/vmzakharov/mutate- test-kata 31

@jeanneboyarsky Demo 32

2019-chicago-jug-mutation-testing.pdf

2019-chicago-jug-mutation-testing.pdf

Jeanne Boyarsky

More Decks by Jeanne Boyarsky

Other Decks in Programming

Featured

Transcript

@jeanneboyarsky 1 Intro to Mutation Testing Jeanne Boyarsky Monday, November

@jeanneboyarsky Shout out to the selection process! 2

@jeanneboyarsky Pause for a Commercial 3

@jeanneboyarsky Why write tests? 4 It used to work!

@jeanneboyarsky How know if tests are good? 5

@jeanneboyarsky If test coverage low 6 Write more tests! Not

@jeanneboyarsky If test coverage high 7 All done! Perfect code!

@jeanneboyarsky Or is it? 8 Why is it saying nobody

@jeanneboyarsky Code Review! 9

@jeanneboyarsky That’s a mutant! 10

@jeanneboyarsky Too many mutants! 11 They’re everywhere!

@jeanneboyarsky PIT to the rescue! 12

@jeanneboyarsky What is PIT? • Originally Parallel Isolated Test •

@jeanneboyarsky Easy to run 14

@jeanneboyarsky Easy to run 15

@jeanneboyarsky The console flies by 16

@jeanneboyarsky We caught 5 mutants 17

@jeanneboyarsky More detail 18

@jeanneboyarsky The mutants live…. 19

@jeanneboyarsky Level up 20

@jeanneboyarsky Now 32 survived 21

@jeanneboyarsky PIT 22 Mutant Generation Bytecode (ASM) Test Selection Coverage

@jeanneboyarsky Mutant Generation Options 1. Bytecode • Faster • May

@jeanneboyarsky Test Selection Options 1. Coverage • Fast - picks

@jeanneboyarsky Mutant Insertion Options 1. Instrumentation • Fast - picks

@jeanneboyarsky Mutant Detection Options 1. Early Exit (fine) • Stops

@jeanneboyarsky JUnit 5 Support • PitClipse assigned to next version

@jeanneboyarsky JUnit 5 Support • No date for IntelliJ 28

@jeanneboyarsky JUnit 5 Support • Supported by • Maven •

@jeanneboyarsky Tips • JUnit tests must pass • Watch console

@jeanneboyarsky For More • Code Kata: https://github.com/vmzakharov/mutate- test-kata 31

@jeanneboyarsky Demo 32