Intro to Mutation Testing

Presented at Oracle Code NYC 2018. This is a repeat of the session I gave at JavaOne in 2017.

Transcript

1. Intro to Mutation Testing speakerdeck.com/boyarsky Twitter @jeanneboyarsky Blog: http://www.selikoff.net Jeanne Boyarsky Oracle Code NYC – March 8, 2018
2. @jeanneboyarsky About Me 15 years as •  Java Developer •  JUnit user
3. The Plan 1.  About mutation testing 2.  Live demo 3.  A look at types of mutants 4.  Profit??? @jeanneboyarsky
4. Why write tests? @jeanneboyarsky It used to work!
5. How know if tests are good? @jeanneboyarsky
6. If test coverage low @jeanneboyarsky Write more tests! Not ready for mutation testing of class
7. If test coverage high @jeanneboyarsky All done! Perfect code!
8. Or is it? Why is it saying nobody is registered? @jeanneboyarsky
9. Code Review! @jeanneboyarsky
10. That’s a mutant! @jeanneboyarsky
11. Too many mutants! They’re everywhere! @jeanneboyarsky
12. PitClipse to the rescue! @jeanneboyarsky
13. @jeanneboyarsky PIT Integration Almost
14. JUnit 5 •  Unofficial Fork: https://github.com/tobiasstadler/pitest-junit5-plugin •  It’s coming: https://github.com/hcoles/pitest/issues/284 https://github.com/hcoles/pitest/issues/398 @jeanneboyarsky
15. Easy to run @jeanneboyarsky
16. The console flies by @jeanneboyarsky
17. Wait, we just had 100% coverage @jeanneboyarsky
18. We caught 5 mutants @jeanneboyarsky
19. More detail @jeanneboyarsky
20. Level up @jeanneboyarsky
21. Now 23 survived @jeanneboyarsky
22. Let’s try it! @jeanneboyarsky
23. Observations •  JUnit tests must pass •  Fixing one may take care of more •  Found a bug •  Better tests @jeanneboyarsky
24. Now with all mutators @jeanneboyarsky
25. Observations •  Better tests •  Unused production code @jeanneboyarsky
26. Examples of Conditional Mutants @jeanneboyarsky Conditionals Boundary <= vs < Negative Conditionals <= vs > Void Method Call Removes method calls
27. Examples of Math Mutants @jeanneboyarsky Math * vs / Increments ++ vs -- Invert Negatives x vs -x
28. Examples of Other Mutants @jeanneboyarsky Return Values x vs x+1 Disabled by default (false positives, duplicates or crashes) Constructor Calls Inline Constants Non Void Method Calls Remove Conditionals Experimental Member Variables Switch
29. Tips for bigger projects •  Watch console to see if still running •  Reports on whole project – drill down/split up •  Exclude long running tests •  Experimental feature: incremental analysis @jeanneboyarsky
30. Review Code Coverage Mutation Testing Requirements
31. More info •  http://pitest.org/ •  https://github.com/philglover/pitclipse •  https://dzone.com/articles/introduction-mutation @jeanneboyarsky
32. @jeanneboyarsky