Intro to Mutation Testing

Transcript

1. Intro to Mutation Testing speakerdeck.com/boyarsky Twitter @jeanneboyarsky Blog: http://www.selikoff.net Jeanne

   Boyarsky Oracle Code NYC – March 8, 2018

2. @jeanneboyarsky About Me 15 years as •  Java Developer • 

   JUnit user

3. The Plan 1.  About mutation testing 2.  Live demo 3. 

   A look at types of mutants 4.  Profit??? @jeanneboyarsky

4. Why write tests? @jeanneboyarsky It used to work!

5. How know if tests are good? @jeanneboyarsky

6. If test coverage low @jeanneboyarsky Write more tests! Not ready

   for mutation testing of class

7. If test coverage high @jeanneboyarsky All done! Perfect code!

8. Or is it? Why is it saying nobody is registered?

   @jeanneboyarsky

9. Code Review! @jeanneboyarsky

10. That’s a mutant! @jeanneboyarsky

11. Too many mutants! They’re everywhere! @jeanneboyarsky

12. PitClipse to the rescue! @jeanneboyarsky

13. @jeanneboyarsky PIT Integration Almost

14. JUnit 5 •  Unofficial Fork: https://github.com/tobiasstadler/pitest-junit5-plugin •  It’s coming: https://github.com/hcoles/pitest/issues/284

    https://github.com/hcoles/pitest/issues/398 @jeanneboyarsky

15. Easy to run @jeanneboyarsky

16. The console flies by @jeanneboyarsky

17. Wait, we just had 100% coverage @jeanneboyarsky

18. We caught 5 mutants @jeanneboyarsky

19. More detail @jeanneboyarsky

20. Level up @jeanneboyarsky

21. Now 23 survived @jeanneboyarsky

22. Let’s try it! @jeanneboyarsky

23. Observations •  JUnit tests must pass •  Fixing one may

    take care of more •  Found a bug •  Better tests @jeanneboyarsky

24. Now with all mutators @jeanneboyarsky

25. Observations •  Better tests •  Unused production code @jeanneboyarsky

26. Examples of Conditional Mutants @jeanneboyarsky Conditionals Boundary <= vs <

    Negative Conditionals <= vs > Void Method Call Removes method calls

27. Examples of Math Mutants @jeanneboyarsky Math * vs / Increments

    ++ vs -- Invert Negatives x vs -x

28. Examples of Other Mutants @jeanneboyarsky Return Values x vs x+1

    Disabled by default (false positives, duplicates or crashes) Constructor Calls Inline Constants Non Void Method Calls Remove Conditionals Experimental Member Variables Switch

29. Tips for bigger projects •  Watch console to see if

    still running •  Reports on whole project – drill down/split up •  Exclude long running tests •  Experimental feature: incremental analysis @jeanneboyarsky

30. Review Code Coverage Mutation Testing Requirements

31. More info •  http://pitest.org/ •  https://github.com/philglover/pitclipse •  https://dzone.com/articles/introduction-mutation @jeanneboyarsky

32. @jeanneboyarsky