Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Intro to Mutation Testing

Intro to Mutation Testing

Presented at Oracle Code NYC 2018. This is a repeat of the session I gave at JavaOne in 2017.


Jeanne Boyarsky

March 05, 2018


  1. Intro to Mutation Testing speakerdeck.com/boyarsky Twitter @jeanneboyarsky Blog: http://www.selikoff.net Jeanne

    Boyarsky Oracle Code NYC – March 8, 2018
  2. @jeanneboyarsky About Me 15 years as •  Java Developer • 

    JUnit user
  3. The Plan 1.  About mutation testing 2.  Live demo 3. 

    A look at types of mutants 4.  Profit??? @jeanneboyarsky
  4. Why write tests? @jeanneboyarsky It used to work!

  5. How know if tests are good? @jeanneboyarsky

  6. If test coverage low @jeanneboyarsky Write more tests! Not ready

    for mutation testing of class
  7. If test coverage high @jeanneboyarsky All done! Perfect code!

  8. Or is it? Why is it saying nobody is registered?

  9. Code Review! @jeanneboyarsky

  10. That’s a mutant! @jeanneboyarsky

  11. Too many mutants! They’re everywhere! @jeanneboyarsky

  12. PitClipse to the rescue! @jeanneboyarsky

  13. @jeanneboyarsky PIT Integration Almost

  14. JUnit 5 •  Unofficial Fork: https://github.com/tobiasstadler/pitest-junit5-plugin •  It’s coming: https://github.com/hcoles/pitest/issues/284

    https://github.com/hcoles/pitest/issues/398 @jeanneboyarsky
  15. Easy to run @jeanneboyarsky

  16. The console flies by @jeanneboyarsky

  17. Wait, we just had 100% coverage @jeanneboyarsky

  18. We caught 5 mutants @jeanneboyarsky

  19. More detail @jeanneboyarsky

  20. Level up @jeanneboyarsky

  21. Now 23 survived @jeanneboyarsky

  22. Let’s try it! @jeanneboyarsky

  23. Observations •  JUnit tests must pass •  Fixing one may

    take care of more •  Found a bug •  Better tests @jeanneboyarsky
  24. Now with all mutators @jeanneboyarsky

  25. Observations •  Better tests •  Unused production code @jeanneboyarsky

  26. Examples of Conditional Mutants @jeanneboyarsky Conditionals Boundary <= vs <

    Negative Conditionals <= vs > Void Method Call Removes method calls
  27. Examples of Math Mutants @jeanneboyarsky Math * vs / Increments

    ++ vs -- Invert Negatives x vs -x
  28. Examples of Other Mutants @jeanneboyarsky Return Values x vs x+1

    Disabled by default (false positives, duplicates or crashes) Constructor Calls Inline Constants Non Void Method Calls Remove Conditionals Experimental Member Variables Switch
  29. Tips for bigger projects •  Watch console to see if

    still running •  Reports on whole project – drill down/split up •  Exclude long running tests •  Experimental feature: incremental analysis @jeanneboyarsky
  30. Review Code Coverage Mutation Testing Requirements

  31. More info •  http://pitest.org/ •  https://github.com/philglover/pitclipse •  https://dzone.com/articles/introduction-mutation @jeanneboyarsky

  32. @jeanneboyarsky