Property-based testing in a nutshell

Talk given at Cork Ministry of Test

Transcript

1. Julien Cretel @_jubobs_ Jubobs Jubobs Property-based testing in a
2. All those example-based tests pass… but the implementation contains a bug!
3. A suite of example-based tests…
4. …can be hard to understand
5. …does not tell the whole truth
6. …can become bloated
7. We want to make sure the tests pull their weight by making them expressive, so that we can tell what's important when we read them and when they fail, and by making sure they don’t become a maintenance drag themselves. Nat Pryce & Steve Freeman (GOOS book, p227)
8. Property-based testing to the rescue! Don’t write tests; generate them!
9. Define properties and let the tool try to falsify them
10. Define how to generate test data
11. “What’s the simplest input causing a failure?” Let the testing library answer this question!
12. The property-based tests highlighted the bug, which is now fixed!
13. Property-based tests… • raise the level of abstraction • increase your confidence in the system under test • reduce the maintenance burden of your tests • free you to focus on quality attributes other than correctness
14. Property-based-testing tools QuickCheck QuickCheck junit-quickcheck ScalaCheck FsCheck Hypothesis elm-check test-check JSVerify
15. “How can I sell it to the team?”
16. Tell your developers about it
17. Set up an internal workshop
18. “Property-test all the things?”
19. “Property-based tests are non-deterministic”
20. “I can’t think of properties about my code”
21. “Increased risk of bugs in testing code”
22. The whole point is that [production code and testing code] double-check on each other. You have a problem when something goes red because they are mismatched. The basic, fundamental thing is that it’s a double-check mechanism. Martin Fowler (https://www.youtube.com/watch? v=dGtasFJnUxI&t=23m)
23. “Too complicated!”
24. Resources Workshop (Cork Software Craftsmanship) Talk (CorkDev)