Metamorphic testing vs property based testing

Transcript

1. Metamorphic vs Property Based Testing Yoriyuki Yamagata
2. Messages ✤ Metamorphic testing is a special case of property based testing ✤ Property based testing is more powerful than metamorphic testing
3. Metamorphic testing ✤ Test a function F if the inputs x, y satisfy a relation R(x, y), then F(x), F(y) satisfy a relation R’(F(x), F(y)) ✤ A way to solve the oracle problem in testing ✤ Oracle problem: we do not know the correct F(x) for x ✤ Example: sin(x) ✤ Hard to know the correct sin(x) for the given x ✤ But sin(x) = sin(x+2π) must always hold
4. Property based testing ✤ Specify a specification of a function F(x), not a correct value of F(x) ✤ Automatic test case generation using type information ✤ Started from Haskell’s QuickCheck but now there is an implementation for almost any language ✤ We introduce Kotlin implementation (kotlintest)
5. Kotlin ✤ Multi-paradigm, cross-platform (JVM, Node.js and LLVM), strongly typed with type inference, and memory-safe language ✤ Full interoperability with Java ✤ Developed by JetBrain, “Preferred language for Android app developers” (Google) ✤ Kotlintest supports property based testing
6. Metamorphic testing
7. Property based testing
8. Property based testing is more general ✤ can test a relation involving more than two functions (e.g. sin and cos, tan and arctan) ✤ can test on a relation between >= 3 elements (e.g. x, y, x+y)
9. Property based testing is more powerful ✤ Random input generation based on types ✤ QuickCheck (for Haskell) defines generators for almost all types (?) ✤ Generators must be defined manually for Classes in Kotlin (but all base types are already covered)