Metamorphic testing vs property based testing

Transcript

1. Metamorphic vs Property Based Testing Yoriyuki Yamagata

2. Messages ✤ Metamorphic testing is a special case of property

   based testing ✤ Property based testing is more powerful than metamorphic testing

3. Metamorphic testing ✤ Test a function F if the inputs

   x, y satisfy a relation R(x, y), then F(x), F(y) satisfy a relation R’(F(x), F(y)) ✤ A way to solve the oracle problem in testing ✤ Oracle problem: we do not know the correct F(x) for x ✤ Example: sin(x) ✤ Hard to know the correct sin(x) for the given x ✤ But sin(x) = sin(x+2π) must always hold

4. Property based testing ✤ Specify a specification of a function

   F(x), not a correct value of F(x) ✤ Automatic test case generation using type information ✤ Started from Haskell’s QuickCheck but now there is an implementation for almost any language ✤ We introduce Kotlin implementation (kotlintest)

5. Kotlin ✤ Multi-paradigm, cross-platform (JVM, Node.js and LLVM), strongly typed

   with type inference, and memory-safe language ✤ Full interoperability with Java ✤ Developed by JetBrain, “Preferred language for Android app developers” (Google) ✤ Kotlintest supports property based testing

6. Metamorphic testing

7. Property based testing

8. Property based testing is more general ✤ can test a

   relation involving more than two functions (e.g. sin and cos, tan and arctan) ✤ can test on a relation between >= 3 elements (e.g. x, y, x+y)

9. Property based testing is more powerful ✤ Random input generation

   based on types ✤ QuickCheck (for Haskell) defines generators for almost all types (?) ✤ Generators must be defined manually for Classes in Kotlin (but all base types are already covered)