Save 37% off PRO during our Black Friday Sale! »

Metamorphic testing vs property based testing

Metamorphic testing vs property based testing


Yoriyuki Yamagata

February 08, 2020


  1. Metamorphic vs Property Based Testing Yoriyuki Yamagata

  2. Messages ✤ Metamorphic testing is a special case of property

    based testing ✤ Property based testing is more powerful than metamorphic testing
  3. Metamorphic testing ✤ Test a function F if the inputs

    x, y satisfy a relation R(x, y), then F(x), F(y) satisfy a relation R’(F(x), F(y)) ✤ A way to solve the oracle problem in testing ✤ Oracle problem: we do not know the correct F(x) for x ✤ Example: sin(x) ✤ Hard to know the correct sin(x) for the given x ✤ But sin(x) = sin(x+2π) must always hold
  4. Property based testing ✤ Specify a specification of a function

    F(x), not a correct value of F(x) ✤ Automatic test case generation using type information ✤ Started from Haskell’s QuickCheck but now there is an implementation for almost any language ✤ We introduce Kotlin implementation (kotlintest)
  5. Kotlin ✤ Multi-paradigm, cross-platform (JVM, Node.js and LLVM), strongly typed

    with type inference, and memory-safe language ✤ Full interoperability with Java ✤ Developed by JetBrain, “Preferred language for Android app developers” (Google) ✤ Kotlintest supports property based testing
  6. Metamorphic testing

  7. Property based testing

  8. Property based testing is more general ✤ can test a

    relation involving more than two functions (e.g. sin and cos, tan and arctan) ✤ can test on a relation between >= 3 elements (e.g. x, y, x+y)
  9. Property based testing is more powerful ✤ Random input generation

    based on types ✤ QuickCheck (for Haskell) defines generators for almost all types (?) ✤ Generators must be defined manually for Classes in Kotlin (but all base types are already covered)