Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Property-Based Testing @ COSCUP 2022

Fd9f21969cd8a0cbf0b2f48a9af0cc28?s=47 cybai
July 29, 2022

Introduction to Property-Based Testing @ COSCUP 2022

Fd9f21969cd8a0cbf0b2f48a9af0cc28?s=128

cybai

July 29, 2022
Tweet

More Decks by cybai

Other Decks in Programming

Transcript

  1. Introduction to Property-Based Testing cybai @ COSCUP 2022

  2. 2 sli.do #coscup22-fp

  3. • @CYBAI • @_cybai • Front-End Engineer @ Autify, Inc.

    • Web Front-End by day, learning Functional Programming by night. whoami 3
  4. Agenda 1. What is Property-Based Testing? 2. More examples for

    Property-Based Testing 3. When to Example-Based Testing? 4. Advanced topics? 5. Q & A 4 Note: PBT refers to Property-Based Testing in this talk.
  5. What is Property-Based Testing? 5

  6. What is Property-Based Testing? 6

  7. What is Property? 7

  8. A property is an invariant of your testing target. What

    is Property? 8
  9. What is Property? 9 A property is an invariant of

    your testing target. What does that mean? Let’s start from the traditional example-based testing.
  10. What is Property-Based Testing? (Cont.) 10 Haskell: QuickCheck, Hedgehog Rust:

    quickcheck, proptest Erlang: PropEr JavaScript: fast-check, jsverify Python: Hypothesis OCaml: popper … and many more PBT frameworks for different languages
  11. 11

  12. The function under test 12

  13. The function under test Sample Inputs 13

  14. The function under test Expected Outputs 14 Sample Inputs

  15. What is Property-Based Testing? (Cont.) 15 E.g. [1,2,3], [0], [],

    [1,2,3,...,100]
  16. What is Property-Based Testing? (Cont.) A test case should be

    like a proof for the invariant for the testing target. 16
  17. A test case should be like a proof for the

    invariant for the testing target. In the example of `reverse` function, the invariant here is “An input array should be kept as same if it’s reversed twice”. What is Property-Based Testing? (Cont.) 17
  18. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

    Shrinkers 18
  19. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

    Shrinkers fc.integer() fc.webUrl() fc.oneof() fc.record() fc.boolean() fc.nat() fc.string() .filter() fc.option() .map() 19 Generate random values
  20. What values will fast-check Arbitraries generate? 20 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.nat() => 0, 1, 2, …, 100000, …
  21. What values will fast-check Arbitraries generate? 21 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", …
  22. What values will fast-check Arbitraries generate? 22 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.oneof(fc.nat(), fc.char()) => Use one of the given arbitraries fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", … .filter() => precondition for the arbitrary (e.g. n < 100) fc.array(fc.nat()) => [], [1], [0,2,2,3], …, [3,5, …, 10,100]
  23. What values will fast-check Arbitraries generate? 23 fc.integer() => -1000,

    -10, …, 0, 1, 3, …, 100, … fc.webUrl() => "https://lo.ui", "https://4.xlm", … fc.oneof(fc.nat(), fc.char()) => Use one of the given arbitraries fc.record({ age: fc.nat(), name: fc.string() }) => Arbitrary for given object model, e.g. { age: 2, name: "John" } fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", … .filter() => precondition for the arbitrary (e.g. n < 100) fc.array(fc.nat()) => [], [1], [0,2,2,3], …, [3,5, …, 10,100] fc.option(fc.nat()) => null, 0, 1, …, 100, …, 100000, …
  24. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

    Shrinkers Run tests for properties and validate specified predicates. Most of PBT frameworks default to 100 number of runs but you can change it. 24
  25. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

    Shrinkers Teach the PBT framework how to shrink the failed test case. This would be helpful for knowing smallest possible case. (but not every PBT framework supports this) 25
  26. More examples for Property-Based Testing 26

  27. More examples - reverse 27

  28. More examples - reverse (Cont.) 28

  29. More examples - reverse (Cont.) 29

  30. More examples - reverse (Cont.) 30 shrunk times and shrunk

    test case
  31. More examples - Calculate Shipping Fee 31

  32. More examples - Calculate Shipping Fee (Cont.) 32

  33. More examples - Calculate Shipping Fee (Cont.) 33

  34. More examples - Calculate Shipping Fee (Cont.) 34

  35. More examples for Property-Based Testing You can find more examples

    on Advent of PBT 2021 created by the author of fast-check. 35
  36. When to Example-Based Testing? You can still write unit tests

    in example-based testing way to make people easier to understand how to use them. I’d highly recommend test them with `doctest`. 36
  37. When to Example-Based Testing? Let’s see how `doctest` is utilized

    in Rust. By default, code block in doc comments can be tested by `cargo test` or `rustdoc --test`. So, it can be tested as a part of document. 37 The calculate shipping fee example in Rust
  38. Advanced Topics? 38

  39. 1. PBT for End-to-End Testing (like this post) Advanced Topics?

    39
  40. 1. PBT for End-to-End Testing (like this post) 2. Model-based

    testing (like XState testing) Advanced Topics? 40
  41. 1. PBT for End-to-End Testing (like this post) 2. Model-based

    testing (like XState testing) 3. PBT for Web Application (like Quickstrom) Advanced Topics? 41
  42. Advanced Topics? 1. PBT for End-to-End Testing (like this post)

    2. Model-based testing (like XState testing) 3. PBT for Web Application (like Quickstrom) 4. How to specify it! 42
  43. Advanced Topics? 1. PBT for End-to-End Testing 2. Model-based testing

    3. PBT for Web Application (like Quickstrom) 4. How to specify it! 43
  44. Advanced Topics? 1. PBT for End-to-End Testing (like this post)

    2. Model-based testing (like XState testing) 3. PBT for Web Application (like Quickstrom) 4. How to specify it! 5. ... and maybe some more I don’t know yet! 44
  45. Test properties (invariants) with property-based testing and move example-based testing

    as document with `doctest`. Conclusion 45
  46. Test properties (invariants) with property-based testing and move example-based testing

    as document with `doctest`. Let’s try to prove our programs by Property-Based Testing! Conclusion 46
  47. Q & A 47