cybai
July 29, 2022
150

Introduction to Property-Based Testing @ COSCUP 2022

July 29, 2022

Transcript

3. • @CYBAI • @_cybai • Front-End Engineer @ Autify, Inc.

• Web Front-End by day, learning Functional Programming by night. whoami 3
4. Agenda 1. What is Property-Based Testing? 2. More examples for

Property-Based Testing 3. When to Example-Based Testing? 4. Advanced topics? 5. Q & A 4 Note: PBT refers to Property-Based Testing in this talk.

8. A property is an invariant of your testing target. What

is Property? 8
9. What is Property? 9 A property is an invariant of

your testing target. What does that mean? Let’s start from the traditional example-based testing.
10. What is Property-Based Testing? (Cont.) 10 Haskell: QuickCheck, Hedgehog Rust:

quickcheck, proptest Erlang: PropEr JavaScript: fast-check, jsverify Python: Hypothesis OCaml: popper … and many more PBT frameworks for different languages

15. What is Property-Based Testing? (Cont.) 15 E.g. [1,2,3], [0], [],

[1,2,3,...,100]
16. What is Property-Based Testing? (Cont.) A test case should be

like a proof for the invariant for the testing target. 16
17. A test case should be like a proof for the

invariant for the testing target. In the example of `reverse` function, the invariant here is “An input array should be kept as same if it’s reversed twice”. What is Property-Based Testing? (Cont.) 17

Shrinkers 18
19. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

Shrinkers fc.integer() fc.webUrl() fc.oneof() fc.record() fc.boolean() fc.nat() fc.string() .filter() fc.option() .map() 19 Generate random values
20. What values will fast-check Arbitraries generate? 20 fc.integer() => -1000,

-10, …, 0, 1, 3, …, 100, … fc.nat() => 0, 1, 2, …, 100000, …
21. What values will fast-check Arbitraries generate? 21 fc.integer() => -1000,

-10, …, 0, 1, 3, …, 100, … fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", …
22. What values will fast-check Arbitraries generate? 22 fc.integer() => -1000,

-10, …, 0, 1, 3, …, 100, … fc.oneof(fc.nat(), fc.char()) => Use one of the given arbitraries fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", … .filter() => precondition for the arbitrary (e.g. n < 100) fc.array(fc.nat()) => [], [1], [0,2,2,3], …, [3,5, …, 10,100]
23. What values will fast-check Arbitraries generate? 23 fc.integer() => -1000,

-10, …, 0, 1, 3, …, 100, … fc.webUrl() => "https://lo.ui", "https://4.xlm", … fc.oneof(fc.nat(), fc.char()) => Use one of the given arbitraries fc.record({ age: fc.nat(), name: fc.string() }) => Arbitrary for given object model, e.g. { age: 2, name: "John" } fc.boolean() => true, false fc.nat() => 0, 1, 2, …, 100000, … fc.string() => "", "C0SCup", "p#@91-+", … .filter() => precondition for the arbitrary (e.g. n < 100) fc.array(fc.nat()) => [], [1], [0,2,2,3], …, [3,5, …, 10,100] fc.option(fc.nat()) => null, 0, 1, …, 100, …, 100000, …
24. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

Shrinkers Run tests for properties and validate specified predicates. Most of PBT frameworks default to 100 number of runs but you can change it. 24
25. What is Property-Based Testing? (Cont.) 1. Arbitraries 2. Runners 3.

Shrinkers Teach the PBT framework how to shrink the failed test case. This would be helpful for knowing smallest possible case. (but not every PBT framework supports this) 25

test case

35. More examples for Property-Based Testing You can find more examples

on Advent of PBT 2021 created by the author of fast-check. 35
36. When to Example-Based Testing? You can still write unit tests

in example-based testing way to make people easier to understand how to use them. I’d highly recommend test them with `doctest`. 36
37. When to Example-Based Testing? Let’s see how `doctest` is utilized

in Rust. By default, code block in doc comments can be tested by `cargo test` or `rustdoc --test`. So, it can be tested as a part of document. 37 The calculate shipping fee example in Rust

39
40. 1. PBT for End-to-End Testing (like this post) 2. Model-based

testing (like XState testing) Advanced Topics? 40
41. 1. PBT for End-to-End Testing (like this post) 2. Model-based

testing (like XState testing) 3. PBT for Web Application (like Quickstrom) Advanced Topics? 41
42. Advanced Topics? 1. PBT for End-to-End Testing (like this post)

2. Model-based testing (like XState testing) 3. PBT for Web Application (like Quickstrom) 4. How to specify it! 42
43. Advanced Topics? 1. PBT for End-to-End Testing 2. Model-based testing

3. PBT for Web Application (like Quickstrom) 4. How to specify it! 43
44. Advanced Topics? 1. PBT for End-to-End Testing (like this post)

2. Model-based testing (like XState testing) 3. PBT for Web Application (like Quickstrom) 4. How to specify it! 5. ... and maybe some more I don’t know yet! 44
45. Test properties (invariants) with property-based testing and move example-based testing

as document with `doctest`. Conclusion 45
46. Test properties (invariants) with property-based testing and move example-based testing

as document with `doctest`. Let’s try to prove our programs by Property-Based Testing! Conclusion 46