Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Property-based Testing in Python with Hypothesis

Property-based Testing in Python with Hypothesis

A talk given at GeekCamp.id, 24 October 2015.

Examples used in the presentation available at https://github.com/michel-slm/talk--property-based-testing--hypothesis/

Michel Alexandre Salim

October 24, 2015

Other Decks in Programming


  1. Property-based Testing with Hypothesis Michel Alexandre Salim • [email protected]

  2. Overview Why Test? From Examples to Properties Taking a Deeper

  3. Why Test?

  4. We are not perfect Mistakes in • Implementation • Assumptions

    • Third-party code! Test as Documentation • Requirements are vague • Tests provide specification • Regression tests
  5. From Examples to Properties

  6. All Swans are White ... or are they?

  7. Property-based Testing Example-Based • Specific scenarios • Must think of

    boundary conditions • White box (mostly) - refactoring? • Deterministic Property-Based • Higher-order properties • Can help find boundary conditions • Black box • Non-deterministic
  8. More non-deterministic tests Property-Based Testing Test cases are randomly generated

    (within given boundaries). Try to find counter-examples to properties. Fuzz Testing Program inputs are randomly generated. Try and cause crash. Mutation Testing Program code is mutated. Verify that test suite kills mutants!.
  9. Taking a Deeper Look

  10. Hypothesis in Context History • 1999: QuickCheck (Haskell) • 2007:

    ScalaCheck • 2013: Hypothesis • 2015: Hypothesis 1.0
  11. Generate Random test inputs generated (+ manual examples) Filter Only

    keep the data with the desired properties Test Run the test for each random data set Simplify On failure, shrink the failing inputs to find simplest failing case Report Report result of test runs
  12. Because a live demonstration is worth a thousand words...

  13. Online Resources Hypothesis Documentation https://hypothesis.readthedocs.org/en/latest/ pytest http://pytest.org/latest/ Examples from this

    presentation https://github.com/michel-slm/talk--property-based-testing--hypothesis/
  14. Questions?