Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Property-based Testing in Python with Hypothesis

Property-based Testing in Python with Hypothesis

A talk given at GeekCamp.id, 24 October 2015.

Examples used in the presentation available at https://github.com/michel-slm/talk--property-based-testing--hypothesis/

Michel Alexandre Salim

October 24, 2015
Tweet

Other Decks in Programming

Transcript

  1. Property-based Testing
    with Hypothesis
    Michel Alexandre Salim • [email protected]

    View Slide

  2. Overview
    Why Test?
    From Examples to Properties
    Taking a Deeper Look

    View Slide

  3. Why Test?

    View Slide

  4. We are not perfect
    Mistakes in
    ● Implementation
    ● Assumptions
    ● Third-party code!
    Test as Documentation
    ● Requirements are vague
    ● Tests provide specification
    ● Regression tests

    View Slide

  5. From Examples to Properties

    View Slide

  6. All Swans are White ... or are they?

    View Slide

  7. Property-based Testing
    Example-Based
    ● Specific scenarios
    ● Must think of boundary conditions
    ● White box (mostly) - refactoring?
    ● Deterministic
    Property-Based
    ● Higher-order properties
    ● Can help find boundary conditions
    ● Black box
    ● Non-deterministic

    View Slide

  8. More non-deterministic tests
    Property-Based Testing
    Test cases are randomly generated (within given boundaries).
    Try to find counter-examples to properties.
    Fuzz Testing
    Program inputs are randomly generated. Try and cause crash.
    Mutation Testing
    Program code is mutated. Verify that test suite kills mutants!.

    View Slide

  9. Taking a Deeper Look

    View Slide

  10. Hypothesis in Context
    History
    ● 1999: QuickCheck (Haskell)
    ● 2007: ScalaCheck
    ● 2013: Hypothesis
    ● 2015: Hypothesis 1.0

    View Slide

  11. Generate
    Random test inputs
    generated (+ manual
    examples)
    Filter
    Only keep the data
    with the desired
    properties
    Test
    Run the test for each
    random data set
    Simplify
    On failure, shrink the
    failing inputs to find
    simplest failing case
    Report
    Report result of test
    runs

    View Slide

  12. Because a live demonstration is worth a thousand words...

    View Slide

  13. Online Resources
    Hypothesis Documentation
    https://hypothesis.readthedocs.org/en/latest/
    pytest
    http://pytest.org/latest/
    Examples from this presentation
    https://github.com/michel-slm/talk--property-based-testing--hypothesis/

    View Slide

  14. Questions?

    View Slide