Quickstart into pytest (tutorial)

This tutorial gives a quick start into pytest. It was presented in german at Pycon DE 2012 in October and works based on the pytest-2.3.2 release.

Transcript

1. quick start into pytest
   If you have time before the tutorial starts, please make
   sure you have installed the latest pytest. You need either:
   setuptools: http:
   //pypi.python.org/pypi/setuptools
   distribute: http:
   //pypi.python.org/pypi/distribute
   and can then type:
   easy_install py # or
   pip install py
   () October 29, 2012 1 / 38
   

   View Slide

2. me
   Holger Krekel
   founded the PyPy project
   developer of py.test and execnet
   Python since around 2000
   Test-driven development since 2001
   operates a small company “merlinux.eu” doing open
   source, Python and test-oriented consulting
   () October 29, 2012 2 / 38
   

   View Slide

3. you
   What is your background?
   python
   testing
   testing tools and libs
   () October 29, 2012 3 / 38
   

   View Slide

4. why automated testing?
   to allow for later changes
   to raise confidence that code works
   to specify and document behaviour
   collaborative and faster development cycles
   () October 29, 2012 4 / 38
   

   View Slide

5. Developer oriented automated tests
   unittest: units react well to input.
   integration: components co-operate nicely
   functional: code changes work out in user
   environments
   () October 29, 2012 5 / 38
   

   View Slide

6. unittest
   assert that functions and classes behave as expected
   () October 29, 2012 6 / 38
   

   View Slide

7. integration
   assert units/components co-operate nicely
   () October 29, 2012 7 / 38
   

   View Slide

8. functional / system
   things work in user target environment
   () October 29, 2012 8 / 38
   

   View Slide

9. The test tool question
   what is the job of automated testing tools?
   () October 29, 2012 9 / 38
   

   View Slide

10. my current answer
    verify code changes work out
    be helpful when tests fail
    () October 29, 2012 10 / 38
    

    View Slide

11. If failures are not helpful ...
    improve the test tool or
    write more or write different tests
    () October 29, 2012 11 / 38
    

    View Slide

12. py.test basics
    http://marikaz.deviantart.com/ CC 3.0
    () October 29, 2012 12 / 38
    

    View Slide

13. py.test fundamental features
    automatic test discovery, no-boilerplate test code
    useful information when a test fails
    cross-project, many options and plugins
    cross-python, python2.5 through to 3.3
    modular and parametrizable fixtures
    distribute tests to multiple hosts
    lots of online and PDF documentation.
    () October 29, 2012 13 / 38
    

    View Slide

14. cross-project test tool
    tests are run via py.test command line tool.
    project specific conftest.py files
    testing starts from files/dirs or current dir
    examples:
    py.test test_file.py
    py.test path/to/tests
    () October 29, 2012 14 / 38
    

    View Slide

15. A Typical Python test layout
    tests included with package:
    mypkg/__init__.py
    ...
    mypkg/tests/test_module.py
    ...
    example invocation:
    py.test mypkg
    () October 29, 2012 15 / 38
    

    View Slide

16. Another typical test layout
    tests separate from package:
    mypkg/__init__.py
    ...
    tests/test_module.py
    example invocations:
    py.test tests
    py.test # starts discovery in cwd
    () October 29, 2012 16 / 38
    

    View Slide

17. automatic test discovery
    py.test walks over the filesystem and:
    discovers test_*.py test files
    discovers test_ functions and Test classes
    automatic discovery avoids boilerplate
    () October 29, 2012 17 / 38
    

    View Slide

18. no boilerplate python test code
    example test functions/methods:
    def test_something():
    x = 3
    assert x == 4
    class TestSomething:
    def test_something(self):
    x = 1
    assert x == 5
    () October 29, 2012 18 / 38
    

    View Slide

19. test classes
    with pytest:
    test classes have a Test prefix, are autodiscovered
    there is no need to subclass anything
    test classes are for logic grouping of your tests
    () October 29, 2012 19 / 38
    

    View Slide

20. Getting Started ...
    Installation:
    easy_install -U py, pip install py or:
    hg clone
    https://bitbucket.org/hpk42/pytest/
    run “python setup.py” with “install” or “develop”
    More notes and info:
    http://pytest.org/latest/
    getting-started.html
    Exercise: install and run “py.test --help“*
    Exercise: Write test_module.py with a “test_“ function
    and a TestClass with test methods and run it
    () October 29, 2012 20 / 38
    

    View Slide

21. assert introspection
    def test_assert_introspection():
    # with unittest.py
    assert x # assertTrue()
    assert x == 1 # assertEqual(x, 1)
    assert x != 2 # assertNotEqual(x,
    assert not x # assertFalse(x)
    () October 29, 2012 21 / 38
    

    View Slide

22. asserting expected exceptions
    Two forms:
    import pytest
    def test_raises_one():
    pytest.raises(ValueError,
    lambda: int("foo"))
    def test_raises_two():
    with pytest.raises(ValueError):
    int("foo")
    () October 29, 2012 22 / 38
    

    View Slide

23. Failure / Traceback Demo
    interactive demo: py.test failure_demo.py
    () October 29, 2012 23 / 38
    

    View Slide

24. print() debugging
    output captured, shown per test on failure:
    def test_something1():
    print "extrainfo"
    assert False
    () October 29, 2012 24 / 38
    

    View Slide

25. some important options
    see also py.test -h:
    -s disable catching of stdout/stderr
    -x exit instantly on first failure
    -k run tests whose names contain a keyword
    -l show locals in tracebacks.
    --pdb start Python debugger on errors
    --tb/fulltrace - control traceback generation.
    Exercise: run tests from last example with these
    options
    () October 29, 2012 25 / 38
    

    View Slide

26. Beyond simple testing: fixtures!
    a test fixture:
    sets up objects or apps for testing
    provides test code with “base” app objects
    very important to avoid repetetive test code
    () October 29, 2012 26 / 38
    

    View Slide

27. test functions and fixtures
    Basic mechanism is simple:
    fixture functions create fixtures values
    test functions or classes use them if needed by
    referencing the function name
    () October 29, 2012 27 / 38
    

    View Slide

28. Example of a simple pytest fixture
    fixture function and test function using it:
    # content of test_module.py
    import pytest
    @pytest.fixture
    def somevalue():
    return 42
    def test_function(somevalue):
    assert somevalue == 42
    Exercise: write a fixture function returning a dummy
    class instance instead of a simple value
    () October 29, 2012 28 / 38
    

    View Slide

29. notes about pytest fixtures
    fixtures are created by a function
    each fixture has a unique name (the function name)
    test functions can get it injected as an argument by
    name
    tests choose which fixtures they need
    move fixture function to a conftest.py file to
    make it available for all test modules
    () October 29, 2012 29 / 38
    

    View Slide

30. Scoping fixture functions
    fixture functions can declare a scope:
    import time, pytest
    @pytest.fixture(scope="module")
    def answer():
    time.sleep(2)
    return 42
    def test_one(answer):
    assert answer == 42
    def test_two(answer):
    assert answer == 42
    Exercise: try it - how long does the test run take?
    () October 29, 2012 30 / 38
    

    View Slide

31. Parametrizing fixtures and tests
    let’s have multiple different “answer” fixtures:
    import time, pytest
    @pytest.fixture(params=[1,2,3])
    def answer(request):
    return request.param
    the request object allows to query test invocation
    information, in this case parameters.
    now any tests using the answer fixture will run three
    times!
    Exercise: write a test function accepting an “answer”
    argument and run it, also write a test that fails
    () October 29, 2012 31 / 38
    

    View Slide

32. Modularity: using fixtures from fixtures
    Just as test functions, fixture functions can use fixtures
    themselves:
    import pytest
    @pytest.fixture
    def answer10(answer):
    return answer * 10
    If you add this to the previous example, you get a
    answer10 fixture which internally uses the answer
    fixture (including parametrization!).
    () October 29, 2012 32 / 38
    

    View Slide

33. Classes can use fixtures as well
    You can use the usefixtures marker to mark all
    methods of a test class:
    import pytest
    @pytest.mark.usefixtures("cleandir")
    class TestCommandline:
    def test_method1(self):
    ...
    def test_method2(self):
    ...
    () October 29, 2012 33 / 38
    

    View Slide

34. implementation of cleandir fixture
    the cleandir fixture runs for each test method where it is an
    argument or where a usefixture marker is present.
    it might look like this:
    # content of conftest.py
    import pytest
    @pytest.fixture
    def cleandir(request, tmpdir):
    old = tmpdir.chdir()
    request.addfinalizer(old.chdir)
    () October 29, 2012 34 / 38
    

    View Slide

35. Networking examples, more docs
    The basic fixture documentation has tutorial-style
    examples on how to manage a SMTP-Server connection
    fixture:
    http:
    //pytest.org/latest/fixture.html
    () October 29, 2012 35 / 38
    

    View Slide

36. More pytest topics
    more interesting topics (find links on webpage):
    distributed testing, looponfailing, timeout
    junitxml, coverage and coverage-html reporting
    marking tests, skipping and xfailing
    conftest and setuptools plugin writing
    understanding hook functions
    see http://pytest.org
    () October 29, 2012 36 / 38
    

    View Slide

37. comparisong with nose
    nose:
    clone of pytest from 2005, introduced plugins
    since developed separately
    shares many ideas and features
    since 2008 pytest also has plugins
    nose no enriched tracebacks, no plain asserts, no
    named fixtures, otherwise similar
    pytest can run most nose-based tests suites by
    default (unittest as well)
    () October 29, 2012 37 / 38
    

    View Slide

38. Feedback and questions
    Questions and feedback round, otherwise meet me on:
    #pylib on irc.freenode.net or
    @hpk42 on twitter
    on stackoverflow with a “pytest” question
    on the mailing list http://codespeak.net/
    mailman/listinfo/py-dev
    () October 29, 2012 38 / 38
    

    View Slide