Quickstart into pytest (tutorial)

This tutorial gives a quick start into pytest. It was presented in german at Pycon DE 2012 in October and works based on the pytest-2.3.2 release.

Transcript

1. quick start into pytest If you have time before the tutorial starts, please make sure you have installed the latest pytest. You need either: setuptools: http: //pypi.python.org/pypi/setuptools distribute: http: //pypi.python.org/pypi/distribute and can then type: easy_install py # or pip install py () October 29, 2012 1 / 38
2. me Holger Krekel <holger@merlinux.eu> founded the PyPy project developer of py.test and execnet Python since around 2000 Test-driven development since 2001 operates a small company “merlinux.eu” doing open source, Python and test-oriented consulting () October 29, 2012 2 / 38
3. you What is your background? python testing testing tools and libs () October 29, 2012 3 / 38
4. why automated testing? to allow for later changes to raise confidence that code works to specify and document behaviour collaborative and faster development cycles () October 29, 2012 4 / 38
5. Developer oriented automated tests unittest: units react well to input. integration: components co-operate nicely functional: code changes work out in user environments () October 29, 2012 5 / 38
6. unittest assert that functions and classes behave as expected () October 29, 2012 6 / 38
7. integration assert units/components co-operate nicely () October 29, 2012 7 / 38
8. functional / system things work in user target environment () October 29, 2012 8 / 38
9. The test tool question what is the job of automated testing tools? () October 29, 2012 9 / 38
10. my current answer verify code changes work out be helpful when tests fail () October 29, 2012 10 / 38
11. If failures are not helpful ... improve the test tool or write more or write different tests () October 29, 2012 11 / 38
12. py.test basics http://marikaz.deviantart.com/ CC 3.0 () October 29, 2012 12 / 38
13. py.test fundamental features automatic test discovery, no-boilerplate test code useful information when a test fails cross-project, many options and plugins cross-python, python2.5 through to 3.3 modular and parametrizable fixtures distribute tests to multiple hosts lots of online and PDF documentation. () October 29, 2012 13 / 38
14. cross-project test tool tests are run via py.test command line tool. project specific conftest.py files testing starts from files/dirs or current dir examples: py.test test_file.py py.test path/to/tests () October 29, 2012 14 / 38
15. A Typical Python test layout tests included with package: mypkg/__init__.py ... mypkg/tests/test_module.py ... example invocation: py.test mypkg () October 29, 2012 15 / 38
16. Another typical test layout tests separate from package: mypkg/__init__.py ... tests/test_module.py example invocations: py.test tests py.test # starts discovery in cwd () October 29, 2012 16 / 38
17. automatic test discovery py.test walks over the filesystem and: discovers test_*.py test files discovers test_ functions and Test classes automatic discovery avoids boilerplate () October 29, 2012 17 / 38
18. no boilerplate python test code example test functions/methods: def test_something(): x = 3 assert x == 4 class TestSomething: def test_something(self): x = 1 assert x == 5 () October 29, 2012 18 / 38
19. test classes with pytest: test classes have a Test prefix, are autodiscovered there is no need to subclass anything test classes are for logic grouping of your tests () October 29, 2012 19 / 38
20. Getting Started ... Installation: easy_install -U py, pip install py or: hg clone https://bitbucket.org/hpk42/pytest/ run “python setup.py” with “install” or “develop” More notes and info: http://pytest.org/latest/ getting-started.html Exercise: install and run “py.test --help“* Exercise: Write test_module.py with a “test_“ function and a TestClass with test methods and run it () October 29, 2012 20 / 38
21. assert introspection def test_assert_introspection(): # with unittest.py assert x # assertTrue() assert x == 1 # assertEqual(x, 1) assert x != 2 # assertNotEqual(x, assert not x # assertFalse(x) () October 29, 2012 21 / 38
22. asserting expected exceptions Two forms: import pytest def test_raises_one(): pytest.raises(ValueError, lambda: int("foo")) def test_raises_two(): with pytest.raises(ValueError): int("foo") () October 29, 2012 22 / 38
23. Failure / Traceback Demo interactive demo: py.test failure_demo.py () October 29, 2012 23 / 38
24. print() debugging output captured, shown per test on failure: def test_something1(): print "extrainfo" assert False () October 29, 2012 24 / 38
25. some important options see also py.test -h: -s disable catching of stdout/stderr -x exit instantly on first failure -k run tests whose names contain a keyword -l show locals in tracebacks. --pdb start Python debugger on errors --tb/fulltrace - control traceback generation. Exercise: run tests from last example with these options () October 29, 2012 25 / 38
26. Beyond simple testing: fixtures! a test fixture: sets up objects or apps for testing provides test code with “base” app objects very important to avoid repetetive test code () October 29, 2012 26 / 38
27. test functions and fixtures Basic mechanism is simple: fixture functions create fixtures values test functions or classes use them if needed by referencing the function name () October 29, 2012 27 / 38
28. Example of a simple pytest fixture fixture function and test function using it: # content of test_module.py import pytest @pytest.fixture def somevalue(): return 42 def test_function(somevalue): assert somevalue == 42 Exercise: write a fixture function returning a dummy class instance instead of a simple value () October 29, 2012 28 / 38
29. notes about pytest fixtures fixtures are created by a function each fixture has a unique name (the function name) test functions can get it injected as an argument by name tests choose which fixtures they need move fixture function to a conftest.py file to make it available for all test modules () October 29, 2012 29 / 38
30. Scoping fixture functions fixture functions can declare a scope: import time, pytest @pytest.fixture(scope="module") def answer(): time.sleep(2) return 42 def test_one(answer): assert answer == 42 def test_two(answer): assert answer == 42 Exercise: try it - how long does the test run take? () October 29, 2012 30 / 38
31. Parametrizing fixtures and tests let’s have multiple different “answer” fixtures: import time, pytest @pytest.fixture(params=[1,2,3]) def answer(request): return request.param the request object allows to query test invocation information, in this case parameters. now any tests using the answer fixture will run three times! Exercise: write a test function accepting an “answer” argument and run it, also write a test that fails () October 29, 2012 31 / 38
32. Modularity: using fixtures from fixtures Just as test functions, fixture functions can use fixtures themselves: import pytest @pytest.fixture def answer10(answer): return answer * 10 If you add this to the previous example, you get a answer10 fixture which internally uses the answer fixture (including parametrization!). () October 29, 2012 32 / 38
33. Classes can use fixtures as well You can use the usefixtures marker to mark all methods of a test class: import pytest @pytest.mark.usefixtures("cleandir") class TestCommandline: def test_method1(self): ... def test_method2(self): ... () October 29, 2012 33 / 38
34. implementation of cleandir fixture the cleandir fixture runs for each test method where it is an argument or where a usefixture marker is present. it might look like this: # content of conftest.py import pytest @pytest.fixture def cleandir(request, tmpdir): old = tmpdir.chdir() request.addfinalizer(old.chdir) () October 29, 2012 34 / 38
35. Networking examples, more docs The basic fixture documentation has tutorial-style examples on how to manage a SMTP-Server connection fixture: http: //pytest.org/latest/fixture.html () October 29, 2012 35 / 38
36. More pytest topics more interesting topics (find links on webpage): distributed testing, looponfailing, timeout junitxml, coverage and coverage-html reporting marking tests, skipping and xfailing conftest and setuptools plugin writing understanding hook functions see http://pytest.org () October 29, 2012 36 / 38
37. comparisong with nose nose: clone of pytest from 2005, introduced plugins since developed separately shares many ideas and features since 2008 pytest also has plugins nose no enriched tracebacks, no plain asserts, no named fixtures, otherwise similar pytest can run most nose-based tests suites by default (unittest as well) () October 29, 2012 37 / 38
38. Feedback and questions Questions and feedback round, otherwise meet me on: #pylib on irc.freenode.net or @hpk42 on twitter on stackoverflow with a “pytest” question on the mailing list http://codespeak.net/ mailman/listinfo/py-dev () October 29, 2012 38 / 38