rapid testing (talk)

The slides for the pytest 1h testing talk at Pycon DE 2012. This talk focuses on assertions and the fixture mechanism introduced with pytest-2.3.2.

Transcript

1. pytest rapid testing Holger Krekel http://merlinux.eu PyCon DE 2012, Leipzig Oct 31st, 2012 H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 1 / 27
2. x H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 1 / 27
3. me Holger Krekel <holger@merlinux.eu>, @hpk42 on twitter founded the PyPy project developer of py.test, tox and execnet Python since around 2000 Test-driven development since 2001 operates a small company “merlinux.eu” doing open source, Python and test-oriented consulting H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 1 / 27
4. why automated testing? to allow for later changes to raise confidence that code works to specify and document behaviour collaborative and faster development cycles H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 2 / 27
5. Developer oriented automated tests unittest: units react well to input. integration: components co-operate nicely functional: code changes work out in user environments H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 3 / 27
6. unittest assert that functions and classes behave as expected H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 4 / 27
7. integration assert units/components co-operate nicely H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 5 / 27
8. functional / system things work in user target environment H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 6 / 27
9. py.test fundamental features automatic test discovery, no-boilerplate test code useful information when a test fails cross-project, many options and plugins modular and parametrizable fixtures distribute tests to multiple hosts H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 7 / 27
10. cross-project test tool tests are run via py.test command line tool. project specific conftest.py files testing starts from files/dirs or current dir examples: py.test test_file.py py.test path/to/tests H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 8 / 27
11. automatic test discovery py.test walks over the filesystem and: discovers test_*.py test files discovers test_ functions and Test classes automatic discovery avoids boilerplate H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 9 / 27
12. no boilerplate python test code example test functions/methods: # content of test_module.py def test_something(): x = 3 assert x == 4 class TestSomething: def test_something(self): x = 1 assert x == 5 H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 10 / 27
13. test classes with pytest: test classes have a Test prefix, are autodiscovered there is no need to subclass anything test classes are for logic grouping of your tests H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 11 / 27
14. assert introspection def test_assert_introspection(): # with unittest.py assert x # assertTrue() assert x == 1 # assertEqual(x, 1) assert x != 2 # assertNotEqual(x, 2 assert not x # assertFalse(x) assert x < 3 and y >5 #? H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 12 / 27
15. asserting expected exceptions import pytest def test_raises_one(): pytest.raises(ValueError, lambda: int("foo")) def test_raises_two(): with pytest.raises(ValueError): int("foo") H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 13 / 27
16. Failure / Traceback Demo interactive demo: py.test failure_demo.py H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 14 / 27
17. Beyond simple testing: fixtures! a test fixture: sets up objects or apps for testing provides test code with “base” app objects very important to avoid repetetive test code in pytest realized via dependency injection: fixture functions create fixtures values test functions and classes can use them H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 15 / 27
18. Example of a simple pytest fixture fixture function and test function using it: # content of test_module.py import pytest @pytest.fixture def somevalue(): return 42 def test_function(somevalue): assert somevalue == 42 H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 16 / 27
19. notes about pytest fixtures fixtures are returned from a fixture function each fixture has a name (the function name) test functions get it injected as an argument by name next: fixtures can be cached on per scope basis fixture functions can be parametrized classes can use @pytest.mark.usefixtures(name1,...). H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 17 / 27
20. Scoping fixture functions fixture functions can declare a caching scope: import time, pytest @pytest.fixture(scope="module") def answer(): time.sleep(2) return 42 def test_one(answer): assert answer == 42 def test_two(answer): assert answer == 42 Takes 2 seconds to run, “answer()” is called once H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 18 / 27
21. Parametrizing fixtures and tests If we have multiple different “answer” fixtures: import pytest @pytest.fixture(params=[1,2]) def answer(request): return request.param now any tests using the answer fixture will run three times! H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 19 / 27
22. Modularity: using fixtures from fixtures Fixture functions can use fixtures themselves: import pytest @pytest.fixture def answer10(answer): return answer * 10 If you add this to the previous example, you get a answer10 fixture which internally uses the answer fixture (including parametrization!). Modularity for the win! H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 20 / 27
23. Classes can use fixtures as well You can use the usefixtures marker to mark all methods of a test class: import pytest @pytest.mark.usefixtures("cleandir") class TestCommandline: def test_method1(self): ... def test_method2(self): ... H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 21 / 27
24. implementation of cleandir fixture the cleandir fixture runs for each test method where it is an argument or where a usefixture marker is present. it might look like this: # content of conftest.py import pytest @pytest.fixture def cleandir(request, tmpdir): old = tmpdir.chdir() request.addfinalizer(old.chdir) H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 22 / 27
25. Other features looponfailing distributed testing coverage and coverage-html reporting timeout marking tests / skip and xfail selectively running tests conftest plugins plugins and hooks H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 23 / 27
26. comparisong with nose nose: clone of pytest from 2005, introduced plugins since developed separately shared many ideas and features since 2008 pytest also has plugins pytest fixtures/asserts not supported in nose pytest can run most nose-based tests suites H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 24 / 27
27. comparisong with unittest unittest: tests need to sublcass unittest.TestCAse no assert statement, >30 assertXYZ methods implicit fixtures (setup/teardown methods) extensions are incompatible to each other no parametrized testing no distributed testing pytest can run most unittest-test suites (and trial) H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 25 / 27
28. plugins, plugins ... pytest-xdist: distributed testing, looponfailing, boxed pytest-capturelog: tests with twisted integration pytest-timeout: time out tests pytest-coverage: test coverage reporting pytest-django: django database testing integration pytest-twisted: twisted testing integration pytest-pep8: configurable pep8-checks pytest-quickcheck: pass typed random data ... H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 26 / 27
29. Feedback and questions am here till friday - talk to me: holger krekel holger@merlinux.eu @hpk42 on twitter available for teaching and consulting H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 27 / 27