rapid testing (talk)

pytest rapid testing
Holger Krekel
http://merlinux.eu
PyCon DE 2012, Leipzig
Oct 31st, 2012
H. Krekel (PyCon DE 2012, Leipzig) py.test - rapid testing Oct 31st, 2012 1 / 27

View Slide

x

View Slide

me
Holger Krekel , @hpk42 on twitter
founded the PyPy project
developer of py.test, tox and execnet
Python since around 2000
Test-driven development since 2001
operates a small company “merlinux.eu” doing open
source, Python and test-oriented consulting

View Slide

why automated testing?
to allow for later changes
to raise conﬁdence that code works
to specify and document behaviour
collaborative and faster development cycles

View Slide

Developer oriented automated tests
unittest: units react well to input.
integration: components co-operate nicely
functional: code changes work out in user
environments

View Slide

unittest
assert that functions and classes behave as expected

View Slide

integration
assert units/components co-operate nicely

View Slide

functional / system
things work in user target environment

View Slide

py.test fundamental features
automatic test discovery, no-boilerplate test code
useful information when a test fails
cross-project, many options and plugins
modular and parametrizable ﬁxtures
distribute tests to multiple hosts

View Slide

cross-project test tool
tests are run via py.test command line tool.
project specific conftest.py files
testing starts from files/dirs or current dir
examples:
py.test test_file.py
py.test path/to/tests

View Slide

automatic test discovery
py.test walks over the ﬁlesystem and:
discovers test_*.py test ﬁles
discovers test_ functions and Test classes
automatic discovery avoids boilerplate

View Slide

no boilerplate python test code
example test functions/methods:
# content of test_module.py
def test_something():
x = 3
assert x == 4
class TestSomething:
def test_something(self):
x = 1
assert x == 5

View Slide

test classes
with pytest:
test classes have a Test preﬁx, are autodiscovered
there is no need to subclass anything
test classes are for logic grouping of your tests

View Slide

assert introspection
def test_assert_introspection():
# with unittest.py
assert x # assertTrue()
assert x == 1 # assertEqual(x, 1)
assert x != 2 # assertNotEqual(x, 2
assert not x # assertFalse(x)
assert x < 3 and y >5 #?

View Slide

asserting expected exceptions
import pytest
def test_raises_one():
pytest.raises(ValueError,
lambda: int("foo"))
def test_raises_two():
with pytest.raises(ValueError):
int("foo")

View Slide

Failure / Traceback Demo
interactive demo: py.test failure_demo.py

View Slide

Beyond simple testing: fixtures!
a test fixture:
sets up objects or apps for testing
provides test code with “base” app objects
very important to avoid repetetive test code
in pytest realized via dependency injection:
fixture functions create fixtures values
test functions and classes can use them

View Slide

Example of a simple pytest ﬁxture
ﬁxture function and test function using it:
# content of test_module.py
import pytest
@pytest.fixture
def somevalue():
return 42
def test_function(somevalue):
assert somevalue == 42

View Slide

notes about pytest fixtures
fixtures are returned from a fixture function
each fixture has a name (the function name)
test functions get it injected as an argument by name
next:
fixtures can be cached on per scope basis
fixture functions can be parametrized
classes can use
@pytest.mark.usefixtures(name1,...).

View Slide

Scoping ﬁxture functions
ﬁxture functions can declare a caching scope:
import time, pytest
@pytest.fixture(scope="module")
def answer():
time.sleep(2)
return 42
def test_one(answer):
assert answer == 42
def test_two(answer):
assert answer == 42
Takes 2 seconds to run, “answer()” is called once

View Slide

Parametrizing fixtures and tests
If we have multiple different “answer” fixtures:
import pytest
@pytest.fixture(params=[1,2])
def answer(request):
return request.param
now any tests using the answer fixture will run three
times!

View Slide

Modularity: using fixtures from fixtures
Fixture functions can use fixtures themselves:
import pytest
@pytest.fixture
def answer10(answer):
return answer * 10
If you add this to the previous example, you get a
answer10 fixture which internally uses the answer
fixture (including parametrization!).
Modularity for the win!

View Slide

Classes can use ﬁxtures as well
You can use the usefixtures marker to mark all
methods of a test class:
import pytest
@pytest.mark.usefixtures("cleandir")
class TestCommandline:
def test_method1(self):
...
def test_method2(self):
...

View Slide

implementation of cleandir ﬁxture
the cleandir ﬁxture runs for each test method where it is an
argument or where a usefixture marker is present.
it might look like this:
# content of conftest.py
import pytest
@pytest.fixture
def cleandir(request, tmpdir):
old = tmpdir.chdir()
request.addfinalizer(old.chdir)

View Slide

Other features
looponfailing
distributed testing
coverage and coverage-html reporting
timeout
marking tests / skip and xfail
selectively running tests
conftest plugins
plugins and hooks

View Slide

comparisong with nose
nose:
clone of pytest from 2005, introduced plugins
since developed separately
shared many ideas and features
since 2008 pytest also has plugins
pytest ﬁxtures/asserts not supported in nose
pytest can run most nose-based tests suites

View Slide

comparisong with unittest
unittest:
tests need to sublcass unittest.TestCAse
no assert statement, >30 assertXYZ methods
implicit ﬁxtures (setup/teardown methods)
extensions are incompatible to each other
no parametrized testing
no distributed testing
pytest can run most unittest-test suites (and trial)

View Slide

plugins, plugins ...
pytest-xdist: distributed testing, looponfailing, boxed
pytest-capturelog: tests with twisted integration
pytest-timeout: time out tests
pytest-coverage: test coverage reporting
pytest-django: django database testing integration
pytest-twisted: twisted testing integration
pytest-pep8: conﬁgurable pep8-checks
pytest-quickcheck: pass typed random data
...

View Slide

Feedback and questions
am here till friday - talk to me:
holger krekel
[email protected]
@hpk42 on twitter
available for teaching and consulting

View Slide

rapid testing (talk)

rapid testing (talk)

holger krekel

More Decks by holger krekel

Other Decks in Technology

Featured

Transcript