Improve Your Testing With Pytest and Mock Pycon SG - @gabehollombe June 19, 2015 

Hello! 

Helps you write better programs “ ” 

It's popular • PyPy - with over 21,000 tests • Sentry • tox • Cloudera • Spotify • many more 

Pytest's Key Benefits • Popular • Concise • Helpful assertion messages • Powerful fixtures • Many plugins 

Also… • runs on Posix/Windows, Python 2.6-3.4, PyPy • well tested with more than a thousand tests against itself • strict backward compatibility policy for safe upgrades • comprehensive online and PDF documentation • many third party plugins and builtin helpers • used in many small and large projects and organisations 

Installation pip install -U pytest py.test —-version 

Our First Test 

def plusone(x): return x + 1 def test_answer(): assert plusone(3) == 5 

py.test py.test py.test # from within a dir Running py.test 

$ py.test =========================== test session starts ============================ platform linux -- Python 3.4.0 -- py-1.4.26 -- pytest-2.6.4 collected 1 items test_sample.py F ================================= FAILURES ================================= _______________________________ test_answer ________________________________ def test_answer(): > assert plusone(3) == 5 E assert 4 == 5 E + where 4 = plusone(3) test_sample.py:5: AssertionError ========================= 1 failed in 0.01 seconds ========================= 

def test_answer(): > assert plusone(3) == 5 E assert 4 == 5 E + where 4 = plusone(3) test_sample.py:5: AssertionError 

Test discovery • collection starts from the initial command line arguments which may be directories, filenames or test ids. • recurse into directories, unless they match norecursedirs • test_*.py or *_test.py files • Test prefixed test classes (without an __init__ method) • test_ prefixed test functions or methods 

Context-Sensitive Comparisons 

assert 'foo 1 bar' == 'foo 2 bar' E - foo 1 bar E ? ^ E + foo 2 bar E ? ^ 

assert 'foo\nspam\nbar' == 'foo\neggs\nbar' E foo E - spam E + eggs E bar 

a = '1'*100 + 'a' + '2'*100 b = '1'*100 + 'b' + '2'*100 assert a == b E assert '111111111111...2222222222222' == '1111111111111...2222222222222' E Skipping 90 identical leading characters in diff, use -v to show E Skipping 91 identical trailing characters in diff, use -v to show E - 1111111111a222222222 E ? ^ E + 1111111111b222222222 E ? ^ 

assert set(['0', '1', '3', '8']) \ == \ set(['0', '3', '5', '8']) E Extra items in the left set: E '1' E Extra items in the right set: E '5' 

assert {'a': 0, 'b': 1, 'c': 0} \ == \ {'a': 0, 'b': 2, 'd': 0} E Omitting 1 identical items, use -v to show E Differing items: E {'b': 1} != {'b': 2} E Left contains more items: E {'c': 0} E Right contains more items: E {'d': 0} E Use -v to get the full diff 

Testing Exceptions 

# content of test_sysexit.py import pytest def f(): raise SystemExit(1) def test_mytest(): with pytest.raises(SystemExit): f() # (this test will pass) 

Classic xUnit
 Setup & Teardown 

# Runs once for this whole module def setup_module(module): print "setup_module" # Runs once for this whole module def teardown_module(module): print "teardown_module" 

# [setup / teardown module] 

# [setup / teardown module] class TestDB: # Runs once for all the tests @classmethod def setup_class(cls): print "\tsetup_class" # Runs once for all the tests @classmethod def teardown_class(cls): print "\t teardown_class" 

# [setup / teardown module] class TestDB: # [setup / teardown class] 

# [setup / teardown module] class TestDB: # [setup / teardown class] # Runs once for each test in this class def setup_method(self, method): print "\t\t setup_method" # Runs once for each test in this class def teardown_method(self, method): print "\t\t teardown_method" 

# [setup / teardown module] class TestDB: # [setup / teardown class] # [setup / teardown method] 

# [setup / teardown module] class TestDB: # [setup / teardown class] # [setup / teardown method] def test_one(self): print "\t\t\t test_one" def test_two(self): print "\t\t\t test_two" 

setup_module setup_class setup_method test_one teardown_method setup_method test_two teardown_method teardown_class teardown_module 

Fixtures 

Fixtures The purpose of test fixtures is to provide a fixed baseline upon which tests can reliably and repeatedly execute. In Pytest, fixtures have explicit names and are activated by declaring their use from test functions, modules, classes or whole projects (via dependency injection). 

# content of ./test_fixture_example.py import pytest class Person: def greet(): return "Hello there!" 

# content of ./test_fixture_example.py import pytest class Person: def greet(): return "Hello there!" @pytest.fixture def person(): return Person() 

# content of ./test_fixture_example.py import pytest class Person: def greet(): return "Hello there!" @pytest.fixture def person(): return Person() def test_greet(person): greeting = person.greet() assert greeting == "Hi there! 

• pytest sees that test_greet needs a function argument named person • A matching fixture function is discovered by looking for a fixture-marked function named person • person() is called to create an instance • test_greet() is called What's Happening 

# content of ./test_fixture_example.py import pytest class Person: def greet(): return "Hello there!" @pytest.fixture def person(): return Person() def test_greet(person): greeting = person.greet() assert greeting == "Hi there! 

> assert greeting == "Hi there!" E assert 'Hello there!' == 'Hi there!' E - Hi there! E + Hello there! 

• Can call other fixtures • Can be used to parameterize a test function against a collection of test values • Can be shared between test files via conftest.py • @pytest.fixture(autouse=True) acts as setup/teardown without explicit request by your test functions • You can even test your fixtures (they’re just functions) Fixtures are awesome 

Verifying Calls 

# content of mock_example.py: class DB: def __init__(self): pass def persist(self, person): pass 

# content of mock_example.py: class DB: def __init__(self): pass def persist(self, person): pass class Person: def __init__(self, name, db): self.name = name self.db = db def save(self): self.db.persist(self) 

# content of mock_test.py import pytest from mock import Mock from mock_example import Person, DB 

# content of mock_test.py import pytest from mock import Mock from mock_example import Person, DB @pytest.fixture def mock_db(): return Mock(spec=DB) # specs to follow . . . 

# content of mock_test.py (continued) def test_save_persists_to_db(mock_db): gabe = Person("Gabe", mock_db) gabe.save() mock_db.persist.assert_called_with(gabe) # Passes 

# content of mock_test.py (continued) def test_fail_not_called(mock_db): mock_db.persist.assert_called_with() # E AssertionError: Expected call: persist() # E Not called 

# content of mock_test.py (continued) def test_fail_called_with_other_arg(mock_db): mock_db.persist(1, 2, 3) mock_db.persist.assert_called_with() # E AssertionError: Expected call: persist() # E Actual call: persist(1, 2, 3) 

# content of mock_test.py (continued) def test_any_call(mock_db): mock_db.persist(1) mock_db.persist(2) mock_db.persist.assert_any_call(1) # Passes 

• assert_called_with only tracks the last call • if you call mock('bar') and then try mock.assert_any_call('foo'), it will fail, but it won’t tell you that mock('bar') was called • the docs are your friend Tiny gotchas (read the docs) 

Stubbing Return Values 

# content of forecaster.py import random class WeatherService: def barometer(self): # some unpredictable result here (live weather) random.choice(['rising', 'falling']) 

# content of forecaster.py continued class Forecaster: def __init__(self, weather_service): self.weather_service = weather_service def forecast(self): reading = self.weather_service.barometer() forecasts = dict( rising='Going to rain', falling='Looks clear', ) return forecasts[reading] 

import pytest from mock import Mock from forecaster import Forecaster, WeatherService @pytest.fixture def mock_ws(): return Mock(spec=WeatherService) 

import pytest from mock import Mock from forecaster import Forecaster, WeatherService @pytest.fixture def mock_ws(): return Mock(spec=WeatherService) def test_rain_when_barometer_rising(mock_ws): forecaster = Forecaster(mock_ws) mock_ws.barometer.return_value = 'rising' assert forecaster.forecast() == 'Going to rain' 

import pytest from mock import Mock from forecaster import Forecaster, WeatherService @pytest.fixture def mock_ws(): return Mock(spec=WeatherService) def test_rain_when_barometer_rising(mock_ws): forecaster = Forecaster(mock_ws) mock_ws.barometer.return_value = 'rising' assert forecaster.forecast() == 'Going to rain' def test_clear_when_barometer_falling(mock_ws): forecaster = Forecaster(mock_ws) mock_ws.barometer.return_value = 'falling' assert forecaster.forecast() == 'Looks clear' 

@pytest.mark.parametrize("reading, expected_forecast", [ ('rising', 'Going to rain'), ('falling', 'Looks clear'), ]) def test_forecast(reading, expected_forecast, mock_ws): forecaster = Forecaster(mock_ws) mock_ws.barometer.return_value = reading assert forecaster.forecast() == expected_forecast Refactored to use a parametrized test 

Monkeypatching 

# content of weather_service.py import random class WeatherService: def barometer(self): # some unpredictable result here (live weather) random.choice(['rising', 'falling']) 

# content of forecaster.py from weather_service import WeatherService class Forecaster: def __init__(self): self.weather_service = WeatherService() # . . . What if I can't inject a mock into my constructor? 

monkeypatch to the rescue • it's a special predefined fixture • use monkeypatch.setattr() to go in to a module and patch a value • pytest removes the patch when the test function returns 

def test_rain_when_barometer_rising(monkeypatch, mock_ws): WS = Mock(return_value=mock_ws) monkeypatch.setattr('forecaster.WeatherService', WS) forecaster = Forecaster() mock_ws.barometer.return_value = 'rising' assert forecaster.forecast() == 'Going to rain' Using the monkeypatch fixture 

Plugins 

pytest-allure-adaptor-1.5.4 pytest-ansible-1.1 pytest-autochecklog-0.1.2 pytest-bdd-2.6.1 pytest-beakerlib-0.5 pytest-beds-0.0.1 pytest-bench-0.3.0 pytest-benchmark-2.3.0 pytest-blockage-0.1 pytest-bpdb-0.1.4 pytest-browsermob-proxy-0.1 pytest-bugzilla-0.2 pytest-cache-1.0 pytest-cagoule-0.2.0 pytest-capturelog-0.7 pytest-catchlog-1.0 pytest-circleci-0.0.2 pytest-cloud-1.0.15 pytest-codecheckers-0.2 pytest-colordots-0.1 pytest-config-0.0.11 pytest-contextfixture-0.1.1 pytest-couchdbkit-0.5.1 pytest-cov-1.8.1 pytest-cpp-0.3.1 pytest-dbfixtures-0.9.0 pytest-dbus-notification-1.0.1 pytest-describe-0.10 pytest-diffeo-0.1.8.dev3 pytest-django-2.8.0 pytest-django-haystack-0.1.1 pytest-django-lite-0.1.1 pytest-django-sqlcount-0.1.0 pytest-echo-1.3 pytest-env-0.5.1 pytest-eradicate-0.0.2 pytest-figleaf-1.0 pytest-fixture-tools-1.0.0 pytest-flakes-0.2 pytest-flask-0.7.2 pytest-greendots-0.3 pytest-growl-0.2 pytest-httpbin-0.0.6 pytest-httpretty-0.2.0 pytest-incremental-0.3.0 pytest-instafail-0.3.0 pytest-ipdb-0.1-prerelease2 pytest-ipynb-0.1.1 pytest-jira-0.01 pytest-knows-0.1.5 pytest-konira-0.2 pytest-localserver-0.3.4 pytest-marker-bugzilla-0.06 pytest-markfiltration-0.8 pytest-marks-0.4 pytest-mock-0.4.0 pytest-monkeyplus-1.1.0 pytest-mozwebqa-1.5 pytest-multihost-0.6 pytest-oerp-0.2.0 pytest-oot-0.3.1 pytest-optional-0.0.2 pytest-ordering-0.3 pytest-osxnotify-0.1.5 pytest-paste-config-0.1 pytest-pep257-0.0.1 pytest-pep8-1.0.6 pytest-pipeline-0.1.0 pytest-poo-0.2 pytest-poo-fail-1.1 pytest-pycharm-0.3.0 pytest-pydev-0.1 pytest-pyq-1.1 pytest-pythonpath-0.6 pytest-qt-1.2.3 pytest-quickcheck-0.8.1 pytest-rage-0.1 pytest-raisesregexp-2.0 pytest-random-0.02 pytest-readme-1.0.0 pytest-regtest-0.5.1 pytest-remove-stale-bytecode-1.0 pytest-rerunfailures-0.05 pytest-runfailed-0.3 pytest-runner-2.3 pytest-services-1.0.4 pytest-sftpserver-1.1.0 pytest-smartcov-0.1 pytest-sourceorder-0.4 pytest-spec-0.2.24 pytest-splinter-1.2.10 pytest-stepwise-0.2 pytest-sugar-0.3.6 pytest-timeout-0.4 pytest-tornado-0.4.2 pytest-translations-0.2.0 pytest-twisted-1.5 pytest-unmarked-1.1 pytest-watch-2.0.0 pytest-xdist-1.11 pytest-xprocess-0.8 pytest-yamlwsgi-0.6 pytest-zap-0.2 

Invoke the python debugger for each failure
 pytest --pdb
 
 Debug first failure, then stop running tests
 pytest -x --pdb
 
 Prefer ipdb? Use mverteuil/pytest-ipdb
 pip install \ 
 git+git://github.com/mverteuil/pytest-ipdb.git 
 pytest --ipdb
 

Time for one more tip? 

import pytest class Developer: def __init__(self, favorite): self.loves=favorite def brag(self): return self.loves + " is the best!" 

# class Developer . . . 

# class Developer . . . @pytest.fixture def language(): return 'python' @pytest.fixture def developer(language): return Developer(language) 

# class Developer . . . # fixtures . . . 

# class Developer . . . # fixtures . . . def test_brag(developer): assert developer.brag() == "python is the best!" 

# class Developer . . . # fixtures . . . # test_brag . . . What if we want to make this developer instance love ruby instead? 

# class Developer . . . # fixtures . . . # test_brag . . . You can parametrize dependent fixtures! 

# class Developer . . . # fixtures . . . # test_brag . . . @pytest.mark.parametrize('language', ['ruby']) def test_brag_works_for_ruby(developer): assert developer.brag() == "ruby is the best!" 

# class Developer . . . @pytest.fixture def language(): return 'python' @pytest.fixture def developer(language): return Developer(language) 

That's all for today. 

Where to read more • pytest.org • www.voidspace.org.uk/python/mock 

Questions? 

Thank you! @gabehollombe [email protected]