Testing in Python: A Primer for Madison Python User's Group

A short introduction to unittest, pytest, and mock for people new to testing in python.

Transcript

1. TESTING IN PYTHON A PRIMER
2. OVERVIEW • Vocabulary • unittest (the module) • pytest • mock
3. VOCABULARY
4. UNIT TEST VOCABULARY • A unit test exercises a unit of work (or code) • Unit tests should be small and thus fast (100s of tests in a couple seconds fast) • Unit tests should ignore collaborators • Unit tests should test one bit of behavior • Units of code should be well factored and will make unit testing easier
5. INTEGRATION VOCABULARY • Tests that work with collaborators • For example: • If class A uses an instance of class B, class B is the collaborator • If class A talks to example.com, example.com is the collaborator • If a unit of code opens a file, that file is the collaborator • Disclaimer: We will not discuss integration testing here
6. INDEPENDENCE VOCABULARY • Each test should not rely on state created by another test • Tests should be able to be run in any order and still pass • Tests should be runnable in isolation
7. TEST PARAMETRIZATION/PARAMETERIZATION VOCABULARY • Yes, you can spell parameterization as parametrization. Yes, it trips me up very often • This allows you to have many different inputs to a test function so you can write fewer lines of code but have more tests. If you have a test function with N items parametrizing it, you will have N tests appear in your test run. 1 for each set of parameters.
8. FIXTURES VOCABULARY • Fixtures are constant bits of data used in tests • Fixtures can also be reusable bits of code (especially when your code is data)
9. COVERAGE VOCABULARY • Test coverage measures what lines of code and branches in your code are exercised by your tests • Many projects aim for 100% test coverage • Test coverage is a lie but still very useful to have
10. TEST DRIVEN DEVELOPMENT VOCABULARY • Also known as “TDD” • Refers to the practice of writing a test before writing the code that the test exercises • If you are adding a new method to a class, you write the test for that method first, then write the method and run the tests to see if the method satisfies the test you wrote
11. HOW MANY IS TOO MANY?
12. EXAMPLE CODE
13. def file_iterator(filename): with open(filename) as fd: for line in fd: yield line.strip()
14. TESTING TOOLS
15. UNITTEST TESTING TOOLS • Benefits: • Standard library • Excellent test collection • Familiarity for most people inside and out of Python (xUnit style)
16. UNITTEST TESTING TOOLS • Disadvantages: • Everything in a class • Tests must be an actual module • Different between versions of Python
17. UNITTEST - EXAMPLE TESTING TOOLS from unittest import TestCase class TestExample(TestCase): def test_strips_newlines(self): for line in file_iterator( ‘example.py’): self.assertFalse( line.endswith(‘\n’))
18. UNITTEST - EXAMPLE TESTING TOOLS def test_strips_leading_ws(self): “””Verify we strip leading whitespace.””” for line in file_iterator( ‘example.py’): self.assertFalse( line.startswith(‘ ’))
19. UNITTEST TESTING TOOLS • Assertions - self.assert* • assertEqual/assertNotEqual • assertTrue/assertFalse/assertIs/assertIsNot • assertIsNone/assertIsNotNone • assertIn/assertNotIn • assertIsInstance/assertNotIsInstance • and more - https://docs.python.org/3/library/unittest.html? highlight=unittest.testcase#unittest.TestCase
20. UNITTEST - EXAMPLE TESTING TOOLS def assertNotEndswith(self, line, text): self.assertFalse(line.endswith( text)) # in our test self.assertNotEndswith(line, ‘\n’)
21. PYTEST TESTING TOOLS • Benefits: • Mature, well maintained, frequently updated • Beautiful test output + tonnes of plugins • Does not require classes (tests can be plain functions) • Uses assert statement, supports parametrization, has test fixtures • Works perfectly with unittest
22. PYTEST TESTING TOOLS • Disadvantages: • Not in the standard library • python -m pip install pytest • Has some very sharp very narrow and very bizarre corner cases
 (not going to go into these now or in questions)
23. UNITTEST - REFRESHER TESTING TOOLS from unittest import TestCase class TestExample(TestCase): def test_strips_newlines(self): for line in file_iterator( ‘example.py’): self.assertFalse( line.endswith(‘\n’))
24. PYTEST - ASSERT EXAMPLE TESTING TOOLS def test_strips_newlines(): for line in file_iterator( ‘example.py’): assert (line.endswith(‘\n’) is False)
25. PYTEST - PARAMETRIZATION EXAMPLE TESTING TOOLS import pytest @pytest.mark.parametrize(‘file’, [ ‘example.py’, ‘example2.py’]) def test_strips_newlines(file): for line in file_iterator(file): assert (line.endswith(‘\n’)) is False)
26. PYTEST - FIXTURE EXAMPLE TESTING TOOLS import pytest @pytest.fixture def file_iter(): return file_iterator(‘ex.py’) def test_strips_nl(file_iter): for line in file_iter: assert (line.endswith(‘\n’)) is False)
27. MOCK TESTING TOOLS • Benefits: • Mature, well maintained, frequently updated • Helps excise pesky collaborators • So valuable it was added to the standard library in Python 3.4 • Actually lives as unittest.mock in Python 3.4+ • Can patch out objects, functions, etc. • Works with unittest, pytest, whatever
28. PYTEST TESTING TOOLS • Disadvantages: • Not in the standard library before Python 3.4 • python -m pip install mock • Some people find it very confusing
29. MOCK - EXAMPLE TESTING TOOLS from mock import Mock, patch def test_strips_nl(file_iter): file = Mock() file.__iter__.return_value = [ ‘0\n’, ‘1\n’] with patch(‘open’) as mockopen: mockopen.return_value = file assert list(file_iter) == [ ‘0’, ‘1’]
30. MOCK - EXAMPLE TESTING TOOLS from mock import Mock, patch def test_strips_nl(file_iter): # … snip … mockopen.assert_called_once_with( ‘ex.py’) assert (file.__iter__.called is True)
31. MOCK - ADDENDUM TESTING TOOLS • There are times to not use mock: • When you’re talking over a network and trying to fake out data
32. THANK YOU