Testing in Python: A Primer for Madison Python User's Group

A short introduction to unittest, pytest, and mock for people new to testing in python.

Transcript

1. TESTING IN PYTHON
   A PRIMER
   

   View Slide

2. OVERVIEW
   • Vocabulary
   • unittest (the module)
   • pytest
   • mock
   

   View Slide

3. VOCABULARY
   

   View Slide

4. UNIT TEST
   VOCABULARY
   • A unit test exercises a unit of work (or code)
   • Unit tests should be small and thus fast (100s of tests in a couple
   seconds fast)
   • Unit tests should ignore collaborators
   • Unit tests should test one bit of behavior
   • Units of code should be well factored and will make unit testing
   easier
   

   View Slide

5. INTEGRATION
   VOCABULARY
   • Tests that work with collaborators
   • For example:
   • If class A uses an instance of class B, class B is the collaborator
   • If class A talks to example.com, example.com is the collaborator
   • If a unit of code opens a file, that file is the collaborator
   • Disclaimer: We will not discuss integration testing here
   

   View Slide

6. INDEPENDENCE
   VOCABULARY
   • Each test should not rely on state created by another test
   • Tests should be able to be run in any order and still pass
   • Tests should be runnable in isolation
   

   View Slide

7. TEST PARAMETRIZATION/PARAMETERIZATION
   VOCABULARY
   • Yes, you can spell parameterization as parametrization. Yes, it
   trips me up very often
   • This allows you to have many different inputs to a test function so
   you can write fewer lines of code but have more tests. If you have
   a test function with N items parametrizing it, you will have N tests
   appear in your test run. 1 for each set of parameters.
   

   View Slide

8. FIXTURES
   VOCABULARY
   • Fixtures are constant bits of data used in tests
   • Fixtures can also be reusable bits of code (especially when your
   code is data)
   

   View Slide

9. COVERAGE
   VOCABULARY
   • Test coverage measures what lines of code and branches in your
   code are exercised by your tests
   • Many projects aim for 100% test coverage
   • Test coverage is a lie but still very useful to have
   

   View Slide

10. TEST DRIVEN DEVELOPMENT
    VOCABULARY
    • Also known as “TDD”
    • Refers to the practice of writing a test before writing the code
    that the test exercises
    • If you are adding a new method to a class, you write the test for
    that method first, then write the method and run the tests to see
    if the method satisfies the test you wrote
    

    View Slide

11. HOW MANY IS
    TOO MANY?
    

    View Slide

12. EXAMPLE CODE
    

    View Slide

13. def file_iterator(filename):
    with open(filename) as fd:
    for line in fd:
    yield line.strip()
    

    View Slide

14. TESTING TOOLS
    

    View Slide

15. UNITTEST
    TESTING TOOLS
    • Benefits:
    • Standard library
    • Excellent test collection
    • Familiarity for most people inside and out of Python (xUnit
    style)
    

    View Slide

16. UNITTEST
    TESTING TOOLS
    • Disadvantages:
    • Everything in a class
    • Tests must be an actual module
    • Different between versions of Python
    

    View Slide

17. UNITTEST - EXAMPLE
    TESTING TOOLS
    from unittest import TestCase
    class TestExample(TestCase):
    def test_strips_newlines(self):
    for line in file_iterator(
    ‘example.py’):
    self.assertFalse(
    line.endswith(‘\n’))
    

    View Slide

18. UNITTEST - EXAMPLE
    TESTING TOOLS
    def test_strips_leading_ws(self):
    “””Verify we strip leading
    whitespace.”””
    for line in file_iterator(
    ‘example.py’):
    self.assertFalse(
    line.startswith(‘ ’))
    

    View Slide

19. UNITTEST
    TESTING TOOLS
    • Assertions - self.assert*
    • assertEqual/assertNotEqual
    • assertTrue/assertFalse/assertIs/assertIsNot
    • assertIsNone/assertIsNotNone
    • assertIn/assertNotIn
    • assertIsInstance/assertNotIsInstance
    • and more - https://docs.python.org/3/library/unittest.html?
    highlight=unittest.testcase#unittest.TestCase
    

    View Slide

20. UNITTEST - EXAMPLE
    TESTING TOOLS
    def assertNotEndswith(self, line,
    text):
    self.assertFalse(line.endswith(
    text))
    # in our test
    self.assertNotEndswith(line, ‘\n’)
    

    View Slide

21. PYTEST
    TESTING TOOLS
    • Benefits:
    • Mature, well maintained, frequently updated
    • Beautiful test output + tonnes of plugins
    • Does not require classes (tests can be plain functions)
    • Uses assert statement, supports parametrization, has test
    fixtures
    • Works perfectly with unittest
    

    View Slide

22. PYTEST
    TESTING TOOLS
    • Disadvantages:
    • Not in the standard library
    • python -m pip install pytest
    • Has some very sharp very narrow and very bizarre corner cases

    (not going to go into these now or in questions)
    

    View Slide

23. UNITTEST - REFRESHER
    TESTING TOOLS
    from unittest import TestCase
    class TestExample(TestCase):
    def test_strips_newlines(self):
    for line in file_iterator(
    ‘example.py’):
    self.assertFalse(
    line.endswith(‘\n’))
    

    View Slide

24. PYTEST - ASSERT EXAMPLE
    TESTING TOOLS
    def test_strips_newlines():
    for line in file_iterator(
    ‘example.py’):
    assert (line.endswith(‘\n’)
    is False)
    

    View Slide

25. PYTEST - PARAMETRIZATION EXAMPLE
    TESTING TOOLS
    import pytest
    @pytest.mark.parametrize(‘file’, [
    ‘example.py’, ‘example2.py’])
    def test_strips_newlines(file):
    for line in file_iterator(file):
    assert (line.endswith(‘\n’))
    is False)
    

    View Slide

26. PYTEST - FIXTURE EXAMPLE
    TESTING TOOLS
    import pytest
    @pytest.fixture
    def file_iter():
    return file_iterator(‘ex.py’)
    def test_strips_nl(file_iter):
    for line in file_iter:
    assert (line.endswith(‘\n’))
    is False)
    

    View Slide

27. MOCK
    TESTING TOOLS
    • Benefits:
    • Mature, well maintained, frequently updated
    • Helps excise pesky collaborators
    • So valuable it was added to the standard library in Python 3.4
    • Actually lives as unittest.mock in Python 3.4+
    • Can patch out objects, functions, etc.
    • Works with unittest, pytest, whatever
    

    View Slide

28. PYTEST
    TESTING TOOLS
    • Disadvantages:
    • Not in the standard library before Python 3.4
    • python -m pip install mock
    • Some people find it very confusing
    

    View Slide

29. MOCK - EXAMPLE
    TESTING TOOLS
    from mock import Mock, patch
    def test_strips_nl(file_iter):
    file = Mock()
    file.__iter__.return_value = [
    ‘0\n’, ‘1\n’]
    with patch(‘open’) as mockopen:
    mockopen.return_value = file
    assert list(file_iter) == [
    ‘0’, ‘1’]
    

    View Slide

30. MOCK - EXAMPLE
    TESTING TOOLS
    from mock import Mock, patch
    def test_strips_nl(file_iter):
    # … snip …
    mockopen.assert_called_once_with(
    ‘ex.py’)
    assert (file.__iter__.called is
    True)
    

    View Slide

31. MOCK - ADDENDUM
    TESTING TOOLS
    • There are times to not use mock:
    • When you’re talking over a network and trying to fake out data
    

    View Slide

32. THANK YOU
    

    View Slide