Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing in Python: A Primer for Madison Python User's Group

Testing in Python: A Primer for Madison Python User's Group

A short introduction to unittest, pytest, and mock for people new to testing in python.

Ian Cordasco

August 11, 2016
Tweet

More Decks by Ian Cordasco

Other Decks in Programming

Transcript

  1. TESTING IN PYTHON
    A PRIMER

    View full-size slide

  2. OVERVIEW
    • Vocabulary
    • unittest (the module)
    • pytest
    • mock

    View full-size slide

  3. UNIT TEST
    VOCABULARY
    • A unit test exercises a unit of work (or code)
    • Unit tests should be small and thus fast (100s of tests in a couple
    seconds fast)
    • Unit tests should ignore collaborators
    • Unit tests should test one bit of behavior
    • Units of code should be well factored and will make unit testing
    easier

    View full-size slide

  4. INTEGRATION
    VOCABULARY
    • Tests that work with collaborators
    • For example:
    • If class A uses an instance of class B, class B is the collaborator
    • If class A talks to example.com, example.com is the collaborator
    • If a unit of code opens a file, that file is the collaborator
    • Disclaimer: We will not discuss integration testing here

    View full-size slide

  5. INDEPENDENCE
    VOCABULARY
    • Each test should not rely on state created by another test
    • Tests should be able to be run in any order and still pass
    • Tests should be runnable in isolation

    View full-size slide

  6. TEST PARAMETRIZATION/PARAMETERIZATION
    VOCABULARY
    • Yes, you can spell parameterization as parametrization. Yes, it
    trips me up very often
    • This allows you to have many different inputs to a test function so
    you can write fewer lines of code but have more tests. If you have
    a test function with N items parametrizing it, you will have N tests
    appear in your test run. 1 for each set of parameters.

    View full-size slide

  7. FIXTURES
    VOCABULARY
    • Fixtures are constant bits of data used in tests
    • Fixtures can also be reusable bits of code (especially when your
    code is data)

    View full-size slide

  8. COVERAGE
    VOCABULARY
    • Test coverage measures what lines of code and branches in your
    code are exercised by your tests
    • Many projects aim for 100% test coverage
    • Test coverage is a lie but still very useful to have

    View full-size slide

  9. TEST DRIVEN DEVELOPMENT
    VOCABULARY
    • Also known as “TDD”
    • Refers to the practice of writing a test before writing the code
    that the test exercises
    • If you are adding a new method to a class, you write the test for
    that method first, then write the method and run the tests to see
    if the method satisfies the test you wrote

    View full-size slide

  10. HOW MANY IS
    TOO MANY?

    View full-size slide

  11. EXAMPLE CODE

    View full-size slide

  12. def file_iterator(filename):
    with open(filename) as fd:
    for line in fd:
    yield line.strip()

    View full-size slide

  13. TESTING TOOLS

    View full-size slide

  14. UNITTEST
    TESTING TOOLS
    • Benefits:
    • Standard library
    • Excellent test collection
    • Familiarity for most people inside and out of Python (xUnit
    style)

    View full-size slide

  15. UNITTEST
    TESTING TOOLS
    • Disadvantages:
    • Everything in a class
    • Tests must be an actual module
    • Different between versions of Python

    View full-size slide

  16. UNITTEST - EXAMPLE
    TESTING TOOLS
    from unittest import TestCase
    class TestExample(TestCase):
    def test_strips_newlines(self):
    for line in file_iterator(
    ‘example.py’):
    self.assertFalse(
    line.endswith(‘\n’))

    View full-size slide

  17. UNITTEST - EXAMPLE
    TESTING TOOLS
    def test_strips_leading_ws(self):
    “””Verify we strip leading
    whitespace.”””
    for line in file_iterator(
    ‘example.py’):
    self.assertFalse(
    line.startswith(‘ ’))

    View full-size slide

  18. UNITTEST
    TESTING TOOLS
    • Assertions - self.assert*
    • assertEqual/assertNotEqual
    • assertTrue/assertFalse/assertIs/assertIsNot
    • assertIsNone/assertIsNotNone
    • assertIn/assertNotIn
    • assertIsInstance/assertNotIsInstance
    • and more - https://docs.python.org/3/library/unittest.html?
    highlight=unittest.testcase#unittest.TestCase

    View full-size slide

  19. UNITTEST - EXAMPLE
    TESTING TOOLS
    def assertNotEndswith(self, line,
    text):
    self.assertFalse(line.endswith(
    text))
    # in our test
    self.assertNotEndswith(line, ‘\n’)

    View full-size slide

  20. PYTEST
    TESTING TOOLS
    • Benefits:
    • Mature, well maintained, frequently updated
    • Beautiful test output + tonnes of plugins
    • Does not require classes (tests can be plain functions)
    • Uses assert statement, supports parametrization, has test
    fixtures
    • Works perfectly with unittest

    View full-size slide

  21. PYTEST
    TESTING TOOLS
    • Disadvantages:
    • Not in the standard library
    • python -m pip install pytest
    • Has some very sharp very narrow and very bizarre corner cases

    (not going to go into these now or in questions)

    View full-size slide

  22. UNITTEST - REFRESHER
    TESTING TOOLS
    from unittest import TestCase
    class TestExample(TestCase):
    def test_strips_newlines(self):
    for line in file_iterator(
    ‘example.py’):
    self.assertFalse(
    line.endswith(‘\n’))

    View full-size slide

  23. PYTEST - ASSERT EXAMPLE
    TESTING TOOLS
    def test_strips_newlines():
    for line in file_iterator(
    ‘example.py’):
    assert (line.endswith(‘\n’)
    is False)

    View full-size slide

  24. PYTEST - PARAMETRIZATION EXAMPLE
    TESTING TOOLS
    import pytest
    @pytest.mark.parametrize(‘file’, [
    ‘example.py’, ‘example2.py’])
    def test_strips_newlines(file):
    for line in file_iterator(file):
    assert (line.endswith(‘\n’))
    is False)

    View full-size slide

  25. PYTEST - FIXTURE EXAMPLE
    TESTING TOOLS
    import pytest
    @pytest.fixture
    def file_iter():
    return file_iterator(‘ex.py’)
    def test_strips_nl(file_iter):
    for line in file_iter:
    assert (line.endswith(‘\n’))
    is False)

    View full-size slide

  26. MOCK
    TESTING TOOLS
    • Benefits:
    • Mature, well maintained, frequently updated
    • Helps excise pesky collaborators
    • So valuable it was added to the standard library in Python 3.4
    • Actually lives as unittest.mock in Python 3.4+
    • Can patch out objects, functions, etc.
    • Works with unittest, pytest, whatever

    View full-size slide

  27. PYTEST
    TESTING TOOLS
    • Disadvantages:
    • Not in the standard library before Python 3.4
    • python -m pip install mock
    • Some people find it very confusing

    View full-size slide

  28. MOCK - EXAMPLE
    TESTING TOOLS
    from mock import Mock, patch
    def test_strips_nl(file_iter):
    file = Mock()
    file.__iter__.return_value = [
    ‘0\n’, ‘1\n’]
    with patch(‘open’) as mockopen:
    mockopen.return_value = file
    assert list(file_iter) == [
    ‘0’, ‘1’]

    View full-size slide

  29. MOCK - EXAMPLE
    TESTING TOOLS
    from mock import Mock, patch
    def test_strips_nl(file_iter):
    # … snip …
    mockopen.assert_called_once_with(
    ‘ex.py’)
    assert (file.__iter__.called is
    True)

    View full-size slide

  30. MOCK - ADDENDUM
    TESTING TOOLS
    • There are times to not use mock:
    • When you’re talking over a network and trying to fake out data

    View full-size slide