Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing Flask Applications with pytest

Patrick
November 29, 2021

Testing Flask Applications with pytest

Testing a Flask application helps ensure that your app will work as expected for your end users. This talk details how to create and run tests with pytest, utilize fixtures to initialize the state of the Flask app, and check the coverage of the tests using coverage.py.

Patrick

November 29, 2021
Tweet

More Decks by Patrick

Other Decks in Programming

Transcript

  1. Testing
    Flask Applications
    with pytest
    Patrick Kennedy
    1

    View full-size slide

  2. Outline
    ● What should be tested in a Flask application?
    ● Writing Flask-specific tests with pytest
    ● Running tests with pytest
    ● Fixtures
    ● Coverage
    2

    View full-size slide

  3. Introduction
    3

    View full-size slide

  4. Why Write Tests?
    4
    ● Testing helps ensure that your app will work as expected for your
    end users
    ○ Fundamental part of the Test-Driven Development (TDD) process
    ● Testing should be combined with a Continuous Integration (CI)
    process
    ○ Tests are automatically run on code changes
    ○ Helps catch problems before end users see them in production
    More about TDD: https://testdriven.io/test-driven-development/

    View full-size slide

  5. What Should be Tested in a Flask App?
    ● Unit tests:
    ○ Focus on testing small units of code in isolation
    ○ Examples:
    ■ Database models
    ■ Utility functions called by view functions
    ● Functional tests:
    ○ Focus on how the view functions operate under different conditions
    ○ Examples:
    ■ Nominal conditions (GET, POST, etc.)
    ■ Invalid HTTP methods
    ■ Invalid input data
    5
    Unit Tests
    Functional Tests
    End-to-End
    Tests
    Info about testing pyramid:
    https://martinfowler.com/articles/
    practical-test-pyramid.html

    View full-size slide

  6. What is pytest?
    ● pytest is a test framework for Python:
    ○ used to write, organize, and run test cases
    ○ provides a lot of flexibility for running the tests
    ● pytest satisfies the key aspects of a great test environment:
    ○ tests are fun to write
    ○ tests can be written quickly by using helper functions (fixtures)
    ○ tests can be executed with a single command
    ○ tests run quickly
    6
    pytest documentation: https://docs.pytest.org/en/latest/
    Installation:
    (venv) $ pip install pytest

    View full-size slide

  7. Writing Tests with pytest
    7

    View full-size slide

  8. Flask User Management Example
    8
    ● Example Flask application:
    ○ Homepage
    ○ Register new users
    ○ Log in/out users
    ○ User Profile
    Source code for examples: https://gitlab.com/patkennedy79/flask_user_management_example

    View full-size slide

  9. ├── app.py
    ├── project
    │ ├── __init__.py
    │ ├── models.py
    │ └── ...blueprint folders...
    ├── requirements.txt
    ├── tests
    │ ├── conftest.py
    │ ├── functional
    │ │ ├── __init__.py
    │ │ ├── test_recipes.py
    │ │ └── test_users.py
    │ └── unit
    │ ├── __init__.py
    │ └── test_models.py
    └── venv
    ● tests folder contains all the
    tests for the project
    ● Tests are divided into different
    folders for:
    ○ functional tests
    ○ unit tests
    Organization
    9

    View full-size slide

  10. Writing a Unit Test (1 / 2)
    from project import db
    from werkzeug.security import generate_password_hash, check_password_hash
    class User(db.Model):
    __tablename__ = 'users'
    id = db.Column(db.Integer, primary_key =True, autoincrement =True)
    email = db.Column(db.String, unique=True, nullable=False)
    password_hashed = db.Column(db.String( 128), nullable=False)
    def __init__(self, email: str, password_plaintext: str):
    """Create a new User object using the email address and hashing the
    plaintext password using Werkzeug.Security.
    """
    self.email = email
    self.password_hashed = self._generate_password_hash(password_plaintext)
    @staticmethod
    def _generate_password_hash (password_plaintext: str):
    return generate_password_hash(password_plaintext)
    10
    Can a User object be
    created successfully?

    View full-size slide

  11. Writing a Unit Test (2 / 2)
    from project.models import User
    def test_new_user():
    """
    GIVEN a User model
    WHEN a new User is created
    THEN check the email and password_hashed fields are defined correctly
    """
    user = User('[email protected]', 'FlaskIsAwesome')
    assert user.email == '[email protected]'
    assert user.password_hashed != 'FlaskIsAwesome'
    11
    Create a new `User` object
    Using `assert`, check the following:
    1. Check that the email is stored correctly
    2. Check that the hashed password does not
    equal the plaintext password
    tests/unit/test_models.py

    View full-size slide

  12. ● Maintenance is a challenging aspect of software development
    ○ Documenting tests is often lacking or missing
    ● GIVEN / WHEN / THEN structure provides a common structure for
    describing tests
    ● GIVEN - what are the initial conditions for the test?
    ● WHEN - what is occurring that needs to be tested?
    ● THEN - what is the expected response?
    Documenting Tests
    12
    More about GIVEN/WHEN/THEN: https://martinfowler.com/bliki/GivenWhenThen.html

    View full-size slide

  13. @recipes_blueprint.route ('/')
    def index():
    return render_template('recipes/index.html')
    Writing a Functional Test (1 / 2)
    13

    View full-size slide

  14. Writing a Functional Test (2 / 2)
    14
    from project import create_app
    def test_home_page():
    """
    GIVEN a Flask application configured for testing
    WHEN the '/' page is requested (GET)
    THEN check that the response is valid
    """
    flask_app = create_app('flask_test.cfg')
    # Create a test client using the Flask application configured for testing
    with flask_app.test_client() as test_client:
    response = test_client.get('/')
    assert response.status_code == 200
    assert b’Flask User Management Example!’ in response.data
    assert b’Need an account?’ in response.data
    assert b’Existing user?’ in response.data
    Create a Flask application
    (assumes the Application
    Factory approach)
    Using `assert`, check the following:
    1. 200 (OK) status returned
    2. Expected text is in the response
    Utilize the `test_client` included with Flask
    to GET the homepage (‘/’ route)
    tests/functional/test_users.py

    View full-size slide

  15. def test_home_page_post():
    """
    GIVEN a Flask application configured for testing
    WHEN the '/' page is is posted to (POST)
    THEN check that a '405' status code is returned
    """
    flask_app = create_app('flask_test.cfg')
    # Create a test client using the Flask application configured for testing
    with flask_app.test_client() as test_client:
    response = test_client.post('/')
    assert response.status_code == 405
    assert b’Flask User Management Example!’ not in response.data
    Writing a Functional Test - Off-Nominal
    15
    Using `assert`, check the following:
    1. 405 (Method Not Allowed) status returned
    2. Header text is NOT in the response
    Utilize the `test_client` included with Flask
    to POST to the homepage (‘/’ route)
    tests/functional/test_users.py

    View full-size slide

  16. Running Tests with pytest
    16

    View full-size slide

  17. $ python -m pytest
    ============================== test session starts ==============================
    tests/functional/test_recipes.py .... [ 26%]
    tests/functional/test_users.py ....... [ 73%]
    tests/unit/test_models.py .... [100%]
    ============================== 15 passed in 1.13s ===============================
    Running pytest (1 / 2)
    17
    ● Recommend running pytest through the Python interpreter
    ○ https://docs.pytest.org/en/6.2.x/usage.html#calling-pytest-through-python-m-pytest

    View full-size slide

  18. Running pytest (2 / 2)
    ● `-v ` flag provides verbose output about the tests run:
    ● Specific types of tests can be run:
    ● Only run the failed tests from the last time pytest was run:
    18
    $ python -m pytest --last-failed
    $ python -m pytest tests/unit
    $ python -m pytest -v
    ========================== test session starts ================================
    tests/functional/test_recipes.py::test_home_page PASSED [ 6%]
    ...
    tests/unit/test_models.py::test_setting_password PASSED [ 93%]
    tests/unit/test_models.py::test_user_id PASSED [100%]
    ============================= 15 passed in 1.05s ==============================

    View full-size slide

  19. What is a Fixture?
    20
    ● Fixtures initialize tests to a known state:
    ○ Run tests in a predictable and repeatable manner
    ● Fixtures are defined as functions (with a descriptive name!)
    ● Multiple fixture can be run to initialize a test
    ○ Fixtures can even call other fixtures!
    ● Fixtures should be defined in `tests/conftest.py`

    View full-size slide

  20. Scope of Fixtures
    21
    ● Scope defines WHEN a fixture will be run
    ● function - run once per test function (default scope)
    ● class - run once per test class
    ● module - run once per module (i.e. test file)
    ● session - run once per session (i.e. per call to pytest)

    View full-size slide

  21. def test_new_user_with_fixture(new_user):
    """
    GIVEN a User model
    WHEN a new User is created
    THEN check the email and hashed_password are defined correctly
    """
    assert new_user.email == '[email protected]'
    assert new_user.hashed_password != 'FlaskIsAwesome'
    from project.models import User
    @pytest.fixture(scope='module')
    def new_user():
    user = User('[email protected]', 'FlaskIsAwesome')
    return user
    Fixture to Help with Unit Testing
    22
    tests/unit/test_models.py
    tests/conftest.py

    View full-size slide

  22. Fixture to Help with Functional Testing
    23
    from project import create_app
    @pytest.fixture(scope ='module')
    def test_client():
    flask_app = create_app( 'flask_test.cfg' )
    # Create a test client using the Flask application configured for testing
    with flask_app.test_client() as testing_client:
    yield testing_client # this is where the testing happens!
    def test_home_page_post_with_fixture (test_client):
    """
    GIVEN a Flask application
    WHEN the '/' page is is posted to (POST)
    THEN check that a '405' status code is returned
    """
    response = test_client .post('/')
    assert response.status_code == 405
    assert b’Flask User Management Example!’ not in response.data
    tests/functional/test_users.py
    tests/conftest.py

    View full-size slide

  23. Fixtures in Action!
    24
    $ python -m pytest --setup-show tests/functional/test_users.py
    ================================== test session starts =================================
    tests/functional/test_recipes.py
    ...
    SETUP M test_client
    functional/test_users.py::test_home_page_with_fixture (fixtures used: test_client).
    ...
    TEARDOWN M test_client
    =================================== 4 passed in 0.18s ==================================
    Step 1 - test_client fixture is executed
    Step 3 - Execution returns to test_client fixture -> Done!
    Step 2 - test executes!

    View full-size slide

  24. ● Code coverage is a metric of how much source code is executed when
    running the tests
    ● Two excellent Python packages:
    ○ coverage.py - tool for measuring code coverage of Python programs
    ○ pytest-cov - pytest plugin for running coverage
    Code Coverage - Introduction
    26
    Coverage.py (by Ned Batcheldor): https://coverage.readthedocs.io/
    Pytest-cov: https://pytest-cov.readthedocs.io/

    View full-size slide

  25. Checking Coverage in pytest
    27
    $ python -m pytest --cov=project
    ================================= test session starts ==================================
    tests/functional/test_recipes.py .... [ 26%]
    tests/functional/test_users.py ....... [ 73%]
    tests/unit/test_models.py .... [100%]
    ---------- coverage: platform darwin, python 3.10.0-final-0 ----------
    Name Stmts Miss Cover
    -------------------------------------------------
    project/__init__.py 24 0 100%
    project/models.py 33 0 100%
    project/recipes/__init__.py 3 0 100%
    project/recipes/routes.py 5 0 100%
    project/users/__init__.py 3 0 100%
    project/users/forms.py 13 0 100%
    project/users/routes.py 47 0 100%
    -------------------------------------------------
    TOTAL 128 0 100%
    ================================ 15 passed in 1.36s ===================================
    Any lines in the source code that are
    not executed are identified here
    Specify the directory containing the source code

    View full-size slide

  26. Conclusion
    28

    View full-size slide

  27. Conclusion
    ● pytest helps simplify testing Flask applications
    ● Fixtures provide a consistent approach for initializing state for tests
    ● Coverage is easy to collect with coverage.py and pytest-cov
    29
    For more complex examples of fixtures and test functions:
    https://gitlab.com/patkennedy79/flask_user_management_example
    “Testing Flask Applications with pytest” blog post:
    https://testdriven.io/blog/flask-pytest/

    View full-size slide

  28. Thank you!
    30
    Patrick Kennedy
    Email: [email protected]
    Twitter: patkennedy79
    Personal Blog: https://www.patricksoftwareblog.com
    Learn Flask by Building and Deploying a Stock Portfolio App:
    https://testdriven.io/courses/learn-flask/

    View full-size slide