Testing Flask Applications with pytest

Testing a Flask application helps ensure that your app will work as expected for your end users. This talk details how to create and run tests with pytest, utilize fixtures to initialize the state of the Flask app, and check the coverage of the tests using coverage.py.

Transcript

1. Testing Flask Applications with pytest Patrick Kennedy 1
2. Outline • What should be tested in a Flask application? • Writing Flask-specific tests with pytest • Running tests with pytest • Fixtures • Coverage 2
3. Introduction 3
4. Why Write Tests? 4 • Testing helps ensure that your app will work as expected for your end users ◦ Fundamental part of the Test-Driven Development (TDD) process • Testing should be combined with a Continuous Integration (CI) process ◦ Tests are automatically run on code changes ◦ Helps catch problems before end users see them in production More about TDD: https://testdriven.io/test-driven-development/
5. What Should be Tested in a Flask App? • Unit tests: ◦ Focus on testing small units of code in isolation ◦ Examples: ▪ Database models ▪ Utility functions called by view functions • Functional tests: ◦ Focus on how the view functions operate under different conditions ◦ Examples: ▪ Nominal conditions (GET, POST, etc.) ▪ Invalid HTTP methods ▪ Invalid input data 5 Unit Tests Functional Tests End-to-End Tests Info about testing pyramid: https://martinfowler.com/articles/ practical-test-pyramid.html
6. What is pytest? • pytest is a test framework for Python: ◦ used to write, organize, and run test cases ◦ provides a lot of flexibility for running the tests • pytest satisfies the key aspects of a great test environment: ◦ tests are fun to write ◦ tests can be written quickly by using helper functions (fixtures) ◦ tests can be executed with a single command ◦ tests run quickly 6 pytest documentation: https://docs.pytest.org/en/latest/ Installation: (venv) $ pip install pytest
7. Writing Tests with pytest 7
8. Flask User Management Example 8 • Example Flask application: ◦ Homepage ◦ Register new users ◦ Log in/out users ◦ User Profile Source code for examples: https://gitlab.com/patkennedy79/flask_user_management_example
9. ├── app.py ├── project │ ├── __init__.py │ ├── models.py │ └── ...blueprint folders... ├── requirements.txt ├── tests │ ├── conftest.py │ ├── functional │ │ ├── __init__.py │ │ ├── test_recipes.py │ │ └── test_users.py │ └── unit │ ├── __init__.py │ └── test_models.py └── venv • tests folder contains all the tests for the project • Tests are divided into different folders for: ◦ functional tests ◦ unit tests Organization 9
10. Writing a Unit Test (1 / 2) from project import db from werkzeug.security import generate_password_hash, check_password_hash class User(db.Model): __tablename__ = 'users' id = db.Column(db.Integer, primary_key =True, autoincrement =True) email = db.Column(db.String, unique=True, nullable=False) password_hashed = db.Column(db.String( 128), nullable=False) def __init__(self, email: str, password_plaintext: str): """Create a new User object using the email address and hashing the plaintext password using Werkzeug.Security. """ self.email = email self.password_hashed = self._generate_password_hash(password_plaintext) @staticmethod def _generate_password_hash (password_plaintext: str): return generate_password_hash(password_plaintext) 10 Can a User object be created successfully?
11. Writing a Unit Test (2 / 2) from project.models import User def test_new_user(): """ GIVEN a User model WHEN a new User is created THEN check the email and password_hashed fields are defined correctly """ user = User('patkennedy79@gmail.com', 'FlaskIsAwesome') assert user.email == 'patkennedy79@gmail.com' assert user.password_hashed != 'FlaskIsAwesome' 11 Create a new `User` object Using `assert`, check the following: 1. Check that the email is stored correctly 2. Check that the hashed password does not equal the plaintext password tests/unit/test_models.py
12. • Maintenance is a challenging aspect of software development ◦ Documenting tests is often lacking or missing • GIVEN / WHEN / THEN structure provides a common structure for describing tests • GIVEN - what are the initial conditions for the test? • WHEN - what is occurring that needs to be tested? • THEN - what is the expected response? Documenting Tests 12 More about GIVEN/WHEN/THEN: https://martinfowler.com/bliki/GivenWhenThen.html
13. @recipes_blueprint.route ('/') def index(): return render_template('recipes/index.html') Writing a Functional Test (1 / 2) 13
14. Writing a Functional Test (2 / 2) 14 from project import create_app def test_home_page(): """ GIVEN a Flask application configured for testing WHEN the '/' page is requested (GET) THEN check that the response is valid """ flask_app = create_app('flask_test.cfg') # Create a test client using the Flask application configured for testing with flask_app.test_client() as test_client: response = test_client.get('/') assert response.status_code == 200 assert b’Flask User Management Example!’ in response.data assert b’Need an account?’ in response.data assert b’Existing user?’ in response.data Create a Flask application (assumes the Application Factory approach) Using `assert`, check the following: 1. 200 (OK) status returned 2. Expected text is in the response Utilize the `test_client` included with Flask to GET the homepage (‘/’ route) tests/functional/test_users.py
15. def test_home_page_post(): """ GIVEN a Flask application configured for testing WHEN the '/' page is is posted to (POST) THEN check that a '405' status code is returned """ flask_app = create_app('flask_test.cfg') # Create a test client using the Flask application configured for testing with flask_app.test_client() as test_client: response = test_client.post('/') assert response.status_code == 405 assert b’Flask User Management Example!’ not in response.data Writing a Functional Test - Off-Nominal 15 Using `assert`, check the following: 1. 405 (Method Not Allowed) status returned 2. Header text is NOT in the response Utilize the `test_client` included with Flask to POST to the homepage (‘/’ route) tests/functional/test_users.py
16. Running Tests with pytest 16
17. $ python -m pytest ============================== test session starts ============================== tests/functional/test_recipes.py .... [ 26%] tests/functional/test_users.py ....... [ 73%] tests/unit/test_models.py .... [100%] ============================== 15 passed in 1.13s =============================== Running pytest (1 / 2) 17 • Recommend running pytest through the Python interpreter ◦ https://docs.pytest.org/en/6.2.x/usage.html#calling-pytest-through-python-m-pytest
18. Running pytest (2 / 2) • `-v ` flag provides verbose output about the tests run: • Specific types of tests can be run: • Only run the failed tests from the last time pytest was run: 18 $ python -m pytest --last-failed $ python -m pytest tests/unit $ python -m pytest -v ========================== test session starts ================================ tests/functional/test_recipes.py::test_home_page PASSED [ 6%] ... tests/unit/test_models.py::test_setting_password PASSED [ 93%] tests/unit/test_models.py::test_user_id PASSED [100%] ============================= 15 passed in 1.05s ==============================
19. Fixtures 19
20. What is a Fixture? 20 • Fixtures initialize tests to a known state: ◦ Run tests in a predictable and repeatable manner • Fixtures are defined as functions (with a descriptive name!) • Multiple fixture can be run to initialize a test ◦ Fixtures can even call other fixtures! • Fixtures should be defined in `tests/conftest.py`
21. Scope of Fixtures 21 • Scope defines WHEN a fixture will be run • function - run once per test function (default scope) • class - run once per test class • module - run once per module (i.e. test file) • session - run once per session (i.e. per call to pytest)
22. def test_new_user_with_fixture(new_user): """ GIVEN a User model WHEN a new User is created THEN check the email and hashed_password are defined correctly """ assert new_user.email == 'patkennedy79@gmail.com' assert new_user.hashed_password != 'FlaskIsAwesome' from project.models import User @pytest.fixture(scope='module') def new_user(): user = User('patkennedy79@gmail.com', 'FlaskIsAwesome') return user Fixture to Help with Unit Testing 22 tests/unit/test_models.py tests/conftest.py
23. Fixture to Help with Functional Testing 23 from project import create_app @pytest.fixture(scope ='module') def test_client(): flask_app = create_app( 'flask_test.cfg' ) # Create a test client using the Flask application configured for testing with flask_app.test_client() as testing_client: yield testing_client # this is where the testing happens! def test_home_page_post_with_fixture (test_client): """ GIVEN a Flask application WHEN the '/' page is is posted to (POST) THEN check that a '405' status code is returned """ response = test_client .post('/') assert response.status_code == 405 assert b’Flask User Management Example!’ not in response.data tests/functional/test_users.py tests/conftest.py
24. Fixtures in Action! 24 $ python -m pytest --setup-show tests/functional/test_users.py ================================== test session starts ================================= tests/functional/test_recipes.py ... SETUP M test_client functional/test_users.py::test_home_page_with_fixture (fixtures used: test_client). ... TEARDOWN M test_client =================================== 4 passed in 0.18s ================================== Step 1 - test_client fixture is executed Step 3 - Execution returns to test_client fixture -> Done! Step 2 - test executes!
25. Coverage 25
26. • Code coverage is a metric of how much source code is executed when running the tests • Two excellent Python packages: ◦ coverage.py - tool for measuring code coverage of Python programs ◦ pytest-cov - pytest plugin for running coverage Code Coverage - Introduction 26 Coverage.py (by Ned Batcheldor): https://coverage.readthedocs.io/ Pytest-cov: https://pytest-cov.readthedocs.io/
27. Checking Coverage in pytest 27 $ python -m pytest --cov=project ================================= test session starts ================================== tests/functional/test_recipes.py .... [ 26%] tests/functional/test_users.py ....... [ 73%] tests/unit/test_models.py .... [100%] ---------- coverage: platform darwin, python 3.10.0-final-0 ---------- Name Stmts Miss Cover ------------------------------------------------- project/__init__.py 24 0 100% project/models.py 33 0 100% project/recipes/__init__.py 3 0 100% project/recipes/routes.py 5 0 100% project/users/__init__.py 3 0 100% project/users/forms.py 13 0 100% project/users/routes.py 47 0 100% ------------------------------------------------- TOTAL 128 0 100% ================================ 15 passed in 1.36s =================================== Any lines in the source code that are not executed are identified here Specify the directory containing the source code
28. Conclusion 28
29. Conclusion • pytest helps simplify testing Flask applications • Fixtures provide a consistent approach for initializing state for tests • Coverage is easy to collect with coverage.py and pytest-cov 29 For more complex examples of fixtures and test functions: https://gitlab.com/patkennedy79/flask_user_management_example “Testing Flask Applications with pytest” blog post: https://testdriven.io/blog/flask-pytest/
30. Thank you! 30 Patrick Kennedy Email: patkennedy79@gmail.com Twitter: patkennedy79 Personal Blog: https://www.patricksoftwareblog.com Learn Flask by Building and Deploying a Stock Portfolio App: https://testdriven.io/courses/learn-flask/