Testing Flask Applications with pytest Patrick Kennedy 1

Outline ● What should be tested in a Flask application? ● Writing Flask-specific tests with pytest ● Running tests with pytest ● Fixtures ● Coverage 2

Introduction 3

Why Write Tests? 4 ● Testing helps ensure that your app will work as expected for your end users ○ Fundamental part of the Test-Driven Development (TDD) process ● Testing should be combined with a Continuous Integration (CI) process ○ Tests are automatically run on code changes ○ Helps catch problems before end users see them in production More about TDD: https://testdriven.io/test-driven-development/

What Should be Tested in a Flask App? ● Unit tests: ○ Focus on testing small units of code in isolation ○ Examples: ■ Database models ■ Utility functions called by view functions ● Functional tests: ○ Focus on how the view functions operate under different conditions ○ Examples: ■ Nominal conditions (GET, POST, etc.) ■ Invalid HTTP methods ■ Invalid input data 5 Unit Tests Functional Tests End-to-End Tests Info about testing pyramid: https://martinfowler.com/articles/ practical-test-pyramid.html

What is pytest? ● pytest is a test framework for Python: ○ used to write, organize, and run test cases ○ provides a lot of flexibility for running the tests ● pytest satisfies the key aspects of a great test environment: ○ tests are fun to write ○ tests can be written quickly by using helper functions (fixtures) ○ tests can be executed with a single command ○ tests run quickly 6 pytest documentation: https://docs.pytest.org/en/latest/ Installation: (venv) $ pip install pytest

Writing Tests with pytest 7

Flask User Management Example 8 ● Example Flask application: ○ Homepage ○ Register new users ○ Log in/out users ○ User Profile Source code for examples: https://gitlab.com/patkennedy79/flask_user_management_example

├── app.py ├── project │ ├── __init__.py │ ├── models.py │ └── ...blueprint folders... ├── requirements.txt ├── tests │ ├── conftest.py │ ├── functional │ │ ├── __init__.py │ │ ├── test_recipes.py │ │ └── test_users.py │ └── unit │ ├── __init__.py │ └── test_models.py └── venv ● tests folder contains all the tests for the project ● Tests are divided into different folders for: ○ functional tests ○ unit tests Organization 9

Writing a Unit Test (1 / 2) from project import db from werkzeug.security import generate_password_hash, check_password_hash class User(db.Model): __tablename__ = 'users' id = db.Column(db.Integer, primary_key =True, autoincrement =True) email = db.Column(db.String, unique=True, nullable=False) password_hashed = db.Column(db.String( 128), nullable=False) def __init__(self, email: str, password_plaintext: str): """Create a new User object using the email address and hashing the plaintext password using Werkzeug.Security. """ self.email = email self.password_hashed = self._generate_password_hash(password_plaintext) @staticmethod def _generate_password_hash (password_plaintext: str): return generate_password_hash(password_plaintext) 10 Can a User object be created successfully?

Writing a Unit Test (2 / 2) from project.models import User def test_new_user(): """ GIVEN a User model WHEN a new User is created THEN check the email and password_hashed fields are defined correctly """ user = User('patkennedy79@gmail.com', 'FlaskIsAwesome') assert user.email == 'patkennedy79@gmail.com' assert user.password_hashed != 'FlaskIsAwesome' 11 Create a new `User` object Using `assert`, check the following: 1. Check that the email is stored correctly 2. Check that the hashed password does not equal the plaintext password tests/unit/test_models.py

● Maintenance is a challenging aspect of software development ○ Documenting tests is often lacking or missing ● GIVEN / WHEN / THEN structure provides a common structure for describing tests ● GIVEN - what are the initial conditions for the test? ● WHEN - what is occurring that needs to be tested? ● THEN - what is the expected response? Documenting Tests 12 More about GIVEN/WHEN/THEN: https://martinfowler.com/bliki/GivenWhenThen.html

@recipes_blueprint.route ('/') def index(): return render_template('recipes/index.html') Writing a Functional Test (1 / 2) 13

Writing a Functional Test (2 / 2) 14 from project import create_app def test_home_page(): """ GIVEN a Flask application configured for testing WHEN the '/' page is requested (GET) THEN check that the response is valid """ flask_app = create_app('flask_test.cfg') # Create a test client using the Flask application configured for testing with flask_app.test_client() as test_client: response = test_client.get('/') assert response.status_code == 200 assert b’Flask User Management Example!’ in response.data assert b’Need an account?’ in response.data assert b’Existing user?’ in response.data Create a Flask application (assumes the Application Factory approach) Using `assert`, check the following: 1. 200 (OK) status returned 2. Expected text is in the response Utilize the `test_client` included with Flask to GET the homepage (‘/’ route) tests/functional/test_users.py

def test_home_page_post(): """ GIVEN a Flask application configured for testing WHEN the '/' page is is posted to (POST) THEN check that a '405' status code is returned """ flask_app = create_app('flask_test.cfg') # Create a test client using the Flask application configured for testing with flask_app.test_client() as test_client: response = test_client.post('/') assert response.status_code == 405 assert b’Flask User Management Example!’ not in response.data Writing a Functional Test - Off-Nominal 15 Using `assert`, check the following: 1. 405 (Method Not Allowed) status returned 2. Header text is NOT in the response Utilize the `test_client` included with Flask to POST to the homepage (‘/’ route) tests/functional/test_users.py

Running Tests with pytest 16

$ python -m pytest ============================== test session starts ============================== tests/functional/test_recipes.py .... [ 26%] tests/functional/test_users.py ....... [ 73%] tests/unit/test_models.py .... [100%] ============================== 15 passed in 1.13s =============================== Running pytest (1 / 2) 17 ● Recommend running pytest through the Python interpreter ○ https://docs.pytest.org/en/6.2.x/usage.html#calling-pytest-through-python-m-pytest

Running pytest (2 / 2) ● `-v ` flag provides verbose output about the tests run: ● Specific types of tests can be run: ● Only run the failed tests from the last time pytest was run: 18 $ python -m pytest --last-failed $ python -m pytest tests/unit $ python -m pytest -v ========================== test session starts ================================ tests/functional/test_recipes.py::test_home_page PASSED [ 6%] ... tests/unit/test_models.py::test_setting_password PASSED [ 93%] tests/unit/test_models.py::test_user_id PASSED [100%] ============================= 15 passed in 1.05s ==============================

Fixtures 19

What is a Fixture? 20 ● Fixtures initialize tests to a known state: ○ Run tests in a predictable and repeatable manner ● Fixtures are defined as functions (with a descriptive name!) ● Multiple fixture can be run to initialize a test ○ Fixtures can even call other fixtures! ● Fixtures should be defined in `tests/conftest.py`

Scope of Fixtures 21 ● Scope defines WHEN a fixture will be run ● function - run once per test function (default scope) ● class - run once per test class ● module - run once per module (i.e. test file) ● session - run once per session (i.e. per call to pytest)

def test_new_user_with_fixture(new_user): """ GIVEN a User model WHEN a new User is created THEN check the email and hashed_password are defined correctly """ assert new_user.email == 'patkennedy79@gmail.com' assert new_user.hashed_password != 'FlaskIsAwesome' from project.models import User @pytest.fixture(scope='module') def new_user(): user = User('patkennedy79@gmail.com', 'FlaskIsAwesome') return user Fixture to Help with Unit Testing 22 tests/unit/test_models.py tests/conftest.py

Fixture to Help with Functional Testing 23 from project import create_app @pytest.fixture(scope ='module') def test_client(): flask_app = create_app( 'flask_test.cfg' ) # Create a test client using the Flask application configured for testing with flask_app.test_client() as testing_client: yield testing_client # this is where the testing happens! def test_home_page_post_with_fixture (test_client): """ GIVEN a Flask application WHEN the '/' page is is posted to (POST) THEN check that a '405' status code is returned """ response = test_client .post('/') assert response.status_code == 405 assert b’Flask User Management Example!’ not in response.data tests/functional/test_users.py tests/conftest.py

Fixtures in Action! 24 $ python -m pytest --setup-show tests/functional/test_users.py ================================== test session starts ================================= tests/functional/test_recipes.py ... SETUP M test_client functional/test_users.py::test_home_page_with_fixture (fixtures used: test_client). ... TEARDOWN M test_client =================================== 4 passed in 0.18s ================================== Step 1 - test_client fixture is executed Step 3 - Execution returns to test_client fixture -> Done! Step 2 - test executes!

Coverage 25

● Code coverage is a metric of how much source code is executed when running the tests ● Two excellent Python packages: ○ coverage.py - tool for measuring code coverage of Python programs ○ pytest-cov - pytest plugin for running coverage Code Coverage - Introduction 26 Coverage.py (by Ned Batcheldor): https://coverage.readthedocs.io/ Pytest-cov: https://pytest-cov.readthedocs.io/

Checking Coverage in pytest 27 $ python -m pytest --cov=project ================================= test session starts ================================== tests/functional/test_recipes.py .... [ 26%] tests/functional/test_users.py ....... [ 73%] tests/unit/test_models.py .... [100%] ---------- coverage: platform darwin, python 3.10.0-final-0 ---------- Name Stmts Miss Cover ------------------------------------------------- project/__init__.py 24 0 100% project/models.py 33 0 100% project/recipes/__init__.py 3 0 100% project/recipes/routes.py 5 0 100% project/users/__init__.py 3 0 100% project/users/forms.py 13 0 100% project/users/routes.py 47 0 100% ------------------------------------------------- TOTAL 128 0 100% ================================ 15 passed in 1.36s =================================== Any lines in the source code that are not executed are identified here Specify the directory containing the source code

Conclusion 28

Conclusion ● pytest helps simplify testing Flask applications ● Fixtures provide a consistent approach for initializing state for tests ● Coverage is easy to collect with coverage.py and pytest-cov 29 For more complex examples of fixtures and test functions: https://gitlab.com/patkennedy79/flask_user_management_example “Testing Flask Applications with pytest” blog post: https://testdriven.io/blog/flask-pytest/

Thank you! 30 Patrick Kennedy Email: patkennedy79@gmail.com Twitter: patkennedy79 Personal Blog: https://www.patricksoftwareblog.com Learn Flask by Building and Deploying a Stock Portfolio App: https://testdriven.io/courses/learn-flask/