Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing Flask Applications

Testing Flask Applications

Testing Flask applications can be done in many ways. This presentation covers the usage of pytest and a few other libraries.

Matt Wright

April 24, 2014
Tweet

More Decks by Matt Wright

Other Decks in Programming

Transcript

  1. Testing Flask Applications (with pytest and other things)

  2. Matt Wright @mattupstate Engineer @ChatID • Python, Ruby, CoffeeScript •

    Flask, Chef, AngularJS Open Source • Flask-Security • Flask-Social • Flask-Mail + a few others
  3. What is a test?

  4. What is a test? The means by which the presence,

    quality, or genuineness of anything is determined; a means of trial [1]
  5. None
  6. Will it blend?

  7. None
  8. Yes! It Blends!

  9. What is software testing?

  10. What is software testing? Software testing can be stated as

    the process of validating and verifying that a computer program/application/product: • meets the requirements that guided its design and development • works as expected • can be implemented with the same characteristics • and satisfies the needs of stakeholders. [2]
  11. None
  12. What is software testing?

  13. What is software testing? The means by which essential or

    distinctive characteristics of a piece of software is determined.
  14. None
  15. Does addition work in Python 3?

  16. # ~/test.py try: assert 1 + 1 == 2 print('Test

    passed!') except AssertionError: print('Test failed!')
  17. barry @ paradise-cafe in ~ $ python3 test.py

  18. barry @ paradise-cafe in ~ $ python3 test.py Test Passed!

  19. Hell yes! Addition works in Python 3!

  20. But why?!?!

  21. 1. Proves that your code works But why?!?!

  22. 1. Proves that your code works 2. Demonstrates thought process

    But why?!?!
  23. 1. Proves that your code works 2. Demonstrates thought process

    3. Refactoring becomes a lot easier But why?!?!
  24. But why?!?! 1. Proves that your code works 2. Demonstrates

    thought process 3. Refactoring becomes a lot easier 4. Tests can serve as example code
  25. But why?!?! 1. Proves that your code works 2. Demonstrates

    thought process 3. Refactoring becomes a lot easier 4. Tests can serve as example code 5. A good challenge is fun!
  26. Testing Flask Apps

  27. Disclaimer!

  28. Getting Started

  29. barry @ pc in ~/myapp $ mkvirtualenv myapp -p `which

    python3` … barry @ pc in ~/myapp workon:myapp $ pip install flask …
  30. # ~/myapp/app.py from flask import Flask app = Flask(__name__) @app.route('/')

    def index(): return 'Hello, world!' @app.route('/contact') def contact(): return 'barry@manilow.com'
  31. Now What?

  32. Werkzeug! http://werkzeug.pocoo.org/docs/test/ also http://flask.pocoo.org/docs/testing/

  33. # ~/myapp/test_index.py from app import app app.testing = True client

    = app.test_client() res = client.get('/') try: assert res.data == b'Hello, world!' print('Test passed!') except AssertionError: print('Test failed!')
  34. barry @ pc in ~/myapp workon:myapp $ python test_index.py Test

    passed!
  35. unittest

  36. # ~/myapp/test_index.py import unittest from app import app app.testing =

    True class IndexTestCase(unittest.TestCase): def test_index(self): client = app.test_client() res = client.get('/') self.assertEqual(res.data, b'Hello, world!') if __name__ == '__main__': unittest.main()
  37. barry @ pc in ~/myapp workon:myapp $ python test_index.py

  38. barry @ pc in ~/myapp workon:myapp $ python test_index.py .

    -------------------------------------- Ran 1 test in 0.020s OK
  39. # ~/myapp/test_contact.py import unittest from app import app app.testing =

    True class ContactTestCase(unittest.TestCase): def test_contact(self): client = app.test_client() res = client.get('/contact') self.assertEqual(res.data, b'barry@manilow.com') if __name__ == '__main__': unittest.main()
  40. barry @ pc in ~/myapp workon:myapp $ python test_contact.py .

    -------------------------------------- Ran 1 test in 0.020s OK
  41. # ~/myapp/test_suite.py import unittest from test_index import IndexTestCase from test_contact

    import ContactTestCase if __name__ == '__main__': unittest.main()
  42. barry @ pc in ~/myapp workon:myapp $ python test_suite.py .

    -------------------------------------- Ran 2 tests in 0.016s OK
  43. pytest!

  44. 1. Easy to get started pytest

  45. barry @ pc in ~/myapp workon:myapp $ pip install pytest

  46. barry @ pc in ~/myapp workon:myapp $ pip install pytest

    barry @ pc in ~/myapp workon:myapp $ py.test
  47. barry @ pc in ~/myapp workon:myapp $ pip install pytest

    barry @ pc in ~/myapp workon:myapp $ py.test =================== test session starts ==================== platform darwin -- Python 3.3.3 -- py-1.4.20 -- pytest-2.5.2 collected 4 items test_contact.py . test_index.py . test_suite.py .. ================ 4 passed in 0.20 seconds ==================
  48. barry @ pc in ~/myapp workon:myapp $ rm test_suite.py

  49. barry @ pc in ~/myapp workon:myapp $ rm test_suite.py barry

    @ pc in ~/myapp workon:myapp $ py.test
  50. barry @ pc in ~/myapp workon:myapp $ rm test_suite.py barry

    @ pc in ~/myapp workon:myapp $ py.test =================== test session starts ==================== platform darwin -- Python 3.3.3 -- py-1.4.20 -- pytest-2.5.2 collected 2 items test_contact.py . test_index.py . ================ 2 passed in 0.10 seconds ==================
  51. 1. Easy to get started 2. Asserting with the assert

    statement! pytest
  52. # ~/myapp/test_index.py import unittest from app import app class IndexTestCase(unittest.TestCase):

    def test_index(self): client = app.test_client() res = client.get('/') self.assertEqual(res.data, b'Hello, world!') if __name__ == '__main__': unittest.main()
  53. # ~/myapp/test_index.py import unittest from app import app class IndexTestCase(unittest.TestCase):

    def test_index(self): client = app.test_client() res = client.get('/') assert res.data == b'Hello, world!' if __name__ == '__main__': unittest.main()
  54. 1. Easy to get started 2. Asserting with the assert

    statement! 3. Helpful traceback and failing assertion reporting pytest
  55. # ~/myapp/test_index.py import unittest from app import app class IndexTestCase(unittest.TestCase):

    def test_index(self): client = app.test_client() res = client.get('/') self.assertEqual(res.data, b'Hello, moon!') if __name__ == '__main__': unittest.main()
  56. barry @ pc in ~/myapp workon:myapp $ py.test ========================= test

    session starts ========================== platform darwin -- Python 3.3.3 -- py-1.4.20 -- pytest-2.5.2 collected 2 items test_contact.py . test_index.py F [CONTINUED...]
  57. =============================== FAILURES =============================== _______________________ IndexTestCase.test_index _______________________ self = <test_index.IndexTestCase testMethod=test_index>

    def test_index(self): client = app.test_client() res = client.get('/') > assert res.data == b'Hello, world!a' E AssertionError: b'Hello, world!' != b'Hello, moon!' test_index.py:9: AssertionError ================== 1 failed, 1 passed in 0.06 seconds ==================
  58. # ~/myapp/test_index.py import unittest from app import app class IndexTestCase(unittest.TestCase):

    def test_index(self): client = app.test_client() res = client.get('/') assert res.data == b'Hello, moon!' if __name__ == '__main__': unittest.main()
  59. barry @ pc in ~/myapp workon:myapp $ py.test ========================= test

    session starts ========================== platform darwin -- Python 3.3.3 -- py-1.4.20 -- pytest-2.5.2 collected 2 items test_contact.py . test_index.py F [CONTINUED...]
  60. =============================== FAILURES =============================== _______________________ IndexTestCase.test_index _______________________ self = <test_index.IndexTestCase testMethod=test_index>

    def test_index(self): client = app.test_client() res = client.get('/') > assert res.data == b'Hello, world!a' E AssertionError: assert b'Hello, world!' == b'Hello, moon!' E At index 7 diff: 119 != 109 E Left contains more items, first extra item: 33 test_index.py:9: AssertionError ================== 1 failed, 1 passed in 0.06 seconds ==================
  61. 1. Easy to get started 2. Asserting with the assert

    statement! 3. Helpful traceback and failing assertion reporting 4. Print debugging and the capturing of standard output during test execution pytest
  62. # ~/myapp/test_index.py import unittest from app import app class IndexTestCase(unittest.TestCase):

    def test_index(self): client = app.test_client() res = client.get('/') print('a debug message') assert res.data == b'Hello, moon!' if __name__ == '__main__': unittest.main()
  63. ... def test_index(self): client = app.test_client() res = client.get('/') >

    assert res.data == b'Hello, world!a' E AssertionError: assert b'Hello, world!' == b'Hello, moon!' E At index 7 diff: 119 != 109 E Left contains more items, first extra item: 33 test_index.py:9: AssertionError --------------------------- Captured stdout ---------------------------- a debug message ================== 1 failed, 1 passed in 0.06 seconds ==================
  64. But wait! There’s more!

  65. Just say NO to boilerplate

  66. # ~/myapp/test_index.py import unittest from app import app class IndexTestCase(unittest.TestCase):

    def test_index(self): client = app.test_client() res = client.get('/') assert res.data == b'Hello, world!' if __name__ == '__main__': unittest.main()
  67. # ~/myapp/test_index.py from app import app app.testing = True def

    test_index(): client = app.test_client() res = client.get('/') assert res.data == b'Hello, world!'
  68. # ~/myapp/test_contact.py import unittest from app import app class ContactTestCase(unittest.TestCase):

    def test_contact(self): client = app.test_client() res = client.get('/contact') assert res.data == b'barry@manilow.com' if __name__ == '__main__': unittest.main()
  69. # ~/myapp/test_contact.py from app import app app.testing = True def

    test_contact(): client = app.test_client() res = client.get('/contact') assert res.data == b'barry@manilow.com'
  70. Fixtures

  71. # ~/myapp/conftest.py import pytest from app import app as _app

    @pytest.fixture() def app(): _app.testing = True return _app @pytest.fixture() def client(app): return app.test_client()
  72. # ~/myapp/test_index.py def test_index(client): res = client.get('/') assert res.data ==

    b'Hello, world!' # ~/myapp/test_contact.py def test_contact(client): res = client.get('/contact') assert res.data == b'barry@manilow.com'
  73. Fixture Scope

  74. # ~/myapp/conftest.py import pytest from app import app as _app

    @pytest.fixture(scope='session') def app(): app.testing = True return _app @pytest.fixture(scope='session') def client(app): return app.test_client()
  75. Protip: See what fixtures are available

  76. barry @ pc in ~/myapp workon:myapp $ py.test --fixtures

  77. barry @ pc in ~/myapp workon:myapp $ py.test --fixtures ========================

    test session starts ========================== platform darwin -- Python 3.3.3 -- py-1.4.20 -- pytest-2.5.2 collected 2 items capsys enables capturing of writes to sys.stdout/sys.stderr and makes captured output available via ``capsys.readouterr()`` method calls which return a ``(out, err)`` tuple. capfd enables capturing of writes to file descriptors 1 and 2 and makes captured output available via ``capsys.readouterr()`` method calls which return a ``(out, err)`` tuple.
  78. tmpdir return a temporary directory path object which is unique

    to each test function invocation, created as a sub directory of the base temporary directory. The returned object is a `py.path.local`_ path object. -------------------- fixtures defined from conftest -------------------- app conftest.py:5: no docstring available client conftest.py:9: no docstring available =========================== in 0.02 seconds ==========================
  79. Project Layout

  80. barry @ pc in ~/myapp workon:myapp $ tree . ├──

    app.py └── tests ├── conftest.py ├── test_contact.py └── test_index.py 1 directory, 4 files
  81. barry @ pc in ~/myapp workon:myapp $ py.test [BLAH BLAH

    BLAH] _path/local.py", line 620, in pyimport __import__(modname) File "/Users/matt/myapp/tests/conftest.py", line 2, in <module> from app import app as _app ImportError: No module named 'app'
  82. # ~/myapp/setup.py from setuptools import setup setup( name='My App', version='0.1.0',

    py_modules=['app'], install_requires=['Flask==0.10.1'] tests_require=['pytest==2.5.2'] )
  83. # ~/myapp/setup.py from setuptools import setup, find_packages setup( name='My App',

    version='0.1.0', packages=find_packages(), install_requires=['Flask==0.10.1'] tests_require=['pytest==2.5.2'] )
  84. barry @ pc in ~/myapp workon:myapp $ pip install -e

    .
  85. barry @ pc in ~/myapp workon:myapp $ pip install -e

    . barry @ pc in ~/myapp workon:myapp $ py.test ========================= test session starts ========================== platform darwin -- Python 3.3.3 -- py-1.4.20 -- pytest-2.5.2 collected 2 items tests/test_contact.py . tests/test_index.py . ======================= 2 passed in 0.04 seconds =======================
  86. Testing Forms (application/x-www-form-urlencoded)

  87. # ~/myapp/app.py from flask import request @app.route('/contact', methods=['POST']) def send_contact():

    msg = request.form.get('msg', None) if msg is not None: return 'Message sent!' else: return 'Invalid message', 400
  88. # ~/myapp/tests/test_contact.py def test_send_contact(client): data = {'msg': 'Hi, Barry!'} res

    = client.post('/contact', data=data) assert res.status_code == 200 assert res.data == b'Message sent!' def test_send_invalid_contact(client): res = client.post('/contact', data={}) assert res.status_code == 400 assert res.data == b'Invalid message'
  89. Testing Forms (application/json)

  90. # ~/myapp/app.py from flask import request, jsonify @app.route('/contact.json', methods=['POST']) def

    send_contact_json(): json = request.get_json() msg = json.get('msg', None) if msg is not None: return jsonify({'success': True}) else: return jsonify({'success': False}), 400
  91. # ~/myapp/tests/test_contact.py from flask import json JSON_MEDIA_TYPE = 'application/json' def

    test_send_contact_json(client): data = json.dumps({'msg': 'Hi, Barry!'}) res = client.post('/contact.json', data=data, content_type=JSON_MEDIA_TYPE) assert res.status_code == 200 assert res.content_type == JSON_MEDIA_TYPE jdata = json.loads(res.data) assert jdata['success'] is True
  92. Dealing with a Database

  93. SQLAlchemy (the easy way)

  94. # ~/myapp/app.py from flask_sqlalchemy import SQLAlchemy db = SQLAlchemy() def

    create_app(database_uri=None): database_uri = database_uri or 'sqlite:////myapp.db' app = Flask(__name__) app.config['SQLALCHEMY_DATABASE_URI'] = database_uri db.init_app(app) app.register_blueprint(my_blueprint) return app
  95. # ~/myapp/tests/conftest.py import tempfile, pytest from app import create_app, db

    @pytest.fixture() def app(tmpdir): f, path = tempfile.mkstemp(suffix='.db', dir=str(tmpdir)) app = create_app(database_uri='sqlite:///' + path) with app.app_context(): db.create_all() return app
  96. http://alexmic.net/flask-sqlalchemy-pytest/

  97. Test Data (with factory-boy)

  98. barry @ pc in ~/myapp workon:myapp $ pip install factory-boy

  99. # ~/myapp/app.py class User(db.Model): id = db.Column(db.Integer, primary_key=True) name =

    db.Column(db.String(255)) email = db.Column(db.String(255)) password = db.Column(db.String(255))
  100. # ~/myapp/app.py from factory import Sequence from factory.alchemy import SQLAlchemyModelFactory

    class ModelFactory(SQLAlchemyModelFactory): FACTORY_SESSION = db.session ABSTRACT_FACTORY = True class UserModelFactory(ModelFactory): FACTORY_FOR = User name = Sequence(lambda n: 'User %s' % n) email = Sequence(lambda n: 'user%s@manilow.com' % n) password = 'password'
  101. # ~/myapp/tests/conftest.py import pytest from app import UserModelFactory, db @pytest.fixture()

    def user(app): with app.app_context() user = UserModelFactory() db.session.commit() return user
  102. # ~/myapp/tests/test_users.py def test_update_user(client, user): res = client.post('/users/%s' % user.id,

    data={ 'name': 'Barry' }) assert res.status_code = 400
  103. Testing ReST APIs

  104. JSON Schema http://json-schema.org/

  105. # ~/myapp/contact_schema.json { "type": "object", "properties": { "success": { "type":

    "boolean" } }, "additionalProperties": false }
  106. barry @ pc in ~/myapp workon:myapp $ pip install jsonschema

  107. # ~/myapp/app.py import os, jsonschema as _jsonschema from flask import

    json class jsonschema: def __init__(self, name): file_name = '%s.json' % name base_dir = os.path.join(os.path.dirname(os.path.realpath(__file__))) with open(os.path.join(base_dir, file_name)) as f: schema = json.load(f) self._schema = schema def __eq__(self, jdata): return _jsonschema.validate(jdata, self._schema) is None
  108. # ~/myapp/tests/test_contact.py from flask import json from app import jsonschema

    JSON_MEDIA_TYPE = 'application/json' def test_send_contact_json(client): data = json.dumps({'msg': 'Hi, Barry!'}) res = client.post('/contact.json', data=data, content_type=JSON_MEDIA_TYPE) assert res.status_code == 200 assert res.content_type == JSON_MEDIA_TYPE assert json.loads(res.data) == jsonschema('contact_schema')
  109. pytest Extensions

  110. Parting Advice

  111. Imagine the desired API Parting Advice

  112. Imagine the desired API Avoid hypothetical edge cases Parting Advice

  113. Parting Advice Imagine the desired API Avoid hypothetical edge cases

    Cover edge cases as they appear
  114. Thank You! mattupstate.com gittip.com/mattupstate github.com/mattupstate twitter.com/mattupstate

  115. Annotations [1] http://dictionary.reference.com/ [2] http://en.wikipedia.org/wiki/Software_testing