Tweet
Tweet

Slide 1

Slide 1 text

Automated Testing Philosophy & Tools

Slide 2

Slide 2 text

I wi" cover • package layout • requirements files • tox • travis

Slide 3

Slide 3 text

I won’t cover • Jenkins • pip and virtualenv • deployment • python 3 compatibility

Slide 4

Slide 4 text

The ideal • Test with every supported python version • Test with every database/search backend • Test with every supported dependency version • Test every possible combination

Slide 5

Slide 5 text

Without automation ~Test with every supported python version ⨉Test with every database/search backend ~Test with every supported dependency version ⨉Test every possible combination

Slide 6

Slide 6 text

With automation ✓Test with every supported python version ~Test with every database/search backend ~Test with every supported dependency version ~Test every possible combination

Slide 7

Slide 7 text

Package layout django-daguerre/ .travis.yml AUTHORS LICENSE MANIFEST.in setup.py tox.ini daguerre/ docs/ test_project/ requirements-1.4.txt requirements-1.5.txt manage.py test_project/

Slide 8

Slide 8 text

Test project django-daguerre/ .travis.yml AUTHORS LICENSE MANIFEST.in setup.py tox.ini daguerre/ docs/ test_project/ requirements-1.4.txt requirements-1.5.txt manage.py test_project/

Slide 9

Slide 9 text

• Sets up django environment • Provides canonical test settings • Hides requirements files

Slide 10

Slide 10 text

Requirements files django-daguerre/ .travis.yml AUTHORS LICENSE MANIFEST.in setup.py tox.ini daguerre/ docs/ test_project/ requirements-1.4.txt requirements-1.5.txt manage.py test_project/

Slide 11

Slide 11 text

• Reproducible test environment(s) • Same today as next week as next year • Precise dependencies

Slide 12

Slide 12 text

⨉ No django-nose south mock six Pillow Pillow>2.0.0 django<1.5 -e git+https://github.com/jbalogh/django-nose.git@ master#egg=django-nose south>0.8 mock six

Slide 13

Slide 13 text

✓ Yes Pillow==2.0.0 nose==1.3.0 django==1.4.5 south==0.8.1 mock==1.0.1 -e git+https://github.com/jbalogh/django-nose.git@ 4908503d475144c91f679e945a6e8d8781291f2a#egg=djang o-nose six==1.3.0

Slide 14

Slide 14 text

tox: local testing django-daguerre/ .travis.yml AUTHORS LICENSE MANIFEST.in setup.py tox.ini daguerre/ docs/ test_project/ requirements-1.4.txt requirements-1.5.txt manage.py test_project/

Slide 15

Slide 15 text

• Isolated virtual environments • Multiple versions of python • Local == pre-commit • .../django-daguerre$ tox

Slide 16

Slide 16 text

tox.ini (basic) [tox] envlist = py26, py27, py33 [testenv] changedir = {toxinidir}/test_project deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.4.txt commands = {envpython} manage.py test daguerre

Slide 17

Slide 17 text

tox.ini (matrix) [tox] envlist = py26-django14, py27-django14, py26-django15, py27-django15, [testenv] changedir = {toxinidir}/test_project commands = {envpython} manage.py test daguerre # Continued

Slide 18

Slide 18 text

tox.ini (matrix) [testenv:py26-django14] basepython=python2.6 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.4.txt [testenv:py27-django14] basepython=python2.7 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.4.txt [testenv:py26-django15] basepython=python2.6 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.5.txt [testenv:py27-django15] basepython=python2.7 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.5.txt

Slide 19

Slide 19 text

travis-ci: remote testing django-daguerre/ .travis.yml AUTHORS LICENSE MANIFEST.in setup.py tox.ini daguerre/ docs/ test_project/ requirements-1.4.txt requirements-1.5.txt manage.py test_project/

Slide 20

Slide 20 text

• Isolated virtual environments • Multiple versions of python • Good matrix support • Built-in support for MySQL, PostgreSQL, Elasticsearch, Redis, and more • Remote == post-push • Integration for public Github repos only

Slide 21

Slide 21 text

.travis.yml (basic) language: python python: - "2.6" - "2.7" - "3.3" install: - pip install . --no-deps - pip install --use-mirrors --no-deps -r test_project/requirements-1.5.txt script: - cd test_project - ./manage.py test --verbosity=2 daguerre

Slide 22

Slide 22 text

tox.ini (basic) [tox] envlist = py26, py27, py33 [testenv] changedir = {toxinidir}/test_project deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.4.txt commands = {envpython} manage.py test daguerre

Slide 23

Slide 23 text

.travis.yml (matrix) language: python python: - "2.6" - "2.7" - "3.3" env: - REQUIREMENTS=test_project/requirements-1.4.txt - REQUIREMENTS=test_project/requirements-1.5.txt matrix: exclude: - python: "3.3" env: REQUIREMENTS=test_project/requirements-1.4.txt install: - pip install . --no-deps - pip install --use-mirrors --no-deps -r $REQUIREMENTS script: - cd test_project - ./manage.py test --verbosity=2 daguerre

Slide 24

Slide 24 text

tox.ini (matrix) [tox] envlist = py26-django14, py27-django14, py26-django15, py27-django15, [testenv] changedir = {toxinidir}/test_project commands = {envpython} manage.py test daguerre # Continued

Slide 25

Slide 25 text

tox.ini (matrix) [testenv:py26-django14] basepython=python2.6 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.4.txt [testenv:py27-django14] basepython=python2.7 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.4.txt [testenv:py26-django15] basepython=python2.6 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.5.txt [testenv:py27-django15] basepython=python2.7 deps = --no-deps --use-mirrors -r{toxinidir}/test_project/requirements-1.5.txt

Slide 26

Slide 26 text

.travis.yml (matrix 2) language: python python: - "2.6" - "2.7" - "3.3" env: - DB=sqlite3 REQUIREMENTS=test_project/requirements-1.4.txt - DB=mysql REQUIREMENTS=test_project/requirements-1.4.txt - DB=sqlite3 REQUIREMENTS=test_project/requirements-1.5.txt - DB=mysql REQUIREMENTS=test_project/requirements-1.5.txt matrix: exclude: - python: "3.3" env: DB=sqlite3 REQUIREMENTS=test_project/requirements-1.4.txt - python: "3.3" env: DB=mysql REQUIREMENTS=test_project/requirements-1.4.txt - python: "3.3" env: DB=mysql REQUIREMENTS=test_project/requirements-1.5.txt install: - pip install . --no-deps - pip install --use-mirrors --no-deps -r $REQUIREMENTS - if [ $DB == 'mysql' ]; then pip install --use-mirrors --no-deps mysql-python; fi before_script: - if [ $DB == 'mysql' ]; then mysql -e 'CREATE DATABASE daguerre_test;'; fi script: - cd test_project - ./manage.py test --verbosity=2 daguerre

Slide 27

Slide 27 text

(in settings.py) import os DB = os.environ.get('DB') if DB == 'mysql': DATABASES = { 'default': { 'ENGINE': 'django.db.backends.mysql', 'NAME': 'daguerre_test', 'USER': 'root', 'TEST_CHARSET': 'utf8', 'TEST_COLLATION': 'utf8_general_ci', } } else: DATABASES = { 'default': { 'ENGINE': 'django.db.backends.sqlite3', 'NAME': os.path.join(PROJECT_DIR, 'db.sl3'), } }

Slide 28

Slide 28 text

Summary (tox) • Multiple versions of python: trivial • Two-dimensional matrix: work, but possible • For example, multiple versions of django • Higher-dimensional matrix: not really feasible • For example multiple versions of django and multiple database settings

Slide 29

Slide 29 text

Summary (travis) • Multiple versions of python: trivial • Two-dimensional matrix: simple • For example, multiple versions of django • Higher-dimensional matrix: work, but possible • For example multiple versions of django and multiple database settings

Slide 30

Slide 30 text

Automated testing?

Slide 31

Slide 31 text

MAKE IT SO

Slide 32

Slide 32 text

Thanks! • Stephen Burrows (melinath) • github.com/melinath/django-daguerre • travis-ci.org • tox.readthedocs.org