Ash Christopher - Speaker Deck

Slide 1

Slide 1 text

TESTING DJANGO PROJECTS AT SCALE Sunday, 11 August, 13

Slide 2

Slide 2 text

This talk isn’t just applicable to Django • Django • Python • Development in general Sunday, 11 August, 13

Slide 3

Slide 3 text

Pragmatic Don’t worry too much about the type of test - worry more about having tests. Sunday, 11 August, 13

Slide 4

Slide 4 text

I don’t have a silver bullet. There is nothing particularly novel about what I am going to talk about. Sunday, 11 August, 13

Slide 5

Slide 5 text

What we will cover • Deﬁnition of a good test. • Canonical form of a test. • 10 guidelines to help promote good tests. Sunday, 11 August, 13

Slide 6

Slide 6 text

What is a good test? Sunday, 11 August, 13

Slide 7

Slide 7 text

What is a good test? Blatantly paraphrased from the Pylons Unit Test documentation. http:/ /j.mp/pylons-unit-testing Sunday, 11 August, 13

Slide 8

Slide 8 text

Tests should be as simple as possible • Be clear - not clever. • Don’t write tests that require their own tests! Sunday, 11 August, 13

Slide 9

Slide 9 text

Tests should run as quickly as possible • Encourage developers to run tests frequently. • Encourage running the whole test suite, rather than a subset. • Instill a culture of conﬁdence and reliance on tests. Sunday, 11 August, 13

Slide 10

Slide 10 text

Tests should avoid coupling with other tests • TestCases should not extend TestCases in other python modules. • Tests should minimize the use helper methods which exist in other python modules. • DRY is well suited for code - but often detrimental for tests. Sunday, 11 August, 13

Slide 11

Slide 11 text

Tests should clearly communicate intent • Tests are explicitly clear about what they are testing. • Related test methods reside in the same TestCase. • Ideally tests should be self documenting, and extra documentation should be a nice-to-have rather than required. Sunday, 11 August, 13

Slide 12

Slide 12 text

Canonical Form How to properly structure a test. Sunday, 11 August, 13

Slide 13

Slide 13 text

Set up pre-conditions Create the environment needed for the test to run. • Create model instances. • Set up the state required for the test. • Create mock objects. • Patch methods, functions and objects. Sunday, 11 August, 13

Slide 14

Slide 14 text

Perform operation under test Run test using the environment established in the pre-conditions. • Call functions/methods. • Modify state. • Record return values. Sunday, 11 August, 13

Slide 15

Slide 15 text

Make assertions Make assertions about the return values or side eﬀects. • Check return values against expected values. • Test that the state has been modiﬁed as expected. • Verify that expected Exceptions were raised. Sunday, 11 August, 13

Slide 16

Slide 16 text

Don’t modify the existing test conditions and make additional assertions! Sunday, 11 August, 13

Slide 17

Slide 17 text

Guidelines Following these guidelines help promote good tests. Sunday, 11 August, 13

Slide 18

Slide 18 text

Sunday, 11 August, 13

Slide 19

Slide 19 text

1 Keep your tests as simple as possible. Sunday, 11 August, 13

Slide 20

Slide 20 text

Complex Tests • Harder for a reader of the test code to understand what is going on. • Complex test code adds functionality which itself should be tested. Sunday, 11 August, 13

Slide 21

Slide 21 text

Sunday, 11 August, 13

Slide 22

Slide 22 text

2 Set up only the minimum needed pre-conditions for your test. Sunday, 11 August, 13

Slide 23

Slide 23 text

Sunday, 11 August, 13

Slide 24

Slide 24 text

Sunday, 11 August, 13

Slide 25

Slide 25 text

Sunday, 11 August, 13

Slide 26

Slide 26 text

What just happened? User UserProﬁle 28 x Business Accounts Site 5x Business Currency Country Customer Vendor StandardAccount Account IncomeAccount ExpenseAccount Product PaymentTerm Tax (GST) Tax (PST) Tax (TEST) Invoice 2x InvoiceItem Bill 2x BillItem Sunday, 11 August, 13

Slide 27

Slide 27 text

What were actually trying to test? Sunday, 11 August, 13

Slide 28

Slide 28 text

3 Test only one piece of functionality in each test method. Sunday, 11 August, 13

Slide 29

Slide 29 text

Don’t do this! • Six assertions testing six diﬀerent behaviors. • Pre-conditions set four times. • Should have split into separate test methods! Sunday, 11 August, 13

Slide 30

Slide 30 text

4 Create your pre-conditions explicitly and resist the urge to use shared helped methods outside your module. Sunday, 11 August, 13

Slide 31

Slide 31 text

Isn’t this anti-DRY? Sunday, 11 August, 13

Slide 32

Slide 32 text

Isn’t this anti-DRY? Yes! • Shared helper methods tend to grow over time. • Shared helper methods tend to cater to the lowest common denominator (try to be generic enough for everyone to use). • Impose a lookup burden on readers of the test code. Sunday, 11 August, 13

Slide 33

Slide 33 text

Isn’t this anti-DRY? Yes! • When you refactor test code, you risk breaking unrelated tests. • You no longer have direct control over your pre- conditions. • Anyone can aﬀect the conﬁdence of your tests. Sunday, 11 August, 13

Slide 34

Slide 34 text

5 Create mixins rather than parent TestCases. Sunday, 11 August, 13

Slide 35

Slide 35 text

BaseTestCase Problems • When a TestCase extends a BaseTestCase (with tests), it will run all tests, even the ones in the BaseTestCase. • Leads to the same tests being run many times unnecessarily. • Use a mixin - it will not be run by the test runner. Sunday, 11 August, 13

Slide 36

Slide 36 text

Sunday, 11 August, 13

Slide 37

Slide 37 text

6 Use factories instead of ﬁxtures. Sunday, 11 August, 13

Slide 38

Slide 38 text

Fixtures are brittle • Fixtures are hard to maintain as your project’s data models change. • Tend to get re-used in tests rather than generating ﬁxtures for each TestCase. • Updating ﬁxtures is a pain - even in JSON. Sunday, 11 August, 13

Slide 39

Slide 39 text

factory_boy • Third-party Django application based on Rails’ factory_girl • Based on your own Django models. • https://github.com/rbarrois/factory_boy Sunday, 11 August, 13

Slide 40

Slide 40 text

factory_boy Sunday, 11 August, 13

Slide 41

Slide 41 text

Benefits of factory_boy • Generate data using the Django ORM. • Give us hooks into data preparation. • Lets us define data across relationships using familiar double-underscore notation. • We can define relationships in FactoryBoy to mimic our business logic. Sunday, 11 August, 13

Slide 42

Slide 42 text

7 Use django.tests.TestCase instead of unittest2.TestCase. Sunday, 11 August, 13

Slide 43

Slide 43 text

Tests pass, but test suite fails! • Unique constraints on database columns. • Signals in Django. Sunday, 11 August, 13

Slide 44

Slide 44 text

Isolated tests make developers happy * Only applies to the `default` database. Django TestCase’s run tests in a database transaction which gets rolled-back*. Actual Wave developers Sunday, 11 August, 13

Slide 45

Slide 45 text

8 Don’t use TestCase.setupClass() and TestCase.tearDownClass(). Sunday, 11 August, 13

Slide 46

Slide 46 text

Cleanup is a pain • Can not guarantee that all the data is cleaned up - especially when using Django signals. • If we want to use multi-processing for tests, we need to be sure test methods can be run in any order. • Over the entire test suite, did not give us enough gains to overcome the potential pitfalls. • Using multi-processing for tests gave us much bigger wins. Sunday, 11 August, 13

Slide 47

Slide 47 text

9 Segment your tests. Sunday, 11 August, 13

Slide 48

Slide 48 text

Use nose and django-nose • Decorate your tests and TestCases with @attr(...). • Segment your tests based on type of tests. • Choose which type of tests to run using -a on the command line. Sunday, 11 August, 13

Slide 49

Slide 49 text

10 Run your test in parallel. Sunday, 11 August, 13

Slide 50

Slide 50 text

Nose and multiprocess Sunday, 11 August, 13

Slide 51

Slide 51 text

Questions? @ashchristopher [email protected] github.com/ashchristopher Sunday, 11 August, 13