smörgåsbord testing of A 

David Winterbottom @codeinthehole /codeinthehole Impending sausage 

David Winterbottom @codeinthehole /codeinthehole Grange Hill (only British people who grew up in the 1990s will understand this) 

Tangent Labs @tangent_labs /tangentlabs See www.tangentlabs.co.uk (PS - we’re hiring!) 

django-oscar @django_oscar /tangentlabs/django-oscar Oscar is the most popular python e-commerce software on Github. It’s code is so beautiful, it’s been known to make grown men weep. oscarcommerce.com 

This is Oscar’s test suite - we can see a few things which betray the biases of this talk 

Nose Nose nose.readthedocs.org ...like nose... 

Django ...and Django. 

Dexterity Readability Speed The talk covers 3 topics 

Who is this? (Hint - it’s not Philip Seymour Hoffman). It’s Kent Beck: creator of jUnit, XP etc 

“I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence ” ...and he’s a pragmatist. Beware of dogma (100% coverage, one assertion per method etc) 

Bad tests exist. They can pour concrete over your codebase. It’s ok to delete tests. 

Dexterity Is there a clever Japanese word for the art of slicing? Dexterity = How to use your test suite effectively 

Basics Love thy test-runner 

Run ALL the tests 

Run tests beneath path Can use your shell’s tab completion for speedy folder selection 

Run test module 

Run test class Colons are a pain (if you are trying to write a autocompleter) as they are word delimiters 

Run test method This is too labour-intensive to type, so we use ... 

nosecomplete github.com/alonho/nosecomplete Someone solved the colon problem! 

nosecomplete github.com/alonho/nosecomplete ...and here it is in action. 

Stories 

Stories Work on a feature Doing TDD, working from a spec etc 

Here’s a sample test run 

https://github.com/asdfasdf/spec Custom test runner You need a custom test-runner 

https://github.com/asdfasdf/spec Testing one module Run your unit tests as you develop 

https://github.com/asdfasdf/spec Spec plugin https://github.com/bitprophet/spec Write your test classes and methods as a spec Based on Pinocchio 

https://github.com/asdfasdf/spec BDD-style output And, write your specs to look like this BDD-style output. 

class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factories.create_order(user=self.user) def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) The Chuckle Brothers are updating their profiles 

class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factoies.create_order(user=self.user) def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) System under test 

class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factories.create_order(user=self.user) def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) Specification On the road towards a full BDD framwork like Lettuce/Behave, but you can still use Nose and all its plugins 

Use-cases Work on a new feature Refactor a feature You’re reviewing a pull request 

https://github.com/nose attribs/spec Run all the shipping tests! 

nose.readthedocs.org/en/latest/plugins/attrib.html Run tagged tests 

from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ... class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... 

from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ... class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... Wrap a class 

from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ... class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... Wrap a method You can tag slow tests 

No content

Pattern match tests github.com/iElectric/nose-selecttests Better for lazy developers 

Use-cases Work on a new feature Refactor a feature Fix failing tests 

Fail fast 

Editor-friendly paths github.com/erikrose/nose-progressive Other nice features: prints out failures/errors as they occur, progress bar 

PDB on failure Need to go up the stack to get to test code 

Only re-run failed tests Needs to be run twice though 

Use-cases Work on a new feature Refactor a feature Fix failing tests Commit Before pushing etc 

Run all the tests as fast as possible 

No plugins 

Parallelise if you use tox, then try detox 

Speed Watch Gary Bernhardt’s talks 

0 4 8 12 16 20 24 Focus / flow / “zone” Test suite time (seconds) 

0 4 8 12 16 20 24 Focus / flow / “zone” Test suite time (seconds) Pre-commit hook? Everything’s rosy when your test suite runs in 100ms 

0 4 8 12 16 20 24 Focus / flow / “zone” Test suite time (seconds) But watch out for the threshold of doom 

0 4 8 12 16 20 24 Test suite time (seconds) Oh no - you’re reading Hacker News! 

Tips for speeding up tests will be in another talk Oscar distinguishes between unit, integration and functional (acceptance) tests to try and steer people towards unit tests 

No content

Scam? blog.thecodewhisperer.com/2010/10/16/integrated-tests-are-a-scam/ Read this polemic 

import mock from django.utils.functional import curry no_database = curry( mock.patch, 'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt 

import mock from django.utils.functional import curry no_database = curry( mock.patch, 'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt Currying mock.patch! All my favourite bits of Python in one curio 

import mock from django.utils.functional import curry no_database = curry( mock.patch, 'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt Wrap classes/methods 

http://bit.ly/156c48x Number of tests Seconds Oscar’s test suite has doubled number of tests while halving the time to run. 

PASSWORD_HASHERS = ['django.contrib.auth.hashers.MD5PasswordHasher'] Using this hasher makes a big difference. 

--processes=8 Maik spotted this option for Nose and we all had a cup of tea to celebrate. 

http://bit.ly/2gZnhj Boy scout rule... Always leave the campsite cleaner than how you found it 

Readability 

http://bit.ly/19ganRJ (Refactoring a Test) Book has a great chapter on refactoring a test from dirty to clean 

class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass( name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) This is a bad test 

class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass( name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Bad naming 

class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass( name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Heavy set-up 

class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass( name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Noise 

class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass( name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Unclear assertions 

class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass( name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Sloppy coding Numbered variable names make baby jesus cry 

class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) This is the same test, but cleaned up 

class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Descriptive names 

class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Factory functions Tests should only show relevant details 

class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Intention revealing Dumb example but a good thing to do in general 

PyHamcrest github.com/hamcrest/PyHamcrest >>> from hamcrest import * >>> assert_that( shipping.charge, is_(equal_to(D('4.99')))) >>> assert_that( "RESTful".lower(), contains_string("stfu")) Hamcrest - more readable but too many parentheses? 

should-dsl github.com/nsi-iff/should-dsl >>> from should_dsl import should, should_not >>> {} |should| be_empty >>> user.email |should| contain("@") >>> basket |should| contain(product) Crazy voodoo here, but quite readable ultimately 

This was the best image of a Smorgasbord I could find 

wiki.python.org/moin/PythonTestingToolsTaxonomy For all your python testing needs.