Share
A smörgåsbord of testing
The controversy over writing tests and TDD is over. The benefits to the design, development and maintenance of your code are real and well known. In 2013, it is simply unprofessional to eschew tests for production code.
Given this, developers need to move beyond the basics of testing and gain a deeper understanding of the art. This entails knowing the patterns and anti-patterns of effective test suites, understanding what to test and how to do it, and a mastery of the many fantastic testing tools in the Python ecosystem.
Bad tests are a genuine menace. A poorly considered test suite pours concrete over your codebase, hindering maintenance and development.
This talk isn't an introduction to testing and TDD. Rather, it is a mixed buffet of tips, anti-patterns, tools and techniques for effective testing.
David Winterbottom
Transcript
79 slides
-
David Winterbottom @codeinthehole /codeinthehole Grange Hill (only British people who
grew up in the 1990s will understand this) -
django-oscar @django_oscar /tangentlabs/django-oscar Oscar is the most popular python e-commerce
software on Github. It’s code is so beautiful, it’s been known to make grown men weep. oscarcommerce.com -
This is Oscar’s test suite - we can see a
few things which betray the biases of this talk -
Who is this? (Hint - it’s not Philip Seymour Hoffman).
It’s Kent Beck: creator of jUnit, XP etc -
“I get paid for code that works, not for tests,
so my philosophy is to test as little as possible to reach a given level of confidence ” ...and he’s a pragmatist. Beware of dogma (100% coverage, one assertion per method etc) -
Bad tests exist. They can pour concrete over your codebase.
It’s ok to delete tests. -
Dexterity Is there a clever Japanese word for the art
of slicing? Dexterity = How to use your test suite effectively -
Run tests beneath path Can use your shell’s tab completion
for speedy folder selection -
Run test class Colons are a pain (if you are
trying to write a autocompleter) as they are word delimiters -
https://github.com/asdfasdf/spec Spec plugin https://github.com/bitprophet/spec Write your test classes and methods
as a spec Based on Pinocchio -
https://github.com/asdfasdf/spec BDD-style output And, write your specs to look like
this BDD-style output. -
class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factories.create_order(user=self.user)
def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) The Chuckle Brothers are updating their profiles -
class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factoies.create_order(user=self.user)
def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) System under test -
class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factories.create_order(user=self.user)
def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) Specification On the road towards a full BDD framwork like Lettuce/Behave, but you can still use Nose and all its plugins -
Use-cases Work on a new feature Refactor a feature You’re
reviewing a pull request -
from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ...
class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... -
from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ...
class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... Wrap a class -
from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ...
class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... Wrap a method You can tag slow tests -
Use-cases Work on a new feature Refactor a feature Fix
failing tests -
PDB on failure Need to go up the stack to
get to test code -
Use-cases Work on a new feature Refactor a feature Fix
failing tests Commit Before pushing etc -
0 4 8 12 16 20 24 Focus / flow
/ “zone” Test suite time (seconds) -
0 4 8 12 16 20 24 Focus / flow
/ “zone” Test suite time (seconds) Pre-commit hook? Everything’s rosy when your test suite runs in 100ms -
0 4 8 12 16 20 24 Focus / flow
/ “zone” Test suite time (seconds) But watch out for the threshold of doom -
0 4 8 12 16 20 24 Test suite time
(seconds) Oh no - you’re reading Hacker News! -
Tips for speeding up tests will be in another talk
Oscar distinguishes between unit, integration and functional (acceptance) tests to try and steer people towards unit tests -
import mock from django.utils.functional import curry no_database = curry( mock.patch,
'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt -
import mock from django.utils.functional import curry no_database = curry( mock.patch,
'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt Currying mock.patch! All my favourite bits of Python in one curio -
import mock from django.utils.functional import curry no_database = curry( mock.patch,
'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt Wrap classes/methods -
http://bit.ly/156c48x Number of tests Seconds Oscar’s test suite has doubled
number of tests while halving the time to run. -
--processes=8 Maik spotted this option for Nose and we all
had a cup of tea to celebrate. -
http://bit.ly/19ganRJ (Refactoring a Test) Book has a great chapter on
refactoring a test from dirty to clean -
class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(
name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) This is a bad test -
class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(
name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Bad naming -
class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(
name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Heavy set-up -
class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(
name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Noise -
class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(
name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Unclear assertions -
class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(
name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Sloppy coding Numbered variable names make baby jesus cry -
class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products
= [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) This is the same test, but cleaned up -
class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products
= [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Descriptive names -
class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products
= [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Factory functions Tests should only show relevant details -
class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products
= [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Intention revealing Dumb example but a good thing to do in general -
PyHamcrest github.com/hamcrest/PyHamcrest >>> from hamcrest import * >>> assert_that( shipping.charge,
is_(equal_to(D('4.99')))) >>> assert_that( "RESTful".lower(), contains_string("stfu")) Hamcrest - more readable but too many parentheses? -
should-dsl github.com/nsi-iff/should-dsl >>> from should_dsl import should, should_not >>> {}
|should| be_empty >>> user.email |should| contain("@") >>> basket |should| contain(product) Crazy voodoo here, but quite readable ultimately