A smörgåsbord of testing

A smörgåsbord of testing

The controversy over writing tests and TDD is over. The benefits to the design, development and maintenance of your code are real and well known. In 2013, it is simply unprofessional to eschew tests for production code.

Given this, developers need to move beyond the basics of testing and gain a deeper understanding of the art. This entails knowing the patterns and anti-patterns of effective test suites, understanding what to test and how to do it, and a mastery of the many fantastic testing tools in the Python ecosystem.

Bad tests are a genuine menace. A poorly considered test suite pours concrete over your codebase, hindering maintenance and development.

This talk isn't an introduction to testing and TDD. Rather, it is a mixed buffet of tips, anti-patterns, tools and techniques for effective testing.

52d39c7b27386ca98bc016119d95b8b8?s=128

David Winterbottom

September 20, 2013
Tweet

Transcript

  1. 5.

    django-oscar @django_oscar /tangentlabs/django-oscar Oscar is the most popular python e-commerce

    software on Github. It’s code is so beautiful, it’s been known to make grown men weep. oscarcommerce.com
  2. 6.

    This is Oscar’s test suite - we can see a

    few things which betray the biases of this talk
  3. 10.

    Who is this? (Hint - it’s not Philip Seymour Hoffman).

    It’s Kent Beck: creator of jUnit, XP etc
  4. 11.

    “I get paid for code that works, not for tests,

    so my philosophy is to test as little as possible to reach a given level of confidence ” ...and he’s a pragmatist. Beware of dogma (100% coverage, one assertion per method etc)
  5. 13.

    Dexterity Is there a clever Japanese word for the art

    of slicing? Dexterity = How to use your test suite effectively
  6. 18.

    Run test class Colons are a pain (if you are

    trying to write a autocompleter) as they are word delimiters
  7. 22.
  8. 29.

    class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factories.create_order(user=self.user)

    def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) The Chuckle Brothers are updating their profiles
  9. 30.

    class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factoies.create_order(user=self.user)

    def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) System under test
  10. 31.

    class TestASignedInUser(WebTestCase): def setUp(self): self.user = G(User) self.order = factories.create_order(user=self.user)

    def test_can_see_their_email_address_on_the_profile_page(self): profile_page = self.app.get( reverse('customer:summary'), user=self.user) self.assertTrue(self.email in profile_page.content) def test_can_update_their_name(self): profile_form_page = self.app.get( reverse('customer:profile-update'), user=self.user) form = profile_form_page.forms['profile_form'] form['first_name'] = 'Barry' form['last_name'] = 'Chuckle' response = form.submit() self.assertRedirects(response, reverse('customer:summary')) # Reload user user = User.objects.get(id=self.user.id) self.assertEquals("Barry", user.first_name) self.assertEquals("Chuckle", user.last_name) Specification On the road towards a full BDD framwork like Lettuce/Behave, but you can still use Nose and all its plugins
  11. 35.

    from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ...

    class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ...
  12. 36.

    from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ...

    class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... Wrap a class
  13. 37.

    from nose.plugins.attrib import attr @attr('shipping') class TestFreeShippping(TestCase): def test_is_free_for_empty_basket(self): ...

    class TestFixedPriceShipping(TestCase): @attr(slow=True) def test_returns_fixed_price_for_basket(self): ... Wrap a method You can tag slow tests
  14. 38.
  15. 41.
  16. 45.

    Use-cases Work on a new feature Refactor a feature Fix

    failing tests Commit Before pushing etc
  17. 50.

    0 4 8 12 16 20 24 Focus / flow

    / “zone” Test suite time (seconds)
  18. 51.

    0 4 8 12 16 20 24 Focus / flow

    / “zone” Test suite time (seconds) Pre-commit hook? Everything’s rosy when your test suite runs in 100ms
  19. 52.

    0 4 8 12 16 20 24 Focus / flow

    / “zone” Test suite time (seconds) But watch out for the threshold of doom
  20. 53.

    0 4 8 12 16 20 24 Test suite time

    (seconds) Oh no - you’re reading Hacker News!
  21. 54.

    Tips for speeding up tests will be in another talk

    Oscar distinguishes between unit, integration and functional (acceptance) tests to try and steer people towards unit tests
  22. 55.
  23. 57.

    import mock from django.utils.functional import curry no_database = curry( mock.patch,

    'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt
  24. 58.

    import mock from django.utils.functional import curry no_database = curry( mock.patch,

    'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt Currying mock.patch! All my favourite bits of Python in one curio
  25. 59.

    import mock from django.utils.functional import curry no_database = curry( mock.patch,

    'django.db.backends.util.CursorWrapper', Mock(side_effect=RuntimeError( "Using the database is not permitted"))) @no_database() class AUnitTest(TestCase): ... http://bit.ly/126vPrt Wrap classes/methods
  26. 60.
  27. 66.

    class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(

    name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) This is a bad test
  28. 67.

    class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(

    name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Bad naming
  29. 68.

    class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(

    name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Heavy set-up
  30. 69.

    class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(

    name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Noise
  31. 70.

    class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(

    name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Unclear assertions
  32. 71.

    class TestBasket(TestCase): def test_add_items(self): # Create products product_class = ProductClass(

    name="Books", require_shipping=True) product1 = Product.objects.create(title="My first book") product2 = Product.objects.create(title="My second book") partner = Partner.objects.create(name="Book Partner") stockrecord1 = StockRecord( price_excl_tax=D('12.00'), partner=partner) stockrecord2 = StockRecord( price_excl_tax=D('14.00'), partner=partner) # Add to basket basket = Basket() basket.add(product1) basket.add(product2) total = basket.total_excl_tax self.assertEqual(D('26.00'), total) self.assertEqual(basket.num_lines, 2) self.assertFalse(basket.is_empty) Sloppy coding Numbered variable names make baby jesus cry
  33. 72.

    class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products

    = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) This is the same test, but cleaned up
  34. 73.

    class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products

    = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Descriptive names
  35. 74.

    class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products

    = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Factory functions Tests should only show relevant details
  36. 75.

    class TestBasketTotal(TestCase): def setUp(self): self.basket = Basket() def test_is_correct_after_adding_multiple_items(self): products

    = [ factory.create_product(price_excl_tax=D('12.00')), factory.create_product(price_excl_tax=D('14.00')), ] for product in products: self.basket.add_product(product) self.assertEqual( D('12.00') + D('14.00'), self.basket.total_excl_tax) Intention revealing Dumb example but a good thing to do in general
  37. 76.

    PyHamcrest github.com/hamcrest/PyHamcrest >>> from hamcrest import * >>> assert_that( shipping.charge,

    is_(equal_to(D('4.99')))) >>> assert_that( "RESTful".lower(), contains_string("stfu")) Hamcrest - more readable but too many parentheses?
  38. 77.

    should-dsl github.com/nsi-iff/should-dsl >>> from should_dsl import should, should_not >>> {}

    |should| be_empty >>> user.email |should| contain("@") >>> basket |should| contain(product) Crazy voodoo here, but quite readable ultimately