Writing Dumb Tests

Writing and debugging code is hard, but testing shouldn't be. This talk will discuss common techniques for writing simpler tests that still exercise your production code while preventing you from spending time debugging test code.

Transcript

1. Writing Dumb Tests Luke Lee References: http://bit.ly/dumber_tests_pytx2014
2. Luke Lee • Scientific Python/Desktop Visualization • Recovering C developer • @durden20 • lukelee.me • codrspace.com/durden References: http://bit.ly/dumber_tests_pytx2014
3. The Last Day of a Sprint References: http://bit.ly/dumber_tests_pytx2014
4. The Last Day of a Sprint References: http://bit.ly/dumber_tests_pytx2014
5. Testing Woes References: http://bit.ly/dumber_tests_pytx2014
6. Smart Tests • Too many details • Data • Too much software References: http://bit.ly/dumber_tests_pytx2014
7. Smart Tests Too Many Details import collections entry = collections.namedtuple('entry', ['name', 'value', 'units']) def lines_to_tuples(filename): tuples = [] with open(filename, 'r') as file_obj: for line in file_obj: fields = line.split() tuples.append(entry(*fields)) return tuples References: http://bit.ly/dumber_tests_pytx2014
8. Smart Tests Too Many Details def test_lines_to_tuples_1(): # Test.txt contains: # Height 60 in # Length 30 in filename = 'test.txt' test_tuples = lines_to_tuples(filename) with open(filename, 'r') as file_obj: for line, test_tuple in zip(file_obj, test_tuples): fields = line.split() assert entry(*fields) == test_tuple References: http://bit.ly/dumber_tests_pytx2014
9. Smart Tests Too Many Details "We've fallen into a trap of testing that the code does what the code says it does, rather than testing functional behavior we care about." -- Daniel Pope, "Every mock.patch is a little smell" References: http://bit.ly/dumber_tests_pytx2014
10. Smart Tests Data Your data is clean, users' data is gold standard References: http://bit.ly/dumber_tests_pytx2014
11. Random Data References: http://bit.ly/dumber_tests_pytx2014
12. Random Data Fails sometimes References: http://bit.ly/dumber_tests_pytx2014
13. Smart Tests Too Much Software • Too easy to change • Too much cognitive overhead References: http://bit.ly/dumber_tests_pytx2014
14. import unittest class TestSequenceFunctions(unittest.TestCase): def setUp(self): self.seq = range(10) def test_shuffle(self): # make sure the shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10)) # should raise an exception for an immutable sequence self.assertRaises(TypeError, random.shuffle, (1,2,3)) def test_sample(self): with self.assertRaises(ValueError): random.sample(self.seq, 20) for element in random.sample(self.seq, 5): self.assertTrue(element in self.seq) References: http://bit.ly/dumber_tests_pytx2014
15. import pytest @pytest.fixture(scope="module", params=["merlinux.eu", "mail.python.org"]) def smtp(request): smtp = smtplib.SMTP(request.param) def fin(): print ("finalizing %s" % smtp) smtp.close() request.addfinalizer(fin) return smtp References: http://bit.ly/dumber_tests_pytx2014
16. Bugs hide in complexity References: http://bit.ly/dumber_tests_pytx2014
17. Saved by the Tests References: http://bit.ly/dumber_tests_pytx2014
18. Smart enough to find bugs but dumb enough to DEBUG References: http://bit.ly/dumber_tests_pytx2014
19. Dumb Tests • Doctests • Data over code • Think goals, not implementation • Be brave References: http://bit.ly/dumber_tests_pytx2014
20. Dumb Tests import doctest • Limited means simple References: http://bit.ly/dumber_tests_pytx2014
21. Dumb Tests Data over code • Commit examples with real data • numpy.savetxt • StringIO References: http://bit.ly/dumber_tests_pytx2014
22. Dumb Tests Data over code • difflib • filecmp • binary data References: http://bit.ly/dumber_tests_pytx2014
23. Dumb Tests Goals, not implementation • Docs guide testing • Postpone code coverage References: http://bit.ly/dumber_tests_pytx2014
24. Dumb Tests Goals, not implementation def test_lines_to_tuples_2(): # Test.txt contains: # Height 60 in # Length 30 in filename = 'test.txt' correct_tuples = [] correct_tuples.append(entry(name='Height', value='60', units='in')) correct_tuples.append(entry(name='Length', value='30', units='in')) test_tuples = lines_to_tuples(filename) assert correct_tuples == test_tuples References: http://bit.ly/dumber_tests_pytx2014
25. Dumb Tests Goals, not implementation Embrace the experience of other people, people not schooled in the details of your implementation, you can learn how to make your work easier to absorb, and therefore more effective at what it's designed to do. -- Dave Winer, Early author of RSS software References: http://bit.ly/dumber_tests_pytx2014
26. Be Brave! References: http://bit.ly/dumber_tests_pytx2014
27. Dumb Tests Be Brave! import pytest @pytest.mark.parametrize("input,expected", [ ("3+5", 8), ("2+4", 6), ("6*9", 42), ]) def test_eval(input, expected): assert eval(input) == expected References: http://bit.ly/dumber_tests_pytx2014
28. Use asserts References: http://bit.ly/dumber_tests_pytx2014
29. Conclusion 1. Uses real world data 2. Focuses on goals not implementation details 3. Lowers cognitive oveheard with brave, simple code References: http://bit.ly/dumber_tests_pytx2014
30. Write Boring Tests References: http://bit.ly/dumber_tests_pytx2014
31. @durden20 www.lukelee.me codrspace.com/durden References: http://bit.ly/dumber_tests_pytx2014