The practice of TDD: tips&tricks

Test Driven Development is a well known practice in software development. However, passing from knowing the principles of TDD to applying them in real world situations is not straightforward: the aim of this talk is to help the audience to fill the gap and apply TDD effectively in Python. The talk will include:

a brief overview of most popular tools and libraries (e.g. unittest, pytest, nose, tox)
useful design patterns
common mistakes and how to avoid them
some real life example taken from the projects the author has worked on in the past 15 years (including PyPy, pdb++, capnpy)

This talk is primarily aimed at beginners.

Transcript

1. The practice of TDD: tips&tricks Antonio Cuni PyCon ZA 2017 antocuni (PyCon ZA 2017) TDD tips&ticks 1 / 41
2. About me PyPy core dev since 2006 pdb++, cffi, vmprof, capnpy, ... @antocuni http://antocuni.eu antocuni (PyCon ZA 2017) TDD tips&ticks 1 / 41
3. About you Either: novice programmer experienced programmer but new to Python and/or TDD antocuni (PyCon ZA 2017) TDD tips&ticks 2 / 41
4. About this talk Two parts General TDD principles Useful patterns and tips antocuni (PyCon ZA 2017) TDD tips&ticks 3 / 41
5. Disclaimer I wanted to show more examples antocuni (PyCon ZA 2017) TDD tips&ticks 4 / 41
6. Disclaimer antocuni (PyCon ZA 2017) TDD tips&ticks 5 / 41
7. Disclaimer antocuni (PyCon ZA 2017) TDD tips&ticks 6 / 41
8. Disclaimer antocuni (PyCon ZA 2017) TDD tips&ticks 7 / 41
9. Disclaimer antocuni (PyCon ZA 2017) TDD tips&ticks 8 / 41
10. Disclaimer Let’s start antocuni (PyCon ZA 2017) TDD tips&ticks 9 / 41
11. The goal of testing Make sure that your code works WRONG! antocuni (PyCon ZA 2017) TDD tips&ticks 10 / 41
12. The goal of testing Make sure that your code works WRONG! antocuni (PyCon ZA 2017) TDD tips&ticks 10 / 41
13. The goal of testing Make sure that your code works Make sure that your code does NOT break antocuni (PyCon ZA 2017) TDD tips&ticks 11 / 41
14. Manual testing (1) Feature A Write the code Start the program Navigate through N steps login click on few links push a button CRASH! (repeat) antocuni (PyCon ZA 2017) TDD tips&ticks 12 / 41
15. Manual testing (1) Feature A Write the code Start the program Navigate through N steps login click on few links push a button CRASH! (repeat) antocuni (PyCon ZA 2017) TDD tips&ticks 12 / 41
16. Manual testing (2) Feature B Modify the code Feature B works! :-) (Feature A no longer works, but you don’t notice) antocuni (PyCon ZA 2017) TDD tips&ticks 13 / 41
17. Manual testing (2) Feature B Modify the code Feature B works! :-) (Feature A no longer works, but you don’t notice) antocuni (PyCon ZA 2017) TDD tips&ticks 13 / 41
18. Automated testing (1) Write the code for Feature A Run a command (~0.5 secs) py.test test_foo.py -k test_feature_A CRASH (repeat) antocuni (PyCon ZA 2017) TDD tips&ticks 14 / 41
19. Automated testing (2) Write the code for Feature B Run a command (~0.5 secs) py.test test_foo.py -k test_feature_B It works py.test test_foo.py CRASH fix Everyone is happy :-) antocuni (PyCon ZA 2017) TDD tips&ticks 15 / 41
20. Automated testing (2) Write the code for Feature B Run a command (~0.5 secs) py.test test_foo.py -k test_feature_B It works py.test test_foo.py CRASH fix Everyone is happy :-) antocuni (PyCon ZA 2017) TDD tips&ticks 15 / 41
21. Automated testing (3) What’s the missing piece? You have to write the test! It’s just a program test frameworks/runners offer a lot of help unittest, unittest2 nose “py.test“ antocuni (PyCon ZA 2017) TDD tips&ticks 16 / 41
22. Automated testing (3) What’s the missing piece? You have to write the test! It’s just a program test frameworks/runners offer a lot of help unittest, unittest2 nose “py.test“ antocuni (PyCon ZA 2017) TDD tips&ticks 16 / 41
23. Test Driven Development Goal: make sure that our code does not break What is not tested is broken (aka: Murphy’s law) Even if it’s not broken right now, it’ll eventually break antocuni (PyCon ZA 2017) TDD tips&ticks 17 / 41
24. Tests first Writing code when no test is failing is forbidden You should write just the code to make the test passing don’t cheat :-) Each test must run in isolation bonus track: VCS commit every time you write/fix a test write meaningful commit messages don’t commit if the tests are broken (unless you are sure it’s the right thing to do :-)) make you confident in your tests {hg,git} bisect antocuni (PyCon ZA 2017) TDD tips&ticks 18 / 41
25. Tests first Writing code when no test is failing is forbidden You should write just the code to make the test passing don’t cheat :-) Each test must run in isolation bonus track: VCS commit every time you write/fix a test write meaningful commit messages don’t commit if the tests are broken (unless you are sure it’s the right thing to do :-)) make you confident in your tests {hg,git} bisect antocuni (PyCon ZA 2017) TDD tips&ticks 18 / 41
26. TDD benefits confidence about the quality of the code easily spot regressions easily find by who/when/why a regression was introduced "Why the hell did I write this piece of code?" look at the commit, and the corresponding test Remove the code, and see if/which tests fail "One of my most productive days was throwing away 1000 lines of code" (Ken Thompson) "Deleted code is debugged code" (Jeff Sickel) The power of refactoring antocuni (PyCon ZA 2017) TDD tips&ticks 19 / 41
27. Properties of a good test It should FAIL before your fix write the test first, then the code Determinism NEVER write a test which fails every other run pay attention e.g. to dictionary order Easy to READ executable documentation antocuni (PyCon ZA 2017) TDD tips&ticks 20 / 41
28. Readability A test tells a story One feature per test No complex control flow Clear failures When a test fails, the poor soul looking at it should be able to understand why antocuni (PyCon ZA 2017) TDD tips&ticks 21 / 41
29. Factorial example Bad def test_factorial(): for n in (5, 7): res = 1 for i in range(1, n+1): res *= i assert factorial(n) == res antocuni (PyCon ZA 2017) TDD tips&ticks 22 / 41
30. Factorial example antocuni (PyCon ZA 2017) TDD tips&ticks 23 / 41
31. Factorial example Better def test_factorial(): assert factorial(5) == 120 assert factorial(7) == 5040 antocuni (PyCon ZA 2017) TDD tips&ticks 24 / 41
32. Factorial example Best def test_factorial(): assert factorial(5) == 2 * 3 * 4 * 5 assert factorial(7) == 2 * 3 * 4 * 5 * 6 * 7 antocuni (PyCon ZA 2017) TDD tips&ticks 25 / 41
33. Easy to write We are all lazy, deal with it The easiest to write a test, the more likely we’ll do Invest time in a proper test infrastructure Test the infrastructure as well :) antocuni (PyCon ZA 2017) TDD tips&ticks 26 / 41
34. test_pypy_c example Bad def test_call_pypy(tmpdir): src = """if 1: def factorial(n): if n in (0, 1): return 1 return n * factorial(n-1) import sys n = eval(sys.argv[1]) print factorial(n) """ pyfile = tmpdir.join("x.py") pyfile.write(src) stdout = subprocess.check_output( ["pypy", str(pyfile), "5"]) res = eval(stdout) assert res == 2*3*4*5 antocuni (PyCon ZA 2017) TDD tips&ticks 27 / 41
35. test_pypy_c example Still bad def execute(tmpdir, src, *args): pyfile = tmpdir.join("x.py") pyfile.write(src) args = ["pypy", str(pyfile)] + list(args) stdout = subprocess.check_output(args) return eval(stdout) def test_call_pypy_2(tmpdir): src = """if 1: def factorial(n): if n in (0, 1): return 1 return n * factorial(n-1) import sys n = eval(sys.argv[1]) print factorial(n) """ res = execute(tmpdir, src, "5") assert res == 2*3*4*5 antocuni (PyCon ZA 2017) TDD tips&ticks 28 / 41
36. test_pypy_c example Good class TestCall(PyPyTest): def test_call(self): def factorial(n): if n in (0, 1): return 1 return n * factorial(n-1) # res = self.run(factorial, 5) assert res == 2*3*4*5 antocuni (PyCon ZA 2017) TDD tips&ticks 29 / 41
37. test_pypy_c example @pytest.mark.usefixtures(’initargs’) class PyPyTest(object): @pytest.fixture def initargs(self, tmpdir): self.tmpdir = tmpdir def run(self, fn, *args): fnsrc = textwrap.dedent(inspect.getsource(fn)) boilerplate = textwrap.dedent(""" import sys args = map(eval, sys.argv[1:]) print {fnname}(*args) """).format(fnname=fn.__name__) src = fnsrc + boilerplate args = map(str, args) return execute(self.tmpdir, src, *args) antocuni (PyCon ZA 2017) TDD tips&ticks 30 / 41
38. test_pypy_c example pypy/module/pypyjit/test/model.py https://bitbucket.org/pypy/pypy/src/ 5a1232a1c2e6e01422713659c57ae3bcd5db3f70/pypy/ module/pypyjit/test_pypy_c/model.py?at=default& fileviewer=file-view-default pypy/module/pypyjit/test/test_00model.py https://bitbucket.org/pypy/pypy/src/ 5a1232a1c2e6e01422713659c57ae3bcd5db3f70/pypy/ module/pypyjit/test_pypy_c/test_00_model.py?at= default&fileviewer=file-view-default capnpy/testing/compiler/support.py https://github.com/antocuni/capnpy/blob/master/ capnpy/testing/compiler/support.py antocuni (PyCon ZA 2017) TDD tips&ticks 31 / 41
39. Decoupling components Useful in general In particular for GUIs Leads to a better design antocuni (PyCon ZA 2017) TDD tips&ticks 32 / 41
40. Decoupling components Google for tkinter tutorial http: //python-textbok.readthedocs.io/en/1. 0/Introduction_to_GUI_Programming.html antocuni (PyCon ZA 2017) TDD tips&ticks 33 / 41
41. Decoupling components class Calculator: def __init__(self, master): ... self.total = 0 self.entered_number = 0 self.total_label_text = IntVar() self.total_label_text.set(self.total) self.add_button = ... self.subtract_button = ... self.reset_button = ... def update(self, method): if method == "add": self.total += self.entered_number elif method == "subtract": self.total -= self.entered_number else: # reset self.total = 0 self.total_label_text.set(self.total) self.entry.delete(0, END) antocuni (PyCon ZA 2017) TDD tips&ticks 34 / 41
42. Decoupling components class Calculator(object): def __init__(self): self.total = 0 def add(self, x): self.total += x def sub(self, x): self.total -= x def reset(self): self.total = 0 antocuni (PyCon ZA 2017) TDD tips&ticks 35 / 41
43. Decoupling components class GUI(object): def __init__(self, master): self.calculator = Calculator() ... self.add_button = ... self.subtract_button = ... self.reset_button = ... self.update_total() def update_total(self): self.total_label_text.set(self.calculator.total) def update(self, method): if method == "add": self.calculator.add(self.entered_number) elif method == "subtract": self.calculator.sub(self.entered_number) else: # reset self.calculator.reset() self.update_total() self.entry.delete(0, END) antocuni (PyCon ZA 2017) TDD tips&ticks 36 / 41
44. Decoupling components def test_Calculator(): calc = Calculator() assert calc.total == 0 calc.add(3) assert calc.total == 3 calc.add(5) assert calc.total == 8 calc.sub(7) assert calc.total == 1 calc.sub(10) assert calc.total == -9 calc.reset() assert calc.total == 0 antocuni (PyCon ZA 2017) TDD tips&ticks 37 / 41
45. Notation Chapter 9: Notation antocuni (PyCon ZA 2017) TDD tips&ticks 38 / 41
46. Notation ops = [ MergePoint(’merge_point’, [sum, n1], []), ResOperation(’guard_class’, [n1, ConstAddr(node_vtable, cpu)], []), ResOperation(’getfield_gc__4’, [n1, ConstInt(ofs_value)], [v]), ResOperation(’int_sub’, [v, ConstInt(1)], [v2]), ResOperation(’int_add’, [sum, v], [sum2]), ResOperation(’new_with_vtable’, [ConstInt(size_of_node), ConstAddr(node_vtable, cpu)], [n2]), ResOperation(’setfield_gc__4’, [n2, ConstInt(ofs_value), v2], []), Jump(’jump’, [sum2, n2], []), ] antocuni (PyCon ZA 2017) TDD tips&ticks 39 / 41
47. Notation https://bitbucket.org/pypy/pypy/src/ 5a1232a1c2e6e01422713659c57ae3bcd5db3f70/rpython/jit/metainterp/ optimizeopt/test/test_optimizeopt.py?at=default&fileviewer= file-view-default def test_remove_guard_class_1(self): ops = """ [p0] guard_class(p0, ConstClass(node_vtable)) [] guard_class(p0, ConstClass(node_vtable)) [] jump(p0) """ preamble = """ [p0] guard_class(p0, ConstClass(node_vtable)) [] jump(p0) """ expected = """ [p0] jump(p0) """ self.optimize_loop(ops, expected, expected_preamble=preamble) antocuni (PyCon ZA 2017) TDD tips&ticks 40 / 41
48. Q&A @antocuni https: //bitbucket.org/antocuni/misc/src/ 704c99980401dbb638387ded33e6c23157a683 talks/tdd-tips-and-tricks/?at= default antocuni (PyCon ZA 2017) TDD tips&ticks 41 / 41