Testing Workshop - Part 2 (PyConUK 2011)

Transcript

1. Testing with Python
2. The REPL (aka the Python Interpreter)
3. Impress your friends: • Read • Eval • Print • Loop
5. Pros • Familiar to Pythonistas • Great for exploration • Simple to use
6. Cons • Manual • Error-prone • Doesn't scale
7. We need automation!
8. And lo, there were Doctests!
9. def double(value): """ >>> double(1) 2 >>> double(2) 4 >>> double("foo") 'foofoo' >>> double(None) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in double TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' """ return value * 2
10. def double(value): """ >>> double(1) 2 >>> double(2) 4 >>> double("foo") 'foofoo' >>> double(None) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in double TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' """ return value * 2 A REPL session "embedded" in a function's doc string
12. Pros • Automated • All the benefits of the REPL • Code and tests together
13. Cons • Code and tests together • Can get big and clunky • Not very flexible
14. We need something better!
15. Behold: Unittest!
16. Unittest is part of the xUnit family xUnit started with SUnit for Smalltalk and has become a de facto standard The API is broadly similar across many different languages.
17. Pros • Automated • Flexible • Scalable • Part of the standard library • Part of the xUnit family
18. Cons • "Unpythonic" Java-esque API • Neglected until recently • Code and tests separate (pro?) • Potential for unwieldy test suites
19. The Building Blocks • Assertions • Test Cases • Test Fixtures • Test Suites • Test Runners
20. Test Runners A test runner is a component which orchestrates the execution of tests and provides the outcome to the user. The runner may use a graphical interface, a textual interface, or return a special value to indicate the results of executing the tests. Source: Python documentation
21. Test Suites A test suite is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together. Source: Python documentation
22. Test Fixtures A test fixture represents the preparation needed to perform one or more tests, and any associate cleanup actions. This may involve, for example, creating temporary or proxy databases, directories, or starting a server process. Source: Python documentation
23. Test Cases A test case is the smallest unit of testing. It checks for a specific response to a particular set of inputs. unittest provides a base class, TestCase, which may be used to create new test cases. Source: Python documentation
24. Assertions In computer programming, an assertion is a predicate (for example a true–false statement) placed in a program to indicate that the developer thinks that the predicate is always true at that place. Source: Wikipedia
25. import unittest class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz') if __name__ == '__main__': unittest.main()
26. import unittest class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz') if __name__ == '__main__': unittest.main() This calls the standard test runner, and gathers up all TestCase classes in the file.
27. import unittest class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz') if __name__ == '__main__': unittest.main() Test cases are subclasses of TestCase.
28. import unittest class TestFoo(unittest.TestCase): def test_something(self): self.assertTrue('x' in 'xyz') if __name__ == '__main__': unittest.main() And there's an assertion. Simples.
29. There are more types of assertion than assertTrue, otherwise things could get awkward.
30. Note: I'm still stuck with Python 2.4, 2.5 and 2.6! Python 2.7 and 3.x have lots of new assertions available.
31. assertTrue has a negative counterpart: assertTrue(x) assertFalse(x) ...which can aid readability, but isn't particularly exciting.
32. We can test for equality: assertEqual(x, y) Which is asserting that x == y Like assertTrue, it has a counterpart: assertNotEqual(x, y)
33. A personal favourite of mine is: fail(message) Which I use as a placeholder: fail("Test not implemented") fail("Good. You do run the tests")
34. Ah, yes... what happens if a test fails?
36. Speaking of messages, most assertions have an optional argument for a message. assertTrue(x, "It lied") assertEqual(x, y, "Blah blah") The message shows if the assertion fails.
37. Another important assertion is: assertRaises(Exception, foo.x, y) In newer versions of Python, this has a much cleaner syntax... with self.assertRaises(Exception): foo.x(y)
38. Keep tests simple Test one thing at a time
39. class TestFoo(unittest.TestCase): def test_double_one(self): foo = MyFoo() self.assertEqual(2, foo.double(1)) def test_double_two(self): foo = MyFoo() self.assertEqual(4, foo.double(2))
40. Did you spot the duplication? Time to introduce Test Fixtures...
41. Fixtures allow us to set up and tear down the environment of the tests. It also allows us to factor out any boilerplate and duplication.
42. class TestFoo(unittest.TestCase): def test_double_one(self): foo = MyFoo() self.assertEqual(2, foo.double(1)) def test_double_two(self): foo = MyFoo() self.assertEqual(4, foo.double(2)) Let's factor out the duplication...
43. class TestFoo(unittest.TestCase): def setUp(self): self.foo = MyFoo() def test_double_one(self): self.assertEqual(2, self.foo.double(1)) def test_double_two(self): self.assertEqual(4, self.foo.double(2))
44. class TestFoo(unittest.TestCase): def setUp(self): self.foo = MyFoo() def test_double_one(self): self.assertEqual(2, self.foo.double(1)) def test_double_two(self): self.assertEqual(4, self.foo.double(2)) setUp is called before every test in the test case.
45. setUp has a partner called tearDown which, if present, is called after every test. Ideal for cleaning up afterwards. def tearDown(self): self.foo.close()
46. Things to think about when testing...
47. Keep each test case focused on a particular unit of code Organise tests within the test case to check individual aspects of the code Helps keep the tests simple and maintainable
48. Multiple test cases can exist in a file. Ideally, break out the test cases into multiple files just as you would with your code.
49. If code is difficult to test: refactor or rewrite to make it easier. This also leads to simpler, more maintainable code.
50. When you write tests, you're adding confidence in the code There's always one more bug to find There's always one more test to write
51. If you find a bug in your code, write a test to replicate it before you implement a fix
52. Even a "bad" test gives value over no test. Bad tests can always be improved as we understand the problem / solution.
53. Code under test should be isolated whenever possible, to minimise external influence We can isolate code from things like databases, network connections, etc. through Fakes and Mocks
54. Fakes are dummy objects that do nothing except stand-in for something Mocks are more interesting. They can capture and respond to messages And now over to Michael...