Good Test, Bad Test

http://late.am/post/2015/04/20/good-test-bad-test.html

In a highly dynamic language like Python, testing is even more critical than in compiled or more static languages. Like any other code we produce, tests can be either good or bad. This talk explores three myths of testing, and the mistakes and bad habits these fallacies encourage; and shows how to write good tests which help assure proper behavior without impeding development progress.

Transcript

1. good test bad test ! ! ! ! ! dan crosta http://late.am/ @lazlofruvous magnetic
2. virtues of good tests
3. fast selective repeatable reliable helpful
4. fast selective repeatable reliable helpful
5. fast selective repeatable reliable helpful
6. fast selective repeatable reliable helpful
7. fast selective repeatable reliable helpful
8. http://www.free-desktop-backgrounds.net/Miscellaneous-pictures/Signs-wallpaper-images/Caution-sign.html opinions ahead
9. myth 1 100% coverage
10. if every line is covered, it must be correct. ! right?
11. https://www.sparkholder.com/wp-content/uploads/2015/01/sparkholder-api.png (modified)
12. https://s-media-cache-ak0.pinimg.com/originals/ca/4c/56/ca4c56fcdc04212f1bd11eca4a2fca36.jpg "mutually assured destruction"
13. ! some code should not
 be tested
14. what would cause
 my test to fail? !
15. keep your “plumbing” dumb. !
16. myth 2 assert about everything
17. more assertions means
 higher confidence ! right?
18. def test_it_parses_log_lines(self): line = '2015-03-11T20:09:25|GET /foo?bar=baz|...' ! parsed = parse_log_line(line) ! ! ! ! ! ! ! ! ! ! ! !
19. def test_it_parses_log_lines(self): line = '2015-03-11T20:09:25|GET /foo?bar=baz|...' ! parsed = parse_log_line(line) ! self.assertEqual({ "date": datetime(2015, 3, 11, 20, 9, 25), "method": "GET", "path": "/foo", "query": "bar=baz", }, parsed) ! ! ! ! !
20. def test_it_parses_log_lines(self): line = '2015-03-11T20:09:25|GET /foo?bar=baz|...' ! parsed = parse_log_line(line) ! self.assertEqual( datetime(2015, 3, 11, 20, 9, 25), parsed["date"], ) self.assertEqual("GET", parsed["method"]) self.assertEqual("/foo", parsed["path"]) self.assertEqual("bar=baz", parsed["query"]) ! ! ! !
21. def test_it_parses_get_request_log_lines(self): # ... self.assertEqual("GET", parsed["method"]) self.assertEqual("/foo", parsed["path"]) self.assertEqual("bar=baz", parsed["query"]) ! def test_it_parses_post_request_log_lines(self): # ... self.assertEqual("POST", parsed["method"]) self.assertEqual("/foo", parsed["path"]) self.assertEqual("bar=baz", parsed["query"]) ! def test_it_parses_log_lines_with_oof_requests(self): # ... self.assertEqual("GET", parsed["method"]) self.assertEqual("/oof", parsed["path"]) self.assertEqual("bar=baz", parsed["query"])
22. what would I do when a test fails? !
23. have your tests tell you what is wrong. !
24. ! only make one assertion per test
25. def test_it_parses_request_method(self): # ... self.assertEqual("GET", parsed["method"]) ! def test_it_parses_request_path(self): # ... self.assertEqual("/foo", parsed["path"]) ! def test_it_parses_query_string(self): # ... self.assertEqual("bar=baz", parsed["query"]) ! def test_it_gives_none_when_no_query_string(self): # ... self.assertIsNone(parsed["query"]) !
26. ! myth 3 mock makes tests better
27. mock helps you isolate tests to just one layer ! right?
28. def authenticate(username): """Return a True if the user is authenticated """ # ... return True ! ! def login(username): if not authenticate(self.params["username"]): return Response(status=401) ! # ... do more stuff ... ! return Response(status=200) !
29. def test_login_returns_200_on_success(self): with mock.patch(app, "authenticate") as auth: auth.return_value = True ! response = app.login("dcrosta") ! self.assertEqual(200, response.status) ! ! def test_login_returns_401_on_failure(self): with mock.patch(app, "authenticate") as auth: auth.return_value = False ! response = app.login("dcrosta") ! self.assertEqual(401, response.status)
30. def authenticate(username): """Return a tuple of (authenticated, user_roles) """ # ... return (True, ["admin"]) ! ! def login(username): if not authenticate(self.params["username"]): return Response(status=401) ! # ... do more stuff ... ! return Response(status=200) !
31. why might my test pass when it really should fail? !
32. use verified doubles
33. class StubDB(object): ! def __init__(self): self.users = {} ! def register_user(self, username, roles): self.users[username] = roles ! def authenticate(self, username): if username not in self.users: return (False, []) return (True, self.users[username]) ! ! !
34. def test_login_returns_200_on_success(self): db = StubDB() db.register_user("dcrosta", ["admin"]) ! app = MyApp(db) response = app.login("dcrosta") ! self.assertEqual(200, response.status) ! def test_login_returns_401_on_failure(self): db = StubDB() ! app = MyApp(db) response = app.login("dcrosta") ! self.assertEqual(401, response.status)
35. http://gingersnapworks.com/wp-content/uploads/2012/12/facebook_like-hang-tag.jpg library authors: ship test doubles
 with your packages
36. tests can smell, too !
37. design your tests !
38. good tests are your best friends
39. good test bad test ! ! ! ! ! dan crosta http://late.am/ @lazlofruvous magnetic