Renee Chu - Unit Tests, Cluster Tests: A Comparative Introduction

You've worked on a shared code base and contributed software tests. Great job! But you don't yet know how to set up the holy grail: Having test coverage that's so complete, you can sleep easy knowing any mistake is caught and spoken for. This talk is a "102" level of test talks, describing the different layers of test protection (unit and cluster) and how to approach writing each.

https://us.pycon.org/2016/schedule/presentation/2182/

Transcript

1. Renee Chu, @reneighbor PyCon 2016, Portland, OR Cluster Tests, Unit Tests A Comparative Introduction
2. This talk is relevant to you if:
3. This talk is relevant to you if you are: • Experienced writing software tests
4. This talk is relevant to you if you are: • Experienced writing software tests • Have not yet owned a project and set up a test suite
5. This talk is relevant to you if you are: • Experienced writing software tests • Have not yet owned a project and set up a test suite “Testing 102”
6. “Testing 102”: Why do you need to know about 2 types of testing?
7. Engineers are lazy, and….
8. Engineers are reliable
9. Laziness + Reliability
10. Your future job:
11. Your future job: Pwitter
12. What does Pwitter do? GET http://pwitter.com/{username}/tweets See a user’s tweets
13. What does Pwitter do? GET http://pwitter.com/ {username}/tweets See a user’s tweets POST -u {user_id}: {auth_token} -d body=”hello world” http://pwitter.com/ {username}/tweets Make a new tweet
14. Who uses Pwitter?
15. Who uses Pwitter? Humans
16. Who uses Pwitter? Websites
17. Who uses Pwitter? Mobile apps
18. What qualities does Pwitter need to have? • Reliability
19. What qualities does Pwitter need to have? • Reliability ◦ Guaranteed Uptime ▪ ie: 99.999% availability (~5min downtime per year)
20. What qualities does Pwitter need to have? • Reliability ◦ Guaranteed Uptime ▪ ie: 99.999% availability (~5min/year) ◦ No bugs ▪ ie: a POST to User A’s account shouldn’t add a tweet to user B.
21. Uptime: Pwitter
22. Uptime: Pwitter broken up
23. Pwitter broken up
24. Pwitter broken up
25. Pwitter broken up
26. Pwitter broken up more
27. Integration Points Auth Service Geo Service { ‘username’: ‘reneighbor’, ‘date_created’: ‘12-12-2015’, ‘user_id’: 10011’, } { ‘name’: ‘portland’, ‘dateCreated’: 1230768000, ‘longitude’: 41.51, ‘latitude’: 127.67, ‘id’: 55505 }
28. Cluster tests- “Thread the needle”
29. Cluster tests- “Thread the needle”
30. GET “/users/{username}/tweets” Unit Tests: Thorough coverage
31. GET “/users/{username}/tweets” 1) Look up user in DB by username 2) Use user’s ID to fetch tweets 3) Create an empty list of results 4) Loop through tweets and append each tweet to it Unit Tests: Through coverage
32. GET “/users/{username}/tweets” 1) Look up user in DB by username a) If DB fetch fails, raise error b) If no user found, raise error 2) Use user’s ID to fetch tweets a) If DB fetch fails, raise error 3) Create an empty list of results 4) Loop through tweets and append each tweet to it Unit Tests: Through coverage
33. GET “/users/{username}/tweets” 1) Look up user in DB by username a) If DB fetch fails, raise error b) If no user found, raise error 2) Use user’s ID to fetch tweets a) If DB fetch fails, raise error 3) Create an empty list of results 4) Loop through tweets and append each tweet to it Unit Tests: Through coverage Test Cases 1. If DB fetches fail, error is thrown 2. If no user is found, error is thrown 3. If no tweets are found, result with an empty list is returned 4. If many tweets are found, result with correct number and content of tweets is returned
34. Unit Tests
35. Inspect Pwitter broken up more
36. Practical Notes:
37. Practical Notes: Where are tests located? • Unit tests: Within same code repo as service code
38. Practical Notes: Where are tests located? • Unit tests: Within same code repo as service code • Cluster Tests: Seperate repo, seperate box
39. Practical Notes: When are tests run? • Unit tests: before merging in new code
40. Practical Notes: When are tests run? • Unit tests: before merging in new code • Cluster tests: continuously
41. Closing pragmatic advice: • Don’t let the perfect be the enemy of the good
42. Closing pragmatic advice: • Don’t let the perfect be the enemy of the good • Too many tests is (almost) as bad as too few tests
43. Reliability Uptime
44. Reliability Correctness
45. Laziness Correctness Uptime