Renee Chu - Unit Tests, Cluster Tests: A Comparative Introduction

You've worked on a shared code base and contributed software tests. Great job! But you don't yet know how to set up the holy grail: Having test coverage that's so complete, you can sleep easy knowing any mistake is caught and spoken for. This talk is a "102" level of test talks, describing the different layers of test protection (unit and cluster) and how to approach writing each.

https://us.pycon.org/2016/schedule/presentation/2182/

Transcript

1. Renee Chu, @reneighbor
   PyCon 2016, Portland, OR
   Cluster Tests, Unit Tests
   A Comparative Introduction
   

   View Slide

2. This talk is relevant to you if:
   

   View Slide

3. This talk is relevant to you if you are:
   ● Experienced writing software tests
   

   View Slide

4. This talk is relevant to you if you are:
   ● Experienced writing software tests
   ● Have not yet owned a project and set
   up a test suite
   

   View Slide

5. This talk is relevant to you if you are:
   ● Experienced writing software tests
   ● Have not yet owned a project and set
   up a test suite
   “Testing 102”
   

   View Slide

6. “Testing 102”: Why do you need to know
   about 2 types of testing?
   

   View Slide

7. Engineers are lazy, and….
   

   View Slide

8. Engineers are reliable
   

   View Slide

9. Laziness + Reliability
   

   View Slide

10. Your future job:
    

    View Slide

11. Your future job: Pwitter
    

    View Slide

12. What does Pwitter do?
    GET http://pwitter.com/{username}/tweets See a user’s tweets
    

    View Slide

13. What does Pwitter do?
    GET http://pwitter.com/
    {username}/tweets
    See a user’s tweets
    POST
    -u {user_id}:
    {auth_token}
    -d body=”hello world”
    http://pwitter.com/
    {username}/tweets
    Make a new tweet
    

    View Slide

14. Who uses Pwitter?
    

    View Slide

15. Who uses Pwitter? Humans
    

    View Slide

16. Who uses Pwitter? Websites
    

    View Slide

17. Who uses Pwitter? Mobile apps
    

    View Slide

18. What qualities does Pwitter need to
    have?
    ● Reliability
    

    View Slide

19. What qualities does Pwitter need to
    have?
    ● Reliability
    ○ Guaranteed Uptime
    ■ ie: 99.999% availability (~5min downtime per year)
    

    View Slide

20. What qualities does Pwitter need to
    have?
    ● Reliability
    ○ Guaranteed Uptime
    ■ ie: 99.999% availability (~5min/year)
    ○ No bugs
    ■ ie: a POST to User A’s account shouldn’t add a
    tweet to user B.
    

    View Slide

21. Uptime: Pwitter
    

    View Slide

22. Uptime: Pwitter broken up
    

    View Slide

23. Pwitter broken up
    

    View Slide

24. Pwitter broken up
    

    View Slide

25. Pwitter broken up
    

    View Slide

26. Pwitter broken up more
    

    View Slide

27. Integration Points
    Auth Service Geo Service
    {
    ‘username’: ‘reneighbor’,
    ‘date_created’: ‘12-12-2015’,
    ‘user_id’: 10011’,
    }
    {
    ‘name’: ‘portland’,
    ‘dateCreated’: 1230768000,
    ‘longitude’: 41.51,
    ‘latitude’: 127.67,
    ‘id’: 55505
    }
    

    View Slide

28. Cluster tests- “Thread the needle”
    

    View Slide

29. Cluster tests- “Thread the needle”
    

    View Slide

30. GET “/users/{username}/tweets”
    Unit Tests: Thorough coverage
    

    View Slide

31. GET “/users/{username}/tweets”
    1) Look up user in DB by username
    2) Use user’s ID to fetch tweets
    3) Create an empty list of results
    4) Loop through tweets and append
    each tweet to it
    Unit Tests: Through coverage
    

    View Slide

32. GET “/users/{username}/tweets”
    1) Look up user in DB by username
    a) If DB fetch fails, raise error
    b) If no user found, raise error
    2) Use user’s ID to fetch tweets
    a) If DB fetch fails, raise error
    3) Create an empty list of results
    4) Loop through tweets and append
    each tweet to it
    Unit Tests: Through coverage
    

    View Slide

33. GET “/users/{username}/tweets”
    1) Look up user in DB by username
    a) If DB fetch fails, raise error
    b) If no user found, raise error
    2) Use user’s ID to fetch tweets
    a) If DB fetch fails, raise error
    3) Create an empty list of results
    4) Loop through tweets and append
    each tweet to it
    Unit Tests: Through coverage
    Test Cases
    1. If DB fetches fail, error is
    thrown
    2. If no user is found, error is
    thrown
    3. If no tweets are found, result
    with an empty list is returned
    4. If many tweets are found,
    result with correct number and
    content of tweets is returned
    

    View Slide

34. Unit Tests
    

    View Slide

35. Inspect Pwitter broken up more
    

    View Slide

36. Practical Notes:
    

    View Slide

37. Practical Notes: Where are tests located?
    ● Unit tests: Within same code repo as
    service code
    

    View Slide

38. Practical Notes: Where are tests located?
    ● Unit tests: Within same code repo as
    service code
    ● Cluster Tests: Seperate repo, seperate
    box
    

    View Slide

39. Practical Notes: When are tests run?
    ● Unit tests: before merging in new code
    

    View Slide

40. Practical Notes: When are tests run?
    ● Unit tests: before merging in new code
    ● Cluster tests: continuously
    

    View Slide

41. Closing pragmatic advice:
    ● Don’t let the perfect be the enemy of
    the good
    

    View Slide

42. Closing pragmatic advice:
    ● Don’t let the perfect be the enemy of
    the good
    ● Too many tests is (almost) as bad as
    too few tests
    

    View Slide

43. Reliability
    Uptime
    

    View Slide

44. Reliability
    Correctness
    

    View Slide

45. Laziness
    Correctness
    Uptime
    

    View Slide