Writing Tests that Stand the Test of Time

This talk was given at DACHFest (dachfest.com) in Munich, Germany.

Video: https://www.youtube.com/watch?v=Hy5ra9qDpEY

Description:
One of the promises of test driven development is the confidence and ease with which we can refactor our production code.

However, sometimes we find ourselves in a position where our tests are highly coupled to the production code and we have to change the tests every time we update our production code. At this point we start wondering if the tests we had are even worth it if we have to spend so much time to maintain them. If you have ever found yourself having to update your tests every time the behavior of a dependency of the class under test has changed, then this talk is for you.

This talk is to share with you some patterns, tools and examples that can guide us to write more maintainable tests.

Transcript

1. Writing Tests that Stand
   the Test of Time
   Segun Famisa
   @segunfamisa
   

   View Slide

2. segunfamisa
   segunfamisa.com
   

   View Slide

3. Outline
   

   View Slide

4. Outline
   Introduction to TDD
   

   View Slide

5. Outline
   Introduction to TDD
   Challenges with TDD
   

   View Slide

6. Outline
   Introduction to TDD
   Challenges with TDD
   Testing tools in practice
   

   View Slide

7. Outline
   Introduction to TDD
   Challenges with TDD
   Testing tools in practice
   Writing maintainable tests
   

   View Slide

8. Outline
   Introduction to TDD
   Challenges with TDD
   Testing tools in practice
   Writing maintainable tests
   Recap
   

   View Slide

9. Introduction
   

   View Slide

10. So, what’s Test Driven
    Development?
    

    View Slide

11. What is TDD?
    ”Test-Driven Development (TDD) is a
    technique for building software that guides
    software development by writing tests”
    https://martinfowler.com/bliki/TestDrivenDevelopment.html
    

    View Slide

12. What is TDD?
    ”TDD is a software development process that
    relies on the repetition of a very short
    development cycle: requirements are turned
    into very specific test cases, then the
    software is improved to pass the new tests,
    only”
    Wikipedia
    

    View Slide

13. View Slide

14. How to do TDD?
    

    View Slide

15. How to do TDD?
    Red - Green - Refactor
    

    View Slide

16. Red - Green - Refactor
    Write
    failing
    test
    

    View Slide

17. Write
    failing
    test
    Write
    code, just
    enough to
    pass
    Red - Green - Refactor
    

    View Slide

18. Red - Green - Refactor
    Write
    failing
    test
    Write
    code, just
    enough to
    pass
    Clean up
    

    View Slide

19. Red - Green - Refactor
    Write
    failing
    test
    Write
    code, just
    enough to
    pass
    Clean up
    

    View Slide

20. How to do TDD?
    TL;DR
    Don’t write new code until you’ve written a test that fails.
    Don’t write more code than you need to make the test pass.
    Repeat.
    

    View Slide

21. Why do we need tests?
    

    View Slide

22. Why do we need tests?
    ● Quick feedback about bugs/errors
    

    View Slide

23. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    

    View Slide

24. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    ● Documentation for code behavior
    

    View Slide

25. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    ● Documentation for code behavior
    ● Confident refactoring
    

    View Slide

26. Challenges with TDD
    

    View Slide

27. If TDD is so nice, why isn’t
    everyone doing it?
    

    View Slide

28. Time
    Challenges with TDD
    

    View Slide

29. Time
    Tests are code too, so they take
    time to write
    

    View Slide

30. Time
    Tests are code too, so they take
    time to write
    However, time spent writing tests
    is an investment in the long run
    

    View Slide

31. Maintenance
    overhead
    Challenges with TDD
    

    View Slide

32. Maintenance
    overhead
    Challenges with TDD
    TDD becomes challenging when:
    ● Old tests break often
    

    View Slide

33. Maintenance
    overhead
    Challenges with TDD
    TDD becomes challenging when:
    ● Old tests break often
    ● Difficult to add new tests
    

    View Slide

34. TDD becomes challenging when:
    ● Old tests break often
    ● Difficult to add new tests
    ● Tests are too tightly coupled to
    production code
    Maintenance
    overhead
    Challenges with TDD
    

    View Slide

35. Testing tools & concepts in
    Practice
    

    View Slide

36. Test doubles
    

    View Slide

37. Test doubles
    ● Like stunt doubles in movies
    

    View Slide

38. Test doubles
    ● Like stunt doubles in movies
    https://people.com/movies/actors-and-their-stunt-doubles-photos
    

    View Slide

39. Test doubles
    ● Like stunt doubles in movies
    ● Objects we can use in our tests in place of real objects
    

    View Slide

40. Test doubles
    ● Like stunt doubles in movies
    ● Objects we can use in our tests in place of real objects
    ● e.g - dummies, stubs, fakes, mocks, spies
    

    View Slide

41. Dummies
    

    View Slide

42. Dummies
    Objects passed around, but not actually used. Often used to
    fill up constructor or method parameters
    

    View Slide

43. Stubs
    

    View Slide

44. Stubs
    ● Objects that return predefined data
    

    View Slide

45. Stubs
    ● Objects that return predefined data
    ● They usually don’t hold state/respond to other actions
    besides the one they are created for
    

    View Slide

46. Stubs - example
    

    View Slide

47. Stubs - example
    interface IUserRepository {...}
    

    View Slide

48. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepository(private val userDao: UserDao) : IUserRepository {
    /**
    * Executes query through the data access object
    */
    override fun getUser(userId: Long): User {
    return userDao.findById(userId = userId)
    }
    }
    

    View Slide

49. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepository(private val userDao: UserDao) : IUserRepository {
    /**
    * Executes query through the data access object
    */
    override fun getUser(userId: Long): User {
    return userDao.findById(userId = userId)
    }
    }
    Real UserRepository class executes DAO queries
    

    View Slide

50. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepositoryStub() : IUserRepository {
    ...
    /*
    * returns preconfigured user
    */
    override fun getUser(userId: Long): User {
    return User(userId = 1, email = "sf@sf.com")
    }
    }
    

    View Slide

51. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepositoryStub() : IUserRepository {
    ...
    /*
    * returns preconfigured user
    */
    override fun getUser(userId: Long): User {
    return User(userId = 1, email = "sf@sf.com")
    }
    }
    Stubbed version of UserRepository returns hardcoded
    value
    

    View Slide

52. Fakes
    

    View Slide

53. Fakes
    ● Similar to stubs, slightly more realistic
    

    View Slide

54. Fakes
    ● Similar to stubs, slightly more realistic
    ● Contain working implementation, but different from real
    version
    

    View Slide

55. Fakes
    ● Similar to stubs, slightly more realistic
    ● Contain working implementation, but different from real
    version
    ● Typically models the behavior of the real class
    

    View Slide

56. Fakes
    ● Similar to stubs, slightly more realistic
    ● Contain working implementation, but different from real
    version
    ● Typically models the behavior of the real class
    ● e.g in-memory-database
    

    View Slide

57. interface UserDao {...}
    Fakes - example
    

    View Slide

58. interface UserDao {...}
    Real UserDao class executes db queries
    Fakes - example
    

    View Slide

59. Fakes - example
    interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    

    View Slide

60. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fakes - example
    

    View Slide

61. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fake data
    source is a
    list instead of
    a database
    Fakes - example
    

    View Slide

62. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fakes - example
    

    View Slide

63. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fake Dao supports
    insert and find
    operations just like
    the real one
    Fakes - example
    

    View Slide

64. Mocks
    

    View Slide

65. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs
    

    View Slide

66. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs
    ● Ability to record method calls and verify them
    

    View Slide

67. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs
    ● Ability to record method calls and verify them
    ● Throw exceptions if wanted method is not called
    

    View Slide

68. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs
    ● Ability to record method calls and verify them
    ● Throw exceptions if wanted method is not called
    ● Different libraries depending on programming language
    

    View Slide

69. Mocks - example
    

    View Slide

70. Mocks - example
    class NotificationService(
    private val client: NotificationClient,
    private val groupRepository: IGroupRepository
    ) {
    fun post(notification: Notification, groupId: Long) {
    client.batchSend(notification, groupRepository.getMembers(groupId))
    }
    }
    

    View Slide

71. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }
    

    View Slide

72. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }
    Create a mock
    

    View Slide

73. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }
    Set
    expectations
    for a mock
    

    View Slide

74. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }
    Verify mock
    interactions
    

    View Slide

75. Spies
    

    View Slide

76. Spies
    ● Hybrid between stubs and mocks
    

    View Slide

77. Spies
    ● Hybrid between stubs and mocks
    ● They are as real as stubs, but also have the ability to
    record interactions like mocks.
    

    View Slide

78. Spies
    ● Hybrid between stubs and mocks
    ● They are as real as stubs, but also have the ability to
    record interactions like mocks.
    

    View Slide

79. Writing maintainable tests
    

    View Slide

80. Maintainable tests:
    

    View Slide

81. Maintainable tests:
    ● Are reliable
    

    View Slide

82. Maintainable tests:
    ● Are reliable
    ● Don’t change as often as the production code changes
    

    View Slide

83. Maintainable tests:
    ● Are reliable
    ● Don’t change as often as the production code changes
    ● Fail for only one reason
    

    View Slide

84. Maintainable tests:
    ● Are reliable
    ● Don’t change as often as the production code changes
    ● Fail for only one reason
    ● Easy to modify & new tests are easy to add
    

    View Slide

85. Use a good test specification system
    

    View Slide

86. Use a good test specification system
    i.e consistent test structure
    

    View Slide

87. Use a good test specification system
    

    View Slide

88. Use a good test specification system
    

    View Slide

89. Use a good test specification system
    

    View Slide

90. Also known as:
    Arrange - Act - Assert
    Or
    Given - When - Then
    Use a good test specification system
    

    View Slide

91. Test behaviour, not implementation
    details
    

    View Slide

92. Test behaviour, not implementation details
    Testing behavior means that:
    ● For non-state changing methods (that return value), you
    only care about the output, not how the output was
    calculated.
    

    View Slide

93. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    }
    

    View Slide

94. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com"))
    }
    Set up test
    preconditions
    

    View Slide

95. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    }
    Exercise the
    scenario under test
    

    View Slide

96. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    verify(cacheSource).getUser(5)
    }
    Verify that we get
    user from cache
    

    View Slide

97. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    verify(cacheSource).getUser(5)
    }
    This tests the implementation details
    Verify that we get
    user from cache
    

    View Slide

98. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    assertEquals(5, user.userId)
    assertEquals("email@email.com", user.email)
    - verify(cacheSource).getUser(5)
    }
    Check that the user
    matches the one
    returned from the
    cache
    

    View Slide

99. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    assertEquals(5, user.userId)
    assertEquals("email@email.com", user.email)
    }
    

    View Slide

100. @Test
     fun testGetUserFromCacheIfUserExistsInCache() {
     val networkSource = mock()
     val cacheSource = mock()
     whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
     whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com"))
     val userRepo = UserRepo(cacheSource, networkSource)
     val user = userRepo.getUser(userId = 5)
     assertEquals(5, user.userId)
     assertEquals("email@email.com", user.email)
     }
     This tests general behavior when cache exists
     

     View Slide

101. Test behaviour, not implementation details
     Testing behavior means that:
     ● For non-state changing methods (that return value), you
     only care about the output, not how the output was
     calculated.
     ● For state changing methods, verify that methods were
     called when using mocks
     

     View Slide

102. Test behaviour, not implementation details
     Testing behavior means that:
     ● For non-state changing methods (that return value), you
     only care about the output, not how the output was
     calculated.
     ● For state changing methods, verify that methods were
     called
     ● Use less of mocks, and more of stubs, fakes and other
     test doubles
     

     View Slide

103. Test behaviour, not implementation details
     Testing behavior means that:
     ● For non-state changing methods (that return value), you
     only care about the output, not how the output was
     calculated.
     ● For state changing methods, verify that methods were
     called
     ● Use less of mocks, and more of stubs, fakes and other
     test doubles
     

     View Slide

104. Avoid assertion roulette
     

     View Slide

105. @Test
     fun createNewUserAndSendWelcomeEmail() {
     // set up dependencies
     ...
     val userService = UserService(userRepository, emailClient)
     val newUser = userService.createUser("user", "user@user.com")
     assertEquals("user", newUser.username)
     assertEquals("user@user.com", newUser.email)
     assertEquals("", newUser.bio)
     assertEquals("", newUser.description)
     assertEquals(true, emailClient.sendWelcomeEmail())
     }
     Avoid assertion roulette
     

     View Slide

106. Avoid assertion roulette
     @Test
     fun createNewUserAndSendWelcomeEmail() {
     // set up dependencies
     ...
     val userService = UserService(userRepository, emailClient)
     val newUser = userService.createUser("user", "user@user.com")
     assertEquals("user", newUser.username)
     assertEquals("user@user.com", newUser.email)
     assertEquals("", newUser.bio)
     assertEquals("", newUser.description)
     assertEquals(true, emailClient.sendWelcomeEmail())
     }
     

     View Slide

107. @Test
     fun createNewUserAndSendWelcomeEmail() {
     // set up dependencies
     ...
     val userService = UserService(userRepository, emailClient)
     val newUser = userService.createUser("user", "user@user.com")
     assertEquals("user", newUser.username)
     assertEquals("user@user.com", newUser.email)
     assertEquals("", newUser.bio)
     assertEquals("", newUser.description)
     assertEquals(true, emailClient.sendWelcomeEmail())
     }
     If test fails, it’s unclear why
     Avoid assertion roulette
     

     View Slide

108. Fixing assertion roulette
     @Test
     fun createNewUserAndSendWelcomeEmail() {
     // set up dependencies
     ...
     val userService = UserService(userRepository, emailClient)
     val newUser = userService.createUser("user", "user@user.com")
     assertEquals("Created username doesn't match input username", "user", newUser.username)
     assertEquals("Created username doesn't match input username", "user@user.com", newUser.email)
     assertEquals("New user's bio is not empty", "", newUser.bio)
     assertEquals("New user's description is not empty", "", newUser.description)
     assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail())
     }
     Simple fix: Add descriptive
     message to assertions
     

     View Slide

109. Fixing assertion roulette
     @Test
     fun testCreateNewUser() {...}
     @Test
     fun testSendWelcomeEmailToNewUser() {
     // set up test
     ...
     assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail())
     }
     Split the assertions into new
     test cases
     

     View Slide

110. More tips for writing maintainable
     tests
     

     View Slide

111. More tips
     ● Use expressive test names
     

     View Slide

112. More tips
     ● Use expressive test names
     ● Avoid logic in your tests -> if/else, loops, etc.
     

     View Slide

113. More tips
     ● Use expressive test names
     ● Avoid logic in your tests -> if/else, loops, etc.
     ● Avoid abstractions in tests
     

     View Slide

114. More tips
     ● Use expressive test names
     ● Avoid logic in your tests -> if/else, loops, etc.
     ● Avoid abstractions in tests
     ● Be generous with comments
     

     View Slide

115. Recap
     

     View Slide

116. Recap
     ● Benefits of TDD
     

     View Slide

117. Recap
     ● Benefits of TDD
     ● Challenges with TDD
     

     View Slide

118. Recap
     ● Benefits of TDD
     ● Challenges with TDD
     ● Mocks, spies, fakes, dummies, stubs - test doubles
     

     View Slide

119. Recap
     ● Benefits of TDD
     ● Challenges with TDD
     ● Mocks, spies, fakes, dummies, stubs - test doubles
     ● Tips for writing maintainable tests
     

     View Slide

120. Resources & Further Reading
     ● https://martinfowler.com/articles/mocksArentStubs.html
     ● http://xunitpatterns.com/
     ● https://testing.googleblog.com/search/label/TotT
     ● https://mtlynch.io/good-developers-bad-tests/
     ● thedoodlelibrary.com (for icons)
     

     View Slide

121. Thank you!
     segunfamisa
     segunfamisa.com
     

     View Slide