Writing Tests that Stand the Test of Time

This talk was given at DACHFest (dachfest.com) in Munich, Germany.

Video: https://www.youtube.com/watch?v=Hy5ra9qDpEY

Description:
One of the promises of test driven development is the confidence and ease with which we can refactor our production code.

However, sometimes we find ourselves in a position where our tests are highly coupled to the production code and we have to change the tests every time we update our production code. At this point we start wondering if the tests we had are even worth it if we have to spend so much time to maintain them. If you have ever found yourself having to update your tests every time the behavior of a dependency of the class under test has changed, then this talk is for you.

This talk is to share with you some patterns, tools and examples that can guide us to write more maintainable tests.

Transcript

1. Writing Tests that Stand the Test of Time Segun Famisa @segunfamisa
2. segunfamisa segunfamisa.com
3. Outline
4. Outline Introduction to TDD
5. Outline Introduction to TDD Challenges with TDD
6. Outline Introduction to TDD Challenges with TDD Testing tools in practice
7. Outline Introduction to TDD Challenges with TDD Testing tools in practice Writing maintainable tests
8. Outline Introduction to TDD Challenges with TDD Testing tools in practice Writing maintainable tests Recap
9. Introduction
10. So, what’s Test Driven Development?
11. What is TDD? ”Test-Driven Development (TDD) is a technique for building software that guides software development by writing tests” https://martinfowler.com/bliki/TestDrivenDevelopment.html
12. What is TDD? ”TDD is a software development process that relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only” Wikipedia
14. How to do TDD?
15. How to do TDD? Red - Green - Refactor
16. Red - Green - Refactor Write failing test
17. Write failing test Write code, just enough to pass Red - Green - Refactor
18. Red - Green - Refactor Write failing test Write code, just enough to pass Clean up
19. Red - Green - Refactor Write failing test Write code, just enough to pass Clean up
20. How to do TDD? TL;DR Don’t write new code until you’ve written a test that fails. Don’t write more code than you need to make the test pass. Repeat.
21. Why do we need tests?
22. Why do we need tests? • Quick feedback about bugs/errors
23. Why do we need tests? • Quick feedback about bugs/errors • Good code design
24. Why do we need tests? • Quick feedback about bugs/errors • Good code design • Documentation for code behavior
25. Why do we need tests? • Quick feedback about bugs/errors • Good code design • Documentation for code behavior • Confident refactoring
26. Challenges with TDD
27. If TDD is so nice, why isn’t everyone doing it?
28. Time Challenges with TDD
29. Time Tests are code too, so they take time to write
30. Time Tests are code too, so they take time to write However, time spent writing tests is an investment in the long run
31. Maintenance overhead Challenges with TDD
32. Maintenance overhead Challenges with TDD TDD becomes challenging when: • Old tests break often
33. Maintenance overhead Challenges with TDD TDD becomes challenging when: • Old tests break often • Difficult to add new tests
34. TDD becomes challenging when: • Old tests break often • Difficult to add new tests • Tests are too tightly coupled to production code Maintenance overhead Challenges with TDD
35. Testing tools & concepts in Practice
36. Test doubles
37. Test doubles • Like stunt doubles in movies
38. Test doubles • Like stunt doubles in movies https://people.com/movies/actors-and-their-stunt-doubles-photos
39. Test doubles • Like stunt doubles in movies • Objects we can use in our tests in place of real objects
40. Test doubles • Like stunt doubles in movies • Objects we can use in our tests in place of real objects • e.g - dummies, stubs, fakes, mocks, spies
41. Dummies
42. Dummies Objects passed around, but not actually used. Often used to fill up constructor or method parameters
43. Stubs
44. Stubs • Objects that return predefined data
45. Stubs • Objects that return predefined data • They usually don’t hold state/respond to other actions besides the one they are created for
46. Stubs - example
47. Stubs - example interface IUserRepository {...}
48. Stubs - example interface IUserRepository {...} ... class UserRepository(private val userDao: UserDao) : IUserRepository { /** * Executes query through the data access object */ override fun getUser(userId: Long): User { return userDao.findById(userId = userId) } }
49. Stubs - example interface IUserRepository {...} ... class UserRepository(private val userDao: UserDao) : IUserRepository { /** * Executes query through the data access object */ override fun getUser(userId: Long): User { return userDao.findById(userId = userId) } } Real UserRepository class executes DAO queries
50. Stubs - example interface IUserRepository {...} ... class UserRepositoryStub() : IUserRepository { ... /* * returns preconfigured user */ override fun getUser(userId: Long): User { return User(userId = 1, email = "sf@sf.com") } }
51. Stubs - example interface IUserRepository {...} ... class UserRepositoryStub() : IUserRepository { ... /* * returns preconfigured user */ override fun getUser(userId: Long): User { return User(userId = 1, email = "sf@sf.com") } } Stubbed version of UserRepository returns hardcoded value
52. Fakes
53. Fakes • Similar to stubs, slightly more realistic
54. Fakes • Similar to stubs, slightly more realistic • Contain working implementation, but different from real version
55. Fakes • Similar to stubs, slightly more realistic • Contain working implementation, but different from real version • Typically models the behavior of the real class
56. Fakes • Similar to stubs, slightly more realistic • Contain working implementation, but different from real version • Typically models the behavior of the real class • e.g in-memory-database
57. interface UserDao {...} Fakes - example
58. interface UserDao {...} Real UserDao class executes db queries Fakes - example
59. Fakes - example interface UserDao {...} ... class FakeUserDao() : UserDao { private val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } }
60. interface UserDao {...} ... class FakeUserDao() : UserDao { private val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fakes - example
61. interface UserDao {...} ... class FakeUserDao() : UserDao { private val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fake data source is a list instead of a database Fakes - example
62. interface UserDao {...} ... class FakeUserDao() : UserDao { private val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fakes - example
63. interface UserDao {...} ... class FakeUserDao() : UserDao { private val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fake Dao supports insert and find operations just like the real one Fakes - example
64. Mocks
65. Mocks • Objects pre-programmed with expected outputs for given inputs
66. Mocks • Objects pre-programmed with expected outputs for given inputs • Ability to record method calls and verify them
67. Mocks • Objects pre-programmed with expected outputs for given inputs • Ability to record method calls and verify them • Throw exceptions if wanted method is not called
68. Mocks • Objects pre-programmed with expected outputs for given inputs • Ability to record method calls and verify them • Throw exceptions if wanted method is not called • Different libraries depending on programming language
69. Mocks - example
70. Mocks - example class NotificationService( private val client: NotificationClient, private val groupRepository: IGroupRepository ) { fun post(notification: Notification, groupId: Long) { client.batchSend(notification, groupRepository.getMembers(groupId)) } }
71. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() { val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) }
72. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() { val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) } Create a mock
73. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() { val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) } Set expectations for a mock
74. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() { val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) } Verify mock interactions
75. Spies
76. Spies • Hybrid between stubs and mocks
77. Spies • Hybrid between stubs and mocks • They are as real as stubs, but also have the ability to record interactions like mocks.
78. Spies • Hybrid between stubs and mocks • They are as real as stubs, but also have the ability to record interactions like mocks.
79. Writing maintainable tests
80. Maintainable tests:
81. Maintainable tests: • Are reliable
82. Maintainable tests: • Are reliable • Don’t change as often as the production code changes
83. Maintainable tests: • Are reliable • Don’t change as often as the production code changes • Fail for only one reason
84. Maintainable tests: • Are reliable • Don’t change as often as the production code changes • Fail for only one reason • Easy to modify & new tests are easy to add
85. Use a good test specification system
86. Use a good test specification system i.e consistent test structure
87. Use a good test specification system
88. Use a good test specification system
89. Use a good test specification system
90. Also known as: Arrange - Act - Assert Or Given - When - Then Use a good test specification system
91. Test behaviour, not implementation details
92. Test behaviour, not implementation details Testing behavior means that: • For non-state changing methods (that return value), you only care about the output, not how the output was calculated.
93. @Test fun testGetUserFromCacheIfUserExistsInCache() { }
94. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) } Set up test preconditions
95. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) } Exercise the scenario under test
96. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) verify(cacheSource).getUser(5) } Verify that we get user from cache
97. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) verify(cacheSource).getUser(5) } This tests the implementation details Verify that we get user from cache
98. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) assertEquals(5, user.userId) assertEquals("email@email.com", user.email) - verify(cacheSource).getUser(5) } Check that the user matches the one returned from the cache
99. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) assertEquals(5, user.userId) assertEquals("email@email.com", user.email) }
100. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) assertEquals(5, user.userId) assertEquals("email@email.com", user.email) } This tests general behavior when cache exists
101. Test behaviour, not implementation details Testing behavior means that: • For non-state changing methods (that return value), you only care about the output, not how the output was calculated. • For state changing methods, verify that methods were called when using mocks
102. Test behaviour, not implementation details Testing behavior means that: • For non-state changing methods (that return value), you only care about the output, not how the output was calculated. • For state changing methods, verify that methods were called • Use less of mocks, and more of stubs, fakes and other test doubles
103. Test behaviour, not implementation details Testing behavior means that: • For non-state changing methods (that return value), you only care about the output, not how the output was calculated. • For state changing methods, verify that methods were called • Use less of mocks, and more of stubs, fakes and other test doubles
104. Avoid assertion roulette
105. @Test fun createNewUserAndSendWelcomeEmail() { // set up dependencies ... val userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("user", newUser.username) assertEquals("user@user.com", newUser.email) assertEquals("", newUser.bio) assertEquals("", newUser.description) assertEquals(true, emailClient.sendWelcomeEmail()) } Avoid assertion roulette
106. Avoid assertion roulette @Test fun createNewUserAndSendWelcomeEmail() { // set up dependencies ... val userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("user", newUser.username) assertEquals("user@user.com", newUser.email) assertEquals("", newUser.bio) assertEquals("", newUser.description) assertEquals(true, emailClient.sendWelcomeEmail()) }
107. @Test fun createNewUserAndSendWelcomeEmail() { // set up dependencies ... val userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("user", newUser.username) assertEquals("user@user.com", newUser.email) assertEquals("", newUser.bio) assertEquals("", newUser.description) assertEquals(true, emailClient.sendWelcomeEmail()) } If test fails, it’s unclear why Avoid assertion roulette
108. Fixing assertion roulette @Test fun createNewUserAndSendWelcomeEmail() { // set up dependencies ... val userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("Created username doesn't match input username", "user", newUser.username) assertEquals("Created username doesn't match input username", "user@user.com", newUser.email) assertEquals("New user's bio is not empty", "", newUser.bio) assertEquals("New user's description is not empty", "", newUser.description) assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail()) } Simple fix: Add descriptive message to assertions
109. Fixing assertion roulette @Test fun testCreateNewUser() {...} @Test fun testSendWelcomeEmailToNewUser() { // set up test ... assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail()) } Split the assertions into new test cases
110. More tips for writing maintainable tests
111. More tips • Use expressive test names
112. More tips • Use expressive test names • Avoid logic in your tests -> if/else, loops, etc.
113. More tips • Use expressive test names • Avoid logic in your tests -> if/else, loops, etc. • Avoid abstractions in tests
114. More tips • Use expressive test names • Avoid logic in your tests -> if/else, loops, etc. • Avoid abstractions in tests • Be generous with comments
115. Recap
116. Recap • Benefits of TDD
117. Recap • Benefits of TDD • Challenges with TDD
118. Recap • Benefits of TDD • Challenges with TDD • Mocks, spies, fakes, dummies, stubs - test doubles
119. Recap • Benefits of TDD • Challenges with TDD • Mocks, spies, fakes, dummies, stubs - test doubles • Tips for writing maintainable tests
120. Resources & Further Reading • https://martinfowler.com/articles/mocksArentStubs.html • http://xunitpatterns.com/ • https://testing.googleblog.com/search/label/TotT • https://mtlynch.io/good-developers-bad-tests/ • thedoodlelibrary.com (for icons)
121. Thank you! segunfamisa segunfamisa.com