Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing Tests that Stand the Test of Time

Writing Tests that Stand the Test of Time

This talk was given at DACHFest (dachfest.com) in Munich, Germany.

Video: https://www.youtube.com/watch?v=Hy5ra9qDpEY

Description:
One of the promises of test driven development is the confidence and ease with which we can refactor our production code.

However, sometimes we find ourselves in a position where our tests are highly coupled to the production code and we have to change the tests every time we update our production code. At this point we start wondering if the tests we had are even worth it if we have to spend so much time to maintain them. If you have ever found yourself having to update your tests every time the behavior of a dependency of the class under test has changed, then this talk is for you.

This talk is to share with you some patterns, tools and examples that can guide us to write more maintainable tests.

9ab0b3b080e75e0c03a0c643333f8b93?s=128

Segun Famisa

November 11, 2018
Tweet

Transcript

  1. Writing Tests that Stand the Test of Time Segun Famisa

    @segunfamisa
  2. segunfamisa segunfamisa.com

  3. Outline

  4. Outline Introduction to TDD

  5. Outline Introduction to TDD Challenges with TDD

  6. Outline Introduction to TDD Challenges with TDD Testing tools in

    practice
  7. Outline Introduction to TDD Challenges with TDD Testing tools in

    practice Writing maintainable tests
  8. Outline Introduction to TDD Challenges with TDD Testing tools in

    practice Writing maintainable tests Recap
  9. Introduction

  10. So, what’s Test Driven Development?

  11. What is TDD? ”Test-Driven Development (TDD) is a technique for

    building software that guides software development by writing tests” https://martinfowler.com/bliki/TestDrivenDevelopment.html
  12. What is TDD? ”TDD is a software development process that

    relies on the repetition of a very short development cycle: requirements are turned into very specific test cases, then the software is improved to pass the new tests, only” Wikipedia
  13. None
  14. How to do TDD?

  15. How to do TDD? Red - Green - Refactor

  16. Red - Green - Refactor Write failing test

  17. Write failing test Write code, just enough to pass Red

    - Green - Refactor
  18. Red - Green - Refactor Write failing test Write code,

    just enough to pass Clean up
  19. Red - Green - Refactor Write failing test Write code,

    just enough to pass Clean up
  20. How to do TDD? TL;DR Don’t write new code until

    you’ve written a test that fails. Don’t write more code than you need to make the test pass. Repeat.
  21. Why do we need tests?

  22. Why do we need tests? • Quick feedback about bugs/errors

  23. Why do we need tests? • Quick feedback about bugs/errors

    • Good code design
  24. Why do we need tests? • Quick feedback about bugs/errors

    • Good code design • Documentation for code behavior
  25. Why do we need tests? • Quick feedback about bugs/errors

    • Good code design • Documentation for code behavior • Confident refactoring
  26. Challenges with TDD

  27. If TDD is so nice, why isn’t everyone doing it?

  28. Time Challenges with TDD

  29. Time Tests are code too, so they take time to

    write
  30. Time Tests are code too, so they take time to

    write However, time spent writing tests is an investment in the long run
  31. Maintenance overhead Challenges with TDD

  32. Maintenance overhead Challenges with TDD TDD becomes challenging when: •

    Old tests break often
  33. Maintenance overhead Challenges with TDD TDD becomes challenging when: •

    Old tests break often • Difficult to add new tests
  34. TDD becomes challenging when: • Old tests break often •

    Difficult to add new tests • Tests are too tightly coupled to production code Maintenance overhead Challenges with TDD
  35. Testing tools & concepts in Practice

  36. Test doubles

  37. Test doubles • Like stunt doubles in movies

  38. Test doubles • Like stunt doubles in movies https://people.com/movies/actors-and-their-stunt-doubles-photos

  39. Test doubles • Like stunt doubles in movies • Objects

    we can use in our tests in place of real objects
  40. Test doubles • Like stunt doubles in movies • Objects

    we can use in our tests in place of real objects • e.g - dummies, stubs, fakes, mocks, spies
  41. Dummies

  42. Dummies Objects passed around, but not actually used. Often used

    to fill up constructor or method parameters
  43. Stubs

  44. Stubs • Objects that return predefined data

  45. Stubs • Objects that return predefined data • They usually

    don’t hold state/respond to other actions besides the one they are created for
  46. Stubs - example

  47. Stubs - example interface IUserRepository {...}

  48. Stubs - example interface IUserRepository {...} ... class UserRepository(private val

    userDao: UserDao) : IUserRepository { /** * Executes query through the data access object */ override fun getUser(userId: Long): User { return userDao.findById(userId = userId) } }
  49. Stubs - example interface IUserRepository {...} ... class UserRepository(private val

    userDao: UserDao) : IUserRepository { /** * Executes query through the data access object */ override fun getUser(userId: Long): User { return userDao.findById(userId = userId) } } Real UserRepository class executes DAO queries
  50. Stubs - example interface IUserRepository {...} ... class UserRepositoryStub() :

    IUserRepository { ... /* * returns preconfigured user */ override fun getUser(userId: Long): User { return User(userId = 1, email = "sf@sf.com") } }
  51. Stubs - example interface IUserRepository {...} ... class UserRepositoryStub() :

    IUserRepository { ... /* * returns preconfigured user */ override fun getUser(userId: Long): User { return User(userId = 1, email = "sf@sf.com") } } Stubbed version of UserRepository returns hardcoded value
  52. Fakes

  53. Fakes • Similar to stubs, slightly more realistic

  54. Fakes • Similar to stubs, slightly more realistic • Contain

    working implementation, but different from real version
  55. Fakes • Similar to stubs, slightly more realistic • Contain

    working implementation, but different from real version • Typically models the behavior of the real class
  56. Fakes • Similar to stubs, slightly more realistic • Contain

    working implementation, but different from real version • Typically models the behavior of the real class • e.g in-memory-database
  57. interface UserDao {...} Fakes - example

  58. interface UserDao {...} Real UserDao class executes db queries Fakes

    - example
  59. Fakes - example interface UserDao {...} ... class FakeUserDao() :

    UserDao { private val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } }
  60. interface UserDao {...} ... class FakeUserDao() : UserDao { private

    val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fakes - example
  61. interface UserDao {...} ... class FakeUserDao() : UserDao { private

    val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fake data source is a list instead of a database Fakes - example
  62. interface UserDao {...} ... class FakeUserDao() : UserDao { private

    val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fakes - example
  63. interface UserDao {...} ... class FakeUserDao() : UserDao { private

    val users = mutableListOf<User>() override fun insert(user: User) { users.add(user) } override fun findById(userId: Long): User { return users.find { it.userId == userId } ?: throw Exception("user not found") } } Fake Dao supports insert and find operations just like the real one Fakes - example
  64. Mocks

  65. Mocks • Objects pre-programmed with expected outputs for given inputs

  66. Mocks • Objects pre-programmed with expected outputs for given inputs

    • Ability to record method calls and verify them
  67. Mocks • Objects pre-programmed with expected outputs for given inputs

    • Ability to record method calls and verify them • Throw exceptions if wanted method is not called
  68. Mocks • Objects pre-programmed with expected outputs for given inputs

    • Ability to record method calls and verify them • Throw exceptions if wanted method is not called • Different libraries depending on programming language
  69. Mocks - example

  70. Mocks - example class NotificationService( private val client: NotificationClient, private

    val groupRepository: IGroupRepository ) { fun post(notification: Notification, groupId: Long) { client.batchSend(notification, groupRepository.getMembers(groupId)) } }
  71. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() {

    val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) }
  72. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() {

    val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) } Create a mock
  73. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() {

    val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) } Set expectations for a mock
  74. Mocks - example class NotificationService(...) {...} @Test fun sendBatchNotifications() {

    val client = mock<NotificationClient>() val groupRepo = mock<IGroupRepository>() whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users) val service = NotificationService(client, groupRepo) service.post(notification, 5) verify(client).batchSend(notification, users) } Verify mock interactions
  75. Spies

  76. Spies • Hybrid between stubs and mocks

  77. Spies • Hybrid between stubs and mocks • They are

    as real as stubs, but also have the ability to record interactions like mocks.
  78. Spies • Hybrid between stubs and mocks • They are

    as real as stubs, but also have the ability to record interactions like mocks.
  79. Writing maintainable tests

  80. Maintainable tests:

  81. Maintainable tests: • Are reliable

  82. Maintainable tests: • Are reliable • Don’t change as often

    as the production code changes
  83. Maintainable tests: • Are reliable • Don’t change as often

    as the production code changes • Fail for only one reason
  84. Maintainable tests: • Are reliable • Don’t change as often

    as the production code changes • Fail for only one reason • Easy to modify & new tests are easy to add
  85. Use a good test specification system

  86. Use a good test specification system i.e consistent test structure

  87. Use a good test specification system

  88. Use a good test specification system

  89. Use a good test specification system

  90. Also known as: Arrange - Act - Assert Or Given

    - When - Then Use a good test specification system
  91. Test behaviour, not implementation details

  92. Test behaviour, not implementation details Testing behavior means that: •

    For non-state changing methods (that return value), you only care about the output, not how the output was calculated.
  93. @Test fun testGetUserFromCacheIfUserExistsInCache() { }

  94. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource

    = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) } Set up test preconditions
  95. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource

    = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) } Exercise the scenario under test
  96. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource

    = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) verify(cacheSource).getUser(5) } Verify that we get user from cache
  97. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource

    = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) verify(cacheSource).getUser(5) } This tests the implementation details Verify that we get user from cache
  98. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource

    = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) assertEquals(5, user.userId) assertEquals("email@email.com", user.email) - verify(cacheSource).getUser(5) } Check that the user matches the one returned from the cache
  99. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource

    = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) assertEquals(5, user.userId) assertEquals("email@email.com", user.email) }
  100. @Test fun testGetUserFromCacheIfUserExistsInCache() { val networkSource = mock<UserNetworkSource>() val cacheSource

    = mock<UserCacheSource>() whenever(cacheSource.userExistsInCache(5)).thenReturn(true) whenever(cacheSource.getUser(5)).thenReturn(User(5, "email@email.com")) val userRepo = UserRepo(cacheSource, networkSource) val user = userRepo.getUser(userId = 5) assertEquals(5, user.userId) assertEquals("email@email.com", user.email) } This tests general behavior when cache exists
  101. Test behaviour, not implementation details Testing behavior means that: •

    For non-state changing methods (that return value), you only care about the output, not how the output was calculated. • For state changing methods, verify that methods were called when using mocks
  102. Test behaviour, not implementation details Testing behavior means that: •

    For non-state changing methods (that return value), you only care about the output, not how the output was calculated. • For state changing methods, verify that methods were called • Use less of mocks, and more of stubs, fakes and other test doubles
  103. Test behaviour, not implementation details Testing behavior means that: •

    For non-state changing methods (that return value), you only care about the output, not how the output was calculated. • For state changing methods, verify that methods were called • Use less of mocks, and more of stubs, fakes and other test doubles
  104. Avoid assertion roulette

  105. @Test fun createNewUserAndSendWelcomeEmail() { // set up dependencies ... val

    userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("user", newUser.username) assertEquals("user@user.com", newUser.email) assertEquals("", newUser.bio) assertEquals("", newUser.description) assertEquals(true, emailClient.sendWelcomeEmail()) } Avoid assertion roulette
  106. Avoid assertion roulette @Test fun createNewUserAndSendWelcomeEmail() { // set up

    dependencies ... val userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("user", newUser.username) assertEquals("user@user.com", newUser.email) assertEquals("", newUser.bio) assertEquals("", newUser.description) assertEquals(true, emailClient.sendWelcomeEmail()) }
  107. @Test fun createNewUserAndSendWelcomeEmail() { // set up dependencies ... val

    userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("user", newUser.username) assertEquals("user@user.com", newUser.email) assertEquals("", newUser.bio) assertEquals("", newUser.description) assertEquals(true, emailClient.sendWelcomeEmail()) } If test fails, it’s unclear why Avoid assertion roulette
  108. Fixing assertion roulette @Test fun createNewUserAndSendWelcomeEmail() { // set up

    dependencies ... val userService = UserService(userRepository, emailClient) val newUser = userService.createUser("user", "user@user.com") assertEquals("Created username doesn't match input username", "user", newUser.username) assertEquals("Created username doesn't match input username", "user@user.com", newUser.email) assertEquals("New user's bio is not empty", "", newUser.bio) assertEquals("New user's description is not empty", "", newUser.description) assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail()) } Simple fix: Add descriptive message to assertions
  109. Fixing assertion roulette @Test fun testCreateNewUser() {...} @Test fun testSendWelcomeEmailToNewUser()

    { // set up test ... assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail()) } Split the assertions into new test cases
  110. More tips for writing maintainable tests

  111. More tips • Use expressive test names

  112. More tips • Use expressive test names • Avoid logic

    in your tests -> if/else, loops, etc.
  113. More tips • Use expressive test names • Avoid logic

    in your tests -> if/else, loops, etc. • Avoid abstractions in tests
  114. More tips • Use expressive test names • Avoid logic

    in your tests -> if/else, loops, etc. • Avoid abstractions in tests • Be generous with comments
  115. Recap

  116. Recap • Benefits of TDD

  117. Recap • Benefits of TDD • Challenges with TDD

  118. Recap • Benefits of TDD • Challenges with TDD •

    Mocks, spies, fakes, dummies, stubs - test doubles
  119. Recap • Benefits of TDD • Challenges with TDD •

    Mocks, spies, fakes, dummies, stubs - test doubles • Tips for writing maintainable tests
  120. Resources & Further Reading • https://martinfowler.com/articles/mocksArentStubs.html • http://xunitpatterns.com/ • https://testing.googleblog.com/search/label/TotT

    • https://mtlynch.io/good-developers-bad-tests/ • thedoodlelibrary.com (for icons)
  121. Thank you! segunfamisa segunfamisa.com