Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing Tests that Stand the Test of Time

Writing Tests that Stand the Test of Time

This talk was given at DACHFest (dachfest.com) in Munich, Germany.

Video: https://www.youtube.com/watch?v=Hy5ra9qDpEY

Description:
One of the promises of test driven development is the confidence and ease with which we can refactor our production code.

However, sometimes we find ourselves in a position where our tests are highly coupled to the production code and we have to change the tests every time we update our production code. At this point we start wondering if the tests we had are even worth it if we have to spend so much time to maintain them. If you have ever found yourself having to update your tests every time the behavior of a dependency of the class under test has changed, then this talk is for you.

This talk is to share with you some patterns, tools and examples that can guide us to write more maintainable tests.

Segun Famisa

November 11, 2018
Tweet

More Decks by Segun Famisa

Other Decks in Programming

Transcript

  1. Writing Tests that Stand
    the Test of Time
    Segun Famisa
    @segunfamisa

    View Slide

  2. segunfamisa
    segunfamisa.com

    View Slide

  3. Outline

    View Slide

  4. Outline
    Introduction to TDD

    View Slide

  5. Outline
    Introduction to TDD
    Challenges with TDD

    View Slide

  6. Outline
    Introduction to TDD
    Challenges with TDD
    Testing tools in practice

    View Slide

  7. Outline
    Introduction to TDD
    Challenges with TDD
    Testing tools in practice
    Writing maintainable tests

    View Slide

  8. Outline
    Introduction to TDD
    Challenges with TDD
    Testing tools in practice
    Writing maintainable tests
    Recap

    View Slide

  9. Introduction

    View Slide

  10. So, what’s Test Driven
    Development?

    View Slide

  11. What is TDD?
    ”Test-Driven Development (TDD) is a
    technique for building software that guides
    software development by writing tests”
    https://martinfowler.com/bliki/TestDrivenDevelopment.html

    View Slide

  12. What is TDD?
    ”TDD is a software development process that
    relies on the repetition of a very short
    development cycle: requirements are turned
    into very specific test cases, then the
    software is improved to pass the new tests,
    only”
    Wikipedia

    View Slide

  13. View Slide

  14. How to do TDD?

    View Slide

  15. How to do TDD?
    Red - Green - Refactor

    View Slide

  16. Red - Green - Refactor
    Write
    failing
    test

    View Slide

  17. Write
    failing
    test
    Write
    code, just
    enough to
    pass
    Red - Green - Refactor

    View Slide

  18. Red - Green - Refactor
    Write
    failing
    test
    Write
    code, just
    enough to
    pass
    Clean up

    View Slide

  19. Red - Green - Refactor
    Write
    failing
    test
    Write
    code, just
    enough to
    pass
    Clean up

    View Slide

  20. How to do TDD?
    TL;DR
    Don’t write new code until you’ve written a test that fails.
    Don’t write more code than you need to make the test pass.
    Repeat.

    View Slide

  21. Why do we need tests?

    View Slide

  22. Why do we need tests?
    ● Quick feedback about bugs/errors

    View Slide

  23. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design

    View Slide

  24. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    ● Documentation for code behavior

    View Slide

  25. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    ● Documentation for code behavior
    ● Confident refactoring

    View Slide

  26. Challenges with TDD

    View Slide

  27. If TDD is so nice, why isn’t
    everyone doing it?

    View Slide

  28. Time
    Challenges with TDD

    View Slide

  29. Time
    Tests are code too, so they take
    time to write

    View Slide

  30. Time
    Tests are code too, so they take
    time to write
    However, time spent writing tests
    is an investment in the long run

    View Slide

  31. Maintenance
    overhead
    Challenges with TDD

    View Slide

  32. Maintenance
    overhead
    Challenges with TDD
    TDD becomes challenging when:
    ● Old tests break often

    View Slide

  33. Maintenance
    overhead
    Challenges with TDD
    TDD becomes challenging when:
    ● Old tests break often
    ● Difficult to add new tests

    View Slide

  34. TDD becomes challenging when:
    ● Old tests break often
    ● Difficult to add new tests
    ● Tests are too tightly coupled to
    production code
    Maintenance
    overhead
    Challenges with TDD

    View Slide

  35. Testing tools & concepts in
    Practice

    View Slide

  36. Test doubles

    View Slide

  37. Test doubles
    ● Like stunt doubles in movies

    View Slide

  38. Test doubles
    ● Like stunt doubles in movies
    https://people.com/movies/actors-and-their-stunt-doubles-photos

    View Slide

  39. Test doubles
    ● Like stunt doubles in movies
    ● Objects we can use in our tests in place of real objects

    View Slide

  40. Test doubles
    ● Like stunt doubles in movies
    ● Objects we can use in our tests in place of real objects
    ● e.g - dummies, stubs, fakes, mocks, spies

    View Slide

  41. Dummies

    View Slide

  42. Dummies
    Objects passed around, but not actually used. Often used to
    fill up constructor or method parameters

    View Slide

  43. Stubs

    View Slide

  44. Stubs
    ● Objects that return predefined data

    View Slide

  45. Stubs
    ● Objects that return predefined data
    ● They usually don’t hold state/respond to other actions
    besides the one they are created for

    View Slide

  46. Stubs - example

    View Slide

  47. Stubs - example
    interface IUserRepository {...}

    View Slide

  48. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepository(private val userDao: UserDao) : IUserRepository {
    /**
    * Executes query through the data access object
    */
    override fun getUser(userId: Long): User {
    return userDao.findById(userId = userId)
    }
    }

    View Slide

  49. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepository(private val userDao: UserDao) : IUserRepository {
    /**
    * Executes query through the data access object
    */
    override fun getUser(userId: Long): User {
    return userDao.findById(userId = userId)
    }
    }
    Real UserRepository class executes DAO queries

    View Slide

  50. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepositoryStub() : IUserRepository {
    ...
    /*
    * returns preconfigured user
    */
    override fun getUser(userId: Long): User {
    return User(userId = 1, email = "[email protected]")
    }
    }

    View Slide

  51. Stubs - example
    interface IUserRepository {...}
    ...
    class UserRepositoryStub() : IUserRepository {
    ...
    /*
    * returns preconfigured user
    */
    override fun getUser(userId: Long): User {
    return User(userId = 1, email = "[email protected]")
    }
    }
    Stubbed version of UserRepository returns hardcoded
    value

    View Slide

  52. Fakes

    View Slide

  53. Fakes
    ● Similar to stubs, slightly more realistic

    View Slide

  54. Fakes
    ● Similar to stubs, slightly more realistic
    ● Contain working implementation, but different from real
    version

    View Slide

  55. Fakes
    ● Similar to stubs, slightly more realistic
    ● Contain working implementation, but different from real
    version
    ● Typically models the behavior of the real class

    View Slide

  56. Fakes
    ● Similar to stubs, slightly more realistic
    ● Contain working implementation, but different from real
    version
    ● Typically models the behavior of the real class
    ● e.g in-memory-database

    View Slide

  57. interface UserDao {...}
    Fakes - example

    View Slide

  58. interface UserDao {...}
    Real UserDao class executes db queries
    Fakes - example

    View Slide

  59. Fakes - example
    interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }

    View Slide

  60. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fakes - example

    View Slide

  61. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fake data
    source is a
    list instead of
    a database
    Fakes - example

    View Slide

  62. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fakes - example

    View Slide

  63. interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    private val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fake Dao supports
    insert and find
    operations just like
    the real one
    Fakes - example

    View Slide

  64. Mocks

    View Slide

  65. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs

    View Slide

  66. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs
    ● Ability to record method calls and verify them

    View Slide

  67. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs
    ● Ability to record method calls and verify them
    ● Throw exceptions if wanted method is not called

    View Slide

  68. Mocks
    ● Objects pre-programmed with expected outputs for given
    inputs
    ● Ability to record method calls and verify them
    ● Throw exceptions if wanted method is not called
    ● Different libraries depending on programming language

    View Slide

  69. Mocks - example

    View Slide

  70. Mocks - example
    class NotificationService(
    private val client: NotificationClient,
    private val groupRepository: IGroupRepository
    ) {
    fun post(notification: Notification, groupId: Long) {
    client.batchSend(notification, groupRepository.getMembers(groupId))
    }
    }

    View Slide

  71. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }

    View Slide

  72. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }
    Create a mock

    View Slide

  73. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }
    Set
    expectations
    for a mock

    View Slide

  74. Mocks - example
    class NotificationService(...) {...}
    @Test
    fun sendBatchNotifications() {
    val client = mock()
    val groupRepo = mock()
    whenever(groupRepo.getMembers(groupId = 5)).thenReturn(users)
    val service = NotificationService(client, groupRepo)
    service.post(notification, 5)
    verify(client).batchSend(notification, users)
    }
    Verify mock
    interactions

    View Slide

  75. Spies

    View Slide

  76. Spies
    ● Hybrid between stubs and mocks

    View Slide

  77. Spies
    ● Hybrid between stubs and mocks
    ● They are as real as stubs, but also have the ability to
    record interactions like mocks.

    View Slide

  78. Spies
    ● Hybrid between stubs and mocks
    ● They are as real as stubs, but also have the ability to
    record interactions like mocks.

    View Slide

  79. Writing maintainable tests

    View Slide

  80. Maintainable tests:

    View Slide

  81. Maintainable tests:
    ● Are reliable

    View Slide

  82. Maintainable tests:
    ● Are reliable
    ● Don’t change as often as the production code changes

    View Slide

  83. Maintainable tests:
    ● Are reliable
    ● Don’t change as often as the production code changes
    ● Fail for only one reason

    View Slide

  84. Maintainable tests:
    ● Are reliable
    ● Don’t change as often as the production code changes
    ● Fail for only one reason
    ● Easy to modify & new tests are easy to add

    View Slide

  85. Use a good test specification system

    View Slide

  86. Use a good test specification system
    i.e consistent test structure

    View Slide

  87. Use a good test specification system

    View Slide

  88. Use a good test specification system

    View Slide

  89. Use a good test specification system

    View Slide

  90. Also known as:
    Arrange - Act - Assert
    Or
    Given - When - Then
    Use a good test specification system

    View Slide

  91. Test behaviour, not implementation
    details

    View Slide

  92. Test behaviour, not implementation details
    Testing behavior means that:
    ● For non-state changing methods (that return value), you
    only care about the output, not how the output was
    calculated.

    View Slide

  93. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    }

    View Slide

  94. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "[email protected]"))
    }
    Set up test
    preconditions

    View Slide

  95. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "[email protected]"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    }
    Exercise the
    scenario under test

    View Slide

  96. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "[email protected]"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    verify(cacheSource).getUser(5)
    }
    Verify that we get
    user from cache

    View Slide

  97. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "[email protected]"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    verify(cacheSource).getUser(5)
    }
    This tests the implementation details
    Verify that we get
    user from cache

    View Slide

  98. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "[email protected]"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    assertEquals(5, user.userId)
    assertEquals("[email protected]", user.email)
    - verify(cacheSource).getUser(5)
    }
    Check that the user
    matches the one
    returned from the
    cache

    View Slide

  99. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "[email protected]"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    assertEquals(5, user.userId)
    assertEquals("[email protected]", user.email)
    }

    View Slide

  100. @Test
    fun testGetUserFromCacheIfUserExistsInCache() {
    val networkSource = mock()
    val cacheSource = mock()
    whenever(cacheSource.userExistsInCache(5)).thenReturn(true)
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "[email protected]"))
    val userRepo = UserRepo(cacheSource, networkSource)
    val user = userRepo.getUser(userId = 5)
    assertEquals(5, user.userId)
    assertEquals("[email protected]", user.email)
    }
    This tests general behavior when cache exists

    View Slide

  101. Test behaviour, not implementation details
    Testing behavior means that:
    ● For non-state changing methods (that return value), you
    only care about the output, not how the output was
    calculated.
    ● For state changing methods, verify that methods were
    called when using mocks

    View Slide

  102. Test behaviour, not implementation details
    Testing behavior means that:
    ● For non-state changing methods (that return value), you
    only care about the output, not how the output was
    calculated.
    ● For state changing methods, verify that methods were
    called
    ● Use less of mocks, and more of stubs, fakes and other
    test doubles

    View Slide

  103. Test behaviour, not implementation details
    Testing behavior means that:
    ● For non-state changing methods (that return value), you
    only care about the output, not how the output was
    calculated.
    ● For state changing methods, verify that methods were
    called
    ● Use less of mocks, and more of stubs, fakes and other
    test doubles

    View Slide

  104. Avoid assertion roulette

    View Slide

  105. @Test
    fun createNewUserAndSendWelcomeEmail() {
    // set up dependencies
    ...
    val userService = UserService(userRepository, emailClient)
    val newUser = userService.createUser("user", "[email protected]")
    assertEquals("user", newUser.username)
    assertEquals("[email protected]", newUser.email)
    assertEquals("", newUser.bio)
    assertEquals("", newUser.description)
    assertEquals(true, emailClient.sendWelcomeEmail())
    }
    Avoid assertion roulette

    View Slide

  106. Avoid assertion roulette
    @Test
    fun createNewUserAndSendWelcomeEmail() {
    // set up dependencies
    ...
    val userService = UserService(userRepository, emailClient)
    val newUser = userService.createUser("user", "[email protected]")
    assertEquals("user", newUser.username)
    assertEquals("[email protected]", newUser.email)
    assertEquals("", newUser.bio)
    assertEquals("", newUser.description)
    assertEquals(true, emailClient.sendWelcomeEmail())
    }

    View Slide

  107. @Test
    fun createNewUserAndSendWelcomeEmail() {
    // set up dependencies
    ...
    val userService = UserService(userRepository, emailClient)
    val newUser = userService.createUser("user", "[email protected]")
    assertEquals("user", newUser.username)
    assertEquals("[email protected]", newUser.email)
    assertEquals("", newUser.bio)
    assertEquals("", newUser.description)
    assertEquals(true, emailClient.sendWelcomeEmail())
    }
    If test fails, it’s unclear why
    Avoid assertion roulette

    View Slide

  108. Fixing assertion roulette
    @Test
    fun createNewUserAndSendWelcomeEmail() {
    // set up dependencies
    ...
    val userService = UserService(userRepository, emailClient)
    val newUser = userService.createUser("user", "[email protected]")
    assertEquals("Created username doesn't match input username", "user", newUser.username)
    assertEquals("Created username doesn't match input username", "[email protected]", newUser.email)
    assertEquals("New user's bio is not empty", "", newUser.bio)
    assertEquals("New user's description is not empty", "", newUser.description)
    assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail())
    }
    Simple fix: Add descriptive
    message to assertions

    View Slide

  109. Fixing assertion roulette
    @Test
    fun testCreateNewUser() {...}
    @Test
    fun testSendWelcomeEmailToNewUser() {
    // set up test
    ...
    assertEquals("Did not send welcome email to new user",true, emailClient.sendWelcomeEmail())
    }
    Split the assertions into new
    test cases

    View Slide

  110. More tips for writing maintainable
    tests

    View Slide

  111. More tips
    ● Use expressive test names

    View Slide

  112. More tips
    ● Use expressive test names
    ● Avoid logic in your tests -> if/else, loops, etc.

    View Slide

  113. More tips
    ● Use expressive test names
    ● Avoid logic in your tests -> if/else, loops, etc.
    ● Avoid abstractions in tests

    View Slide

  114. More tips
    ● Use expressive test names
    ● Avoid logic in your tests -> if/else, loops, etc.
    ● Avoid abstractions in tests
    ● Be generous with comments

    View Slide

  115. Recap

    View Slide

  116. Recap
    ● Benefits of TDD

    View Slide

  117. Recap
    ● Benefits of TDD
    ● Challenges with TDD

    View Slide

  118. Recap
    ● Benefits of TDD
    ● Challenges with TDD
    ● Mocks, spies, fakes, dummies, stubs - test doubles

    View Slide

  119. Recap
    ● Benefits of TDD
    ● Challenges with TDD
    ● Mocks, spies, fakes, dummies, stubs - test doubles
    ● Tips for writing maintainable tests

    View Slide

  120. Resources & Further Reading
    ● https://martinfowler.com/articles/mocksArentStubs.html
    ● http://xunitpatterns.com/
    ● https://testing.googleblog.com/search/label/TotT
    ● https://mtlynch.io/good-developers-bad-tests/
    ● thedoodlelibrary.com (for icons)

    View Slide

  121. Thank you!
    segunfamisa
    segunfamisa.com

    View Slide