Writing Tests that Stand the Test of Time - droidcon Boston

This talk was given at droidcon Boston 2019 (https://www.droidcon-boston.com/) in Boston, MA, United States.

Video: Not available yet.

Description:

One of the promises of test-driven development is the confidence and ease with which we can refactor our production code.

However, sometimes we find ourselves in situations where our tests are too coupled to our code and we have to change the tests every time we update the code. At this point, we start wondering if the tests are worth it.

If you have ever found yourself having to update your tests every time the behavior of a dependency of a class changes, then this talk is for you.

This talk is to share some patterns, tools, and examples that can guide us to write more maintainable tests.

We will look at why we need tests in the first place, how writing tests is an investment that will save time in the future. We will also look at some causes of unmaintainable tests (for example using a wrong test double, testing implementation details and not general behavior) and how to overcome these problems. With the use of examples from everyday Android development, we will learn how to avoid brittle tests.

You will leave the talk having a clearer understanding of why tests are important and worth the time and ultimately be able to apply the tips to write robust and more maintainable tests.

Transcript

1. Writing tests that stand
   the test of time
   Segun Famisa
   Android GDE
   

   View Slide

2. segunfamisa
   segunfamisa.com
   

   View Slide

3. Outline
   Introduction to TDD
   Challenges with TDD
   Testing tools in practice
   Writing maintainable tests
   Recap
   

   View Slide

4. Introduction
   

   View Slide

5. So, what’s Test Driven Development?
   

   View Slide

6. TDD is a software development process that relies on the
   repetition of a very short development cycle: requirements are
   turned into very specific test cases, then the software is
   improved to pass the new tests, only
   Wikipedia
   “
   

   View Slide

7. TDD is a software development process that relies on the
   repetition of a very short development cycle: requirements are
   turned into very specific test cases, then the software is
   improved to pass the new tests, only
   Wikipedia
   

   View Slide

8. TDD is a software development process that relies on the
   repetition of a very short development cycle: requirements are
   turned into very specific test cases, then the software is
   improved to pass the new tests, only
   Wikipedia
   

   View Slide

9. How to do TDD?
   

   View Slide

10. Red - Green - Refactor
    

    View Slide

11. Red - Green - Refactor
    Write
    failing test
    

    View Slide

12. Red - Green - Refactor
    Write
    failing test
    Write
    code, just
    enough to
    pass
    

    View Slide

13. Red - Green - Refactor
    Write
    failing test
    Write
    code, just
    enough to
    pass
    Clean up
    

    View Slide

14. Red - Green - Refactor
    Write
    failing test
    Write
    code, just
    enough to
    pass
    Clean up
    

    View Slide

15. Why do we need tests?
    

    View Slide

16. Why do we need tests?
    ● Quick feedback about bugs/errors
    

    View Slide

17. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    

    View Slide

18. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    ● Documentation for code behavior
    

    View Slide

19. Why do we need tests?
    ● Quick feedback about bugs/errors
    ● Good code design
    ● Documentation for code behavior
    ● Confident refactoring
    

    View Slide

20. Challenges in practicing TDD
    

    View Slide

21. Challenges in practicing TDD
    

    View Slide

22. Challenges in practicing TDD
    Legacy code
    ● Legacy code is difficult to test.
    

    View Slide

23. Challenges in practicing TDD
    Time
    ● Tests are code too, so they take time to write
    

    View Slide

24. Challenges in practicing TDD
    Bad tests
    ● Fragile and obscure tests defeat the purpose of TDD
    

    View Slide

25. Challenges in practicing TDD
    Bad tests
    ● Fragile and obscure tests defeat the purpose of TDD
    ● “Bad tests” is worse than no tests - time and effort wasted without results
    

    View Slide

26. Tools & concepts for writing maintainable
    tests
    

    View Slide

27. Test doubles
    

    View Slide

28. Test doubles
    Just like stunt doubles
    https://people.com/movies/actors-and-their-stunt-doubles-photos
    

    View Slide

29. Test doubles - dummies
    

    View Slide

30. Test doubles - dummies
    ● Dummies are like placeholders. Just to fill in parameters.
    

    View Slide

31. Test doubles - stubs
    

    View Slide

32. Test doubles - stubs
    ● Objects that return predefined data
    ● They usually don’t hold state/respond to other actions besides the one they
    are created for
    

    View Slide

33. Test doubles - stubs
    interface IUserRepository {...}
    

    View Slide

34. Test doubles - stubs
    interface IUserRepository {...}
    ...
    class UserRepository(private val userDao: UserDao) : IUserRepository {
    override fun getUser(userId: Long): User {
    return userDao.findById(userId = userId)
    }
    }
    

    View Slide

35. Test doubles - stubs
    interface IUserRepository {...}
    ...
    class UserRepository(private val userDao: UserDao) : IUserRepository {
    override fun getUser(userId: Long): User {
    return userDao.findById(userId = userId)
    }
    }
    

    View Slide

36. Test doubles - stubs
    interface IUserRepository {...}
    ...
    class UserRepositoryStub() : IUserRepository {
    override fun getUser(userId: Long): User {
    return User(userId = 1, email = "sf@sf.com")
    }
    }
    Stub returns a preconfigured user
    

    View Slide

37. Test doubles - fakes
    

    View Slide

38. Test doubles - fakes
    ● Similar to stubs, slightly more realistic
    ● Contain working implementation, but different from real version
    ● Typically models the behavior of the real class
    

    View Slide

39. Test doubles - fakes
    interface UserDao {...}
    

    View Slide

40. Test doubles - fakes
    interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    

    View Slide

41. Test doubles - fakes
    interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fake dao uses a list instead
    of a db
    

    View Slide

42. Test doubles - fakes
    interface UserDao {...}
    ...
    class FakeUserDao() : UserDao {
    val users = mutableListOf()
    override fun insert(user: User) {
    users.add(user)
    }
    override fun findById(userId: Long): User {
    return users.find { it.userId == userId }
    ?: throw Exception("user not found")
    }
    }
    Fake dao supports the
    same operations
    

    View Slide

43. Test doubles - mocks
    

    View Slide

44. Test doubles - mocks
    ● Objects pre-programmed with expected outputs for given inputs
    ● Ability to record method calls and verify them
    ● Throw exceptions if wanted method is not called
    

    View Slide

45. Test doubles - mocks
    @Test
    fun userShouldBeReturnedFromDao() {
    val dao: UserDao = mock()
    whenever(dao.getUser(userId = 5)).thenReturn(User(5, "sf@sf.com"))
    ...
    }
    

    View Slide

46. Test doubles - mocks
    @Test
    fun userShouldBeReturnedFromDao() {
    val dao: UserDao = mock()
    whenever(dao.getUser(userId = 5)).thenReturn(User(5, "sf@sf.com"))
    ...
    } Mock pre-programmed with
    input/output
    

    View Slide

47. Test doubles - mocks
    // SettingsPresenterTest.kt
    @Test
    fun clickingIconShouldOpenProfileScreen() {
    ...
    val view: SettingsContract.View = mock()
    val presenter = SettingsPresenter(view, userRepo)
    presenter.profileIconClicked()
    verify(view).openProfileScreen()
    }
    

    View Slide

48. Test doubles - mocks
    Ability to verify interactions
    // SettingsPresenterTest.kt
    @Test
    fun clickingIconShouldOpenProfileScreen() {
    ...
    val view: SettingsContract.View = mock()
    val presenter = SettingsPresenter(view, userRepo)
    presenter.profileIconClicked()
    verify(view).openProfileScreen()
    }
    

    View Slide

49. Test doubles - spies
    

    View Slide

50. Test doubles - spies
    ● Hybrid between stubs, fakes and mocks
    

    View Slide

51. Test doubles - spies
    ● Hybrid between stubs, fakes and mocks
    ● They are as real as stubs, but also have the ability to record interactions like
    mocks.
    

    View Slide

52. What are maintainable tests?
    

    View Slide

53. View Slide

54. What are maintainable tests?
    Tests are maintainable when:
    ● Old tests do not break often
    

    View Slide

55. What are maintainable tests?
    Tests are maintainable when:
    ● Old tests do not break often
    ● Old tests are easy to update
    

    View Slide

56. What are maintainable tests?
    Tests are maintainable when:
    ● Old tests do not break often
    ● Old tests are easy to update
    ● Easy to add new tests
    

    View Slide

57. Writing maintainable tests
    

    View Slide

58. 1. Use a good test specification system
    Set up dependencies
    Exercise
    Verify
    

    View Slide

59. 1. Use a good test specification system
    Also known as
    Arrange - Act - Assert
    Or
    Given - When - Then
    Set up dependencies
    Exercise
    Verify
    

    View Slide

60. 1. Use a good test specification system
    @Test
    fun scenarioX() {
    // Given the dependencies/behavior
    // When we act on the scenario
    // Then assert that expected behavior happens
    }
    

    View Slide

61. 2. Test behavior, not implementation details
    

    View Slide

62. 2. Test behavior, not implementation details
    ● For methods that return value, you should care only about the output, not
    how it was calculated.
    

    View Slide

63. 2. Test behavior, not implementation details
    @Test
    fun `get user details from cache if available`() {
    ...
    val userRepo = UserRepository(cacheSource, networkSource)
    // given that a user exists in cache
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "sf@sf.com"))
    // when we get user from repository
    val user = userRepo.getUser(userId = 5)
    // then verify that the cache source was called
    verify(cacheSource).getUser(5)
    }
    

    View Slide

64. 2. Test behavior, not implementation details
    @Test
    fun `get user details from cache if available`() {
    ...
    val userRepo = UserRepository(cacheSource, networkSource)
    // given that a user exists in cache
    whenever(cacheSource.getUser(5)).thenReturn(User(5, "sf@sf.com"))
    // when we get user from repository
    val user = userRepo.getUser(userId = 5)
    // then verify that the cache source was called
    verify(cacheSource).getUser(5)
    }
    This tests implementation
    details
    

    View Slide

65. 2. Test behavior, not implementation details
    @Test
    fun `get user details from cache if available`() {
    ...
    // given that a user exists in cache
    val cachedUser = User(5, "sf@sf.com")
    whenever(cacheSource.getUser(5)).thenReturn(cachedUser)
    // when we get user from repository
    val returnedUser = userRepo.getUser(userId = 5)
    // then verify that the returned user is the one from cache
    assertEquals(cachedUser, returnedUser)
    }
    

    View Slide

66. 2. Test behavior, not implementation details
    @Test
    fun `get user details from cache if available`() {
    ...
    // given that a user exists in cache
    val cachedUser = User(5, "sf@sf.com")
    whenever(cacheSource.getUser(5)).thenReturn(cachedUser)
    // when we get user from repository
    val returnedUser = userRepo.getUser(userId = 5)
    // then verify that the returned user is the one from cache
    assertEquals(cachedUser, returnedUser)
    }
    This tests general behavior of this
    repository in this scenario.
    

    View Slide

67. 2. Test behavior, not implementation details
    ● For methods that return value, one should care only about the output, not
    how it was calculated.
    ● For methods that do not return any value, verify interactions with
    dependencies
    

    View Slide

68. 2. Test behavior, not implementation details
    ● For methods that return value, one should care only about the output, not
    how it was calculated.
    ● For methods that do not return any value, verify interactions with
    dependencies
    ● Be careful about overusing mocks.
    

    View Slide

69. 3. Assert/verify only one thing per test
    

    View Slide

70. 3. Assert/verify only one thing per test
    In most cases, only one assert / verify should be done in each test.
    

    View Slide

71. 3. Assert/verify only one thing per test
    In most cases, only one assert / verify should be done in each test.
    A test should fail for only 1 reason
    

    View Slide

72. 3. Assert/verify only one thing per test
    @Test
    fun `enabling setting updates preference and sends tracking`() {
    ...
    // when user enables the setting
    viewModel.enableSetting()
    // then verify that we set preference
    verify(userPreference).enableSetting()
    // then verify that we send tracking
    verify(trackingUtils).trackUserEnabledSetting()
    }
    

    View Slide

73. 3. Assert/verify only one thing per test
    @Test
    fun `enabling setting updates preference and sends tracking`() {
    ...
    // when user enables the setting
    viewModel.enableSetting()
    // then verify that we set preference
    verify(userPreference).enableSetting()
    // then verify that we send tracking
    verify(trackingUtils).trackUserEnabledSetting()
    }
    

    View Slide

74. 3. Assert/verify only one thing per test
    @Test
    fun `enabling setting updates preference and sends tracking`() {
    ...
    // when user enables the setting
    viewModel.enableSetting()
    // then verify that we set preference
    verify(userPreference).enableSetting()
    // then verify that we send tracking
    verify(trackingUtils).trackUserEnabledSetting()
    }
    The use of “and” suggests
    that the test is testing more
    than one thing
    

    View Slide

75. 3. Assert/verify only one thing per test
    @Test
    fun `enabling setting updates preference`() {
    ...
    // then verify that we set preference
    verify(userPreference).enableSetting()
    }
    @Test
    fun `enabling setting posts tracking`() {
    ...
    // then verify that we post tracking
    verify(trackingUtils).trackUserEnabledSetting()
    }
    

    View Slide

76. 4. Use descriptive test names
    

    View Slide

77. 4. Use descriptive test names
    From the test name, we should be able to tell why the test failed.
    

    View Slide

78. 4. Use descriptive test names
    From the test name, we should be able to tell why the test failed.
    @Test
    fun `search field is updated correctly when user has search history`() {
    ...
    }
    

    View Slide

79. 4. Use descriptive test names
    From the test name, we should be able to tell why the test failed.
    @Test
    fun `search field is updated correctly when user has search history`() {
    ...
    }
    

    View Slide

80. 4. Use descriptive test names
    From the test name, we should be able to tell why the test failed.
    @Test
    fun `search field is updated correctly when user has search history`() {
    ...
    }
    @Test
    fun `search field is updated with last search when user has search history`() {
    ...
    }
    

    View Slide

81. 4. Use descriptive test names
    From the test name, we should be able to tell why the test failed.
    @Test
    fun `search field is updated correctly when user has search history`() {
    ...
    }
    @Test
    fun `search field is updated with last search when user has search history`() {
    ...
    }
    

    View Slide

82. 4. Use descriptive test names
    Kotlin allows us to use to write test function names with spaces
    @Test
    fun `welcome dialog should be shown on first log in`() {
    // test goes here
    }
    

    View Slide

83. 4. Use descriptive test names
    JUnit 5 allows to specify a custom display name for the test
    @Test
    @DisplayName("welcome dialog should be shown on first log in")
    void showWelcomeDialogOnFirstLogin() {
    // test goes here
    }
    

    View Slide

84. 5. More tips
    ● Avoid logic in your tests -> if/else, loops, etc.
    

    View Slide

85. 5. More tips
    ● Avoid logic in your tests -> if/else, loops, etc.
    ● Avoid abstractions in tests
    

    View Slide

86. 5. More tips
    ● Avoid logic in your tests -> if/else, loops, etc.
    ● Avoid abstractions in tests
    ● Be generous with comments
    

    View Slide

87. 5. More tips
    ● Avoid logic in your tests -> if/else, loops, etc.
    ● Avoid abstractions in tests
    ● Be generous with comments
    ● Use parameterized tests
    

    View Slide

88. Resources
    ● https://martinfowler.com/articles/mocksArentStubs.html
    ● http://xunitpatterns.com/
    ● https://testing.googleblog.com/search/label/TotT
    ● https://mtlynch.io/good-developers-bad-tests/
    ● https://jeroenmols.com/blog/2018/12/06/fixthetest/
    

    View Slide

89. Thank you!
    @segunfamisa
    

    View Slide