Write awesome unit tests

5f57d2d205e77e185986459c1b89a874?s=47 Jeroen Mols
November 15, 2018

Write awesome unit tests

Slides from my talk at Devoxx 2018
Video: https://www.youtube.com/watch?v=F8Gc8Nwf0yk

While there are plenty of resources about writing awesome code, the same doesn't quite exist for tests. So how do you write great unit tests? And how do you ensure failures are easy to fix? What are the best practices? And most importantly what are the pitfalls to look out for?

If you're already comfortable writing tests, come level up your testing skills with deep insights into the "Why" and "How" of fundamental testing principles. You will learn:

- What three criteria make tests awesome
- What to test and what not to test
- Why you should optimise tests for failure
- How many tests you should write per method
- How to test literally everything
- What is code coverage and why it can be deceiving
- How to use AssertJ to make test super readable
- What TDD is and what its benefit is
- ...

5f57d2d205e77e185986459c1b89a874?s=128

Jeroen Mols

November 15, 2018
Tweet

Transcript

  1. @MOLSJEROEN WRITE AWESOME UNIT TESTS

  2. @MOLSJEROEN @MOLSJEROEN

  3. @MOLSJEROEN CORE PRINCIPLES THREE GROUND RULES CODE COVERAGE TEST DRIVEN

    DEVELOPMENT TESTING MYTHS TEST VS PRODUCTION CODE TIPS
  4. CORE PRINCIPLES

  5. @MOLSJEROEN WHY DO WE WRITE TESTS? Proof of working code

    Safeguard that code keeps on working Documentation Enables refactoring Release faster Easiest form of testing …
  6. WE WRITE TESTS TO RUN THEM

  7. @MOLSJEROEN TESTING IS A LONG TERM COMMITMENT Fix failing test

    Remove obsolete tests Refactor existing tests Upgrade test frameworks Fix bugs in tests …
  8. TESTS ARE PRODUCTION C0DE

  9. @MOLSJEROEN TESTS REQUIRE INFRASTRUCTURE CI server to run tests Dependency

    injection framework Tools to analyze code coverage Slack integration when tests fail Build command to run tests Hardware to run tests on …
  10. OPTIMIZE TESTS FOR FAILURE

  11. THREE GROUND RULES

  12. @MOLSJEROEN 1 - RUN LUDICROUSLY FAST Safety net against regression

    Detect bugs early and cheaply Rapid feedback loop is required Run tests on every change
  13. @MOLSJEROEN

  14. @MOLSJEROEN 2 - SMALL AND FOCUSSED Only few lines of

    code Clear structure: Assert/Act/Arrange Only one failing test for each bug Test name indicates the problem
  15. @MOLSJEROEN @Test fun `login should fail with wrong password`() {

    // Test code }
  16. @MOLSJEROEN 3 - 100% RELIABLE Drop everything to fix failure

    Reason must be obvious Failures should be rare Correlation with modified code “Magical rerun” drives developers mad Lose of trust in safety net
  17. @MOLSJEROEN @Test fun `should properly format time`() { val expectedTime

    = FORMAT.format(Date()) val formattedTime = TimeFormatter().currentFormattedTime assertEquals(expectedTime, formattedTime) } companion object { private val FORMAT = SimpleDateFormat("HH:mm:ss:SSS") }
  18. @MOLSJEROEN @Test fun `should properly format time`() { val expectedTime

    = FORMAT.format(Date()) val formattedTime = TimeFormatter().currentFormattedTime assertEquals(expectedTime, formattedTime) } companion object { private val FORMAT = SimpleDateFormat("HH:mm:ss:SSS") }
  19. @MOLSJEROEN @Test fun `should properly format time`() { val now

    = Date() val expectedTime = FORMAT.format(now) val formattedTime = TimeFormatter().currentFormattedTime(now) assertEquals(expectedTime, formattedTime) } companion object { private val FORMAT = SimpleDateFormat("HH:mm:ss:SSS") }
  20. CODE COVERAGE

  21. IF YOUR APP HAS 100% CODE COVERAGE, CAN IT STILL

    HAVE BUGS?
  22. @MOLSJEROEN 100% APP COVERAGE Does not mean app is bug

    free! UI inconsistencies Interaction between classes Interaction with external services ….
  23. IF A CLASS HAS 100% CODE COVERAGE, CAN THAT STILL

    HAVE BUGS?
  24. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int

    { return a + b } }
  25. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int

    { return a + b } } @Test fun `sum of one and three is four`() { val result = Calculator().sum(1,3) assertEquals(4, result) }
  26. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int

    { return a + b } } @Test fun `sum of one and three is four`() { val result = Calculator().sum(1,3) }
  27. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int

    { return 4 } } @Test fun `sum of one and three is four`() { val result = Calculator().sum(1,3) assertEquals(4, result) }
  28. @MOLSJEROEN 100% COVERAGE Doesn’t mean anything Agnostic about quality of

    the tests Unaware of functionality coverage Not even check for asserts
  29. @MOLSJEROEN Coverage Effort

  30. @MOLSJEROEN

  31. TEST DRIVEN DEVELOPMENT

  32. @MOLSJEROEN WRITE TESTS BEFORE OR AFTER CODE Doesn’t matter! But,

    writing before: …guarantees test would fail …reduces manual testing …is easier …forces you to think better about requirements …helps split a problem in smaller parts
  33. @MOLSJEROEN TDD - WHAT (Extreme) way of working Flow: 1.

    Write no code except to make a test pass or failing test 2. Write minimum code to make test fail 3. Write minimum code to make test pass
  34. @MOLSJEROEN Green Red Refactor

  35. @MOLSJEROEN TEST DRIVEN … Development … Design … Divide and

    conquer … Documentation … DevOps … Determination … Dream
  36. @MOLSJEROEN class CalculatorTest { }

  37. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator()

    } }
  38. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator()

    } } class Calculator { }
  39. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator()

    } @Test fun `one plus three is four`() { Calculator().sum(1, 3) } } class Calculator { }
  40. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator()

    } @Test fun `one plus three is four`() { Calculator().sum(1, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { TODO("not implemented") } }
  41. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator()

    } @Test fun `one plus three is four`() { Calculator().sum(1, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return 4 } }
  42. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is

    four`() { Calculator().sum(1, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return 4 } }
  43. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is

    four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return 4 } }
  44. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is

    four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return if (a == 1) 4 else 5 } }
  45. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is

    four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } @Test fun `three plus three is six`() { Calculator().sum(3, 3) } class Calculator { fun sum(a: Int, b: Int) : Int { return if (a == 1) 4 else 5 } }
  46. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is

    four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } @Test fun `three plus three is six`() { Calculator().sum(3, 3) } class Calculator { fun sum(a: Int, b: Int) : Int { return if (a == 1) 4 else if (a == 2) 5 else 6 } }
  47. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is

    four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } @Test fun `three plus three is six`() { Calculator().sum(3, 3) } class Calculator { fun sum(a: Int, b: Int) : Int { return a + b } }
  48. TESTING MYTHS

  49. TESTING SLOWS DOWN DEVELOPMENT

  50. TESTING IS HARD

  51. TEST CODE IS NOT PRODUCTION CODE

  52. SIMPLE CODE SHOULDN’T BE TESTED

  53. TESTING NEGATIVELY IMPACTS THE PRODUCTION CODE

  54. EVERYTHING SHOULD BE TESTED

  55. TEST VS PRODUCTION CODE

  56. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class

    CalculatorTest : TestCase() { fun test_sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
  57. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class

    CalculatorTest : TestCase() { fun test_sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
  58. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class

    CalculatorTest { @Test fun sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
  59. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class

    CalculatorTest { @Test fun sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
  60. @MOLSJEROEN class CalculatorTest { @Test fun sumShouldAddNumbers() { val sum

    = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
  61. USE JUNIT 4 SYNTAX

  62. @MOLSJEROEN class WebServiceTest { lateinit var webService: WebService @Before fun

    setUp() { webService = WebServiceTestHelper.createWebService() } @Test fun loginHasFailed() { val result = webService.login() checkLoginFailed(result) } }
  63. @MOLSJEROEN class WebServiceTest { lateinit var webService: WebService @Before fun

    setUp() { webService = WebServiceTestHelper.createWebService() } @Test fun loginHasFailed() { val result = webService.login() checkLoginFailed(result) } }
  64. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService

    = WebServiceTestHelper.createWebService() val result = webService.login() checkLoginFailed(result) } }
  65. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService

    = WebServiceTestHelper.createWebService() val result = webService.login() checkLoginFailed(result) } }
  66. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService

    = WebService() webService.setUserCredentials("email@google.com", “wrong_pwd”) val result = webService.login() checkLoginFailed(result) } }
  67. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService

    = WebService() webService.setUserCredentials("email@google.com", “wrong_pwd”) val result = webService.login() checkLoginFailed(result) } }
  68. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService

    = WebService() webService.setUserCredentials("email@google.com", “wrong_pwd”) val result = webService.login() assertThat(result.isSuccess).isFalse() } }
  69. KEEP READER IN TEST

  70. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator

    : Calculator /* Test that perimeter is calculated correctly */ @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
  71. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator

    : Calculator /* Test that perimeter is calculated correctly */ @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
  72. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator

    : Calculator @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
  73. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator

    : Calculator @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
  74. @MOLSJEROEN class CalculatorTest { @Mock lateinit var calculator : Calculator

    @Test fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { const val WIDTH = 5 const val HEIGHT = 10 }
  75. @MOLSJEROEN class CalculatorTest { @Mock lateinit var calculator : Calculator

    @Test fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { const val WIDTH = 5 const val HEIGHT = 10 }
  76. @MOLSJEROEN class CalculatorTest { @Mock lateinit var calculator : Calculator

    @Test fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(5, 10) assertThat(circumference).isEqualTo(30) } }
  77. NO CLUTTER

  78. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService

    = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.isSuccess).isFalse() assertThat(result.user).isNull() } }
  79. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService

    = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.isSuccess).isFalse() assertThat(result.user).isNull() } }
  80. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService

    = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
  81. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService

    = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
  82. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService

    = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
  83. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService

    = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
  84. @MOLSJEROEN class WebServiceTest { @Test fun `user is null when

    login fails`() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
  85. OPTIMIZE TEST FOR FAILURE

  86. @MOLSJEROEN DO REPEAT YOURSELF USE MAGIC INTS AND STRINGS NO

    ENCAPSULATION DON’T LAYER TEST CLASSES LONG TEST METHOD NAMES
  87. TIPS

  88. @MOLSJEROEN TESTING UI View ViewModel Repository LiveData

  89. @MOLSJEROEN assertEquals(expected, actual) assertNull(actual) assertNotNull(actual) assertTrue(condition) assertFalse(condition) assertSame(expected, actual) assertNotSame(expected,

    actual) ASSERTJ
  90. @MOLSJEROEN assertEquals(expected, actual) assertNull(actual) assertNotNull(actual) assertTrue(condition) assertFalse(condition) assertSame(expected, actual) assertNotSame(expected,

    actual) assertThat(actual).isEqualTo(…) assertThat(actual).isNull() assertThat(actual).isNotNull() assertThat(condition).isTrue() assertThat(condition).isFalse() assertThat(actual).isSameAs(…) assertThat(actual).isNotSameAs(…) ASSERTJ
  91. @MOLSJEROEN assertThat(number).isGreaterThan(2) assertThat(string).endsWith("mols") assertThat(string).isEmpty() assertThat(tomorrow).isAfter(today) assertThat(list).contains("a") assertThat(list).containsExactly(first, second) assertThat(list).hasSize(9) assertThat(map).containsKey(2)

    ASSERTJ
  92. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`()

    { val userManager = mock(UserManager::class.java) val user = mock(User::class.java) val userConfig = mock(UserConfiguration::class.java) `when`(userManager.getCurrentUser()).thenReturn(user) `when`(user.getConfiguration()).thenReturn(userConfig) `when`(userConfig.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(userManager) assertThat(allowed).isTrue() }
  93. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`()

    { val userManager = mock(UserManager::class.java) val user = mock(User::class.java) val userConfig = mock(UserConfiguration::class.java) `when`(userManager.getCurrentUser()).thenReturn(user) `when`(user.getConfiguration()).thenReturn(userConfig) `when`(userConfig.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(userManager) assertThat(allowed).isTrue() }
  94. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`()

    { val user = mock(User::class.java) val userConfig = mock(UserConfiguration::class.java) `when`(user.getConfiguration()).thenReturn(userConfig) `when`(userConfig.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(user) assertThat(allowed).isTrue() }
  95. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`()

    { val user = mock(User::class.java) `when`(user.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(user) assertThat(allowed).isTrue() }
  96. @MOLSJEROEN PROVIDING TEST DATA @Test fun `should use mock data`()

    { val mock = mock(UserData::class.java) `when`(mock.firstName).thenReturn("FirstName") `when`(mock.lastName).thenReturn("LastName") `when`(mock.userId).thenReturn(42) `when`(mock.street).thenReturn("Street") `when`(mock.houseNumber).thenReturn(1) `when`(mock.city).thenReturn("City") `when`(mock.country).thenReturn("Country") // Test code here }
  97. @MOLSJEROEN PROVIDING TEST DATA @Test fun `should use test data`()

    { val mock = createFakeUserData() // Test code here } fun createFakeUserData() : UserData { return UserData("FirstName", "LastName", is 42, "Street", 1, "City", "Country") }
  98. @MOLSJEROEN spy(UserManager::class.java)

  99. @MOLSJEROEN UNTESTABLE DEPENDENCIES Camera ViewModel

  100. @MOLSJEROEN UNTESTABLE DEPENDENCIES Camera ViewModel CameraWrapper

  101. @MOLSJEROEN class NativeCamera { lateinit var nativeCamera: Camera private set

    @Throws(RuntimeException::class) fun openNativeCamera() { nativeCamera = Camera.open(CameraInfo.CAMERA_FACING_BACK) } fun releaseNativeCamera() { nativeCamera.release() } } UNTESTABLE DEPENDENCIES
  102. @MOLSJEROEN class NativeCamera { lateinit var nativeCamera: Camera private set

    @Throws(RuntimeException::class) fun openNativeCamera() { nativeCamera = Camera.open(CameraInfo.CAMERA_FACING_BACK) } fun releaseNativeCamera() { nativeCamera.release() } } UNTESTABLE DEPENDENCIES
  103. @MOLSJEROEN WHAT NOT TO UNIT TEST UI Layer: Views, Fragments,

    Activities,… Handovers between Threads File IO operations Database IO operations Dependency wrappers Deep OS integrations: camera, sensors,… ….
  104. WRAP UP

  105. WE NEED TO BE AS CONFIDENT IN THE TESTS WE

    CODE AS WE ARE IN THE CODE WE TEST Xavier F. Gouchet
  106. @MOLSJEROEN TESTS ARE PRODUCTION CODE DIFFERENT RULES APPLY (E.G. NO

    DRY) AWESOME TESTS ARE FAST, FOCUSSED AND NON FLAKY CODE COVERAGE IS MEANINGLESS NOT EVERYTHING NEEDS TO BE TESTED
  107. @MOLSJEROEN HTTPS://JEROENMOLS.COM/BLOG

  108. @MOLSJEROEN IMAGE CREDITS Welcome image by Clement12
 https://www.flickr.com/photos/clement127/20626915084/in/photolist-Lrdy6N- rk5BW3-xqJkdY-Qn7xsS-QS4TQX-RYAGuU-qymdNL-vAg6ro-sa1Wh5-ruMfjM- rh9fKo-vTgeEP-qW7Kmv-rVaPtk-rNEdMh-pd8is6-CdNeWN-BkhgMC-xzkZKR-

    rZ6aWZ Material design icons by Google
 https://material.io/tools/icons Incognito Incognito by Vaibhav Radhakrishnan
 https://thenounproject.com/term/incognito/404950
  109. MOLSJEROEN