Write awesome unit tests

Slides from my talk at Devoxx 2018
Video: https://www.youtube.com/watch?v=F8Gc8Nwf0yk

While there are plenty of resources about writing awesome code, the same doesn't quite exist for tests. So how do you write great unit tests? And how do you ensure failures are easy to fix? What are the best practices? And most importantly what are the pitfalls to look out for?

If you're already comfortable writing tests, come level up your testing skills with deep insights into the "Why" and "How" of fundamental testing principles. You will learn:

- What three criteria make tests awesome
- What to test and what not to test
- Why you should optimise tests for failure
- How many tests you should write per method
- How to test literally everything
- What is code coverage and why it can be deceiving
- How to use AssertJ to make test super readable
- What TDD is and what its benefit is
- ...

Transcript

1. @MOLSJEROEN WRITE AWESOME UNIT TESTS
2. @MOLSJEROEN @MOLSJEROEN
3. @MOLSJEROEN CORE PRINCIPLES THREE GROUND RULES CODE COVERAGE TEST DRIVEN DEVELOPMENT TESTING MYTHS TEST VS PRODUCTION CODE TIPS
4. CORE PRINCIPLES
5. @MOLSJEROEN WHY DO WE WRITE TESTS? Proof of working code Safeguard that code keeps on working Documentation Enables refactoring Release faster Easiest form of testing …
6. WE WRITE TESTS TO RUN THEM
7. @MOLSJEROEN TESTING IS A LONG TERM COMMITMENT Fix failing test Remove obsolete tests Refactor existing tests Upgrade test frameworks Fix bugs in tests …
8. TESTS ARE PRODUCTION C0DE
9. @MOLSJEROEN TESTS REQUIRE INFRASTRUCTURE CI server to run tests Dependency injection framework Tools to analyze code coverage Slack integration when tests fail Build command to run tests Hardware to run tests on …
10. OPTIMIZE TESTS FOR FAILURE
11. THREE GROUND RULES
12. @MOLSJEROEN 1 - RUN LUDICROUSLY FAST Safety net against regression Detect bugs early and cheaply Rapid feedback loop is required Run tests on every change
13. @MOLSJEROEN
14. @MOLSJEROEN 2 - SMALL AND FOCUSSED Only few lines of code Clear structure: Assert/Act/Arrange Only one failing test for each bug Test name indicates the problem
15. @MOLSJEROEN @Test fun `login should fail with wrong password`() { // Test code }
16. @MOLSJEROEN 3 - 100% RELIABLE Drop everything to fix failure Reason must be obvious Failures should be rare Correlation with modified code “Magical rerun” drives developers mad Lose of trust in safety net
17. @MOLSJEROEN @Test fun `should properly format time`() { val expectedTime = FORMAT.format(Date()) val formattedTime = TimeFormatter().currentFormattedTime assertEquals(expectedTime, formattedTime) } companion object { private val FORMAT = SimpleDateFormat("HH:mm:ss:SSS") }
18. @MOLSJEROEN @Test fun `should properly format time`() { val expectedTime = FORMAT.format(Date()) val formattedTime = TimeFormatter().currentFormattedTime assertEquals(expectedTime, formattedTime) } companion object { private val FORMAT = SimpleDateFormat("HH:mm:ss:SSS") }
19. @MOLSJEROEN @Test fun `should properly format time`() { val now = Date() val expectedTime = FORMAT.format(now) val formattedTime = TimeFormatter().currentFormattedTime(now) assertEquals(expectedTime, formattedTime) } companion object { private val FORMAT = SimpleDateFormat("HH:mm:ss:SSS") }
20. CODE COVERAGE
21. IF YOUR APP HAS 100% CODE COVERAGE, CAN IT STILL HAVE BUGS?
22. @MOLSJEROEN 100% APP COVERAGE Does not mean app is bug free! UI inconsistencies Interaction between classes Interaction with external services ….
23. IF A CLASS HAS 100% CODE COVERAGE, CAN THAT STILL HAVE BUGS?
24. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int { return a + b } }
25. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int { return a + b } } @Test fun `sum of one and three is four`() { val result = Calculator().sum(1,3) assertEquals(4, result) }
26. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int { return a + b } } @Test fun `sum of one and three is four`() { val result = Calculator().sum(1,3) }
27. @MOLSJEROEN class Calculator { fun sum(a: Int, b: Int): Int { return 4 } } @Test fun `sum of one and three is four`() { val result = Calculator().sum(1,3) assertEquals(4, result) }
28. @MOLSJEROEN 100% COVERAGE Doesn’t mean anything Agnostic about quality of the tests Unaware of functionality coverage Not even check for asserts
29. @MOLSJEROEN Coverage Effort
30. @MOLSJEROEN
31. TEST DRIVEN DEVELOPMENT
32. @MOLSJEROEN WRITE TESTS BEFORE OR AFTER CODE Doesn’t matter! But, writing before: …guarantees test would fail …reduces manual testing …is easier …forces you to think better about requirements …helps split a problem in smaller parts
33. @MOLSJEROEN TDD - WHAT (Extreme) way of working Flow: 1. Write no code except to make a test pass or failing test 2. Write minimum code to make test fail 3. Write minimum code to make test pass
34. @MOLSJEROEN Green Red Refactor
35. @MOLSJEROEN TEST DRIVEN … Development … Design … Divide and conquer … Documentation … DevOps … Determination … Dream
36. @MOLSJEROEN class CalculatorTest { }
37. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator() } }
38. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator() } } class Calculator { }
39. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator() } @Test fun `one plus three is four`() { Calculator().sum(1, 3) } } class Calculator { }
40. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator() } @Test fun `one plus three is four`() { Calculator().sum(1, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { TODO("not implemented") } }
41. @MOLSJEROEN class CalculatorTest { @Test fun `can instantiate`() { Calculator() } @Test fun `one plus three is four`() { Calculator().sum(1, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return 4 } }
42. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is four`() { Calculator().sum(1, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return 4 } }
43. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return 4 } }
44. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } } class Calculator { fun sum(a: Int, b: Int) : Int { return if (a == 1) 4 else 5 } }
45. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } @Test fun `three plus three is six`() { Calculator().sum(3, 3) } class Calculator { fun sum(a: Int, b: Int) : Int { return if (a == 1) 4 else 5 } }
46. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } @Test fun `three plus three is six`() { Calculator().sum(3, 3) } class Calculator { fun sum(a: Int, b: Int) : Int { return if (a == 1) 4 else if (a == 2) 5 else 6 } }
47. @MOLSJEROEN class CalculatorTest { @Test fun `one plus three is four`() { Calculator().sum(1, 3) } @Test fun `two plus three is five`() { Calculator().sum(2, 3) } @Test fun `three plus three is six`() { Calculator().sum(3, 3) } class Calculator { fun sum(a: Int, b: Int) : Int { return a + b } }
48. TESTING MYTHS
49. TESTING SLOWS DOWN DEVELOPMENT
50. TESTING IS HARD
51. TEST CODE IS NOT PRODUCTION CODE
52. SIMPLE CODE SHOULDN’T BE TESTED
53. TESTING NEGATIVELY IMPACTS THE PRODUCTION CODE
54. EVERYTHING SHOULD BE TESTED
55. TEST VS PRODUCTION CODE
56. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class CalculatorTest : TestCase() { fun test_sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
57. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class CalculatorTest : TestCase() { fun test_sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
58. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class CalculatorTest { @Test fun sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
59. @MOLSJEROEN // Instrumented test, run on Android device. @RunWith(AndroidJUnit4::class) class CalculatorTest { @Test fun sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
60. @MOLSJEROEN class CalculatorTest { @Test fun sumShouldAddNumbers() { val sum = Calculator().sum(1, 2) assertThat(sum).isEqualTo(3) } }
61. USE JUNIT 4 SYNTAX
62. @MOLSJEROEN class WebServiceTest { lateinit var webService: WebService @Before fun setUp() { webService = WebServiceTestHelper.createWebService() } @Test fun loginHasFailed() { val result = webService.login() checkLoginFailed(result) } }
63. @MOLSJEROEN class WebServiceTest { lateinit var webService: WebService @Before fun setUp() { webService = WebServiceTestHelper.createWebService() } @Test fun loginHasFailed() { val result = webService.login() checkLoginFailed(result) } }
64. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService = WebServiceTestHelper.createWebService() val result = webService.login() checkLoginFailed(result) } }
65. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService = WebServiceTestHelper.createWebService() val result = webService.login() checkLoginFailed(result) } }
66. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService = WebService() webService.setUserCredentials("email@google.com", “wrong_pwd”) val result = webService.login() checkLoginFailed(result) } }
67. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService = WebService() webService.setUserCredentials("email@google.com", “wrong_pwd”) val result = webService.login() checkLoginFailed(result) } }
68. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { var webService = WebService() webService.setUserCredentials("email@google.com", “wrong_pwd”) val result = webService.login() assertThat(result.isSuccess).isFalse() } }
69. KEEP READER IN TEST
70. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator : Calculator /* Test that perimeter is calculated correctly */ @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
71. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator : Calculator /* Test that perimeter is calculated correctly */ @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
72. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator : Calculator @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
73. @MOLSJEROEN internal class CalculatorTest { @Mock private lateinit var calculator : Calculator @Test internal fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { private const val WIDTH = 5 private const val HEIGHT = 10 }
74. @MOLSJEROEN class CalculatorTest { @Mock lateinit var calculator : Calculator @Test fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { const val WIDTH = 5 const val HEIGHT = 10 }
75. @MOLSJEROEN class CalculatorTest { @Mock lateinit var calculator : Calculator @Test fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(WIDTH, HEIGHT) assertThat(circumference).isEqualTo(30) } companion object { const val WIDTH = 5 const val HEIGHT = 10 }
76. @MOLSJEROEN class CalculatorTest { @Mock lateinit var calculator : Calculator @Test fun calculatePerimeterOfRectangle() { val geoCalculator = GeometryCalculator(calculator) val circumference = geoCalculator.perimeter(5, 10) assertThat(circumference).isEqualTo(30) } }
77. NO CLUTTER
78. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.isSuccess).isFalse() assertThat(result.user).isNull() } }
79. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.isSuccess).isFalse() assertThat(result.user).isNull() } }
80. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
81. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
82. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
83. @MOLSJEROEN class WebServiceTest { @Test fun loginHasFailed() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
84. @MOLSJEROEN class WebServiceTest { @Test fun `user is null when login fails`() { val webService = WebService() webService.setUserCredentials("email@google.com", "wrong_pwd") val result = webService.login() assertThat(result.user).isNull() } }
85. OPTIMIZE TEST FOR FAILURE
86. @MOLSJEROEN DO REPEAT YOURSELF USE MAGIC INTS AND STRINGS NO ENCAPSULATION DON’T LAYER TEST CLASSES LONG TEST METHOD NAMES
87. TIPS
88. @MOLSJEROEN TESTING UI View ViewModel Repository LiveData
89. @MOLSJEROEN assertEquals(expected, actual) assertNull(actual) assertNotNull(actual) assertTrue(condition) assertFalse(condition) assertSame(expected, actual) assertNotSame(expected, actual) ASSERTJ
90. @MOLSJEROEN assertEquals(expected, actual) assertNull(actual) assertNotNull(actual) assertTrue(condition) assertFalse(condition) assertSame(expected, actual) assertNotSame(expected, actual) assertThat(actual).isEqualTo(…) assertThat(actual).isNull() assertThat(actual).isNotNull() assertThat(condition).isTrue() assertThat(condition).isFalse() assertThat(actual).isSameAs(…) assertThat(actual).isNotSameAs(…) ASSERTJ
91. @MOLSJEROEN assertThat(number).isGreaterThan(2) assertThat(string).endsWith("mols") assertThat(string).isEmpty() assertThat(tomorrow).isAfter(today) assertThat(list).contains("a") assertThat(list).containsExactly(first, second) assertThat(list).hasSize(9) assertThat(map).containsKey(2) ASSERTJ
92. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`() { val userManager = mock(UserManager::class.java) val user = mock(User::class.java) val userConfig = mock(UserConfiguration::class.java) `when`(userManager.getCurrentUser()).thenReturn(user) `when`(user.getConfiguration()).thenReturn(userConfig) `when`(userConfig.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(userManager) assertThat(allowed).isTrue() }
93. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`() { val userManager = mock(UserManager::class.java) val user = mock(User::class.java) val userConfig = mock(UserConfiguration::class.java) `when`(userManager.getCurrentUser()).thenReturn(user) `when`(user.getConfiguration()).thenReturn(userConfig) `when`(userConfig.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(userManager) assertThat(allowed).isTrue() }
94. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`() { val user = mock(User::class.java) val userConfig = mock(UserConfiguration::class.java) `when`(user.getConfiguration()).thenReturn(userConfig) `when`(userConfig.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(user) assertThat(allowed).isTrue() }
95. @MOLSJEROEN NESTING MOCKS fun `can show alcohol ads when adult`() { val user = mock(User::class.java) `when`(user.getAge()).thenReturn(21) val allowed = Advertisements().alcoholAdsAllowed(user) assertThat(allowed).isTrue() }
96. @MOLSJEROEN PROVIDING TEST DATA @Test fun `should use mock data`() { val mock = mock(UserData::class.java) `when`(mock.firstName).thenReturn("FirstName") `when`(mock.lastName).thenReturn("LastName") `when`(mock.userId).thenReturn(42) `when`(mock.street).thenReturn("Street") `when`(mock.houseNumber).thenReturn(1) `when`(mock.city).thenReturn("City") `when`(mock.country).thenReturn("Country") // Test code here }
97. @MOLSJEROEN PROVIDING TEST DATA @Test fun `should use test data`() { val mock = createFakeUserData() // Test code here } fun createFakeUserData() : UserData { return UserData("FirstName", "LastName", is 42, "Street", 1, "City", "Country") }
98. @MOLSJEROEN spy(UserManager::class.java)
99. @MOLSJEROEN UNTESTABLE DEPENDENCIES Camera ViewModel
100. @MOLSJEROEN UNTESTABLE DEPENDENCIES Camera ViewModel CameraWrapper
101. @MOLSJEROEN class NativeCamera { lateinit var nativeCamera: Camera private set @Throws(RuntimeException::class) fun openNativeCamera() { nativeCamera = Camera.open(CameraInfo.CAMERA_FACING_BACK) } fun releaseNativeCamera() { nativeCamera.release() } } UNTESTABLE DEPENDENCIES
102. @MOLSJEROEN class NativeCamera { lateinit var nativeCamera: Camera private set @Throws(RuntimeException::class) fun openNativeCamera() { nativeCamera = Camera.open(CameraInfo.CAMERA_FACING_BACK) } fun releaseNativeCamera() { nativeCamera.release() } } UNTESTABLE DEPENDENCIES
103. @MOLSJEROEN WHAT NOT TO UNIT TEST UI Layer: Views, Fragments, Activities,… Handovers between Threads File IO operations Database IO operations Dependency wrappers Deep OS integrations: camera, sensors,… ….
104. WRAP UP
105. WE NEED TO BE AS CONFIDENT IN THE TESTS WE CODE AS WE ARE IN THE CODE WE TEST Xavier F. Gouchet
106. @MOLSJEROEN TESTS ARE PRODUCTION CODE DIFFERENT RULES APPLY (E.G. NO DRY) AWESOME TESTS ARE FAST, FOCUSSED AND NON FLAKY CODE COVERAGE IS MEANINGLESS NOT EVERYTHING NEEDS TO BE TESTED
107. @MOLSJEROEN HTTPS://JEROENMOLS.COM/BLOG
108. @MOLSJEROEN IMAGE CREDITS Welcome image by Clement12
 https://www.flickr.com/photos/clement127/20626915084/in/photolist-Lrdy6N- rk5BW3-xqJkdY-Qn7xsS-QS4TQX-RYAGuU-qymdNL-vAg6ro-sa1Wh5-ruMfjM- rh9fKo-vTgeEP-qW7Kmv-rVaPtk-rNEdMh-pd8is6-CdNeWN-BkhgMC-xzkZKR- rZ6aWZ Material design icons by Google
 https://material.io/tools/icons Incognito Incognito by Vaibhav Radhakrishnan
 https://thenounproject.com/term/incognito/404950
109. MOLSJEROEN