It's time to up your test game

On the paper, everyone agrees that tests are important. And yet, more often than none, tests are still considered the poor cousin in the development industry. In the rare case when we actually take time to write them, we more often than not consider it a cumbersome task, and give it less thoughts than the "real code".

But tests are often as important, if not more, as the code that will be shipped on the Play Store. After briefly explaining the basics of testing, this talk will go through various common mistakes and bad practices found in real-life unit tests, and will give techniques and tips to make those tests cleaner and better.

Transcript

1. It's time to step up your test game Apps Conf 2018
2. About me… • Xavier F. Gouchet • Writing Android apps since Android 1.5 (Cupcake) • Lead Android Engineer at WorkWell • @xgouchet on Github / StackOverflow / LinkedIn
4. Testing : the basics
5. What are the goals of tests ? • “Verify that the code works” • Make the contract explicit • Guide the development (TDD) • Prevent regression caused by ◦ Other devs ◦ Ourselves in the future ◦ Changes in 1ˢᵗ / 3ʳᵈ party dependencies • Ensure retrocompatibility & ease maintenance • Increase our confidence in the code
6. “Your code is meant to be executed. Testing is the best way to make sure it actually is executed.”
7. The different kinds of testing ♫
8. Unit testing
9. Integration Testing
10. Functional Testing
11. Test Driven Development
12. Code coverage
13. TL;DR; There are no good reasons not to test Why do you not write tests ? It’s 2018, and yet some people still find reasons not to test ?
14. Reasons not to test • I don’t need tests • I don’t have time ◦ I must ship the app today ◦ It takes too long to write tests • I don’t know how to write test ◦ I don’t know what to test ◦ I don’t know the tools
15. The ‘UNIT’ in Unit testing
16. What not to test ? • 3ʳᵈ party libraries (network, database, Rx, …) • Framework (Android / Linux, iOS, …) • Network / API • Language (Java/Kotlin, ObjC/Swift, toolchain, …) • Trivial code (getter/setters, …)
17. What to test ? • Decision logic (if, switch, when, …) • Input → Output • (De)serialization / transformation (manual only) • Callbacks events • Critical code
18. What’s a Unit ? The smallest piece of code which can be tested decoupled from all other code in the system. Could be a single method or a group of classes.
19. • Isolate the Unit ◦ Dependency Injection ◦ Extract irrelevant code to delegates / listeners • Write down the Unit’s purpose • Write down known (and unknown) use cases How to write Unit tests ?
20. • The good ◦ Proper inputs (“someone@mycompany.io”, 42, 3.1) ◦ System in a proper state • The bad ◦ Unusual inputs (“a."(),;"."c@d".e@[15.16.23.42]”, 4815162, 0.0000013) ◦ System in other state than expected • The ugly ◦ Unexpected inputs (null, -INFINITY, NaN, …) ◦ System in invalid state Consider all the edge cases
21. The Test Structure (AAA / GWT / SEVT) 1. Prepare the unit under test and test data 2. Execute the unit 3. Verify the outputs / state / callbacks
22. Test Conventions • Guides the writing of tests • Ensure coherence over the code base • Helps with readability → maintainability
23. The golden rules of (Unit) tests A test must be : • fast to execute; • reproducible; • reliable
24. TDD, the proper way
25. “The programmer should let correctness proof and program grow hand in hand.” Edsger W. Dijkstra, 1972
26. The TDD Loop Write the simplest test 1 Verify the test fails 2 Write the simplest code 3 Verify the test succeeds 4 Refactor. Enhance. Comment. Repeat. 5
27. Writing the test first makes you think about the solution. How is my Unit going to be used in the system ? Write the simplest test
28. Making sure the test fails first ensures the reliability of your test. Verify it fails
29. This helps only writing relevant code, instead of creating patterns that won’t be needed. Write the simplest code
30. Do I really need to explain this one? Verify the test succeeds
31. From this step, you know that your code and your test work. Don’t stop there. Refactor. Enhance. Comment.
32. Don’t stop at the first checkpoint. You can always add more test cases. You can always improve your code. Repeat !
33. When the tests go wrong
34. All my tests involved users named Bob. Aged 42. Working at FooBar Inc.
35. When all your fake data are “foo” and 42, how can you be sure that your tests are valid ?
36. Using fuzzy data in your tests • Random data • Help define boundaries of valid / invalid inputs • Help find edge cases
37. Elmyr libary class FooTest { @JvmField @Rule val forger = new JUnitForger() @Test fun testSomething() { val id = forger.anHexidecimalString() val email = forger.anEmail() val phone = forger.aStringMatching("\+\\d{1,3} \\d{12}") // … } }
38. <FooTest.testSomething()> failed with fake seed 0x4815162342 Add the following line in your setUp method to reproduce : forger.reset(0x4815162342L) Elmyr libary
39. https://github.com/xgouchet/Elmyr
40. About mocks Fake implementation of class / interface Can be driven to “mock” the behavior of a class
41. • Mocks are designed to make the tests pass • How can you tell if your mocks are consistent across your tests, or if they’re consistent with the real implementation ? How your mocks can go wrong
42. A standard test class class QueueContract { val mockedQueue : Queue = mock() fun prepareEmpty() { whenever(mockedQueue.isEmpty()).thenReturn(true) whenever(mockedQueue.getFirst().thenThrow(Exception()) } }
43. Code coverage doesn’t guarantee you test anything 100%
44. SUCCESS: 26/26 (100%) Tests passed — @bloerwald
45. Are your tests testing anything • TDD’s “fail first” ensure they do • You need at least one verify / assert / … statement
46. A code coverage of 20% doesn’t mean 20% of your code is tested. It means at least 80% of your code is not tested. Xavier F. Gouchet
47. Practical Process
48. Create a tests
49. Example fun computeAge_ok() { val user = User(name = “Bob”, birthDate = “20/04/1984”) val age = computeAge(user) assert(age == 34) }
50. Example fun computeAge_impossible() { val user = User(name = “Bob”, birthDate = “20/04/2084”) val age = try { computeAge(fakeUser) } catch (e: Exception) { e } assert(age is Exception) }
51. Quick Test Actions
52. During development time • Launch locally regularly ◦ Avoid PR validation failure ◦ Make it a coding habit • Run all tests and static analysis on each PR ◦ Travis, Bitrise, Circle CI, Jenkins, …
53. Frameworks (Android) • Unit Testing / Integration Testing ◦ Java : JUnit ◦ Kotlin : Spek, KotlinTest ◦ Mocking : Mockito, EasyMock, Mockk ◦ Fuzzy Testing : Elmyr • Functional testing : ◦ Espresso, Robotium, Appium
54. Rules of thumb
55. “A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in.” John D. Carmack
56. for loop • count = 0 • count < 0 • count = 1 • count = 10 • count = 65536 for (i in 0..count) { // … }
57. for loop • null list • empty list • list size = 1 • list size = 7 • list size = 65536 • sorted / unsorted ? • duplicate values / ref ? for (item in list) { // … }
58. if • a = true; b = true • a = true; b = false • a = false; b = true • a = false; b = false if (a || !b) { // … }
59. if … else if • a = true; b = true • a = true; b = false • a = false; b = true • a = false; b = false if (a) { // … } else if (b) { // … }
60. • All known cases • null • all/multiple ‘else’ cases switch / when when (dow) { MONDAY -> {} // … else -> {} }
61. • Combinations of : ◦ i < 0 ? i == 0 ? i > 0 ◦ s = null ? s = “” ? s = “नम ते” ◦ f < 0 ? f = NaN ? f = 0.000001 ? function parameters fun x(i: Int, s: String?, f: Float): Any
62. Conclusion
63. The proper mindset • Bugs are not failure but insights When you fix a bug, write a test for it • Grow your code and tests hand in hand… … one step at a time • Always try to think about what can go wrong • Test is code ◦ Use patterns, inheritance, … ◦ It can have bugs too
64. “You need to be as confident in the tests we code as we are in the code we test.” Xavier F. Gouchet
65. Going further • Look at your platform’s best practices and libraries • Lots of patterns to explore : ◦ Mutation / Fault Injection testing ◦ Property based testing ◦ Behavior Driven Development ◦ Automated Contract Testing
66. Thanks for your attention Any question ? @xgouchet http://github.com/xgouchet/Elmyr