It's time to up your test game

It's time to up your test game

On the paper, everyone agrees that tests are important. And yet, more often than none, tests are still considered the poor cousin in the development industry. In the rare case when we actually take time to write them, we more often than not consider it a cumbersome task, and give it less thoughts than the "real code".

But tests are often as important, if not more, as the code that will be shipped on the Play Store. After briefly explaining the basics of testing, this talk will go through various common mistakes and bad practices found in real-life unit tests, and will give techniques and tips to make those tests cleaner and better.

Cf95f93e78f6d6dd0630049396f723c6?s=128

Xavier Gouchet

October 09, 2018
Tweet

Transcript

  1. 2.

    About me… • Xavier F. Gouchet • Writing Android apps

    since Android 1.5 (Cupcake) • Lead Android Engineer at WorkWell • @xgouchet on Github / StackOverflow / LinkedIn
  2. 3.
  3. 5.

    What are the goals of tests ? • “Verify that

    the code works” • Make the contract explicit • Guide the development (TDD) • Prevent regression caused by ◦ Other devs ◦ Ourselves in the future ◦ Changes in 1ˢᵗ / 3ʳᵈ party dependencies • Ensure retrocompatibility & ease maintenance • Increase our confidence in the code
  4. 6.

    “Your code is meant to be executed. Testing is the

    best way to make sure it actually is executed.”
  5. 13.

    TL;DR; There are no good reasons not to test Why

    do you not write tests ? It’s 2018, and yet some people still find reasons not to test ?
  6. 14.

    Reasons not to test • I don’t need tests •

    I don’t have time ◦ I must ship the app today ◦ It takes too long to write tests • I don’t know how to write test ◦ I don’t know what to test ◦ I don’t know the tools
  7. 16.

    What not to test ? • 3ʳᵈ party libraries (network,

    database, Rx, …) • Framework (Android / Linux, iOS, …) • Network / API • Language (Java/Kotlin, ObjC/Swift, toolchain, …) • Trivial code (getter/setters, …)
  8. 17.

    What to test ? • Decision logic (if, switch, when,

    …) • Input → Output • (De)serialization / transformation (manual only) • Callbacks events • Critical code
  9. 18.

    What’s a Unit ? The smallest piece of code which

    can be tested decoupled from all other code in the system. Could be a single method or a group of classes.
  10. 19.

    • Isolate the Unit ◦ Dependency Injection ◦ Extract irrelevant

    code to delegates / listeners • Write down the Unit’s purpose • Write down known (and unknown) use cases How to write Unit tests ?
  11. 20.

    • The good ◦ Proper inputs (“someone@mycompany.io”, 42, 3.1) ◦

    System in a proper state • The bad ◦ Unusual inputs (“a."(),;"."c@d".e@[15.16.23.42]”, 4815162, 0.0000013) ◦ System in other state than expected • The ugly ◦ Unexpected inputs (null, -INFINITY, NaN, …) ◦ System in invalid state Consider all the edge cases
  12. 21.

    The Test Structure (AAA / GWT / SEVT) 1. Prepare

    the unit under test and test data 2. Execute the unit 3. Verify the outputs / state / callbacks
  13. 22.

    Test Conventions • Guides the writing of tests • Ensure

    coherence over the code base • Helps with readability → maintainability
  14. 23.

    The golden rules of (Unit) tests A test must be

    : • fast to execute; • reproducible; • reliable
  15. 26.

    The TDD Loop Write the simplest test 1 Verify the

    test fails 2 Write the simplest code 3 Verify the test succeeds 4 Refactor. Enhance. Comment. Repeat. 5
  16. 27.

    Writing the test first makes you think about the solution.

    How is my Unit going to be used in the system ? Write the simplest test
  17. 29.

    This helps only writing relevant code, instead of creating patterns

    that won’t be needed. Write the simplest code
  18. 31.

    From this step, you know that your code and your

    test work. Don’t stop there. Refactor. Enhance. Comment.
  19. 32.

    Don’t stop at the first checkpoint. You can always add

    more test cases. You can always improve your code. Repeat !
  20. 35.

    When all your fake data are “foo” and 42, how

    can you be sure that your tests are valid ?
  21. 36.

    Using fuzzy data in your tests • Random data •

    Help define boundaries of valid / invalid inputs • Help find edge cases
  22. 37.

    Elmyr libary class FooTest { @JvmField @Rule val forger =

    new JUnitForger() @Test fun testSomething() { val id = forger.anHexidecimalString() val email = forger.anEmail() val phone = forger.aStringMatching("\+\\d{1,3} \\d{12}") // … } }
  23. 38.

    <FooTest.testSomething()> failed with fake seed 0x4815162342 Add the following line

    in your setUp method to reproduce : forger.reset(0x4815162342L) Elmyr libary
  24. 40.

    About mocks Fake implementation of class / interface Can be

    driven to “mock” the behavior of a class
  25. 41.

    • Mocks are designed to make the tests pass •

    How can you tell if your mocks are consistent across your tests, or if they’re consistent with the real implementation ? How your mocks can go wrong
  26. 42.

    A standard test class class QueueContract { val mockedQueue :

    Queue = mock() fun prepareEmpty() { whenever(mockedQueue.isEmpty()).thenReturn(true) whenever(mockedQueue.getFirst().thenThrow(Exception()) } }
  27. 45.

    Are your tests testing anything • TDD’s “fail first” ensure

    they do • You need at least one verify / assert / … statement
  28. 46.

    A code coverage of 20% doesn’t mean 20% of your

    code is tested. It means at least 80% of your code is not tested. Xavier F. Gouchet
  29. 49.

    Example fun computeAge_ok() { val user = User(name = “Bob”,

    birthDate = “20/04/1984”) val age = computeAge(user) assert(age == 34) }
  30. 50.

    Example fun computeAge_impossible() { val user = User(name = “Bob”,

    birthDate = “20/04/2084”) val age = try { computeAge(fakeUser) } catch (e: Exception) { e } assert(age is Exception) }
  31. 52.

    During development time • Launch locally regularly ◦ Avoid PR

    validation failure ◦ Make it a coding habit • Run all tests and static analysis on each PR ◦ Travis, Bitrise, Circle CI, Jenkins, …
  32. 53.

    Frameworks (Android) • Unit Testing / Integration Testing ◦ Java

    : JUnit ◦ Kotlin : Spek, KotlinTest ◦ Mocking : Mockito, EasyMock, Mockk ◦ Fuzzy Testing : Elmyr • Functional testing : ◦ Espresso, Robotium, Appium
  33. 55.

    “A large fraction of the flaws in software development are

    due to programmers not fully understanding all the possible states their code may execute in.” John D. Carmack
  34. 56.

    for loop • count = 0 • count < 0

    • count = 1 • count = 10 • count = 65536 for (i in 0..count) { // … }
  35. 57.

    for loop • null list • empty list • list

    size = 1 • list size = 7 • list size = 65536 • sorted / unsorted ? • duplicate values / ref ? for (item in list) { // … }
  36. 58.

    if • a = true; b = true • a

    = true; b = false • a = false; b = true • a = false; b = false if (a || !b) { // … }
  37. 59.

    if … else if • a = true; b =

    true • a = true; b = false • a = false; b = true • a = false; b = false if (a) { // … } else if (b) { // … }
  38. 60.

    • All known cases • null • all/multiple ‘else’ cases

    switch / when when (dow) { MONDAY -> {} // … else -> {} }
  39. 61.

    • Combinations of : ◦ i < 0 ? i

    == 0 ? i > 0 ◦ s = null ? s = “” ? s = “नम ते” ◦ f < 0 ? f = NaN ? f = 0.000001 ? function parameters fun x(i: Int, s: String?, f: Float): Any
  40. 63.

    The proper mindset • Bugs are not failure but insights

    When you fix a bug, write a test for it • Grow your code and tests hand in hand… … one step at a time • Always try to think about what can go wrong • Test is code ◦ Use patterns, inheritance, … ◦ It can have bugs too
  41. 64.

    “You need to be as confident in the tests we

    code as we are in the code we test.” Xavier F. Gouchet
  42. 65.

    Going further • Look at your platform’s best practices and

    libraries • Lots of patterns to explore : ◦ Mutation / Fault Injection testing ◦ Property based testing ◦ Behavior Driven Development ◦ Automated Contract Testing