It's time to up your test game

On the paper, everyone agrees that tests are important. And yet, more often than none, tests are still considered the poor cousin in the development industry. In the rare case when we actually take time to write them, we more often than not consider it a cumbersome task, and give it less thoughts than the "real code".

But tests are often as important, if not more, as the code that will be shipped on the Play Store. After briefly explaining the basics of testing, this talk will go through various common mistakes and bad practices found in real-life unit tests, and will give techniques and tips to make those tests cleaner and better.

Transcript

1. It's time to step up
   your test game
   Apps Conf 2018
   

   View Slide

2. About me…
   ● Xavier F. Gouchet
   ● Writing Android apps since Android 1.5 (Cupcake)
   ● Lead Android Engineer at WorkWell
   ● @xgouchet on Github / StackOverflow / LinkedIn
   

   View Slide

3. View Slide

4. Testing : the basics
   

   View Slide

5. What are the goals of tests ?
   ● “Verify that the code works”
   ● Make the contract explicit
   ● Guide the development (TDD)
   ● Prevent regression caused by
   ○ Other devs
   ○ Ourselves in the future
   ○ Changes in 1ˢᵗ / 3ʳᵈ party dependencies
   ● Ensure retrocompatibility & ease maintenance
   ● Increase our confidence in the code
   

   View Slide

6. “Your code is meant to be
   executed.
   Testing is the best way to make
   sure it actually is executed.”
   

   View Slide

7. The different kinds of testing ♫
   

   View Slide

8. Unit testing
   

   View Slide

9. Integration Testing
   

   View Slide

10. Functional Testing
    

    View Slide

11. Test Driven Development
    

    View Slide

12. Code coverage
    

    View Slide

13. TL;DR;
    There are no good reasons not
    to test
    Why do you not
    write tests ?
    It’s 2018, and yet some people still find
    reasons not to test ?
    

    View Slide

14. Reasons not to test
    ● I don’t need tests
    ● I don’t have time
    ○ I must ship the app today
    ○ It takes too long to write tests
    ● I don’t know how to write test
    ○ I don’t know what to test
    ○ I don’t know the tools
    

    View Slide

15. The ‘UNIT’ in Unit testing
    

    View Slide

16. What not to test ?
    ● 3ʳᵈ party libraries (network, database, Rx, …)
    ● Framework (Android / Linux, iOS, …)
    ● Network / API
    ● Language (Java/Kotlin, ObjC/Swift, toolchain, …)
    ● Trivial code (getter/setters, …)
    

    View Slide

17. What to test ?
    ● Decision logic (if, switch, when, …)
    ● Input → Output
    ● (De)serialization / transformation (manual only)
    ● Callbacks events
    ● Critical code
    

    View Slide

18. What’s a Unit ?
    The smallest piece of code
    which can be tested decoupled
    from all other code in the
    system.
    Could be a single method or a
    group of classes.
    

    View Slide

19. ● Isolate the Unit
    ○ Dependency Injection
    ○ Extract irrelevant code to delegates / listeners
    ● Write down the Unit’s purpose
    ● Write down known (and unknown) use cases
    How to write Unit tests ?
    

    View Slide

20. ● The good
    ○ Proper inputs (“someone@mycompany.io”, 42, 3.1)
    ○ System in a proper state
    ● The bad
    ○ Unusual inputs (“a."(),;"."c@d".e@[15.16.23.42]”, 4815162, 0.0000013)
    ○ System in other state than expected
    ● The ugly
    ○ Unexpected inputs (null, -INFINITY, NaN, …)
    ○ System in invalid state
    Consider all the edge cases
    

    View Slide

21. The Test Structure (AAA / GWT / SEVT)
    1. Prepare the unit under test and test data
    2. Execute the unit
    3. Verify the outputs / state / callbacks
    

    View Slide

22. Test Conventions
    ● Guides the writing of tests
    ● Ensure coherence over the code base
    ● Helps with readability → maintainability
    

    View Slide

23. The golden rules
    of (Unit) tests
    A test must be :
    ● fast to execute;
    ● reproducible;
    ● reliable
    

    View Slide

24. TDD, the proper way
    

    View Slide

25. “The programmer should let
    correctness proof and program
    grow hand in hand.”
    Edsger W. Dijkstra, 1972
    

    View Slide

26. The TDD Loop
    Write the
    simplest test
    1
    Verify the
    test fails
    2
    Write the
    simplest code
    3
    Verify the
    test succeeds
    4
    Refactor. Enhance.
    Comment. Repeat.
    5
    

    View Slide

27. Writing the test first makes you
    think about the solution.
    How is my Unit going to be
    used in the system ?
    Write the
    simplest test
    

    View Slide

28. Making sure the test fails first
    ensures the reliability of your
    test.
    Verify it fails
    

    View Slide

29. This helps only writing relevant
    code, instead of creating
    patterns that won’t be needed.
    Write the
    simplest code
    

    View Slide

30. Do I really need to explain this
    one?
    Verify the test
    succeeds
    

    View Slide

31. From this step, you know that
    your code and your test work.
    Don’t stop there.
    Refactor.
    Enhance.
    Comment.
    

    View Slide

32. Don’t stop at the first
    checkpoint.
    You can always add more test
    cases. You can always improve
    your code.
    Repeat !
    

    View Slide

33. When the tests go wrong
    

    View Slide

34. All my tests involved
    users named Bob.
    Aged 42.
    Working at FooBar Inc.
    

    View Slide

35. When all your fake data
    are “foo” and 42, how
    can you be sure that
    your tests are valid ?
    

    View Slide

36. Using fuzzy data in your tests
    ● Random data
    ● Help define boundaries of valid / invalid inputs
    ● Help find edge cases
    

    View Slide

37. Elmyr libary
    class FooTest {
    @JvmField @Rule val forger = new JUnitForger()
    @Test fun testSomething() {
    val id = forger.anHexidecimalString()
    val email = forger.anEmail()
    val phone = forger.aStringMatching("\+\\d{1,3} \\d{12}")
    // …
    }
    }
    

    View Slide

38. failed with fake seed 0x4815162342
    Add the following line in your setUp method to reproduce :
    forger.reset(0x4815162342L)
    Elmyr libary
    

    View Slide

39. https://github.com/xgouchet/Elmyr
    

    View Slide

40. About mocks
    Fake implementation of class /
    interface
    Can be driven to “mock” the
    behavior of a class
    

    View Slide

41. ● Mocks are designed to make the tests pass
    ● How can you tell if your mocks are consistent across your tests, or if
    they’re consistent with the real implementation ?
    How your mocks can go wrong
    

    View Slide

42. A standard test class
    class QueueContract {
    val mockedQueue : Queue = mock()
    fun prepareEmpty() {
    whenever(mockedQueue.isEmpty()).thenReturn(true)
    whenever(mockedQueue.getFirst().thenThrow(Exception())
    }
    }
    

    View Slide

43. Code coverage doesn’t guarantee you test anything
    100%
    

    View Slide

44. SUCCESS: 26/26 (100%) Tests passed — @bloerwald
    

    View Slide

45. Are your tests testing anything
    ● TDD’s “fail first” ensure they do
    ● You need at least one verify / assert / … statement
    

    View Slide

46. A code coverage of 20% doesn’t
    mean 20% of your code is
    tested.
    It means at least 80% of your
    code is not tested.
    Xavier F. Gouchet
    

    View Slide

47. Practical Process
    

    View Slide

48. Create a tests
    

    View Slide

49. Example
    fun computeAge_ok() {
    val user = User(name = “Bob”, birthDate = “20/04/1984”)
    val age = computeAge(user)
    assert(age == 34)
    }
    

    View Slide

50. Example
    fun computeAge_impossible() {
    val user = User(name = “Bob”, birthDate = “20/04/2084”)
    val age = try {
    computeAge(fakeUser)
    } catch (e: Exception) { e }
    assert(age is Exception)
    }
    

    View Slide

51. Quick Test Actions
    

    View Slide

52. During development time
    ● Launch locally regularly
    ○ Avoid PR validation failure
    ○ Make it a coding habit
    ● Run all tests and static analysis on each PR
    ○ Travis, Bitrise, Circle CI, Jenkins, …
    

    View Slide

53. Frameworks (Android)
    ● Unit Testing / Integration Testing
    ○ Java : JUnit
    ○ Kotlin : Spek, KotlinTest
    ○ Mocking : Mockito, EasyMock, Mockk
    ○ Fuzzy Testing : Elmyr
    ● Functional testing :
    ○ Espresso, Robotium, Appium
    

    View Slide

54. Rules of thumb
    

    View Slide

55. “A large fraction of the flaws in
    software development are due to
    programmers not fully
    understanding all the possible
    states their code may execute
    in.”
    John D. Carmack
    

    View Slide

56. for loop
    ● count = 0
    ● count < 0
    ● count = 1
    ● count = 10
    ● count = 65536
    for (i in 0..count) {
    // …
    }
    

    View Slide

57. for loop
    ● null list
    ● empty list
    ● list size = 1
    ● list size = 7
    ● list size = 65536
    ● sorted / unsorted ?
    ● duplicate values / ref ?
    for (item in list) {
    // …
    }
    

    View Slide

58. if
    ● a = true; b = true
    ● a = true; b = false
    ● a = false; b = true
    ● a = false; b = false
    if (a || !b) {
    // …
    }
    

    View Slide

59. if … else if
    ● a = true; b = true
    ● a = true; b = false
    ● a = false; b = true
    ● a = false; b = false
    if (a) {
    // …
    } else if (b) {
    // …
    }
    

    View Slide

60. ● All known cases
    ● null
    ● all/multiple ‘else’ cases
    switch / when
    when (dow) {
    MONDAY -> {}
    // …
    else -> {}
    }
    

    View Slide

61. ● Combinations of :
    ○ i < 0 ? i == 0 ? i > 0
    ○ s = null ? s = “” ? s = “नम ते”
    ○ f < 0 ? f = NaN ? f = 0.000001 ?
    function
    parameters
    fun x(i: Int,
    s: String?,
    f: Float): Any
    

    View Slide

62. Conclusion
    

    View Slide

63. The proper mindset
    ● Bugs are not failure but insights
    When you fix a bug, write a test for it
    ● Grow your code and tests hand in hand…
    … one step at a time
    ● Always try to think about what can go wrong
    ● Test is code
    ○ Use patterns, inheritance, …
    ○ It can have bugs too
    

    View Slide

64. “You need to be as confident in
    the tests we code as we are in
    the code we test.”
    Xavier F. Gouchet
    

    View Slide

65. Going further
    ● Look at your platform’s best practices and libraries
    ● Lots of patterns to explore :
    ○ Mutation / Fault Injection testing
    ○ Property based testing
    ○ Behavior Driven Development
    ○ Automated Contract Testing
    

    View Slide

66. Thanks for your
    attention Any question ?
    @xgouchet
    http://github.com/xgouchet/Elmyr
    

    View Slide