Upgrade to Pro — share decks privately, control downloads, hide ads and more …

It's time to up your test game

It's time to up your test game

On the paper, everyone agrees that tests are important. And yet, more often than none, tests are still considered the poor cousin in the development industry. In the rare case when we actually take time to write them, we more often than not consider it a cumbersome task, and give it less thoughts than the "real code".

But tests are often as important, if not more, as the code that will be shipped on the Play Store. After briefly explaining the basics of testing, this talk will go through various common mistakes and bad practices found in real-life unit tests, and will give techniques and tips to make those tests cleaner and better.

Xavier Gouchet

October 09, 2018
Tweet

More Decks by Xavier Gouchet

Other Decks in Programming

Transcript

  1. It's time to step up
    your test game
    Apps Conf 2018

    View Slide

  2. About me…
    ● Xavier F. Gouchet
    ● Writing Android apps since Android 1.5 (Cupcake)
    ● Lead Android Engineer at WorkWell
    ● @xgouchet on Github / StackOverflow / LinkedIn

    View Slide

  3. View Slide

  4. Testing : the basics

    View Slide

  5. What are the goals of tests ?
    ● “Verify that the code works”
    ● Make the contract explicit
    ● Guide the development (TDD)
    ● Prevent regression caused by
    ○ Other devs
    ○ Ourselves in the future
    ○ Changes in 1ˢᵗ / 3ʳᵈ party dependencies
    ● Ensure retrocompatibility & ease maintenance
    ● Increase our confidence in the code

    View Slide

  6. “Your code is meant to be
    executed.
    Testing is the best way to make
    sure it actually is executed.”

    View Slide

  7. The different kinds of testing ♫

    View Slide

  8. Unit testing

    View Slide

  9. Integration Testing

    View Slide

  10. Functional Testing

    View Slide

  11. Test Driven Development

    View Slide

  12. Code coverage

    View Slide

  13. TL;DR;
    There are no good reasons not
    to test
    Why do you not
    write tests ?
    It’s 2018, and yet some people still find
    reasons not to test ?

    View Slide

  14. Reasons not to test
    ● I don’t need tests
    ● I don’t have time
    ○ I must ship the app today
    ○ It takes too long to write tests
    ● I don’t know how to write test
    ○ I don’t know what to test
    ○ I don’t know the tools

    View Slide

  15. The ‘UNIT’ in Unit testing

    View Slide

  16. What not to test ?
    ● 3ʳᵈ party libraries (network, database, Rx, …)
    ● Framework (Android / Linux, iOS, …)
    ● Network / API
    ● Language (Java/Kotlin, ObjC/Swift, toolchain, …)
    ● Trivial code (getter/setters, …)

    View Slide

  17. What to test ?
    ● Decision logic (if, switch, when, …)
    ● Input → Output
    ● (De)serialization / transformation (manual only)
    ● Callbacks events
    ● Critical code

    View Slide

  18. What’s a Unit ?
    The smallest piece of code
    which can be tested decoupled
    from all other code in the
    system.
    Could be a single method or a
    group of classes.

    View Slide

  19. ● Isolate the Unit
    ○ Dependency Injection
    ○ Extract irrelevant code to delegates / listeners
    ● Write down the Unit’s purpose
    ● Write down known (and unknown) use cases
    How to write Unit tests ?

    View Slide

  20. ● The good
    ○ Proper inputs (“[email protected]”, 42, 3.1)
    ○ System in a proper state
    ● The bad
    ○ Unusual inputs (“a."(),;"."c@d".e@[15.16.23.42]”, 4815162, 0.0000013)
    ○ System in other state than expected
    ● The ugly
    ○ Unexpected inputs (null, -INFINITY, NaN, …)
    ○ System in invalid state
    Consider all the edge cases

    View Slide

  21. The Test Structure (AAA / GWT / SEVT)
    1. Prepare the unit under test and test data
    2. Execute the unit
    3. Verify the outputs / state / callbacks

    View Slide

  22. Test Conventions
    ● Guides the writing of tests
    ● Ensure coherence over the code base
    ● Helps with readability → maintainability

    View Slide

  23. The golden rules
    of (Unit) tests
    A test must be :
    ● fast to execute;
    ● reproducible;
    ● reliable

    View Slide

  24. TDD, the proper way

    View Slide

  25. “The programmer should let
    correctness proof and program
    grow hand in hand.”
    Edsger W. Dijkstra, 1972

    View Slide

  26. The TDD Loop
    Write the
    simplest test
    1
    Verify the
    test fails
    2
    Write the
    simplest code
    3
    Verify the
    test succeeds
    4
    Refactor. Enhance.
    Comment. Repeat.
    5

    View Slide

  27. Writing the test first makes you
    think about the solution.
    How is my Unit going to be
    used in the system ?
    Write the
    simplest test

    View Slide

  28. Making sure the test fails first
    ensures the reliability of your
    test.
    Verify it fails

    View Slide

  29. This helps only writing relevant
    code, instead of creating
    patterns that won’t be needed.
    Write the
    simplest code

    View Slide

  30. Do I really need to explain this
    one?
    Verify the test
    succeeds

    View Slide

  31. From this step, you know that
    your code and your test work.
    Don’t stop there.
    Refactor.
    Enhance.
    Comment.

    View Slide

  32. Don’t stop at the first
    checkpoint.
    You can always add more test
    cases. You can always improve
    your code.
    Repeat !

    View Slide

  33. When the tests go wrong

    View Slide

  34. All my tests involved
    users named Bob.
    Aged 42.
    Working at FooBar Inc.

    View Slide

  35. When all your fake data
    are “foo” and 42, how
    can you be sure that
    your tests are valid ?

    View Slide

  36. Using fuzzy data in your tests
    ● Random data
    ● Help define boundaries of valid / invalid inputs
    ● Help find edge cases

    View Slide

  37. Elmyr libary
    class FooTest {
    @JvmField @Rule val forger = new JUnitForger()
    @Test fun testSomething() {
    val id = forger.anHexidecimalString()
    val email = forger.anEmail()
    val phone = forger.aStringMatching("\+\\d{1,3} \\d{12}")
    // …
    }
    }

    View Slide

  38. failed with fake seed 0x4815162342
    Add the following line in your setUp method to reproduce :
    forger.reset(0x4815162342L)
    Elmyr libary

    View Slide

  39. https://github.com/xgouchet/Elmyr

    View Slide

  40. About mocks
    Fake implementation of class /
    interface
    Can be driven to “mock” the
    behavior of a class

    View Slide

  41. ● Mocks are designed to make the tests pass
    ● How can you tell if your mocks are consistent across your tests, or if
    they’re consistent with the real implementation ?
    How your mocks can go wrong

    View Slide

  42. A standard test class
    class QueueContract {
    val mockedQueue : Queue = mock()
    fun prepareEmpty() {
    whenever(mockedQueue.isEmpty()).thenReturn(true)
    whenever(mockedQueue.getFirst().thenThrow(Exception())
    }
    }

    View Slide

  43. Code coverage doesn’t guarantee you test anything
    100%

    View Slide

  44. SUCCESS: 26/26 (100%) Tests passed — @bloerwald

    View Slide

  45. Are your tests testing anything
    ● TDD’s “fail first” ensure they do
    ● You need at least one verify / assert / … statement

    View Slide

  46. A code coverage of 20% doesn’t
    mean 20% of your code is
    tested.
    It means at least 80% of your
    code is not tested.
    Xavier F. Gouchet

    View Slide

  47. Practical Process

    View Slide

  48. Create a tests

    View Slide

  49. Example
    fun computeAge_ok() {
    val user = User(name = “Bob”, birthDate = “20/04/1984”)
    val age = computeAge(user)
    assert(age == 34)
    }

    View Slide

  50. Example
    fun computeAge_impossible() {
    val user = User(name = “Bob”, birthDate = “20/04/2084”)
    val age = try {
    computeAge(fakeUser)
    } catch (e: Exception) { e }
    assert(age is Exception)
    }

    View Slide

  51. Quick Test Actions

    View Slide

  52. During development time
    ● Launch locally regularly
    ○ Avoid PR validation failure
    ○ Make it a coding habit
    ● Run all tests and static analysis on each PR
    ○ Travis, Bitrise, Circle CI, Jenkins, …

    View Slide

  53. Frameworks (Android)
    ● Unit Testing / Integration Testing
    ○ Java : JUnit
    ○ Kotlin : Spek, KotlinTest
    ○ Mocking : Mockito, EasyMock, Mockk
    ○ Fuzzy Testing : Elmyr
    ● Functional testing :
    ○ Espresso, Robotium, Appium

    View Slide

  54. Rules of thumb

    View Slide

  55. “A large fraction of the flaws in
    software development are due to
    programmers not fully
    understanding all the possible
    states their code may execute
    in.”
    John D. Carmack

    View Slide

  56. for loop
    ● count = 0
    ● count < 0
    ● count = 1
    ● count = 10
    ● count = 65536
    for (i in 0..count) {
    // …
    }

    View Slide

  57. for loop
    ● null list
    ● empty list
    ● list size = 1
    ● list size = 7
    ● list size = 65536
    ● sorted / unsorted ?
    ● duplicate values / ref ?
    for (item in list) {
    // …
    }

    View Slide

  58. if
    ● a = true; b = true
    ● a = true; b = false
    ● a = false; b = true
    ● a = false; b = false
    if (a || !b) {
    // …
    }

    View Slide

  59. if … else if
    ● a = true; b = true
    ● a = true; b = false
    ● a = false; b = true
    ● a = false; b = false
    if (a) {
    // …
    } else if (b) {
    // …
    }

    View Slide

  60. ● All known cases
    ● null
    ● all/multiple ‘else’ cases
    switch / when
    when (dow) {
    MONDAY -> {}
    // …
    else -> {}
    }

    View Slide

  61. ● Combinations of :
    ○ i < 0 ? i == 0 ? i > 0
    ○ s = null ? s = “” ? s = “नम ते”
    ○ f < 0 ? f = NaN ? f = 0.000001 ?
    function
    parameters
    fun x(i: Int,
    s: String?,
    f: Float): Any

    View Slide

  62. Conclusion

    View Slide

  63. The proper mindset
    ● Bugs are not failure but insights
    When you fix a bug, write a test for it
    ● Grow your code and tests hand in hand…
    … one step at a time
    ● Always try to think about what can go wrong
    ● Test is code
    ○ Use patterns, inheritance, …
    ○ It can have bugs too

    View Slide

  64. “You need to be as confident in
    the tests we code as we are in
    the code we test.”
    Xavier F. Gouchet

    View Slide

  65. Going further
    ● Look at your platform’s best practices and libraries
    ● Lots of patterns to explore :
    ○ Mutation / Fault Injection testing
    ○ Property based testing
    ○ Behavior Driven Development
    ○ Automated Contract Testing

    View Slide

  66. Thanks for your
    attention Any question ?
    @xgouchet
    http://github.com/xgouchet/Elmyr

    View Slide