Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Effective Android Testing - A value driven approach

Effective Android Testing - A value driven approach

Companion slides from my talk with same name.

Delivered in the following events

- PicPay Special Webinar (Online) - October/2021

Ubiratan Soares
PRO

October 27, 2021
Tweet

More Decks by Ubiratan Soares

Other Decks in Technology

Transcript

  1. A value-driven approach
    Ubiratan Soares
    October / 2021
    EFFECTIVE
    ANDROID TESTING

    View Slide

  2. CHALLENGES
    FROM TESTING
    ANDROID APPS

    View Slide

  3. Testing can be hard

    View Slide

  4. Testing can be hard
    For Android projects,
    it can be even harder

    View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. CONCEPTS
    REVIEW

    View Slide

  9. Dimensions of testing
    • Which type of code a test touches
    • Which boundaries of the system it touches
    • Which real flows a test exercises
    • Which level of isolation are in place (if any)
    • Etc

    View Slide

  10. Unit
    Integration
    E2E
    +
    +
    Value Ensured
    Complexity Involved

    View Slide

  11. Subject Under Test
    Input Output
    Classical understanding
    Single Class or Single Function

    View Slide

  12. Entry point Exit point
    Not necessarily 1:1 relationship to classes / functions
    SUT

    View Slide

  13. https://youtu.be/6ndAWzc2F-I

    View Slide

  14. Entry point Exit point
    SUT
    Subject under test is not supposed to be the whole thing

    View Slide

  15. Entry point Exit point
    SUT
    Boundary
    Owned by
    Others
    Owned by
    Others
    Abstractions you own
    within the app boundaries
    Boundary

    View Slide

  16. Unit tests
    • Well-defined single entry point
    • Well-defined single exit point
    • Usually a small subject
    • Exercise abstractions you own
    • Not necessary 1:1 relationship to classes / functions
    • Never touches infrastructure ⚠

    View Slide

  17. View Slide

  18. Entry point
    Boundary
    Owned by
    Others
    Owned by
    You
    Exit point

    View Slide

  19. Entry point
    Boundary
    Owned by
    Others
    Owned by
    You
    Boundary
    Owned by
    Others
    Exit point

    View Slide

  20. Entry point
    Boundary
    Owned by
    Others
    Owned by
    You
    Boundary
    Owned by
    Others
    Exit point

    View Slide

  21. Entry point
    Boundary
    Owned by
    Others
    Owned by
    You
    Boundary
    Owned by
    Others
    Exit point

    View Slide

  22. View Slide

  23. Integration tests
    • Exercise entry points and/or exit points beyond your
    application boundaries
    • Always enforce some degree of isolation (ideally fully isolated)
    • Categorized according the number of integrations under test
    • Categorized according user-value under test

    View Slide

  24. Integration Level Rationale
    Single Exercises only one well-defined system boundary
    Multiple Exercises several well-defined system boundaries
    Component Exercises one well-defined component (user perspective)
    Acceptance Exercises several components with real user flows

    View Slide

  25. E2E tests
    • Black-box instrumentation over a deliverable artefact
    • Exercise user flows over a pre-production environment
    • The most expensive ones to write + maintain

    View Slide

  26. AN
    EFFECTIVE
    STRATEGY

    View Slide

  27. Expectations 😎
    Code

    View Slide

  28. Reality 😅
    Code

    View Slide

  29. Bussines Rules App
    Micro
    service
    Nano
    service
    Monolith

    View Slide

  30. App
    Realization of value
    Enabler of value
    !=
    To avoid

    View Slide

  31. app Product
    onboarding
    login
    search

    Features
    Platform
    core-persistance
    core-rest
    core-location
    core-translation
    common-android
    common-di
    common-assets

    View Slide

  32. Entrypoint / Glue
    Features
    Platform
    Codebase contribution
    ** Practical observations from previous projects

    View Slide

  33. “If the vast majority of a
    codebase from a Mobile
    product deals with coordination
    over middleware and/or
    infrastructure, how do we do
    unit testing ?”

    View Slide

  34. “Write tests.
    Not too many.
    Mostly integration”
    Kent Dodds
    https://kentcdodds.com/blog/write-tests

    View Slide

  35. https://youtu.be/Fha2bVoC8SE

    View Slide

  36. https://martinfowler.com/articles/2021-test-shapes.html

    View Slide

  37. Integration
    Unit
    E2E
    Contribution to Product Codebase
    Contribution to Test Suite

    View Slide

  38. Product
    Features
    Platform
    Unit tests (stable core)
    Integration tests
    Unit tests (on-demand)
    Contract tests
    Component tests
    Screenshot tests
    E2E tests
    Acceptance tests

    View Slide

  39. Unit Tests
    that bring
    value
    Tests for your
    implementation for HTTP
    authentication (eg, Oauth)

    View Slide

  40. Unit Tests
    that bring
    value
    Tests for your retry-
    strategy over HTTP (eg,
    exponential backoff)

    View Slide

  41. Unit Tests
    that bring
    value
    Tests over non-trivial
    validations over data
    provided by users via your
    App (eg, regexes)

    View Slide

  42. Unit Tests
    that bring
    value
    Tests over non-trivial data
    parsing that happens at Mobile
    level (eg URI / DeepLinks)

    View Slide

  43. Unit Tests
    that don’t
    bring value
    Tests over REST integration
    by mocking a Retrofit API

    View Slide

  44. Unit Tests
    that don’t
    bring value
    Tests over ViewModels by
    mocking its colaborators

    View Slide

  45. Unit Tests
    that don’t
    bring value
    Tests over trivial data
    mappers used within your
    abstractions

    View Slide

  46. Unit Tests
    that don’t
    bring value
    Tests over sandwitch layers

    View Slide

  47. Are you ready to let
    Mockito go ???
    Android Engineer

    View Slide

  48. Integrated
    tests that
    bring value
    Comprehenstive tests over
    data sources, exercising
    thirdy-party transformation /
    serialization engines
    alongside with your own error
    handlers and/or data
    transformers

    View Slide

  49. class FactsDataSourceTests {
    @get:Rule val restInfrastructure = RestInfrastructureRule()
    private lateinit var dataSource: FactsDataSource
    @Before fun `before each test`() {
    val api = restInfrastructure.server.wireRestApi()
    dataSource = FactsDataSource(api)
    }
    @Test fun `should handle no results properly`() {
    restInfrastructure.restScenario(
    status = 200,
    response = loadFile("200_search_no_results.json")
    )
    val noFacts = emptyList()
    assertThat(simpleSearch()).isEqualTo(noFacts)
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  50. class FactsDataSourceTests {
    @get:Rule val restInfrastructure = RestInfrastructureRule()
    private lateinit var dataSource: FactsDataSource
    @Before fun `before each test`() {
    val api = restInfrastructure.server.wireRestApi()
    dataSource = FactsDataSource(api)
    }
    @Test fun `should handle no results properly`() {
    restInfrastructure.restScenario(
    status = 200,
    response = loadFile("200_search_no_results.json")
    )
    val noFacts = emptyList()
    assertThat(simpleSearch()).isEqualTo(noFacts)
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  51. class FactsDataSourceTests {
    @get:Rule val restInfrastructure = RestInfrastructureRule()
    private lateinit var dataSource: FactsDataSource
    @Before fun `before each test`() {
    val api = restInfrastructure.server.wireRestApi()
    dataSource = FactsDataSource(api)
    }
    @Test fun `should handle no results properly`() {
    restInfrastructure.restScenario(
    status = 200,
    response = loadFile("200_search_no_results.json")
    )
    val noFacts = emptyList()
    assertThat(simpleSearch()).isEqualTo(noFacts)
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  52. @Test fun `should handle downstream error`() {
    restInfrastructure.restScenario(status = 500)
    assertErrorTransformed(
    whenRunning = this::simpleSearch,
    expected = RemoteServiceIntegrationError.RemoteSystem
    )
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  53. @Test fun `should fetch facts with valid query term`() {
    restInfrastructure.restScenario(
    status = 200,
    response = loadFile("200_search_with_results.json")
    )
    val facts = listOf(
    ChuckNorrisFact(
    id = "2wzginmks8azrbaxnamxdw",
    shareableUrl = "https://api.chucknorris.io/jokes/2wzginmks",
    textual = "Chuck Norris divides by zero”,
    )
    )
    assertThat(simpleSearch()).isEqualTo(facts)
    }
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  54. Integrated
    tests that
    bring value
    Component tests over entire
    screens, capturing the
    asynchonicity of data
    operations without coupling
    with implementation details
    (eg, Coroutines)

    View Slide

  55. Activity
    View

    View Slide

  56. Activity
    Screen Interface
    View or Wrapper

    View Slide

  57. Activity
    Screen Interface
    View or Wrapper Test Abstraction

    View Slide

  58. @RunWith(AndroidJUnit4::class)
    class FactsActivityTests {
    private lateinit var screen: FakeFactsScreen
    @get:Rule val restInfrastructure = RestInfrastructureRule()
    @Before fun `before each test`() {
    // Switching over fake screen wrapper with DI support
    val testApp = TestApplication.setupWith(
    factsModule,
    factsTestModule,
    RestInfrastructureTestModule(restInfrastructure.server)
    )
    screen = testApp.factsScreen()
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  59. @Test fun `at first lunch, should start on empty state`() {
    whenActivityResumed {
    assertThat(screen.isLinked).isTrue()
    val expectedStates = listOf(Idle, Loading, FactsScreenState.Empty)
    assertThat(screen.trackedStates).isEqualTo(expectedStates)
    }
    }
    // More tests
    @Test fun `when remote service fails, should display the error`() {
    restInfrastructure.restScenario(status = 503)
    PersistanceHelper.registerNewSearch("code")
    whenActivityResumed {
    val error = Failed(RemoteServiceIntegrationError.RemoteSystem)
    val expectedStates = listOf(Idle, Loading, error)
    assertThat(screen.trackedStates).isEqualTo(expectedStates)
    }
    }
    } https://github.com/dotanuki-labs/norris

    View Slide

  60. @Test fun `at first lunch, should start on empty state`() {
    whenActivityResumed {
    assertThat(screen.isLinked).isTrue()
    val expectedStates = listOf(Idle, Loading, FactsScreenState.Empty)
    assertThat(screen.trackedStates).isEqualTo(expectedStates)
    }
    }
    // More tests
    @Test fun `when remote service fails, should display the error`() {
    restInfrastructure.restScenario(status = 503)
    PersistanceHelper.registerNewSearch("code")
    whenActivityResumed {
    val error = Failed(RemoteServiceIntegrationError.RemoteSystem)
    val expectedStates = listOf(Idle, Loading, error)
    assertThat(screen.trackedStates).isEqualTo(expectedStates)
    }
    }
    } https://github.com/dotanuki-labs/norris

    View Slide

  61. Screenshots tests over your
    UI delivery, decoupled from
    any data operations
    Integrated
    tests that
    bring value

    View Slide

  62. Activity
    Screen Interface
    View or Wrapper Test Abstraction

    View Slide

  63. Activity
    Screen Interface
    View or Wrapper Test Abstraction
    TestActivity

    View Slide

  64. @LargeTest
    class FactsScreenshotTests : ScreenshotTest {
    private fun checkScreenshot(targetState: FactsScreenState) {
    val testActivity = prepareToCaptureScreenshot { target ->
    listOf(FactsScreenState.Idle, targetState).forEach {
    target.screen.updateWith(it)
    }
    }
    compareScreenshot(testActivity)
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  65. @Test fun emptyState() {
    checkScreenshot(Empty)
    }
    @Test fun errorState() {
    val error = RemoteServiceIntegrationError.RemoteSystem
    checkScreenshot(Failed(error))
    }
    @Test fun successState() {
    val facts = listOf(
    FactDisplayRow(
    url = "https://chucknorris.io/jokes/998877",
    fact = "Chuck Norris merges before commit anything on Git",
    displayWithSmallerFontSize = false
    )
    )
    val presentation = FactsPresentation("humor", facts)
    checkScreenshot(Success(presentation))
    }
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  66. @Test fun emptyState() {
    checkScreenshot(Empty)
    }
    @Test fun errorState() {
    val error = RemoteServiceIntegrationError.RemoteSystem
    checkScreenshot(Failed(error))
    }
    @Test fun successState() {
    val facts = listOf(
    FactDisplayRow(
    url = "https://chucknorris.io/jokes/998877",
    fact = "Chuck Norris merges before commit anything on Git",
    displayWithSmallerFontSize = false
    )
    )
    val presentation = FactsPresentation("humor", facts)
    checkScreenshot(Success(presentation))
    }
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  67. Integrated
    tests that
    bring value
    Acceptance tests over a fake
    ready-to-deliver artefact,
    exercising real user flows in
    isolation

    View Slide

  68. @RunWith(AndroidJUnit4!"class)
    class NorrisAcceptanceTests {
    private val restInfrastructure = RestInfrastructureRule(customPort = 4242)
    private val stressedExecution = FlakyTestRule()
    @get:Rule val rules: RuleChain =
    RuleChain.outerRule(stressedExecution).around(restInfrastructure)
    @Before fun beforeEachTest() {
    PersistanceHelper.clearStorage()
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  69. @Repeat @Test fun shouldPerformFirstSearch_AfterFirstLunch_ByTypingATerm() {
    val searchTerm = "math"
    val suggestions = listOf("career", "dev", "humor")
    val fact = "Chuck Norris can divide by zero"
    val suggestionsPayload = RestDataBuilder.suggestionsPayload(suggestions)
    val factsPayload = RestDataBuilder.factsPayload(searchTerm, fact)
    restInfrastructure.run {
    restScenario(200, suggestionsPayload)
    restScenario(200, factsPayload)
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  70. startingFrom {
    checkEmptyState()
    clickOnSearchIcon()
    awaitTransition()
    checkSuggestions(suggestions)
    performSearch(searchTerm)
    awaitTransition()
    checkDisplayed(fact)
    }
    }
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  71. fun Project.applyAndroidApplicationConventions() {
    applyAndroidStandardConventions()
    val android = extensions.findByName("android") as ApplicationExtension
    android.apply {
    testBuildType = when {
    isTestMode() -> "release"
    else -> "debug"
    }
    defaultConfig {
    testInstrumentationRunner = AndroidDefinitions.instrumentationTestRunner
    }
    // More plugin-driven conventions
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  72. fun Project.applyAndroidApplicationConventions() {
    applyAndroidStandardConventions()
    val android = extensions.findByName("android") as ApplicationExtension
    android.apply {
    testBuildType = when {
    isTestMode() -> "release"
    else -> "debug"
    }
    defaultConfig {
    testInstrumentationRunner = AndroidDefinitions.instrumentationTestRunner
    }
    // More plugin-driven conventions
    }
    https://github.com/dotanuki-labs/norris

    View Slide

  73. Integration Test Proposed Value
    Single integration over
    data sources
    Ensures data transformations + error handling
    over the target boundary
    Component Tests over
    entire screens
    Enables refactoring over target screen
    Screenshot Tests over
    entire screens
    Ensures association between state and Views
    without any assertions
    Acceptance
    Exercises several components with real user flows over a
    production-like artefact, emulating E2E at integration level

    View Slide

  74. https://github.com/dotanuki-labs/norris

    View Slide

  75. View Slide

  76. PLUS
    ULTRA !!!

    View Slide

  77. Coverage
    on Sonarqube
    Actual tests

    View Slide

  78. View Slide

  79. View Slide

  80. Standards help large teams since
    they reduce both accidental
    complexity and also cognitive load
    when switching between contexts
    inside the same codebase

    View Slide

  81. Purpose Recommendation Reason
    Regular assertions Google Truth Max compatibility for JVM + Roboletric + Instrumentation
    Test Runner jUnit4 Junit5 is not supported for Android/Instrumentation
    Screenshot Tests Karumi/Shot Nice wrapper over Facebook tooling + useful add-ons
    Espresso Tests Barista Nice wrapper over Espresso tooling + useful add-ons
    Compose Tests Standard tooling Provided since day-zero
    Contract Tests Pact/JVM Most adopted solution
    E2E 👀 Up to you, as long it runs for as part of your pipeline

    View Slide

  82. What really matters is focusing on
    writting good tests that allow
    Engineers to move faster by
    maximizing the correlation between
    test code and value proposition (ie,
    probability of catching bugs)

    View Slide

  83. Software
    Engineer
    Quality
    Engineer

    View Slide

  84. Team work makes the dream work!

    View Slide

  85. FINAL
    REMARKS

    View Slide

  86. TL;DR
    • Be aware of unit tests that don’t bring any value
    • If the case for you, invest energy on integration tests
    specially over Data Sources and Screen Components
    • There is no such thing like effective Espresso tests over
    non-release builds
    • Let Mocks fade away from your codebase (and your life!)
    • Mind the conciliation between your Test Strategy and your
    Quality Strategy

    View Slide

  87. UBIRATAN
    SOARES
    Brazilian Computer Scientist
    Senior Software Engineer @ N26
    GDE for Android and Kotlin
    @ubiratanfsoares
    ubiratansoares.dev

    View Slide

  88. THANKS

    View Slide