Save 37% off PRO during our Black Friday Sale! »

Effective Android Testing - A value driven approach

Effective Android Testing - A value driven approach

Companion slides from my talk with same name.

Delivered in the following events

- PicPay Special Webinar (Online) - October/2021

D4b7a3e2ed10f86e0b52498713ba2601?s=128

Ubiratan Soares
PRO

October 27, 2021
Tweet

Transcript

  1. A value-driven approach Ubiratan Soares October / 2021 EFFECTIVE ANDROID

    TESTING
  2. CHALLENGES FROM TESTING ANDROID APPS

  3. Testing can be hard

  4. Testing can be hard For Android projects, it can be

    even harder
  5. None
  6. None
  7. None
  8. CONCEPTS REVIEW

  9. Dimensions of testing • Which type of code a test

    touches • Which boundaries of the system it touches • Which real flows a test exercises • Which level of isolation are in place (if any) • Etc
  10. Unit Integration E2E + + Value Ensured Complexity Involved

  11. Subject Under Test Input Output Classical understanding Single Class or

    Single Function
  12. Entry point Exit point Not necessarily 1:1 relationship to classes

    / functions SUT
  13. https://youtu.be/6ndAWzc2F-I

  14. Entry point Exit point SUT Subject under test is not

    supposed to be the whole thing
  15. Entry point Exit point SUT Boundary Owned by Others Owned

    by Others Abstractions you own within the app boundaries Boundary
  16. Unit tests • Well-defined single entry point • Well-defined single

    exit point • Usually a small subject • Exercise abstractions you own • Not necessary 1:1 relationship to classes / functions • Never touches infrastructure ⚠
  17. None
  18. Entry point Boundary Owned by Others Owned by You Exit

    point
  19. Entry point Boundary Owned by Others Owned by You Boundary

    Owned by Others Exit point
  20. Entry point Boundary Owned by Others Owned by You Boundary

    Owned by Others Exit point
  21. Entry point Boundary Owned by Others Owned by You Boundary

    Owned by Others Exit point
  22. None
  23. Integration tests • Exercise entry points and/or exit points beyond

    your application boundaries • Always enforce some degree of isolation (ideally fully isolated) • Categorized according the number of integrations under test • Categorized according user-value under test
  24. Integration Level Rationale Single Exercises only one well-defined system boundary

    Multiple Exercises several well-defined system boundaries Component Exercises one well-defined component (user perspective) Acceptance Exercises several components with real user flows
  25. E2E tests • Black-box instrumentation over a deliverable artefact •

    Exercise user flows over a pre-production environment • The most expensive ones to write + maintain
  26. AN EFFECTIVE STRATEGY

  27. Expectations 😎 Code

  28. Reality 😅 Code

  29. Bussines Rules App Micro service Nano service Monolith

  30. App Realization of value Enabler of value != To avoid

  31. app Product onboarding login search … Features Platform core-persistance core-rest

    core-location core-translation common-android common-di common-assets …
  32. Entrypoint / Glue Features Platform Codebase contribution ** Practical observations

    from previous projects
  33. “If the vast majority of a codebase from a Mobile

    product deals with coordination over middleware and/or infrastructure, how do we do unit testing ?”
  34. “Write tests. Not too many. Mostly integration” Kent Dodds https://kentcdodds.com/blog/write-tests

  35. https://youtu.be/Fha2bVoC8SE

  36. https://martinfowler.com/articles/2021-test-shapes.html

  37. Integration Unit E2E Contribution to Product Codebase Contribution to Test

    Suite
  38. Product Features Platform Unit tests (stable core) Integration tests Unit

    tests (on-demand) Contract tests Component tests Screenshot tests E2E tests Acceptance tests
  39. Unit Tests that bring value Tests for your implementation for

    HTTP authentication (eg, Oauth)
  40. Unit Tests that bring value Tests for your retry- strategy

    over HTTP (eg, exponential backoff)
  41. Unit Tests that bring value Tests over non-trivial validations over

    data provided by users via your App (eg, regexes)
  42. Unit Tests that bring value Tests over non-trivial data parsing

    that happens at Mobile level (eg URI / DeepLinks)
  43. Unit Tests that don’t bring value Tests over REST integration

    by mocking a Retrofit API
  44. Unit Tests that don’t bring value Tests over ViewModels by

    mocking its colaborators
  45. Unit Tests that don’t bring value Tests over trivial data

    mappers used within your abstractions
  46. Unit Tests that don’t bring value Tests over sandwitch layers

  47. Are you ready to let Mockito go ??? Android Engineer

  48. Integrated tests that bring value Comprehenstive tests over data sources,

    exercising thirdy-party transformation / serialization engines alongside with your own error handlers and/or data transformers
  49. class FactsDataSourceTests { @get:Rule val restInfrastructure = RestInfrastructureRule() private lateinit

    var dataSource: FactsDataSource @Before fun `before each test`() { val api = restInfrastructure.server.wireRestApi() dataSource = FactsDataSource(api) } @Test fun `should handle no results properly`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_no_results.json") ) val noFacts = emptyList<ChuckNorrisFact>() assertThat(simpleSearch()).isEqualTo(noFacts) } https://github.com/dotanuki-labs/norris
  50. class FactsDataSourceTests { @get:Rule val restInfrastructure = RestInfrastructureRule() private lateinit

    var dataSource: FactsDataSource @Before fun `before each test`() { val api = restInfrastructure.server.wireRestApi() dataSource = FactsDataSource(api) } @Test fun `should handle no results properly`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_no_results.json") ) val noFacts = emptyList<ChuckNorrisFact>() assertThat(simpleSearch()).isEqualTo(noFacts) } https://github.com/dotanuki-labs/norris
  51. class FactsDataSourceTests { @get:Rule val restInfrastructure = RestInfrastructureRule() private lateinit

    var dataSource: FactsDataSource @Before fun `before each test`() { val api = restInfrastructure.server.wireRestApi() dataSource = FactsDataSource(api) } @Test fun `should handle no results properly`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_no_results.json") ) val noFacts = emptyList<ChuckNorrisFact>() assertThat(simpleSearch()).isEqualTo(noFacts) } https://github.com/dotanuki-labs/norris
  52. @Test fun `should handle downstream error`() { restInfrastructure.restScenario(status = 500)

    assertErrorTransformed( whenRunning = this::simpleSearch, expected = RemoteServiceIntegrationError.RemoteSystem ) } https://github.com/dotanuki-labs/norris
  53. @Test fun `should fetch facts with valid query term`() {

    restInfrastructure.restScenario( status = 200, response = loadFile("200_search_with_results.json") ) val facts = listOf( ChuckNorrisFact( id = "2wzginmks8azrbaxnamxdw", shareableUrl = "https://api.chucknorris.io/jokes/2wzginmks", textual = "Chuck Norris divides by zero”, ) ) assertThat(simpleSearch()).isEqualTo(facts) } } https://github.com/dotanuki-labs/norris
  54. Integrated tests that bring value Component tests over entire screens,

    capturing the asynchonicity of data operations without coupling with implementation details (eg, Coroutines)
  55. Activity View

  56. Activity Screen Interface View or Wrapper

  57. Activity Screen Interface View or Wrapper Test Abstraction

  58. @RunWith(AndroidJUnit4::class) class FactsActivityTests { private lateinit var screen: FakeFactsScreen @get:Rule

    val restInfrastructure = RestInfrastructureRule() @Before fun `before each test`() { // Switching over fake screen wrapper with DI support val testApp = TestApplication.setupWith( factsModule, factsTestModule, RestInfrastructureTestModule(restInfrastructure.server) ) screen = testApp.factsScreen() } https://github.com/dotanuki-labs/norris
  59. @Test fun `at first lunch, should start on empty state`()

    { whenActivityResumed<FactsActivity> { assertThat(screen.isLinked).isTrue() val expectedStates = listOf(Idle, Loading, FactsScreenState.Empty) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } // More tests @Test fun `when remote service fails, should display the error`() { restInfrastructure.restScenario(status = 503) PersistanceHelper.registerNewSearch("code") whenActivityResumed<FactsActivity> { val error = Failed(RemoteServiceIntegrationError.RemoteSystem) val expectedStates = listOf(Idle, Loading, error) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } } https://github.com/dotanuki-labs/norris
  60. @Test fun `at first lunch, should start on empty state`()

    { whenActivityResumed<FactsActivity> { assertThat(screen.isLinked).isTrue() val expectedStates = listOf(Idle, Loading, FactsScreenState.Empty) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } // More tests @Test fun `when remote service fails, should display the error`() { restInfrastructure.restScenario(status = 503) PersistanceHelper.registerNewSearch("code") whenActivityResumed<FactsActivity> { val error = Failed(RemoteServiceIntegrationError.RemoteSystem) val expectedStates = listOf(Idle, Loading, error) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } } https://github.com/dotanuki-labs/norris
  61. Screenshots tests over your UI delivery, decoupled from any data

    operations Integrated tests that bring value
  62. Activity Screen Interface View or Wrapper Test Abstraction

  63. Activity Screen Interface View or Wrapper Test Abstraction TestActivity

  64. @LargeTest class FactsScreenshotTests : ScreenshotTest { private fun checkScreenshot(targetState: FactsScreenState)

    { val testActivity = prepareToCaptureScreenshot<FactsTestActivity> { target -> listOf(FactsScreenState.Idle, targetState).forEach { target.screen.updateWith(it) } } compareScreenshot(testActivity) } https://github.com/dotanuki-labs/norris
  65. @Test fun emptyState() { checkScreenshot(Empty) } @Test fun errorState() {

    val error = RemoteServiceIntegrationError.RemoteSystem checkScreenshot(Failed(error)) } @Test fun successState() { val facts = listOf( FactDisplayRow( url = "https://chucknorris.io/jokes/998877", fact = "Chuck Norris merges before commit anything on Git", displayWithSmallerFontSize = false ) ) val presentation = FactsPresentation("humor", facts) checkScreenshot(Success(presentation)) } } https://github.com/dotanuki-labs/norris
  66. @Test fun emptyState() { checkScreenshot(Empty) } @Test fun errorState() {

    val error = RemoteServiceIntegrationError.RemoteSystem checkScreenshot(Failed(error)) } @Test fun successState() { val facts = listOf( FactDisplayRow( url = "https://chucknorris.io/jokes/998877", fact = "Chuck Norris merges before commit anything on Git", displayWithSmallerFontSize = false ) ) val presentation = FactsPresentation("humor", facts) checkScreenshot(Success(presentation)) } } https://github.com/dotanuki-labs/norris
  67. Integrated tests that bring value Acceptance tests over a fake

    ready-to-deliver artefact, exercising real user flows in isolation
  68. @RunWith(AndroidJUnit4!"class) class NorrisAcceptanceTests { private val restInfrastructure = RestInfrastructureRule(customPort =

    4242) private val stressedExecution = FlakyTestRule() @get:Rule val rules: RuleChain = RuleChain.outerRule(stressedExecution).around(restInfrastructure) @Before fun beforeEachTest() { PersistanceHelper.clearStorage() } https://github.com/dotanuki-labs/norris
  69. @Repeat @Test fun shouldPerformFirstSearch_AfterFirstLunch_ByTypingATerm() { val searchTerm = "math" val

    suggestions = listOf("career", "dev", "humor") val fact = "Chuck Norris can divide by zero" val suggestionsPayload = RestDataBuilder.suggestionsPayload(suggestions) val factsPayload = RestDataBuilder.factsPayload(searchTerm, fact) restInfrastructure.run { restScenario(200, suggestionsPayload) restScenario(200, factsPayload) } https://github.com/dotanuki-labs/norris
  70. startingFrom<FactsActivity> { checkEmptyState() clickOnSearchIcon() awaitTransition() checkSuggestions(suggestions) performSearch(searchTerm) awaitTransition() checkDisplayed(fact) }

    } } https://github.com/dotanuki-labs/norris
  71. fun Project.applyAndroidApplicationConventions() { applyAndroidStandardConventions() val android = extensions.findByName("android") as ApplicationExtension

    android.apply { testBuildType = when { isTestMode() -> "release" else -> "debug" } defaultConfig { testInstrumentationRunner = AndroidDefinitions.instrumentationTestRunner } // More plugin-driven conventions } https://github.com/dotanuki-labs/norris
  72. fun Project.applyAndroidApplicationConventions() { applyAndroidStandardConventions() val android = extensions.findByName("android") as ApplicationExtension

    android.apply { testBuildType = when { isTestMode() -> "release" else -> "debug" } defaultConfig { testInstrumentationRunner = AndroidDefinitions.instrumentationTestRunner } // More plugin-driven conventions } https://github.com/dotanuki-labs/norris
  73. Integration Test Proposed Value Single integration over data sources Ensures

    data transformations + error handling over the target boundary Component Tests over entire screens Enables refactoring over target screen Screenshot Tests over entire screens Ensures association between state and Views without any assertions Acceptance Exercises several components with real user flows over a production-like artefact, emulating E2E at integration level
  74. https://github.com/dotanuki-labs/norris

  75. None
  76. PLUS ULTRA !!!

  77. Coverage on Sonarqube Actual tests

  78. None
  79. None
  80. Standards help large teams since they reduce both accidental complexity

    and also cognitive load when switching between contexts inside the same codebase
  81. Purpose Recommendation Reason Regular assertions Google Truth Max compatibility for

    JVM + Roboletric + Instrumentation Test Runner jUnit4 Junit5 is not supported for Android/Instrumentation Screenshot Tests Karumi/Shot Nice wrapper over Facebook tooling + useful add-ons Espresso Tests Barista Nice wrapper over Espresso tooling + useful add-ons Compose Tests Standard tooling Provided since day-zero Contract Tests Pact/JVM Most adopted solution E2E 👀 Up to you, as long it runs for as part of your pipeline
  82. What really matters is focusing on writting good tests that

    allow Engineers to move faster by maximizing the correlation between test code and value proposition (ie, probability of catching bugs)
  83. Software Engineer Quality Engineer

  84. Team work makes the dream work!

  85. FINAL REMARKS

  86. TL;DR • Be aware of unit tests that don’t bring

    any value • If the case for you, invest energy on integration tests specially over Data Sources and Screen Components • There is no such thing like effective Espresso tests over non-release builds • Let Mocks fade away from your codebase (and your life!) • Mind the conciliation between your Test Strategy and your Quality Strategy
  87. UBIRATAN SOARES Brazilian Computer Scientist Senior Software Engineer @ N26

    GDE for Android and Kotlin @ubiratanfsoares ubiratansoares.dev
  88. THANKS