Effective Android Testing - A value driven approach

Slide 1

Slide 1 text

A value-driven approach Ubiratan Soares October / 2021 EFFECTIVE ANDROID TESTING

Slide 2

Slide 2 text

CHALLENGES FROM TESTING ANDROID APPS

Slide 3

Slide 3 text

Testing can be hard

Slide 4

Slide 4 text

Testing can be hard For Android projects, it can be even harder

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

CONCEPTS REVIEW

Slide 9

Slide 9 text

Dimensions of testing • Which type of code a test touches • Which boundaries of the system it touches • Which real ﬂows a test exercises • Which level of isolation are in place (if any) • Etc

Slide 10

Slide 10 text

Unit Integration E2E + + Value Ensured Complexity Involved

Slide 11

Slide 11 text

Subject Under Test Input Output Classical understanding Single Class or Single Function

Slide 12

Slide 12 text

Entry point Exit point Not necessarily 1:1 relationship to classes / functions SUT

Slide 13

Slide 13 text

https://youtu.be/6ndAWzc2F-I

Slide 14

Slide 14 text

Entry point Exit point SUT Subject under test is not supposed to be the whole thing

Slide 15

Slide 15 text

Entry point Exit point SUT Boundary Owned by Others Owned by Others Abstractions you own within the app boundaries Boundary

Slide 16

Slide 16 text

Unit tests • Well-deﬁned single entry point • Well-deﬁned single exit point • Usually a small subject • Exercise abstractions you own • Not necessary 1:1 relationship to classes / functions • Never touches infrastructure ⚠

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Entry point Boundary Owned by Others Owned by You Exit point

Slide 19

Slide 19 text

Entry point Boundary Owned by Others Owned by You Boundary Owned by Others Exit point

Slide 20

Slide 20 text

Entry point Boundary Owned by Others Owned by You Boundary Owned by Others Exit point

Slide 21

Slide 21 text

Entry point Boundary Owned by Others Owned by You Boundary Owned by Others Exit point

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Integration tests • Exercise entry points and/or exit points beyond your application boundaries • Always enforce some degree of isolation (ideally fully isolated) • Categorized according the number of integrations under test • Categorized according user-value under test

Slide 24

Slide 24 text

Integration Level Rationale Single Exercises only one well-defined system boundary Multiple Exercises several well-defined system boundaries Component Exercises one well-defined component (user perspective) Acceptance Exercises several components with real user flows

Slide 25

Slide 25 text

E2E tests • Black-box instrumentation over a deliverable artefact • Exercise user ﬂows over a pre-production environment • The most expensive ones to write + maintain

Slide 26

Slide 26 text

AN EFFECTIVE STRATEGY

Slide 27

Slide 27 text

Expectations 😎 Code

Slide 28

Slide 28 text

Reality 😅 Code

Slide 29

Slide 29 text

Bussines Rules App Micro service Nano service Monolith

Slide 30

Slide 30 text

App Realization of value Enabler of value != To avoid

Slide 31

Slide 31 text

app Product onboarding login search … Features Platform core-persistance core-rest core-location core-translation common-android common-di common-assets …

Slide 32

Slide 32 text

Entrypoint / Glue Features Platform Codebase contribution ** Practical observations from previous projects

Slide 33

Slide 33 text

“If the vast majority of a codebase from a Mobile product deals with coordination over middleware and/or infrastructure, how do we do unit testing ?”

Slide 34

Slide 34 text

“Write tests. Not too many. Mostly integration” Kent Dodds https://kentcdodds.com/blog/write-tests

Slide 35

Slide 35 text

https://youtu.be/Fha2bVoC8SE

Slide 36

Slide 36 text

https://martinfowler.com/articles/2021-test-shapes.html

Slide 37

Slide 37 text

Integration Unit E2E Contribution to Product Codebase Contribution to Test Suite

Slide 38

Slide 38 text

Product Features Platform Unit tests (stable core) Integration tests Unit tests (on-demand) Contract tests Component tests Screenshot tests E2E tests Acceptance tests

Slide 39

Slide 39 text

Unit Tests that bring value Tests for your implementation for HTTP authentication (eg, Oauth)

Slide 40

Slide 40 text

Unit Tests that bring value Tests for your retry- strategy over HTTP (eg, exponential backoff)

Slide 41

Slide 41 text

Unit Tests that bring value Tests over non-trivial validations over data provided by users via your App (eg, regexes)

Slide 42

Slide 42 text

Unit Tests that bring value Tests over non-trivial data parsing that happens at Mobile level (eg URI / DeepLinks)

Slide 43

Slide 43 text

Unit Tests that don’t bring value Tests over REST integration by mocking a Retroﬁt API

Slide 44

Slide 44 text

Unit Tests that don’t bring value Tests over ViewModels by mocking its colaborators

Slide 45

Slide 45 text

Unit Tests that don’t bring value Tests over trivial data mappers used within your abstractions

Slide 46

Slide 46 text

Unit Tests that don’t bring value Tests over sandwitch layers

Slide 47

Slide 47 text

Are you ready to let Mockito go ??? Android Engineer

Slide 48

Slide 48 text

Integrated tests that bring value Comprehenstive tests over data sources, exercising thirdy-party transformation / serialization engines alongside with your own error handlers and/or data transformers

Slide 49

Slide 49 text

class FactsDataSourceTests { @get:Rule val restInfrastructure = RestInfrastructureRule() private lateinit var dataSource: FactsDataSource @Before fun `before each test`() { val api = restInfrastructure.server.wireRestApi() dataSource = FactsDataSource(api) } @Test fun `should handle no results properly`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_no_results.json") ) val noFacts = emptyList() assertThat(simpleSearch()).isEqualTo(noFacts) } https://github.com/dotanuki-labs/norris

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

@Test fun `should handle downstream error`() { restInfrastructure.restScenario(status = 500) assertErrorTransformed( whenRunning = this::simpleSearch, expected = RemoteServiceIntegrationError.RemoteSystem ) } https://github.com/dotanuki-labs/norris

Slide 53

Slide 53 text

@Test fun `should fetch facts with valid query term`() { restInfrastructure.restScenario( status = 200, response = loadFile("200_search_with_results.json") ) val facts = listOf( ChuckNorrisFact( id = "2wzginmks8azrbaxnamxdw", shareableUrl = "https://api.chucknorris.io/jokes/2wzginmks", textual = "Chuck Norris divides by zero”, ) ) assertThat(simpleSearch()).isEqualTo(facts) } } https://github.com/dotanuki-labs/norris

Slide 54

Slide 54 text

Integrated tests that bring value Component tests over entire screens, capturing the asynchonicity of data operations without coupling with implementation details (eg, Coroutines)

Slide 55

Slide 55 text

Activity View

Slide 56

Slide 56 text

Activity Screen Interface View or Wrapper

Slide 57

Slide 57 text

Activity Screen Interface View or Wrapper Test Abstraction

Slide 58

Slide 58 text

@RunWith(AndroidJUnit4::class) class FactsActivityTests { private lateinit var screen: FakeFactsScreen @get:Rule val restInfrastructure = RestInfrastructureRule() @Before fun `before each test`() { // Switching over fake screen wrapper with DI support val testApp = TestApplication.setupWith( factsModule, factsTestModule, RestInfrastructureTestModule(restInfrastructure.server) ) screen = testApp.factsScreen() } https://github.com/dotanuki-labs/norris

Slide 59

Slide 59 text

@Test fun `at first lunch, should start on empty state`() { whenActivityResumed { assertThat(screen.isLinked).isTrue() val expectedStates = listOf(Idle, Loading, FactsScreenState.Empty) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } // More tests @Test fun `when remote service fails, should display the error`() { restInfrastructure.restScenario(status = 503) PersistanceHelper.registerNewSearch("code") whenActivityResumed { val error = Failed(RemoteServiceIntegrationError.RemoteSystem) val expectedStates = listOf(Idle, Loading, error) assertThat(screen.trackedStates).isEqualTo(expectedStates) } } } https://github.com/dotanuki-labs/norris

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Screenshots tests over your UI delivery, decoupled from any data operations Integrated tests that bring value

Slide 62

Slide 62 text

Activity Screen Interface View or Wrapper Test Abstraction

Slide 63

Slide 63 text

Activity Screen Interface View or Wrapper Test Abstraction TestActivity

Slide 64

Slide 64 text

@LargeTest class FactsScreenshotTests : ScreenshotTest { private fun checkScreenshot(targetState: FactsScreenState) { val testActivity = prepareToCaptureScreenshot { target -> listOf(FactsScreenState.Idle, targetState).forEach { target.screen.updateWith(it) } } compareScreenshot(testActivity) } https://github.com/dotanuki-labs/norris

Slide 65

Slide 65 text

@Test fun emptyState() { checkScreenshot(Empty) } @Test fun errorState() { val error = RemoteServiceIntegrationError.RemoteSystem checkScreenshot(Failed(error)) } @Test fun successState() { val facts = listOf( FactDisplayRow( url = "https://chucknorris.io/jokes/998877", fact = "Chuck Norris merges before commit anything on Git", displayWithSmallerFontSize = false ) ) val presentation = FactsPresentation("humor", facts) checkScreenshot(Success(presentation)) } } https://github.com/dotanuki-labs/norris

Slide 66

Slide 66 text

Slide 67

Slide 67 text

Integrated tests that bring value Acceptance tests over a fake ready-to-deliver artefact, exercising real user ﬂows in isolation

Slide 68

Slide 68 text

@RunWith(AndroidJUnit4!"class) class NorrisAcceptanceTests { private val restInfrastructure = RestInfrastructureRule(customPort = 4242) private val stressedExecution = FlakyTestRule() @get:Rule val rules: RuleChain = RuleChain.outerRule(stressedExecution).around(restInfrastructure) @Before fun beforeEachTest() { PersistanceHelper.clearStorage() } https://github.com/dotanuki-labs/norris

Slide 69

Slide 69 text

@Repeat @Test fun shouldPerformFirstSearch_AfterFirstLunch_ByTypingATerm() { val searchTerm = "math" val suggestions = listOf("career", "dev", "humor") val fact = "Chuck Norris can divide by zero" val suggestionsPayload = RestDataBuilder.suggestionsPayload(suggestions) val factsPayload = RestDataBuilder.factsPayload(searchTerm, fact) restInfrastructure.run { restScenario(200, suggestionsPayload) restScenario(200, factsPayload) } https://github.com/dotanuki-labs/norris

Slide 70

Slide 70 text

startingFrom { checkEmptyState() clickOnSearchIcon() awaitTransition() checkSuggestions(suggestions) performSearch(searchTerm) awaitTransition() checkDisplayed(fact) } } } https://github.com/dotanuki-labs/norris

Slide 71

Slide 71 text

fun Project.applyAndroidApplicationConventions() { applyAndroidStandardConventions() val android = extensions.findByName("android") as ApplicationExtension android.apply { testBuildType = when { isTestMode() -> "release" else -> "debug" } defaultConfig { testInstrumentationRunner = AndroidDefinitions.instrumentationTestRunner } // More plugin-driven conventions } https://github.com/dotanuki-labs/norris

Slide 72

Slide 72 text

Slide 73

Slide 73 text

Integration Test Proposed Value Single integration over data sources Ensures data transformations + error handling over the target boundary Component Tests over entire screens Enables refactoring over target screen Screenshot Tests over entire screens Ensures association between state and Views without any assertions Acceptance Exercises several components with real user ﬂows over a production-like artefact, emulating E2E at integration level

Slide 74

Slide 74 text

https://github.com/dotanuki-labs/norris

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

PLUS ULTRA !!!

Slide 77

Slide 77 text

Coverage on Sonarqube Actual tests

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

No content

Slide 80

Slide 80 text

Standards help large teams since they reduce both accidental complexity and also cognitive load when switching between contexts inside the same codebase

Slide 81

Slide 81 text

Purpose Recommendation Reason Regular assertions Google Truth Max compatibility for JVM + Roboletric + Instrumentation Test Runner jUnit4 Junit5 is not supported for Android/Instrumentation Screenshot Tests Karumi/Shot Nice wrapper over Facebook tooling + useful add-ons Espresso Tests Barista Nice wrapper over Espresso tooling + useful add-ons Compose Tests Standard tooling Provided since day-zero Contract Tests Pact/JVM Most adopted solution E2E 👀 Up to you, as long it runs for as part of your pipeline

Slide 82

Slide 82 text

What really matters is focusing on writting good tests that allow Engineers to move faster by maximizing the correlation between test code and value proposition (ie, probability of catching bugs)

Slide 83

Slide 83 text

Software Engineer Quality Engineer

Slide 84

Slide 84 text

Team work makes the dream work!

Slide 85

Slide 85 text

FINAL REMARKS

Slide 86

Slide 86 text

TL;DR • Be aware of unit tests that don’t bring any value • If the case for you, invest energy on integration tests specially over Data Sources and Screen Components • There is no such thing like effective Espresso tests over non-release builds • Let Mocks fade away from your codebase (and your life!) • Mind the conciliation between your Test Strategy and your Quality Strategy

Slide 87

Slide 87 text

UBIRATAN SOARES Brazilian Computer Scientist Senior Software Engineer @ N26 GDE for Android and Kotlin @ubiratanfsoares ubiratansoares.dev

Slide 88

Slide 88 text

THANKS