Maintaining E2E Test Automation as We Transition from View to Compose

Maintaining E2E Test Automation as We Transition from View to
Compose @shinmiy / @kenken 2023-09-14 DroidKaigi 2023 Day.1

Who are we again? 2 @kenken Android Engineers @ Merpay
by day, DroidKaigi Committee staffs by night @shinmiy

Agenda • The Mercari app scenario • Maintain and migrate
E2E test automation • General tips 3

The Mercari App • 22.6M customers using the service each
month • Buying and selling over 254B JPY (~1.7B USD) worth of goods • Payment features ◦ Barcode payments, credit cards, lending… 4 Source: Mercari Business Annual Financial Overview (FY2023.6, JP ver)

Large scale releases 5

1 week release train 6

• Started automating release judgement tests (E2E tests) ◦ Reducing
release cycle from 2 weeks to 1 week • End-to-End testing ◦ Tests that simulate a real word scenario ◦ Black box tests that check the entire app as a system to ensure it’s working ◦ ✅ Comprehensive, looks at the entire system ◦ 😞 Doesn’t test inner workings ◦ 😞 Time consuming ◦ → combined with other types of tests E2E tests to judge releases 7

E2E tests to judge releases 8 // Login with a
user TestAccount( userName = "some-preexisting-test-account", ).login(loginUseCase) TopPage .goToDashboardPage() // Move to target page .assertIsPointValueDisplayed() // Check behavior

E2E testing for release 9 https://engineering.mercari.com/en/blog/entry/20211210-merpay-android-test-automation/

TestRail 10 • Tool used to track all tests =
Source of Truth • All automated tests map 1-to-1 to a TestRail test case • Use TestRail API to update automated test results (from Results – TestRail Support Center)

Firebase Test Lab 11 • Used to conduct tests remotely
on real devices • Flank: test runner allowing parallel tests https://github.com/Flank/flank

Ground Up App (GU App) 12 • Full rewrite of
Mercari app • Planned from ~2020, released in September 2022 • New Design System • Android View -> Jetpack Compose • Gradually migrating existing features

So our scenario: 13 • App that has both Views
and Jetpack Compose • Ongoing feature development (with both Jetpack Compose and Views) • New features: Jetpack Compose • Existing features: gradually migrate to New App system (= Jetpack Compose)

Testing in this scenario 14 • Android View features using
Espresso • Jetpack Compose features using Compose testing library • Mixed tests • Page Object models ◦ https://www.selenium.dev/documentation/test_practices/encouraged/page_obje ct_models/

Recap) How to write UI tests for Android View 15
• Espresso is used in many cases ☕ • Find components with their View IDs, attributes and/or their hierarchy etc, then interact with them, and finally assert them

Cheat sheet for Espresso 16 • Android Developers provides a
useful cheat sheet ◦ https://developer.android.com/training/testing/espresso/cheat-sheet

Sample screen (Android View-based) 17 <?xml version="1.0" encoding="utf-8"?> <androidx.constraintlayout.widget.ConstraintLayout> <com.google.android.material.textview.MaterialTextView
android:id="@+id/screen_name_label" android:text="AndroidViewActivity" /> <com.google.android.material.textview.MaterialTextView android:id="@+id/counter_label" android:text="count: 0" /> <com.google.android.material.button.MaterialButton android:id="@+id/count_up_button" android:text="count up" /> </androidx.constraintlayout.widget.ConstraintLayout>

UI test implementation for Android View 18 class AndroidViewActivityInstrumentedTest {
@get:Rule val activityScenarioRule = ActivityScenarioRule(AndroidViewActivity::class.java) @Test fun counterTest() { Espresso.onView(ViewMatchers.withId(R.id.counter_label)) .check(ViewAssertions.matches(ViewMatchers.withText("count: 0"))) Espresso.onView(ViewMatchers.withId(R.id.count_up_button)) .perform(ViewActions.click()) Espresso.onView(ViewMatchers.withId(R.id.counter_label)) .check(ViewAssertions.matches(ViewMatchers.withText("count: 1"))) } }

• Compose testing library is used in most cases •
Composables do not have IDs • Test Tags can be used like Android View’s ID, but never abuse them! ◦ https://developer.android.com/codelabs/jetpack-compose-testing#3 How to write UI tests for Compose 23

How to write UI tests for Compose • Interaction with
components through semantics tree ◦ Represent semantic meaning and convey to accessibility services and testing tools ◦ https://developer.android.com/jetpack/compose/semantics 24 UI hierarchy and its semantics tree

• Android Developers provides an useful cheat sheet ◦ https://developer.android.com/jetpack/compose/testing-cheatsheet
Cheat sheet for Compose testing library 25

Sample screen (Compose-based) 26 @Composable fun ComposeScreen() { var counter
by remember { mutableIntStateOf(0) } Box(modifier = Modifier.fillMaxSize().padding(16.dp)) { Text("ComposeActivity", modifier = Modifier.align(Alignment.TopStart)) Column( modifier = Modifier.align(Alignment.Center), verticalArrangement = Arrangement.Center, horizontalAlignment = Alignment.CenterHorizontally, ) { Text("count: $counter") Spacer(modifier = Modifier.height(16.dp)) Button(onClick = { counter++ }) { Text("count up") } } }

UI test implementation for Compose 27 class ComposeActivityInstrumentedTest { @get:Rule
val composeTestRule = createAndroidComposeRule<ComposeActivity>() @Test fun counterTest() { composeTestRule.onNodeWithText("count: 0").assertIsDisplayed() composeTestRule.onNodeWithText("count up").performClick() composeTestRule.onNodeWithText("count: 1").assertIsDisplayed() } }

val composeTestRule = createAndroidComposeRule<ComposeActivity>() @Test fun counterTest() { composeTestRule.onNodeWithText("count: 0").assertIsDisplayed() composeTestRule.onNodeWithText("count up").performClick() composeTestRule.onNodeWithText("count: 1").assertIsDisplayed() } } ?

Page Object Model (POM) 34 • Design pattern to isolate
UI from test classes • Page Objects can represent the whole screen, fragments, or smaller components • Interact with UIs through the Page Object • Readable, reusable and maintanable • Method chaining

Page Object Model (POM) 35 Test class 1 Test class
3 Test class 2 A Page Object Screen A Screen B B Page Object

POM implementation for Android View 36 object AndroidViewPage { private
val counterLabel get() = Espresso.onView(ViewMatchers.withId(R.id.counter_label)) private val countUpButton get() = Espresso.onView(ViewMatchers.withId(R.id.count_up_button)) fun clickCountUpButton(): AndroidViewPage = this.also { countUpButton.perform(ViewActions.click()) } fun assertCounterLabel(expected: Int): AndroidViewPage = this.also { counterLabel.check(ViewAssertions.matches(ViewMatchers.withText("count: $expected"))) } }

POM implementation for Compose 40 object ComposePage { lateinit var
composeTestRule: AndroidComposeTestRule<ActivityScenarioRule<ComposeActivity>, ComposeActivity> private val counterLabel get() = composeTestRule.onNodeWithText("count:", substring = true) private val countUpButton get() = composeTestRule.onNodeWithText("count up") fun clickCountUpButton(): ComposePage = this.also { countUpButton.performClick() } fun assertCounterLabel(expected: Int): ComposePage = this.also { counterLabel.assertTextEquals("count: $expected") } }

Comparing test codes for Android View 43 class AndroidViewActivityInstrumentedTest {
// ... @Test fun counterTestByUsingPageObject() { AndroidViewPage .assertCounterLabel(0) .clickCountUpButton() .assertCounterLabel(1) } } class AndroidViewActivityInstrumentedTest { // ... @Test fun counterTest() { Espresso.onView(withId(R.id.counter_label)) .check(matches(withText("count: 0"))) Espresso.onView(withId(R.id.count_up_button)) .perform(click()) Espresso.onView(withId(R.id.counter_label)) .check(matches(withText("count: 1"))) } }

Comparing test codes for Compose 44 class ComposeActivityInstrumentedTest { //
... @Before fun setUp() { ComposePage.composeTestRule = composeTestRule } @Test fun counterTestByUsingPageObject() { ComposePage .assertCounterLabel(0) .clickCountUpButton() .assertCounterLabel(1) } } class ComposeActivityInstrumentedTest { // ... @Test fun counterTest() { composeTestRule .onNodeWithText("count: 0") .assertIsDisplayed() composeTestRule .onNodeWithText("count up") .performClick() composeTestRule .onNodeWithText("count: 1") .assertIsDisplayed() } }

Comparing test codes with Page Objects 45 class AndroidViewActivityInstrumentedTest {
// ... @Test fun counterTestByUsingPageObject() { AndroidViewPage // Given: assert initial count .assertCounterLabel(0) // When: click count up button .clickCountUpButton() // Then: assert that count is incremented .assertCounterLabel(1) } } class ComposeActivityInstrumentedTest { // ... @Before fun setUp() { ComposePage.composeTestRule = composeTestRule } @Test fun counterTestByUsingPageObject() { ComposePage // Given: assert initial count .assertCounterLabel(0) // When: click count up button .clickCountUpButton() // Then: assert that count is incremented .assertCounterLabel(1) } } ↓Android View Compose→

• Prerequisites 1. Transition E2E from Android View to Compose
WITHOUT changing any features 2. Switching both UIs by using feature flags ▪ Transitioning in stages, NOT all at once Migrating E2E tests from Android View to Compose 46

1. Create a new screen with Compose 2. Introduce Feature
Flags mechanism 3. Switch both screens with the Feature Flags 4. Create a Page Object interface for these screens 5. Implement the interface for Android View and Compose 6. Create Fake Feature Flags and replace with it 7. Switch these Page Objects with the Feature Flags Transition steps 47

Flags mechanism 3. Switch both screens with the Feature Flags 4. Create a Page Object interface for these screens 5. Implement the interface for Android View and Compose 6. Create Fake Feature Flags and replace with it 7. Switch these Page Objects with the Feature Flags Transition steps for production codes 48

Flags mechanism 3. Switch both screens with the Feature Flags 4. Create a Page Object interface for these screens 5. Implement the interface for Android View and Compose 6. Create Fake Feature Flags and replace with it 7. Switch these Page Objects with the Feature Flags Transition steps for E2E test codes 49

1. Create a new screen with Compose 50 @Composable fun
ComposeScreen() { var counter by remember { mutableIntStateOf(0) } Box(modifier = Modifier.fillMaxSize().padding(16.dp)) { Text("ComposeActivity", modifier = Modifier.align(Alignment.TopStart)) Column( modifier = Modifier.align(Alignment.Center), verticalArrangement = Arrangement.Center, horizontalAlignment = Alignment.CenterHorizontally, ) { Text("count: $counter") Spacer(modifier = Modifier.height(16.dp)) Button(onClick = { counter++ }) { Text("count up") } } }

2-1. Introduce Feature Flags mechanism 51 interface FeatureFlagProvider { fun
shouldUseComposeScreen(): Boolean } class FeatureFlagProviderImpl @Inject constructor(): FeatureFlagProvider { // * In an actual development, it should be received from external sources override fun shouldUseComposeScreen(): Boolean = true }

2-2. Configure dependency injection 52 @Module @InstallIn(SingletonComponent::class) abstract class DiModule
{ @Singleton @Binds abstract fun bindFeatureFlagProvider( impl: FeatureFlagProviderImpl, ): FeatureFlagProvider }

3. Switch both screens with the Feature Flags 53 @AndroidEntryPoint
class MainActivity : ComponentActivity() { @Inject lateinit var featureFlagProvider: FeatureFlagProvider override fun onCreate(savedInstanceState: Bundle?) { // In setContent() val shouldUseComposeScreen = remember { featureFlagProvider.shouldUseComposeScreen() } Button( onClick = { if (shouldUseComposeScreen) { startActivity(ComposeActivity.createIntent(this@MainActivity)) } else { startActivity(AndroidViewActivity.createIntent(this@MainActivity)) } } ) { Text("Go to next activity") } // ... } }

Flags mechanism 3. Switch both screens with the Feature Flags 4. Create a Page Object interface for these screens 5. Implement the interface for Android View and Compose 6. Create Fake Feature Flags and replace with it 7. Switch these Page Objects with the Feature Flags Transition steps for E2E test codes 54

4. Create a Page Object interface 55 interface TransitionTargetPage {
fun clickCountUpButton(): TransitionTargetPage fun assertCounterLabel(expected: Int): TransitionTargetPage // New assertion for checking if screen is switched successfully fun assertScreenName(): TransitionTargetPage }

5-1. Implement the interface for Android View 56 object AndroidViewPage
: TransitionTargetPage { // ... override fun clickCountUpButton(): TransitionTargetPage = this.also { countUpButton.perform(click()) } override fun assertCounterLabel(expected: Int): TransitionTargetPage = this.also { counterLabel.check(matches(withText("count: $expected"))) } override fun assertScreenName(): TransitionTargetPage = this.also { screenNameLabel.check(matches(withText("AndroidViewActivity"))) } }

5-1. Implement the interface for Android View 57 object AndroidViewPage
: TransitionTargetPage { // ... override fun clickCountUpButton(): TransitionTargetPage = this.also { countUpButton.perform(click()) } override fun assertCounterLabel(expected: Int): TransitionTargetPage = this.also { counterLabel.check(matches(withText("count: $expected"))) } override fun assertScreenName(): TransitionTargetPage = this.also { screenNameLabel.check(matches(withText("AndroidViewActivity"))) } }

5-2. Implement the interface for Compose 58 object ComposePage :
TransitionTargetPage { // ... override fun clickCountUpButton(): TransitionTargetPage = this.also { countUpButton.performClick() } override fun assertCounterLabel(expected: Int): TransitionTargetPage = this.also { counterLabel.assertTextEquals("count: $expected") } override fun assertScreenName(): TransitionTargetPage = this.also { screenNameLabel.assertIsDisplayed() } }

5-3. Add a method to switch them 59 interface TransitionTargetPage
{ // ... companion object { fun get(isComposeScreen: Boolean): TransitionTargetPage { return if (isComposeScreen) ComposePage else AndroidViewPage } } }

6-1. Replace Feature Flag with Fake 60 class FakeFeatureFlagResolver @Inject
constructor() : FeatureFlagProvider { override fun shouldUseComposeScreen(): Boolean = false } @Module @TestInstallIn(components = [SingletonComponent::class], replaces = [DiModule::class]) abstract class FakeDiModule { @Singleton @Binds abstract fun bindFeatureFlagResolver( impl: FakeFeatureFlagResolver, ): FeatureFlagProvider }

6-2. Setup for Fake Feature Flag 61 @HiltAndroidTest class MainActivityInstrumentedTest
{ @get:Rule(order = 0) val hiltRule = HiltAndroidRule(this) // ... @Inject lateinit var featureFlagProvider: FeatureFlagProvider @Before fun setUp() { hiltRule.inject() } }

7-1. Create MainPage for MainActivity 62 object MainPage { lateinit
var composeTestRule: AndroidComposeTestRule<ActivityScenarioRule<MainActivity>, MainActivity> private val button get() = composeTestRule.onNodeWithText("Go to next activity") fun clickNavigationButton(isComposeScreen: Boolean): TransitionTargetPage = TransitionTargetPage.get(isComposeScreen).also { button.performClick() } }

7-2. Create E2E tests that can switch screens 63 @HiltAndroidTest
class MainActivityInstrumentedTest { // ... @Test fun navigationTest() { MainPage .clickNavigationButton(featureFlagProvider.shouldUseComposeScreen()) .assertScreenName() } }

64 Complete 🎉 true false Value of Fake FeatureFlagProvider Compose
Page Android View Page TransitionTargetPage TransitionTargetPage

Working with the QA team 65 QA Create Test Case
Add to TestRail Create Task ticket Implement E2E test Update TestRail QA QA • Create a task ticket for devs to work on • Flag TestRail test case as automated • Create new test cases • Update existing test case Engineer • Flag TestRail case as a manual test • 💪💪💪💪 Engineer Engineer

Maintaining tests as a team 66 • Tackling similar tasks:
Great opportunity to share tips • Work as a team! Pair pro, mob pro it, talk while you wait for builds • Keep a tips & tricks document and write whenever you find something

General tips from our list of hacks, gotchas, and other
things we’ve noticed 67

Waiting for tests 68 • Waiting for network calls, animations,
transitions… • waitUntil to wait for something to be completed composeTestRule .waitUntil(timeoutMillis = 5000L) { composeTestRule .onNodeWithText("Completed!") .isVisible() }

Waiting for tests 69 • Starting Compose 1.4 we have
some newer apis: @OptIn(ExperimentalTestApi::class) composeTestRule.waitUntilNodeCount(hasText("Completed!"), count = 1) composeTestRule.waitUntilAtLeastOneExists(hasTestTag("done")) composeTestRule.waitUntilExactlyOneExists(hasText("Completed!")) composeTestRule.waitUntilDoesNotExist(hasText("Loading..."))

onNodeWithTag vs onNodeWithText 70 • Similar name, similar arguments =
**very** easy to mix up! • Read what you write! (especially when it’s autocompleted) composeTestRule .onNodeWithText(SomeScreenTestTags.COMPONENT_A) // 🥹

71 • Look for composables using multiple matchers • Function
of SemanticsMatcher Multiple matchers using and/or composeTestRule .onNodeWithTag("card_a") .onChildren() .filterToOne( hasTestTag("bottom_button") and hasText("Details") )

72 class SemanticsMatcher( val description: String, private val matcher: (SemanticsNode)
-> Boolean ) { infix fun and(other: SemanticsMatcher): SemanticsMatcher { return SemanticsMatcher( description = "($description) && (${other.description})" ) { matcher(it) && other.matches(it) } } } Multiple matchers using and/or

73 • Multiple nodes may exist when working with SubcomposeLayouts
Working with SubcomposeLayouts composeTestRule .onNodeWithTag("content") .assertIsDisplayed() java.lang.AssertionError: Failed to perform isDisplayed check. Reason: Expected exactly '1' node but found '2' nodes that satisfy: (TestTag = 'content')

74 Working with SubcomposeLayouts SubcomposeLayout(modifier) { constraints -> val mainPlaceables
= subcompose(Enum.Main, content).map { it.measure(Constraints()) } val maxSize = mainPlaceables.fold(IntSize.Zero) { currentMax, placeable -> /* calculate size */ } val resizedPlaceables = subcompose(Enum.Dependent, content).map { it.measure(Constraints(minWidth = maxSize.width)) } layout(constraints.maxWidth, constraints.maxHeight) { /* layout resizedPlaceables */ } }

75 Working with SubcomposeLayouts Only one of them is displayed:
composeTestRule .onAllNodesWithTag("content")[0] .assertIsDisplayed() // ✅ composeTestRule .onAllNodesWithTag("content")[1] .assertIsDisplayed() java.lang.AssertionError: Assert failed: The component is not displayed!

76 Working with SubcomposeLayouts composeTestRule .onAllNodesWithTag("content") val resizedPlaceables = subcompose(Enum.Dependent)
{ Box(Modifier.testTag("displayed")) { content() } } Fetch all nodes: Add a Test Tag to the one displayed when subcomposing:

77 • Very good experience with smart code completion so
far • Especially for repetitive code (like tests!) Using GitHub Copilot

• A good base to start: 78 Using GitHub Copilot

• Filling in data classes, naming enums… 79 Using GitHub
Copilot

In Conclusion 80 • Automating E2E test can save resources
and help with streamlining the release process • Migrating E2E tests from Android View to Compose ◦ Introduce Feature Flags mechanism to switch 2 screens in production code ◦ Use Page Object model for both test implementation ◦ Create Fake and overwrite the configuration of the Feature Flags for E2E tests ◦ Switch the Page Objects based on the value of the Feature Flags • Work with the QA team to divide and conquer • Many tips to share within the team, great opportunity to have fun!

Thanks! 81 @shinmiy / @kenken Enjoy the rest of DroidKaigi
2023!

Maintaining E2E Test Automation as We Transition from View to Compose

Maintaining E2E Test Automation as We Transition from View to Compose

More Decks by kenken

Other Decks in Programming

Featured

Transcript