Maintaining E2E Test Automation as We Transition from View to Compose

Maintaining E2E Test Automation as
We Transition from View to
Compose
@shinmiy / @kenken
2023-09-14 DroidKaigi 2023 Day.1

View Slide

Who are we again?
2
@kenken
Android Engineers @ Merpay by day,
DroidKaigi Committee staffs by night
@shinmiy

View Slide

Agenda
● The Mercari app scenario
● Maintain and migrate E2E test automation
● General tips
3

View Slide

The Mercari App
● 22.6M customers using the
service each month
● Buying and selling over
254B JPY (~1.7B USD) worth
of goods
● Payment features
○ Barcode payments, credit
cards, lending…
4
Source: Mercari Business Annual Financial Overview (FY2023.6, JP ver)

View Slide

Large scale releases
5

View Slide

1 week release train
6

View Slide

● Started automating release judgement tests (E2E tests)
○ Reducing release cycle from 2 weeks to 1 week
● End-to-End testing
○ Tests that simulate a real word scenario
○ Black box tests that check the entire app as a system to ensure it’s working
○ ✅ Comprehensive, looks at the entire system
○ 😞 Doesn’t test inner workings
○ 😞 Time consuming
○ → combined with other types of tests
E2E tests to judge releases
7

View Slide

E2E tests to judge releases
8
// Login with a user
TestAccount(
userName = "some-preexisting-test-account",
).login(loginUseCase)
TopPage
.goToDashboardPage() // Move to target page
.assertIsPointValueDisplayed() // Check behavior

View Slide

E2E testing for release
9
https://engineering.mercari.com/en/blog/entry/20211210-merpay-android-test-automation/

View Slide

TestRail
10
● Tool used to track all tests = Source of Truth
● All automated tests map 1-to-1 to a TestRail test case
● Use TestRail API to update automated test results
(from Results – TestRail Support Center)

View Slide

Firebase Test Lab
11
● Used to conduct tests remotely on real devices
● Flank: test runner allowing parallel tests
https://github.com/Flank/flank

View Slide

Ground Up App (GU App)
12
● Full rewrite of Mercari app
● Planned from ~2020, released in September 2022
● New Design System
● Android View -> Jetpack Compose
● Gradually migrating existing features

View Slide

So our scenario:
13
● App that has both Views and Jetpack Compose
● Ongoing feature development
(with both Jetpack Compose and Views)
● New features: Jetpack Compose
● Existing features: gradually migrate to New App system
(= Jetpack Compose)

View Slide

Testing in this scenario
14
● Android View features using Espresso
● Jetpack Compose features using Compose testing library
● Mixed tests
● Page Object models
○ https://www.selenium.dev/documentation/test_practices/encouraged/page_obje
ct_models/

View Slide

Recap) How to write UI tests for Android View
15
● Espresso is used in many cases ☕
● Find components with their View IDs, attributes and/or their hierarchy
etc, then interact with them, and finally assert them

View Slide

Cheat sheet for Espresso
16
● Android Developers provides a useful cheat sheet
○ https://developer.android.com/training/testing/espresso/cheat-sheet

View Slide

Sample screen (Android View-based)
17

android:id="@+id/screen_name_label"
android:text="AndroidViewActivity"
/>
android:id="@+id/counter_label"
android:text="count: 0"
/>
android:id="@+id/count_up_button"
android:text="count up"
/>

View Slide

UI test implementation for Android View
18
class AndroidViewActivityInstrumentedTest {
@get:Rule
val activityScenarioRule = ActivityScenarioRule(AndroidViewActivity::class.java)
@Test
fun counterTest() {
Espresso.onView(ViewMatchers.withId(R.id.counter_label))
.check(ViewAssertions.matches(ViewMatchers.withText("count: 0")))
Espresso.onView(ViewMatchers.withId(R.id.count_up_button))
.perform(ViewActions.click())
}
}

View Slide

19
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

20
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

21
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

22
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

● Compose testing library is used in most cases
● Composables do not have IDs
● Test Tags can be used like Android View’s ID, but never abuse them!
○ https://developer.android.com/codelabs/jetpack-compose-testing#3
How to write UI tests for Compose
23

View Slide

How to write UI tests for Compose
● Interaction with components through semantics tree
○ Represent semantic meaning and convey to accessibility services and testing tools
○ https://developer.android.com/jetpack/compose/semantics
24
UI hierarchy and its semantics tree

View Slide

● Android Developers provides an useful cheat sheet
○ https://developer.android.com/jetpack/compose/testing-cheatsheet
Cheat sheet for Compose testing library
25

View Slide

Sample screen (Compose-based)
26
@Composable
fun ComposeScreen() {
var counter by remember { mutableIntStateOf(0) }
Box(modifier = Modifier.fillMaxSize().padding(16.dp)) {
Text("ComposeActivity", modifier =
Modifier.align(Alignment.TopStart))
Column(
modifier = Modifier.align(Alignment.Center),
verticalArrangement = Arrangement.Center,
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text("count: $counter")
Spacer(modifier = Modifier.height(16.dp))
Button(onClick = { counter++ }) {
Text("count up")
}
}
}

View Slide

UI test implementation for Compose
27
class ComposeActivityInstrumentedTest {
@get:Rule
val composeTestRule = createAndroidComposeRule()
@Test
fun counterTest() {
composeTestRule.onNodeWithText("count: 0").assertIsDisplayed()
composeTestRule.onNodeWithText("count up").performClick()
}
}

View Slide

28
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

29
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

30
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

31
@get:Rule
@Test
fun counterTest() {
}
}
?

View Slide

32
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

33
@get:Rule
@Test
fun counterTest() {
}
}

View Slide

Page Object Model (POM)
34
● Design pattern to isolate UI from test classes
● Page Objects can represent the whole screen, fragments, or smaller
components
● Interact with UIs through the Page Object
● Readable, reusable and maintanable
● Method chaining

View Slide

Page Object Model (POM)
35
Test
class 1
Test
class 3
Test
class 2
A Page
Object
Screen
A
Screen
B
B Page
Object

View Slide

POM implementation for Android View
36
object AndroidViewPage {
private val counterLabel
get() = Espresso.onView(ViewMatchers.withId(R.id.counter_label))
private val countUpButton
get() = Espresso.onView(ViewMatchers.withId(R.id.count_up_button))
fun clickCountUpButton(): AndroidViewPage = this.also {
countUpButton.perform(ViewActions.click())
}
fun assertCounterLabel(expected: Int): AndroidViewPage = this.also {
counterLabel.check(ViewAssertions.matches(ViewMatchers.withText("count: $expected")))
}
}

View Slide

37
}
}
}

View Slide

38
}
}
}

View Slide

39
}
}
}

View Slide

POM implementation for Compose
40
object ComposePage {
lateinit var composeTestRule: AndroidComposeTestRule,
ComposeActivity>
get() = composeTestRule.onNodeWithText("count:", substring = true)
get() = composeTestRule.onNodeWithText("count up")
fun clickCountUpButton(): ComposePage = this.also {
countUpButton.performClick()
}
fun assertCounterLabel(expected: Int): ComposePage = this.also {
counterLabel.assertTextEquals("count: $expected")
}
}

View Slide

41
ComposeActivity>
}
}
}

View Slide

42
ComposeActivity>
}
}
}

View Slide

Comparing test codes for Android View
43
// ...
@Test
fun counterTestByUsingPageObject() {
AndroidViewPage
.assertCounterLabel(0)
.clickCountUpButton()
}
}
// ...
@Test
fun counterTest() {
Espresso.onView(withId(R.id.counter_label))
.check(matches(withText("count: 0")))
Espresso.onView(withId(R.id.count_up_button))
.perform(click())
Espresso.onView(withId(R.id.counter_label))
.check(matches(withText("count: 1")))
}
}

View Slide

Comparing test codes for Compose
44
// ...
@Before
fun setUp() {
ComposePage.composeTestRule = composeTestRule
}
@Test
ComposePage
}
}
// ...
@Test
fun counterTest() {
composeTestRule
.onNodeWithText("count: 0")
.assertIsDisplayed()
composeTestRule
.onNodeWithText("count up")
.performClick()
composeTestRule
.onNodeWithText("count: 1")
}
}

View Slide

Comparing test codes with Page Objects
45
// ...
@Test
AndroidViewPage
// Given: assert initial count
// When: click count up button
// Then: assert that count is incremented
}
}
// ...
@Before
fun setUp() {
ComposePage.composeTestRule = composeTestRule
}
@Test
ComposePage
// Given: assert initial count
// When: click count up button
// Then: assert that count is incremented
}
}
↓Android View
Compose→

View Slide

● Prerequisites
1. Transition E2E from Android View to
Compose WITHOUT changing any
features
2. Switching both UIs by using feature
flags
■ Transitioning in stages, NOT all
at once
Migrating E2E tests from Android View to Compose
46

View Slide

1. Create a new screen with Compose
2. Introduce Feature Flags mechanism
3. Switch both screens with the Feature Flags
4. Create a Page Object interface for these screens
5. Implement the interface for Android View and Compose
6. Create Fake Feature Flags and replace with it
7. Switch these Page Objects with the Feature Flags
Transition steps
47

View Slide

Transition steps for production codes
48

View Slide

Transition steps for E2E test codes
49

View Slide

50
@Composable
fun ComposeScreen() {
var counter by remember { mutableIntStateOf(0) }
Box(modifier = Modifier.fillMaxSize().padding(16.dp)) {
Text("ComposeActivity", modifier =
Modifier.align(Alignment.TopStart))
Column(
modifier = Modifier.align(Alignment.Center),
verticalArrangement = Arrangement.Center,
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text("count: $counter")
Spacer(modifier = Modifier.height(16.dp))
Button(onClick = { counter++ }) {
Text("count up")
}
}
}

View Slide

2-1. Introduce Feature Flags mechanism
51
interface FeatureFlagProvider {
fun shouldUseComposeScreen(): Boolean
}
class FeatureFlagProviderImpl @Inject constructor(): FeatureFlagProvider {
// * In an actual development, it should be received from external sources
override fun shouldUseComposeScreen(): Boolean = true
}

View Slide

2-2. Configure dependency injection
52
@Module
@InstallIn(SingletonComponent::class)
abstract class DiModule {
@Singleton
@Binds
abstract fun bindFeatureFlagProvider(
impl: FeatureFlagProviderImpl,
): FeatureFlagProvider
}

View Slide

53
@AndroidEntryPoint
class MainActivity : ComponentActivity() {
@Inject
lateinit var featureFlagProvider: FeatureFlagProvider
override fun onCreate(savedInstanceState: Bundle?) {
// In setContent()
val shouldUseComposeScreen = remember { featureFlagProvider.shouldUseComposeScreen() }
Button(
onClick = {
if (shouldUseComposeScreen) {
startActivity(ComposeActivity.createIntent(this@MainActivity))
} else {
startActivity(AndroidViewActivity.createIntent(this@MainActivity))
}
}
) {
Text("Go to next activity")
}
// ...
}
}

View Slide

Transition steps for E2E test codes
54

View Slide

4. Create a Page Object interface
55
interface TransitionTargetPage {
fun clickCountUpButton(): TransitionTargetPage
fun assertCounterLabel(expected: Int): TransitionTargetPage
// New assertion for checking if screen is switched successfully
fun assertScreenName(): TransitionTargetPage
}

View Slide

5-1. Implement the interface for Android View
56
object AndroidViewPage : TransitionTargetPage {
// ...
override fun clickCountUpButton(): TransitionTargetPage = this.also {
countUpButton.perform(click())
}
override fun assertCounterLabel(expected: Int): TransitionTargetPage = this.also {
counterLabel.check(matches(withText("count: $expected")))
}
override fun assertScreenName(): TransitionTargetPage = this.also {
screenNameLabel.check(matches(withText("AndroidViewActivity")))
}
}

View Slide

5-1. Implement the interface for Android View
57
object AndroidViewPage : TransitionTargetPage {
// ...
countUpButton.perform(click())
}
counterLabel.check(matches(withText("count: $expected")))
}
screenNameLabel.check(matches(withText("AndroidViewActivity")))
}
}

View Slide

5-2. Implement the interface for Compose
58
object ComposePage : TransitionTargetPage {
// ...
}
}
screenNameLabel.assertIsDisplayed()
}
}

View Slide

5-3. Add a method to switch them
59
interface TransitionTargetPage {
// ...
companion object {
fun get(isComposeScreen: Boolean): TransitionTargetPage {
return if (isComposeScreen) ComposePage else AndroidViewPage
}
}
}

View Slide

6-1. Replace Feature Flag with Fake
60
class FakeFeatureFlagResolver @Inject constructor() : FeatureFlagProvider {
override fun shouldUseComposeScreen(): Boolean = false
}
@Module
@TestInstallIn(components = [SingletonComponent::class], replaces = [DiModule::class])
abstract class FakeDiModule {
@Singleton
@Binds
abstract fun bindFeatureFlagResolver(
impl: FakeFeatureFlagResolver,
): FeatureFlagProvider
}

View Slide

6-2. Setup for Fake Feature Flag
61
@HiltAndroidTest
class MainActivityInstrumentedTest {
@get:Rule(order = 0)
val hiltRule = HiltAndroidRule(this)
// ...
@Inject
lateinit var featureFlagProvider: FeatureFlagProvider
@Before
fun setUp() {
hiltRule.inject()
}
}

View Slide

7-1. Create MainPage for MainActivity
62
object MainPage {
lateinit var composeTestRule: AndroidComposeTestRule, MainActivity>
private val button
get() = composeTestRule.onNodeWithText("Go to next activity")
fun clickNavigationButton(isComposeScreen: Boolean): TransitionTargetPage =
TransitionTargetPage.get(isComposeScreen).also {
button.performClick()
}
}

View Slide

7-2. Create E2E tests that can switch screens
63
@HiltAndroidTest
class MainActivityInstrumentedTest {
// ...
@Test
fun navigationTest() {
MainPage
.clickNavigationButton(featureFlagProvider.shouldUseComposeScreen())
.assertScreenName()
}
}

View Slide

64
Complete 🎉
true
false
Value of Fake
FeatureFlagProvider
Compose
Page
Android View
Page
TransitionTargetPage
TransitionTargetPage

View Slide

Working with the QA team
65
QA
Create
Test Case
Add to
TestRail
Create
Task ticket
Implement
E2E test
Update
TestRail
QA QA
● Create a task
ticket for devs to
work on
● Flag TestRail test
case as
automated
● Create new test
cases
● Update existing
test case
Engineer
● Flag TestRail
case as a
manual test
● 💪💪💪💪
Engineer Engineer

View Slide

Maintaining tests as a team
66
● Tackling similar tasks: Great opportunity to share tips
● Work as a team! Pair pro, mob pro it, talk while you wait for builds
● Keep a tips & tricks document and write whenever you find
something

View Slide

General tips
from our list of hacks, gotchas, and other things we’ve noticed
67

View Slide

Waiting for tests
68
● Waiting for network calls, animations, transitions…
● waitUntil to wait for something to be completed
composeTestRule
.waitUntil(timeoutMillis = 5000L) {
composeTestRule
.onNodeWithText("Completed!")
.isVisible()
}

View Slide

Waiting for tests
69
● Starting Compose 1.4 we have some newer apis:
@OptIn(ExperimentalTestApi::class)
composeTestRule.waitUntilNodeCount(hasText("Completed!"), count = 1)
composeTestRule.waitUntilAtLeastOneExists(hasTestTag("done"))
composeTestRule.waitUntilExactlyOneExists(hasText("Completed!"))
composeTestRule.waitUntilDoesNotExist(hasText("Loading..."))

View Slide

onNodeWithTag vs onNodeWithText
70
● Similar name, similar arguments = **very** easy to mix up!
● Read what you write! (especially when it’s autocompleted)
composeTestRule
.onNodeWithText(SomeScreenTestTags.COMPONENT_A) // 🥹

View Slide

71
● Look for composables using multiple matchers
● Function of SemanticsMatcher
Multiple matchers using and/or
composeTestRule
.onNodeWithTag("card_a")
.onChildren()
.filterToOne(
hasTestTag("bottom_button") and hasText("Details")
)

View Slide

72
class SemanticsMatcher(
val description: String,
private val matcher: (SemanticsNode) -> Boolean
) {
infix fun and(other: SemanticsMatcher): SemanticsMatcher {
return SemanticsMatcher(
description = "($description) && (${other.description})"
) {
matcher(it) && other.matches(it)
}
}
}
Multiple matchers using and/or

View Slide

73
● Multiple nodes may exist when working with SubcomposeLayouts
Working with SubcomposeLayouts
composeTestRule
.onNodeWithTag("content")
java.lang.AssertionError: Failed to perform isDisplayed check.
Reason: Expected exactly '1' node but found '2' nodes that satisfy:
(TestTag = 'content')

View Slide

74
SubcomposeLayout(modifier) { constraints ->
val mainPlaceables = subcompose(Enum.Main, content).map {
it.measure(Constraints())
}
val maxSize = mainPlaceables.fold(IntSize.Zero) { currentMax, placeable ->
/* calculate size */
}
val resizedPlaceables = subcompose(Enum.Dependent, content).map {
it.measure(Constraints(minWidth = maxSize.width))
}
layout(constraints.maxWidth, constraints.maxHeight) {
/* layout resizedPlaceables */
}
}

View Slide

75
Only one of them is displayed:
composeTestRule
.onAllNodesWithTag("content")[0]
.assertIsDisplayed() // ✅
composeTestRule
.onAllNodesWithTag("content")[1]
java.lang.AssertionError: Assert failed: The component is
not displayed!

View Slide

76
composeTestRule
.onAllNodesWithTag("content")
val resizedPlaceables = subcompose(Enum.Dependent) {
Box(Modifier.testTag("displayed")) { content() }
}
Fetch all nodes:
Add a Test Tag to the one displayed when subcomposing:

View Slide

77
● Very good experience with smart code completion so far
● Especially for repetitive code (like tests!)
Using GitHub Copilot

View Slide

● A good base to start:
78

View Slide

● Filling in data classes, naming enums…
79

View Slide

In Conclusion
80
● Automating E2E test can save resources and help with streamlining
the release process
● Migrating E2E tests from Android View to Compose
○ Introduce Feature Flags mechanism to switch 2 screens in production code
○ Use Page Object model for both test implementation
○ Create Fake and overwrite the configuration of the Feature Flags for E2E tests
○ Switch the Page Objects based on the value of the Feature Flags
● Work with the QA team to divide and conquer
● Many tips to share within the team, great opportunity to have fun!

View Slide

Thanks!
81
@shinmiy / @kenken
Enjoy the rest of DroidKaigi 2023!

View Slide

Maintaining E2E Test Automation as We Transition from View to Compose

Maintaining E2E Test Automation as We Transition from View to Compose

kenken

More Decks by kenken

Other Decks in Programming

Featured

Transcript