Introducing UI Testing to Your Team

Transcript

Introducing UI Testing to Your Team Mustafa Mutlu

@bekobird Android Engineer at N26 Mustafa MoneyBeam, FPS and BACS

Transactions for the UK

5 2018 15 Number of Android devs

32 15 Now Number of Android devs 2019 5

E2E Tests Integration Tests Unit Tests Testing Strategy

E2E Tests Integration Tests Unit Tests Testing Strategy Slower Faster

More Integration More Isolation

E2E Tests Integration Tests Testing Strategy

A Typical Test Case • ID and name of the

test • Preconditions • Steps with expected result

A Typical Test Case • Step: When I enter MoneyBeam

and select a contact • Expected: Then I see the money amount entrance screen and the screen has these: • Top left: origin account's name and balance • Top right: destination account's name • Center: money input • Center: description text • Bottom: Continue button that is not in enabled/clickable state

• 3200+ test cases • Testing 500 to 850 cases,

in each release • Releasing every once in 3 weeks • This is a lot of work!

• Automated end-to-end UI tests • Both for Android and

iOS • Black-box testing • Uses Appium Photo by ActionVance on Unsplash Batman Project

• Slow test execution • Not included in our CI

pipeline • Diﬃcult to maintain • Flakiness caused by E2E Batman Project

Espresso • Focused on testing the UI (not E2E) •

Maintained by the same feature developers

Robot Pattern https://jakewharton.com/testing-robots/ Recommended talk by Jake Wharton

@Test fun moneyBeamSample() { apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify()) val intent = MoneyBeamTransferActivity.newIntent(targetContext,

"Namey McNameface") setupUser { firstName("John") lastName("Doe") currency("GBP") balance(10.toBigDecimal()) } activityScenarioRule.launchActivity(intent) ...

@Test fun moneyBeamSample() { apiServer.registerJsonResponse(MoneyBeamApiRequests.MyApiRequest("a-param"), responseRaw().jsonify()) val intent = MoneyBeamTransferActivity.newIntent(targetContext,

... assertClickableNavigateUpButton() onView(allOf(withParent(withId(originContainer)), withId(nameText))) .check(matches(allOf(isDisplayed(), withText("John Doe")))) onView(allOf(withParent(withId(originContainer)), withId(initialsText))) .check(matches(allOf(isDisplayed(),

withText("JD")))) onView(allOf(withParent(withId(originContainer)), withId(balanceText))) .check(matches(allOf(isDisplayed(), withText("£10.00")))) assertDisplayed(descriptionText, "Send to Namey") assertNotDisplayed(errorText) isButtonEnabled(continueButton) writeTo(amountEditText, "1100") assertDisplayed(errorText, "You can only send MoneyBeams up to £1,000") assertNotDisplayed(descriptionText) isButtonDisabled(continueButton) }

... assertClickableNavigateUpButton() onView(allOf(withParent(withId(originContainer)), withId(nameText))) .check(matches(allOf(isDisplayed(), withText("John Doe")))) onView(allOf(withParent(withId(originContainer)), withId(initialsText))) .check(matches(allOf(isDisplayed(),

Robot Pattern View ViewModel Domain Data What How How N26

Android App

Robot Pattern View ViewModel Domain Data What How How N26

Android App Test Test What & How

Robot Pattern View ViewModel Domain Data What How How N26

Android App Robot How Test What

Robot Pattern View ViewModel Domain Data What How How N26

Android App Robot Test How What

... assertClickableNavigateUpButton() onView(allOf(withParent(withId(originContainer)), withId(nameText))) .check(matches(allOf(isDisplayed(), withText("John Doe")))) onView(allOf(withParent(withId(originContainer)), withId(initialsText))) .check(matches(allOf(isDisplayed(),

... moneyBeamSampleRobot { assertClickableNavigateUpButton() isOriginDisplayed("John Doe", "JD", "£10.00") isDestinationDisplayed("Namey McNameface",

"NM") isDescriptionDisplayed("Send to Namey") isContinueButtonDisabled() enterAmount("1100") isErrorDisplayed("You can only send MoneyBeams up to £1,000") isContinueButtonDisabled() enterAmount("10") isDescriptionDisplayed("Send to Namey") isContinueButtonEnabled() } }

... moneyBeamSampleRobot { assertClickableNavigateUpButton() isOriginDisplayed("John Doe", "JD", "£10.00") isDestinationDisplayed("Namey McNameface",

class MoneyBeamSampleRobot : BaseRobot() { private val amountEditText = R.id.editMoneyBeamTransferAmount

private val descriptionText = R.id.textMoneyBeamTransferAmountDescription private val errorText = R.id.textMoneyBeamTransferAmountError // other view IDs fun isDescriptionDisplayed(text: String) { assertDisplayed(descriptionText, text) assertNotDisplayed(errorText) } fun isErrorDisplayed(text: String) { ... } fun isOriginDisplayed(name: String, initials: String, balance: String) { onView(allOf(withParent(withId(originContainer)), withId(nameText))).check(matches(allOf(isDisplayed(), withText(name)))) onView(allOf(withParent(withId(originContainer)), withId(initialsText))).check(matches(allOf(isDisplayed(), withText(initials)))) onView(allOf(withParent(withId(originContainer)), withId(balanceText))).check(matches(allOf(isDisplayed(), withText(balance)))) } fun isDestinationDisplayed(name: String, initials: String) { ... } fun isContinueButtonEnabled() = isButtonEnabled(continueButton) fun isContinueButtonDisabled() = isButtonDisabled(continueButton) fun enterAmount(amount: String) = writeTo(amountEditText, amount) fun clickOnContinueButton() = clickOn(continueButton) }

class MoneyBeamSampleRobot : BaseRobot() { private val amountEditText = R.id.editMoneyBeamTransferAmount

Challenges

Animations "To avoid ﬂakiness, we highly recommend that you turn

off system animations on the virtual or physical devices used for testing." - Espresso docs

Animations • System animations • Dynamic Animations from AndroidX Our

solution: add a switch to turn off custom animations

Dealing Asynchronicity • Tests need to know when to wait

and proceed • Causing ﬂakiness Our solution: make use of Idling Resources • OkHttpIdlingResource • RxIdler

Mocking Data Sources

Mocking Data Sources Domain Data

Mocking Data Sources Domain Data Repository Remote Data Source

Mocking Data Sources Domain Data Repository Remote Data Source Framework

Data Source

Mocking Data Sources Domain Data Repository Remote Data Source Fake

Data Source

Dealing with Localization English (US) English (Canada) • Same account

(United States) • Same currency (USD) • Same language (English) • Different device locale • Result: different money representation

Dealing with Localization • Using device locale for formatting monetary

amounts • Test device's locale is unpredictable • Causing ﬂaky tests Our solution: add a test rule to deﬁne the device locale

Dealing With Flakiness • Identify the cause and update the

test • Add retry for failed tests (FlakyTestRule from the Barista library)

Integration with CI Pipeline

Integration with CI Pipeline 3 min

Integration with CI Pipeline 5 min

Integration with CI Pipeline 10 min

Integration with CI Pipeline 20 min

Integration with CI Pipeline Then what?

Selectively Running the Tests • Find changed ﬁles from the

Git history • Find which features are changed • Run the tests of the changed features • If there is a change in a common module, trigger the full run • See: goo.gle/androidx-dependency-tracker

20+ minutes UI Testing Pipeline

3 min UI Testing Pipeline

android-project git:(my-branch) ./run-on-remote.sh -h | Running on Remote usage: -s

| --skip-build Skip building tasks -n name | --test-name name Specify the name of the test. Default is branch name -d device | --device device Specify the name of the device on which the tests will be run. Default is random -p packages | --packages packages The list of packages to run the tests (comma separated). Default is all -l | --log Log the output to log.txt -h | --help Display help

Introducing UI Testing to Your Team

Introducing UI Testing to Your Team

More Decks by Mustafa Berkay Mutlu

Other Decks in Programming

Featured

Transcript