Efficient Android UI Testing @alex_zhukovich https://alexzh.com/ 

ANDROID TESTING UI & NON-UI TESTS DEVICES 

Local and remote data sources. DATA LAYERS Vendors produce devices with different OS versions. FRAGMENTATION Phones, Tablets, Watches, TVs and custom devices. DIFFERENT DEVICES Notifications, Widgets, Runtime permissions, App Shortcuts, etc. OS FEATURES 

APPLICATION IS GROWING Users Features 

LOCAL UI* Non-UI INSTRUMENTATION UI Non-UI ANDROID TESTS 

APPROACHES TOOLS LAYOUT INSPECTOR UI TEST RECORDERS FRAMEWORKS TYPES OF UI TESTS 

VIEW INFORMATION 

LAYOUT INSPECTOR DETAIL VIEW INFORMATION We can get information about text, color, alpha, margins, paddings, etc. PIXEL PERFECTNESS We can use built-in tools for comparing design and rendered layouts. 3D VISUALIZATION We can analyze layouts of the application in 3D to a better understanding of nesting layouts. 

ANALYZE ANY APP We can analyze any application without root permissions. UI AUTOMATOR VIEWER 

TEST RECORDER 

APPIUM RECORDER CREATE APPIUM TESTS It allows us to create simple Appium tests in few clicks. GENERATE CODE Appium Recorder generates a code of test case that use an Appium framework. PROBLEM WITH WAITING Generated code has a problem with waiting data from web servers, etc. 

GENERATED CODE BY APPIUM RECORDER public class SampleTest { private AndroidDriver driver; @Before public void setUp() throws MalformedURLException { DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); desiredCapabilities.setCapability("platformName", "Android"); desiredCapabilities.setCapability("ensureWebviewsHavePages", true); URL remoteUrl = new URL("http://localhost:4723/wd/hub"); driver = new AndroidDriver(remoteUrl, desiredCapabilities); } @Test public void sampleTest() { MobileElement el7 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/emailEditText"); el7.sendKeys("[email protected]"); MobileElement el8 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/passwordEditText"); el8.sendKeys("test"); MobileElement el9 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/loginButton"); el9.click(); MobileElement el10 = (MobileElement) driver.findElementByAccessibilityId("Profile"); el10.click(); MobileElement el11 = (MobileElement) driver.findElementByAccessibilityId("More options"); el11.click(); MobileElement el12 = (MobileElement) driver.findElementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/ android.widget.ListView/android.widget.LinearLayout[2]/android.widget.LinearLayout/android.widget.RelativeLayout/android.widget.TextView"); el12.click(); } @After public void tearDown() { driver.quit(); } } 

CREATE ESPRESSO TESTS It allows us to create simple Espresso tests in few clicks. GENERATE CODE Espresso Test Recorder generates a code of test case that use an Espresso framework. PROBLEM WITH WAITING Generated code has a problem with waiting data from web servers, etc ESPRESSO TEST RECORDER 

GENERATED CODE BY ESPRESSO TEST RECORDER @RunWith(AndroidJUnit4.class) public class RecorderActivityTest { @Rule public ActivityTestRule mActivityTestRule = new ActivityTestRule<>(SplashActivity.class); @Test public void recorderActivityTest() { ViewInteraction textInputEditText = onView( allOf(withId(R.id.emailEditText), childAtPosition(childAtPosition(withId(R.id.emailInputLayout),0),0))); textInputEditText.perform(scrollTo(), replaceText("[email protected]"), closeSoftKeyboard()); ViewInteraction textInputEditText2 = onView( allOf(withId(R.id.passwordEditText),childAtPosition(childAtPosition(withId(R.id.passwordInputLayout), 0), 0))); textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard()); ViewInteraction materialButton = onView( allOf(withId(R.id.loginButton), withText("Login"), childAtPosition(childAtPosition(withId(R.id.scrollView2), 0), 4))); materialButton.perform(scrollTo(), click()); ViewInteraction bottomNavigationItemView = onView( allOf( withId(R.id.navigation_profile), withContentDescription("Profile"), childAtPosition(childAtPosition(withId(R.id.navigation), 0), 2), isDisplayed() )); bottomNavigationItemView.perform(click()); ViewInteraction overflowMenuButton = onView( allOf(withContentDescription("More options"), childAtPosition(childAtPosition(withId(R.id.toolbar), 1), 0), isDisplayed())); overflowMenuButton.perform(click()); ViewInteraction appCompatTextView = onView( allOf(withId(R.id.title), withText("Log Out"), childAtPosition(childAtPosition(withId(R.id.content), 0), 0), isDisplayed())); appCompatTextView.perform(click()); } ... } 

UI TESTING FRAMEWORKS 

ESPRESSO APPIUM UI AUTOMATOR 

ESPRESSO START ANY ACTIVITY Espresso allows us to start a test from any Activity. We can also test any fragment in isolation with an additional library. SYNCHRONIZATION Espresso performs synchronization with an app after each onView call. IDLING RESOURCES The Idling Resources mechanism allows the Espresso to wait until the asynchronous operation is finished. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) 

INTERACTION WITH ANY APP Ui Automator framework can interact with any application installed on the device/emulator. INTERACTION WITH OS Ui Automator framework can interact with OS features, like notification, app shortcuts, permissions, etc. INTERACTION WITH DEVICE Ui Automator framework can interact with device buttons, emulate screen rotation, etc. UI AUTOMATOR val email = device.findObject(By.res(PACKAGE, "email")) email.text = EMAIL val password = device.findObject(By.res(PACKAGE, "password")) email.text = PASSWORD val login = device.findObject(By.res(PACKAGE, "login")) if (login != null) { ... } 

APPIUM SUPPORT MANY LANGUAGES We can use different programming languages to create tests using the Apium framework. like Java, Python, Ruby, C#, etc. INTERACTION WITH ANY APP Appium framework can interact with any application installed on the device/ emulator. INTERACTION WITH OS, DEVICE Appium framework can interact with OS features and device buttons, screen rotation, etc. val email = driver.findElement(By.id("email")) email.value = EMAIL val password = driver.findElement(By.id("password")) password.value = PASSWORD val login = driver.findElement(By.id("login")) login.isDisplayed() 

ESPRESSO UI AUTOMATOR APPIUM No SUPPORT OF RESOURCES TEST CASES TYPE EXECUTION TIME Yes Gray box 0.946 s No Black box 8.743 s Black box 12.154 s 

TYPES OF UI TESTS 

END-TO-END TESTS FUNCTIONAL TESTS COMPONENT TESTS VIEW COMPARISON TESTS 

END-TO-END TESTS SCENARIO: Splash Screen 1. Login to application 2. Analyse coffee drink screen 3. Open Profile screen 4. Open Menu 5. Logout from application Open Login screen 

END-TO-END TESTS INTERACTION WITH SERVER Interaction with production server. ENTRY POINT Similar to entry point of app. 

INTERACTION WITH SERVER Interaction with the non-production server. NAVIGATION Usually, navigation is not needed. ENTRY POINT We can start test from required Activity/ Fragment. FUNCTIONAL TESTS 

COMPONENT TESTS SHORT EXECUTION TIME Usually, we have no interaction with the server, no navigation, etc. APP INDEPENDANT TESTS Test cases verify the rendering of the view and usually use fake data. 

DEVICE DEPENDENT Test cases based on screenshots; different devices can have other screen resolutions. MIGRATION REQURE SCRIPTS We need to update all reference screenshots from one screen resolution to another one. VIEW COMPARISON TESTS 

END TO END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar to entry point of app. APP VERIFICATION Slow verification of application. NAVIGATION Navigate to required screen. SERVER INTERACTION Interaction with production server. ENTRY POINT Start required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast verification of components or screens. 

EFFICIENCY READABILITY TIME EFFICIENCY COMPLEXITY 

READABILITY DIFFERENT APPROACHES TIME EFFICIENCY COMPLEXITY 

READABILITY 

ESPRESSO // Enter email onView(withId(R.id.email)) .perform(replaceText(EMAIL)) // Enter password onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) // Press login button onView(withId(R.id.login)) .perform(click()) 

ESPRESSO onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ... 

ESPRESSO // Login screen onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) // Home screen val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ... 

onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) 

onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) } 

onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } 

onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } } 

onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } } fun loginScreen(func: LoginScreenRobot.() -> Unit) = LoginScreenRobot().apply { func() } 

BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } ... } ESPRESSO 

BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { val view = device.findObject(By.res(resId(viewId))) view.text = text } fun clickOnView(viewId: Int) { device.findObject(By.res(resIf(viewId))) .click() } ... } UI AUTOMATOR 

loginScreen { enterEmail(EMAIL) enterPassword(PASSWORD) pressLogin() } homeScreen { waitCoffeeDrinks() pressProfile() } profileScreen { waitUserData() hasEmail(EMAIL) hasUserName(USER_NAME) openMenu() pressLogout() } loginScreen { isOpen() } loginScreen { login(EMAIL, PASSWORD) } homeScreen { navigateToProfile() } profileScreen { hasEmail(EMAIL) hasUserName(USER_NAME) logout() } loginScreen { isOpen() } 

DIFFERENT APPROACHES 

DEBUG SERVER APPLICABLE TO E2E TESTS We can update the URL constant during compilation APK for instrumentation test cases. MAKE TESTS MORE STABLE We can have stable data on the debug server. DATA SYNCHRONIZATION We need to synchronize data between the production and debug server. Production Debug 

SHORT EXECUTION TIME Interaction with files and databases is faster than using UI for data changes. PRE-POPULATING DATA Server Database 

MOCK API RESPONSES MAKE TESTS MORE STABLE We can use predefined fake responses instead of using a production server. NO INTERACTION WITH SERVER Responses from the production server can differ from predefined data. DATA SYNCHRONIZATION We need to synchronize predefined responses with responses from the production server. Mock-Server 

FLEXIBILITY We can define what we want to test and which layers should be mocked. SHORT EXECUTION TIME Short execution time can be achieved if we mock top layers of the application. MOCK APP LAYERS APP DOMAIN DATA 

Activities Fragments 

Fragments @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun superImportantTest() { ... launchFragmentInContainer() ... } } 

TIME EFFICIENCY 

END TO END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar to entry point of app. APP VERIFICATION Slow verification of application. NAVIGATION Navigate to required screen. SERVER INTERACTION Interaction with production server. ENTRY POINT Start required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast verification of components or screens. 

END-TO-END TESTS FUNCTIONAL TESTS 

COMPLEXITY 

FRAMEWORKS DOMAIN SPECIFIC LANGUAGE TEST INFRASRUCTURE KNOWN APPROACHES 

UI TESTING FRAMEWORKS APPIUM ESPRESSO KAKAO BARISTA KASPERSSO UI AUTOMATOR 

APPIUM UI AUTOMATOR ESPRESSO KAKAO KASPRESSO BARISTA KAUTOMATOR 

BARISTA LESS BOILERPLATE CODE Barista API is removing a lot of boilerplate code with its own abstraction over Espresso. USEFUL TEST RULES Barista library provides JUnit rules for clearing data on device/emulation, like ClearFilesRule, ClearPreferencesRule, etc. HANDLING FLAKY TESTS Barista library provides annotations for handling flaky tests, like AllowFlaky, Repeat. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL) 

PREDEFINED DSL Kakao framework provides an abstraction on top of the Espresso framework. INTERCEPTORS MECHANISM An interceptor allows you to inject other logic between Kakao → Espresso call chain. KAKAO onScreen { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } 

KASPRESSO BASED ON KAKAO We have access to all features from Kakao and Espresso frameworks. ADVANCED REPORTING SYSTEM We can generate human-readable reports with the Kaspresso framework. HANDLING FLAKY TESTS Kaspresso support functions for handling flaky operation based on interceptor mechanism, like flakySafely. beforeTest { activityTestRule.launchActivity(intent) }.afterTest { ... }.run { onScreen { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } } 

BEST PRACTICES NAMING QUALITY FLAKY TESTS 

LEARN YOUR TOOLS 

PAY ATTENTION TO NAMING 

NO "SLEEP" IN TESTS 

ALL TEST CASES SHOULD BE INDEPENDENT 

MAKE AN ABSTRACTION OVER THE FRAMEWORK 

FOLLOW THE "NO FLAKY TESTS" POLICY 

DON’T RELY ONLY ON UI TEST AUTOMATION 

THANK YOU FOR LISTENING! @alex_zhukovich https://alexzh.com/ alex-zhukovich