Efficient Android UI Testing

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate automated tests to your project in an efficient way because they should be fast and stable.

This talk will cover the following topics:
– DSL (Domain Specific Language) in UI Testing
– Efficient combination of different types of UI tests
– Best practices for creating test suit for Android application
– Popular mistakes in UI Testing

Twitter: https://twitter.com/alex_zhukovich
LinkedIn: https://www.linkedin.com/in/alex-zhukovich/
Blog: https://alexzh.com

Transcript

1. Efficient Android UI Testing @alex_zhukovich https://alexzh.com/
2. ANDROID TESTING UI & NON-UI TESTS DEVICES
3. Local and remote data sources. DATA LAYERS Vendors produce devices with different OS versions. FRAGMENTATION Phones, Tablets, Watches, TVs and custom devices. DIFFERENT DEVICES Notifications, Widgets, Runtime permissions, App Shortcuts, etc. OS FEATURES
4. APPLICATION IS GROWING Users Features
5. LOCAL UI* Non-UI INSTRUMENTATION UI Non-UI ANDROID TESTS
6. APPROACHES TOOLS LAYOUT INSPECTOR UI TEST RECORDERS FRAMEWORKS TYPES OF UI TESTS
7. VIEW INFORMATION
8. LAYOUT INSPECTOR DETAIL VIEW INFORMATION We can get information about text, color, alpha, margins, paddings, etc. PIXEL PERFECTNESS We can use built-in tools for comparing design and rendered layouts. 3D VISUALIZATION We can analyze layouts of the application in 3D to a better understanding of nesting layouts.
9. ANALYZE ANY APP We can analyze any application without root permissions. UI AUTOMATOR VIEWER
10. TEST RECORDER
11. APPIUM RECORDER CREATE APPIUM TESTS It allows us to create simple Appium tests in few clicks. GENERATE CODE Appium Recorder generates a code of test case that use an Appium framework. PROBLEM WITH WAITING Generated code has a problem with waiting data from web servers, etc.
12. GENERATED CODE BY APPIUM RECORDER public class SampleTest { private AndroidDriver driver; @Before public void setUp() throws MalformedURLException { DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); desiredCapabilities.setCapability("platformName", "Android"); desiredCapabilities.setCapability("ensureWebviewsHavePages", true); URL remoteUrl = new URL("http://localhost:4723/wd/hub"); driver = new AndroidDriver(remoteUrl, desiredCapabilities); } @Test public void sampleTest() { MobileElement el7 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/emailEditText"); el7.sendKeys("test@alexzh.com"); MobileElement el8 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/passwordEditText"); el8.sendKeys("test"); MobileElement el9 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/loginButton"); el9.click(); MobileElement el10 = (MobileElement) driver.findElementByAccessibilityId("Profile"); el10.click(); MobileElement el11 = (MobileElement) driver.findElementByAccessibilityId("More options"); el11.click(); MobileElement el12 = (MobileElement) driver.findElementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/ android.widget.ListView/android.widget.LinearLayout[2]/android.widget.LinearLayout/android.widget.RelativeLayout/android.widget.TextView"); el12.click(); } @After public void tearDown() { driver.quit(); } }
13. CREATE ESPRESSO TESTS It allows us to create simple Espresso tests in few clicks. GENERATE CODE Espresso Test Recorder generates a code of test case that use an Espresso framework. PROBLEM WITH WAITING Generated code has a problem with waiting data from web servers, etc ESPRESSO TEST RECORDER
14. GENERATED CODE BY ESPRESSO TEST RECORDER @RunWith(AndroidJUnit4.class) public class RecorderActivityTest { @Rule public ActivityTestRule<SplashActivity> mActivityTestRule = new ActivityTestRule<>(SplashActivity.class); @Test public void recorderActivityTest() { ViewInteraction textInputEditText = onView( allOf(withId(R.id.emailEditText), childAtPosition(childAtPosition(withId(R.id.emailInputLayout),0),0))); textInputEditText.perform(scrollTo(), replaceText("test@alexzh.com"), closeSoftKeyboard()); ViewInteraction textInputEditText2 = onView( allOf(withId(R.id.passwordEditText),childAtPosition(childAtPosition(withId(R.id.passwordInputLayout), 0), 0))); textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard()); ViewInteraction materialButton = onView( allOf(withId(R.id.loginButton), withText("Login"), childAtPosition(childAtPosition(withId(R.id.scrollView2), 0), 4))); materialButton.perform(scrollTo(), click()); ViewInteraction bottomNavigationItemView = onView( allOf( withId(R.id.navigation_profile), withContentDescription("Profile"), childAtPosition(childAtPosition(withId(R.id.navigation), 0), 2), isDisplayed() )); bottomNavigationItemView.perform(click()); ViewInteraction overflowMenuButton = onView( allOf(withContentDescription("More options"), childAtPosition(childAtPosition(withId(R.id.toolbar), 1), 0), isDisplayed())); overflowMenuButton.perform(click()); ViewInteraction appCompatTextView = onView( allOf(withId(R.id.title), withText("Log Out"), childAtPosition(childAtPosition(withId(R.id.content), 0), 0), isDisplayed())); appCompatTextView.perform(click()); } ... }
15. UI TESTING FRAMEWORKS
16. ESPRESSO APPIUM UI AUTOMATOR
17. ESPRESSO START ANY ACTIVITY Espresso allows us to start a test from any Activity. We can also test any fragment in isolation with an additional library. SYNCHRONIZATION Espresso performs synchronization with an app after each onView call. IDLING RESOURCES The Idling Resources mechanism allows the Espresso to wait until the asynchronous operation is finished. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed()))
18. INTERACTION WITH ANY APP Ui Automator framework can interact with any application installed on the device/emulator. INTERACTION WITH OS Ui Automator framework can interact with OS features, like notification, app shortcuts, permissions, etc. INTERACTION WITH DEVICE Ui Automator framework can interact with device buttons, emulate screen rotation, etc. UI AUTOMATOR val email = device.findObject(By.res(PACKAGE, "email")) email.text = EMAIL val password = device.findObject(By.res(PACKAGE, "password")) email.text = PASSWORD val login = device.findObject(By.res(PACKAGE, "login")) if (login != null) { ... }
19. APPIUM SUPPORT MANY LANGUAGES We can use different programming languages to create tests using the Apium framework. like Java, Python, Ruby, C#, etc. INTERACTION WITH ANY APP Appium framework can interact with any application installed on the device/ emulator. INTERACTION WITH OS, DEVICE Appium framework can interact with OS features and device buttons, screen rotation, etc. val email = driver.findElement(By.id("email")) email.value = EMAIL val password = driver.findElement(By.id("password")) password.value = PASSWORD val login = driver.findElement(By.id("login")) login.isDisplayed()
20. ESPRESSO UI AUTOMATOR APPIUM No SUPPORT OF RESOURCES TEST CASES TYPE EXECUTION TIME Yes Gray box 0.946 s No Black box 8.743 s Black box 12.154 s
21. TYPES OF UI TESTS
22. END-TO-END TESTS FUNCTIONAL TESTS COMPONENT TESTS VIEW COMPARISON TESTS
23. END-TO-END TESTS SCENARIO: Splash Screen 1. Login to application 2. Analyse coffee drink screen 3. Open Profile screen 4. Open Menu 5. Logout from application Open Login screen
24. END-TO-END TESTS INTERACTION WITH SERVER Interaction with production server. ENTRY POINT Similar to entry point of app.
25. INTERACTION WITH SERVER Interaction with the non-production server. NAVIGATION Usually, navigation is not needed. ENTRY POINT We can start test from required Activity/ Fragment. FUNCTIONAL TESTS
26. COMPONENT TESTS SHORT EXECUTION TIME Usually, we have no interaction with the server, no navigation, etc. APP INDEPENDANT TESTS Test cases verify the rendering of the view and usually use fake data.
27. DEVICE DEPENDENT Test cases based on screenshots; different devices can have other screen resolutions. MIGRATION REQURE SCRIPTS We need to update all reference screenshots from one screen resolution to another one. VIEW COMPARISON TESTS
28. END TO END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar to entry point of app. APP VERIFICATION Slow verification of application. NAVIGATION Navigate to required screen. SERVER INTERACTION Interaction with production server. ENTRY POINT Start required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast verification of components or screens.
29. EFFICIENCY READABILITY TIME EFFICIENCY COMPLEXITY
30. READABILITY DIFFERENT APPROACHES TIME EFFICIENCY COMPLEXITY
31. READABILITY
32. ESPRESSO // Enter email onView(withId(R.id.email)) .perform(replaceText(EMAIL)) // Enter password onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) // Press login button onView(withId(R.id.login)) .perform(click())
33. ESPRESSO onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
34. ESPRESSO // Login screen onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) // Home screen val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
35. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed()))
36. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) }
37. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } }
38. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } }
39. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } } fun loginScreen(func: LoginScreenRobot.() -> Unit) = LoginScreenRobot().apply { func() }
40. BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } ... } ESPRESSO
41. BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { val view = device.findObject(By.res(resId(viewId))) view.text = text } fun clickOnView(viewId: Int) { device.findObject(By.res(resIf(viewId))) .click() } ... } UI AUTOMATOR
42. loginScreen { enterEmail(EMAIL) enterPassword(PASSWORD) pressLogin() } homeScreen { waitCoffeeDrinks() pressProfile() } profileScreen { waitUserData() hasEmail(EMAIL) hasUserName(USER_NAME) openMenu() pressLogout() } loginScreen { isOpen() } loginScreen { login(EMAIL, PASSWORD) } homeScreen { navigateToProfile() } profileScreen { hasEmail(EMAIL) hasUserName(USER_NAME) logout() } loginScreen { isOpen() }
43. DIFFERENT APPROACHES
44. DEBUG SERVER APPLICABLE TO E2E TESTS We can update the URL constant during compilation APK for instrumentation test cases. MAKE TESTS MORE STABLE We can have stable data on the debug server. DATA SYNCHRONIZATION We need to synchronize data between the production and debug server. Production Debug
45. SHORT EXECUTION TIME Interaction with files and databases is faster than using UI for data changes. PRE-POPULATING DATA Server Database
46. MOCK API RESPONSES MAKE TESTS MORE STABLE We can use predefined fake responses instead of using a production server. NO INTERACTION WITH SERVER Responses from the production server can differ from predefined data. DATA SYNCHRONIZATION We need to synchronize predefined responses with responses from the production server. Mock-Server
47. FLEXIBILITY We can define what we want to test and which layers should be mocked. SHORT EXECUTION TIME Short execution time can be achieved if we mock top layers of the application. MOCK APP LAYERS APP DOMAIN DATA
48. Activities Fragments
49. Fragments @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun superImportantTest() { ... launchFragmentInContainer<LoginFragment>() ... } }
50. TIME EFFICIENCY
51. END TO END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar to entry point of app. APP VERIFICATION Slow verification of application. NAVIGATION Navigate to required screen. SERVER INTERACTION Interaction with production server. ENTRY POINT Start required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast verification of components or screens.
52. END-TO-END TESTS FUNCTIONAL TESTS
53. COMPLEXITY
54. FRAMEWORKS DOMAIN SPECIFIC LANGUAGE TEST INFRASRUCTURE KNOWN APPROACHES
55. UI TESTING FRAMEWORKS APPIUM ESPRESSO KAKAO BARISTA KASPERSSO UI AUTOMATOR
56. APPIUM UI AUTOMATOR ESPRESSO KAKAO KASPRESSO BARISTA KAUTOMATOR
57. BARISTA LESS BOILERPLATE CODE Barista API is removing a lot of boilerplate code with its own abstraction over Espresso. USEFUL TEST RULES Barista library provides JUnit rules for clearing data on device/emulation, like ClearFilesRule, ClearPreferencesRule, etc. HANDLING FLAKY TESTS Barista library provides annotations for handling flaky tests, like AllowFlaky, Repeat. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL)
58. PREDEFINED DSL Kakao framework provides an abstraction on top of the Espresso framework. INTERCEPTORS MECHANISM An interceptor allows you to inject other logic between Kakao → Espresso call chain. KAKAO onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() }
59. KASPRESSO BASED ON KAKAO We have access to all features from Kakao and Espresso frameworks. ADVANCED REPORTING SYSTEM We can generate human-readable reports with the Kaspresso framework. HANDLING FLAKY TESTS Kaspresso support functions for handling flaky operation based on interceptor mechanism, like flakySafely. beforeTest { activityTestRule.launchActivity(intent) }.afterTest { ... }.run { onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } }
60. BEST PRACTICES NAMING QUALITY FLAKY TESTS
61. LEARN YOUR TOOLS
62. PAY ATTENTION TO NAMING
63. NO "SLEEP" IN TESTS
64. ALL TEST CASES SHOULD BE INDEPENDENT
65. MAKE AN ABSTRACTION OVER THE FRAMEWORK
66. FOLLOW THE "NO FLAKY TESTS" POLICY
67. DON’T RELY ONLY ON UI TEST AUTOMATION
68. THANK YOU FOR LISTENING! @alex_zhukovich https://alexzh.com/ alex-zhukovich