Writing Robust Android UI Tests

Mobile apps are growing. They are becoming more complex and require more testing. This means that it is the perfect time to integrate automated tests in your project to efficiently make them fast and stable.

During this talk, we will explore how to create a lot of test cases efficiently. We will start with a fast introduction to Espresso and UiAutomator, Kakao frameworks. Afterwards, we will move to write different types of UI tests, and finally, we create a DSL which helps us to write stable test cases with minimum noise.

Transcript

1. Writing Robust Android UI Tests @Alex_Zhukovich
3. Non-UI testing UI testing Stress testing View Comparison testing
4. Android tests Local Instrumentation
5. Android tests Local Instrumentation Non-UI UI* UI Non-UI
6. adb shell am instrument Instrumentation >_ Test app Dev app
7. https://github.com/AlexZhukovich/ImBarista-App
9. public class SampleTest { private AndroidDriver driver; @Before public void setUp() throws MalformedURLException { DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); desiredCapabilities.setCapability("platformName", "Android"); desiredCapabilities.setCapability("ensureWebviewsHavePages", true); URL remoteUrl = new URL("http://localhost:4723/wd/hub"); driver = new AndroidDriver(remoteUrl, desiredCapabilities); } @Test public void sampleTest() { MobileElement el7 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/emailEditText"); el7.sendKeys("test@alexzh.com"); MobileElement el8 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/passwordEditText"); el8.sendKeys("test"); MobileElement el9 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/loginButton"); el9.click(); MobileElement el10 = (MobileElement) driver.findElementByAccessibilityId("Profile"); el10.click(); MobileElement el11 = (MobileElement) driver.findElementByAccessibilityId("More options"); el11.click(); MobileElement el12 = (MobileElement) driver.findElementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/android.widget.ListView/android.widget.LinearLayout[2] /android.widget.LinearLayout/android.widget.RelativeLayout/android.widget.TextView"); el12.click(); } @After public void tearDown() { driver.quit(); } } Appium Recorder
10. @RunWith(AndroidJUnit4.class) public class RecorderActivityTest { @Rule public ActivityTestRule<SplashActivity> mActivityTestRule = new ActivityTestRule<>(SplashActivity.class); @Test public void recorderActivityTest() { ViewInteraction textInputEditText = onView( allOf(withId(R.id.emailEditText), childAtPosition(childAtPosition(withId(R.id.emailInputLayout),0),0))); textInputEditText.perform(scrollTo(), replaceText("test@alexzh.com"), closeSoftKeyboard()); ViewInteraction textInputEditText2 = onView( allOf(withId(R.id.passwordEditText),childAtPosition(childAtPosition(withId(R.id.passwordInputLayout), 0), 0))); textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard()); ViewInteraction materialButton = onView( allOf(withId(R.id.loginButton), withText("Login"), childAtPosition(childAtPosition(withId(R.id.scrollView2), 0), 4))); materialButton.perform(scrollTo(), click()); ViewInteraction bottomNavigationItemView = onView( allOf( withId(R.id.navigation_profile), withContentDescription("Profile"), childAtPosition(childAtPosition(withId(R.id.navigation), 0), 2), isDisplayed() )); bottomNavigationItemView.perform(click()); ViewInteraction overflowMenuButton = onView( allOf(withContentDescription("More options"), childAtPosition(childAtPosition(withId(R.id.toolbar), 1), 0), isDisplayed())); overflowMenuButton.perform(click()); ViewInteraction appCompatTextView = onView( allOf(withId(R.id.title), withText("Log Out"), childAtPosition(childAtPosition(withId(R.id.content), 0), 0), isDisplayed())); appCompatTextView.perform(click()); } ... } Espresso Test Recorder
11. val textInputEditText = onView( allOf( withId(R.id.emailEditText), childAtPosition( childAtPosition( withId(R.id.emailInputLayout), 0 ), 0 ) ) ) textInputEditText.perform(scrollTo(), replaceText("test@alexzh.com"), closeSoftKeyboard()) val textInputEditText2 = onView( allOf( withId(R.id.passwordEditText), childAtPosition( childAtPosition( withId(R.id.passwordInputLayout), 0 ), 0 ) ) ) textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard()) val materialButton = onView( allOf( withId(R.id.loginButton), withText("Login"), childAtPosition( childAtPosition( withId(R.id.scrollView2), 0 ), 4 ) ) ) materialButton.perform(scrollTo(), click()) onView(withId(R.id.emailEditText)) .perform(replaceText("test@alexzh.com")) onView(withId(R.id.passwordEditText)) .perform(replaceText("test")) onView(withId(R.id.loginButton)) .perform(click())
12. Changes in app Network speed Unreadability Test data
13. Layout Inspector UI Automator Viewer
14. @Test fun shouldDisplayErrorWhenPasswordIsBlank() { val email = "test-account@alexzh.com" val password = "" onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password)) onView(withId(R.id.loginButton)) .perform(click()) onView(withText(R.string.password_is_blank)) .check(matches(isDisplayed())) }
15. onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
16. // -- LOGIN -- onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) // -- HOME SCREEN -- val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
17. Domain Specific Language
18. Scenario: 1. Login to application
19. Scenario: 1. Login to application 2. Open Profile screen 3. Check name and email
20. Scenario: 1. Login to application 2. Open Profile screen 3. Check name and email 4. Open menu 5. Logout from application
21. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click())
22. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click()) open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } }
23. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click()) open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } }
24. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click()) loginScreen { login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } } fun loginScreen(func: LoginScreenRobot.() -> Unit) = LoginScreenRobot().apply { func() }
25. loginScreen { enterEmail(EMAIL) enterPassword(PASSWORD) pressLogin() } homeScreen { waitCoffeeDrinks() pressProfile() } profileScreen { waitUserData() hasEmail(EMAIL) hasUserName(USER_NAME) openMenu() pressLogout() } loginScreen { isOpen() } loginScreen { login(EMAIL, PASSWORD) } homeScreen { navigateToProfile() } profileScreen { hasEmail(EMAIL) hasUserName(USER_NAME) logout() } loginScreen { isOpen() }
26. Test case Screen Robot Basic operations
27. Readability Reusability Building complexity Framework independant
28. Approaches and tools
30. Clear data
32. Mock Fake
33. End to End tests Functional tests 1 1 Entry point Start test from the main screen 2 Interaction with server Verification interaction with server 3 Navigation Verification navigation between screens Entry point Start application from any screen 2 Interaction with server Usually no interaction with prod server 3 Navigation Usually no navigation between screen Navigation between neighbor screens 4 UI verification Fast verification of UI components 4 App verification Slow verification of entire product
34. Espresso UiAutomator Appium onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) Synchronization IdlingResources Run any Activity
35. Espresso UiAutomator Appium val email = device.findObject(By.res(PACKAGE, "email")) email.text = EMAIL val password = device.findObject(By.res(PACKAGE, "password")) email.text = PASSWORD val login = device.findObject(By.res(PACKAGE, "login")) if (login != null) { ... } Interact with OS Interact with any application
36. Espresso UiAutomator Appium val email = driver.findElement(By.id("email")) email.value = EMAIL val password = driver.findElement(By.id("password")) password.value = PASSWORD val login = driver.findElement(By.id("login")) login.isDisplayed() Interact with OS Interact with any application Support many languages
37. Espresso UiAutomator Appium Support of Android resources Yes No No
38. Espresso UiAutomator Appium Support of Android resources Yes No No Test cases type GrayBox BlackBox BlackBox
39. Espresso UiAutomator Appium Support of Android resources Yes No No Test cases type GrayBox BlackBox BlackBox Execution time 0.976 s 8.743 s 12.154 s
40. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL) Kakao Barista Kaspresso
41. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL) @Rule ClearPreferencesRule @Rule ClearDatabaseRule @Rule ClearFilesRule Kakao Barista Kaspresso
42. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL) @Rule ClearPreferencesRule @Rule ClearDatabaseRule @Rule ClearFilesRule @Test @AllowFlaky(attempts = 5) @Repeat(times = 5) fun flaky_test() { ... } Kakao Barista Kaspresso
43. @Test fun should_verify_account_information() { val toolbarVisibilityIR = ViewVisibilityIdlingResource(R.id.toolbar, View.VISIBLE) IdlingRegistry.getInstance().register(toolbarVisibilityIR) assertDisplayed(R.id.toolbar) IdlingRegistry.getInstance().unregister(toolbarVisibilityIR) // LOGIN writeTo(R.id.emailEditText, EMAIL) writeTo(R.id.passwordEditText, PASSWORD) closeKeyboard() clickOn(R.id.loginButton) // NAVIGATE TO PROFILE val progressBarIR = ViewVisibilityIdlingResource(R.id.progressBar, View.GONE) IdlingRegistry.getInstance().register(progressBarIR) assertDisplayed(R.id.recyclerView) IdlingRegistry.getInstance().unregister(progressBarIR) clickOn(R.id.navigation_profile) // VERIFY PROFILE INFO val progressProfileIR = ViewVisibilityIdlingResource(R.id.progressBar, View.GONE) IdlingRegistry.getInstance().register(progressProfileIR) assertDisplayed(R.id.nameTextView, USER_NAME) assertDisplayed(R.id.emailTextView, EMAIL) IdlingRegistry.getInstance().unregister(progressProfileIR) // LOGOUT clickMenu(R.id.logout_action) // LOGIN SCREEN intended(hasComponent(LoginActivity::class.java.name)) } Kakao Barista Kaspresso
44. Kakao Barista Kaspresso onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() }
45. onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } Interceptors Kakao Barista Kaspresso
46. @Test fun should_verify_account_information() { onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } onScreen<HomeScreen> { waitCoffeeDrinkList() profileMenuItem.click() } onScreen<ProfileScreen> { waitForProfileInfo() name.hasText(USER_NAME) email.hasText(EMAIL) openMenu() logout.click() } onScreen<LoginScreen> { loginScreenIntent.intended() } } Kakao Barista Kaspresso
47. onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } Include KAutomator Injecting data DSL Interaction with Device Advanced reporting system Handling flaky tests Kakao Barista Kaspresso
48. Kakao @Test fun should_verify_account_information() = beforeTest { activityTestRule.launchActivity(intent) }.afterTest { ... }.run { onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } onScreen<HomeScreen> { waitCoffeeDrinkList() profileMenuItem.click() } onScreen<ProfileScreen> { waitForProfileInfo() name.hasText(USER_NAME) email.hasText(EMAIL) openMenu() logout.click() } onScreen<LoginScreen> { loginScreenIntent.intended() } } Barista Kaspresso
49. Pre-populating WireMock RestMock MockWebServer Fake Mock
50. Activity Fragment
51. Activity @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun superImportantTest() { ... ActivityScenario.launch( LoginFragment::class.java ) ... } } Fragment @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun superImportantTest() { ... launchFragmentInContainer<LoginFragment>() ... } }
52. End To End tests Functional tests
53. Shared tests
54. Local tests Instrumentation tests Shared tests
55. android { ... sourceSets { androidTest { java.srcDirs += "src/sharedTest/java" } test { java.srcDirs += "src/sharedTest/java" } } } androidTest sharedTest test
56. How to start
57. App flow Priorities Analytical data
58. App flow Priorities Analytical data
59. App flow Priorities Analytical data
60. Flaky tests
61. Testing approaches Tools Parallel execution Test data
62. FAILED TESTS FLAKY TESTS
63. Android Test Orchestrator @Rule ClearPreferencesRule ClearDatabaseRule ClearFilesRule Interceptors @AllowFlaky @
64. #ExploreMore Android UI testing alexzh.com @AlexZhukovich