Writing Robust Android UI Tests

Writing Robust Android UI Tests

Mobile apps are growing. They are becoming more complex and require more testing. This means that it is the perfect time to integrate automated tests in your project to efficiently make them fast and stable.

During this talk, we will explore how to create a lot of test cases efficiently. We will start with a fast introduction to Espresso and UiAutomator, Kakao frameworks. Afterwards, we will move to write different types of UI tests, and finally, we create a DSL which helps us to write stable test cases with minimum noise.

2b0404a5db1a74f01bf3bf94d142e28c?s=128

Alex Zhukovich

September 10, 2020
Tweet

Transcript

  1. Writing Robust Android UI Tests @Alex_Zhukovich

  2. None
  3. Non-UI testing UI testing Stress testing View Comparison testing

  4. Android tests Local Instrumentation

  5. Android tests Local Instrumentation Non-UI UI* UI Non-UI

  6. adb shell am instrument Instrumentation >_ Test app Dev app

  7. https://github.com/AlexZhukovich/ImBarista-App

  8. None
  9. public class SampleTest { private AndroidDriver driver; @Before public void

    setUp() throws MalformedURLException { DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); desiredCapabilities.setCapability("platformName", "Android"); desiredCapabilities.setCapability("ensureWebviewsHavePages", true); URL remoteUrl = new URL("http://localhost:4723/wd/hub"); driver = new AndroidDriver(remoteUrl, desiredCapabilities); } @Test public void sampleTest() { MobileElement el7 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/emailEditText"); el7.sendKeys("test@alexzh.com"); MobileElement el8 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/passwordEditText"); el8.sendKeys("test"); MobileElement el9 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/loginButton"); el9.click(); MobileElement el10 = (MobileElement) driver.findElementByAccessibilityId("Profile"); el10.click(); MobileElement el11 = (MobileElement) driver.findElementByAccessibilityId("More options"); el11.click(); MobileElement el12 = (MobileElement) driver.findElementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/android.widget.ListView/android.widget.LinearLayout[2] /android.widget.LinearLayout/android.widget.RelativeLayout/android.widget.TextView"); el12.click(); } @After public void tearDown() { driver.quit(); } } Appium Recorder
  10. @RunWith(AndroidJUnit4.class) public class RecorderActivityTest { @Rule public ActivityTestRule<SplashActivity> mActivityTestRule =

    new ActivityTestRule<>(SplashActivity.class); @Test public void recorderActivityTest() { ViewInteraction textInputEditText = onView( allOf(withId(R.id.emailEditText), childAtPosition(childAtPosition(withId(R.id.emailInputLayout),0),0))); textInputEditText.perform(scrollTo(), replaceText("test@alexzh.com"), closeSoftKeyboard()); ViewInteraction textInputEditText2 = onView( allOf(withId(R.id.passwordEditText),childAtPosition(childAtPosition(withId(R.id.passwordInputLayout), 0), 0))); textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard()); ViewInteraction materialButton = onView( allOf(withId(R.id.loginButton), withText("Login"), childAtPosition(childAtPosition(withId(R.id.scrollView2), 0), 4))); materialButton.perform(scrollTo(), click()); ViewInteraction bottomNavigationItemView = onView( allOf( withId(R.id.navigation_profile), withContentDescription("Profile"), childAtPosition(childAtPosition(withId(R.id.navigation), 0), 2), isDisplayed() )); bottomNavigationItemView.perform(click()); ViewInteraction overflowMenuButton = onView( allOf(withContentDescription("More options"), childAtPosition(childAtPosition(withId(R.id.toolbar), 1), 0), isDisplayed())); overflowMenuButton.perform(click()); ViewInteraction appCompatTextView = onView( allOf(withId(R.id.title), withText("Log Out"), childAtPosition(childAtPosition(withId(R.id.content), 0), 0), isDisplayed())); appCompatTextView.perform(click()); } ... } Espresso Test Recorder
  11. val textInputEditText = onView( allOf( withId(R.id.emailEditText), childAtPosition( childAtPosition( withId(R.id.emailInputLayout), 0

    ), 0 ) ) ) textInputEditText.perform(scrollTo(), replaceText("test@alexzh.com"), closeSoftKeyboard()) val textInputEditText2 = onView( allOf( withId(R.id.passwordEditText), childAtPosition( childAtPosition( withId(R.id.passwordInputLayout), 0 ), 0 ) ) ) textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard()) val materialButton = onView( allOf( withId(R.id.loginButton), withText("Login"), childAtPosition( childAtPosition( withId(R.id.scrollView2), 0 ), 4 ) ) ) materialButton.perform(scrollTo(), click()) onView(withId(R.id.emailEditText)) .perform(replaceText("test@alexzh.com")) onView(withId(R.id.passwordEditText)) .perform(replaceText("test")) onView(withId(R.id.loginButton)) .perform(click())
  12. Changes in app Network speed Unreadability Test data

  13. Layout Inspector UI Automator Viewer

  14. @Test fun shouldDisplayErrorWhenPasswordIsBlank() { val email = "test-account@alexzh.com" val password

    = "" onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password)) onView(withId(R.id.loginButton)) .perform(click()) onView(withText(R.string.password_is_blank)) .check(matches(isDisplayed())) }
  15. onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) val progressBarIR =

    ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
  16. // -- LOGIN -- onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton))

    .perform(click()) // -- HOME SCREEN -- val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
  17. Domain Specific Language

  18. Scenario: 1. Login to application

  19. Scenario: 1. Login to application 2. Open Profile screen 3.

    Check name and email
  20. Scenario: 1. Login to application 2. Open Profile screen 3.

    Check name and email 4. Open menu 5. Logout from application
  21. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click())

  22. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click()) open class

    BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } }
  23. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click()) open class

    BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } }
  24. // LOGIN onView(withId(R.id.email)) .perform(replaceText(EMAIL)) onView(withId(R.id.password)) .perform(replaceText(PASSWORD)) onView(withId(R.id.login)) .perform(click()) loginScreen {

    login(EMAIL, PASSWORD) } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } } fun loginScreen(func: LoginScreenRobot.() -> Unit) = LoginScreenRobot().apply { func() }
  25. loginScreen { enterEmail(EMAIL) enterPassword(PASSWORD) pressLogin() } homeScreen { waitCoffeeDrinks() pressProfile()

    } profileScreen { waitUserData() hasEmail(EMAIL) hasUserName(USER_NAME) openMenu() pressLogout() } loginScreen { isOpen() } loginScreen { login(EMAIL, PASSWORD) } homeScreen { navigateToProfile() } profileScreen { hasEmail(EMAIL) hasUserName(USER_NAME) logout() } loginScreen { isOpen() }
  26. Test case Screen Robot Basic operations

  27. Readability Reusability Building complexity Framework independant

  28. Approaches and tools

  29. None
  30. Clear data

  31. None
  32. Mock Fake

  33. End to End tests Functional tests 1 1 Entry point

    Start test from the main screen 2 Interaction with server Verification interaction with server 3 Navigation Verification navigation between screens Entry point Start application from any screen 2 Interaction with server Usually no interaction with prod server 3 Navigation Usually no navigation between screen Navigation between neighbor screens 4 UI verification Fast verification of UI components 4 App verification Slow verification of entire product
  34. Espresso UiAutomator Appium onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) Synchronization

    IdlingResources Run any Activity
  35. Espresso UiAutomator Appium val email = device.findObject(By.res(PACKAGE, "email")) email.text =

    EMAIL val password = device.findObject(By.res(PACKAGE, "password")) email.text = PASSWORD val login = device.findObject(By.res(PACKAGE, "login")) if (login != null) { ... } Interact with OS Interact with any application
  36. Espresso UiAutomator Appium val email = driver.findElement(By.id("email")) email.value = EMAIL

    val password = driver.findElement(By.id("password")) password.value = PASSWORD val login = driver.findElement(By.id("login")) login.isDisplayed() Interact with OS Interact with any application Support many languages
  37. Espresso UiAutomator Appium Support of Android resources Yes No No

  38. Espresso UiAutomator Appium Support of Android resources Yes No No

    Test cases type GrayBox BlackBox BlackBox
  39. Espresso UiAutomator Appium Support of Android resources Yes No No

    Test cases type GrayBox BlackBox BlackBox Execution time 0.976 s 8.743 s 12.154 s
  40. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL) Kakao

    Barista Kaspresso
  41. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL) @Rule

    ClearPreferencesRule @Rule ClearDatabaseRule @Rule ClearFilesRule Kakao Barista Kaspresso
  42. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL) @Rule

    ClearPreferencesRule @Rule ClearDatabaseRule @Rule ClearFilesRule @Test @AllowFlaky(attempts = 5) @Repeat(times = 5) fun flaky_test() { ... } Kakao Barista Kaspresso
  43. @Test fun should_verify_account_information() { val toolbarVisibilityIR = ViewVisibilityIdlingResource(R.id.toolbar, View.VISIBLE) IdlingRegistry.getInstance().register(toolbarVisibilityIR)

    assertDisplayed(R.id.toolbar) IdlingRegistry.getInstance().unregister(toolbarVisibilityIR) // LOGIN writeTo(R.id.emailEditText, EMAIL) writeTo(R.id.passwordEditText, PASSWORD) closeKeyboard() clickOn(R.id.loginButton) // NAVIGATE TO PROFILE val progressBarIR = ViewVisibilityIdlingResource(R.id.progressBar, View.GONE) IdlingRegistry.getInstance().register(progressBarIR) assertDisplayed(R.id.recyclerView) IdlingRegistry.getInstance().unregister(progressBarIR) clickOn(R.id.navigation_profile) // VERIFY PROFILE INFO val progressProfileIR = ViewVisibilityIdlingResource(R.id.progressBar, View.GONE) IdlingRegistry.getInstance().register(progressProfileIR) assertDisplayed(R.id.nameTextView, USER_NAME) assertDisplayed(R.id.emailTextView, EMAIL) IdlingRegistry.getInstance().unregister(progressProfileIR) // LOGOUT clickMenu(R.id.logout_action) // LOGIN SCREEN intended(hasComponent(LoginActivity::class.java.name)) } Kakao Barista Kaspresso
  44. Kakao Barista Kaspresso onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard()

    } login.click() }
  45. onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() }

    Interceptors Kakao Barista Kaspresso
  46. @Test fun should_verify_account_information() { onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD)

    closeSoftKeyboard() } login.click() } onScreen<HomeScreen> { waitCoffeeDrinkList() profileMenuItem.click() } onScreen<ProfileScreen> { waitForProfileInfo() name.hasText(USER_NAME) email.hasText(EMAIL) openMenu() logout.click() } onScreen<LoginScreen> { loginScreenIntent.intended() } } Kakao Barista Kaspresso
  47. onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() }

    Include KAutomator Injecting data DSL Interaction with Device Advanced reporting system Handling flaky tests Kakao Barista Kaspresso
  48. Kakao @Test fun should_verify_account_information() = beforeTest { activityTestRule.launchActivity(intent) }.afterTest {

    ... }.run { onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } onScreen<HomeScreen> { waitCoffeeDrinkList() profileMenuItem.click() } onScreen<ProfileScreen> { waitForProfileInfo() name.hasText(USER_NAME) email.hasText(EMAIL) openMenu() logout.click() } onScreen<LoginScreen> { loginScreenIntent.intended() } } Barista Kaspresso
  49. Pre-populating WireMock RestMock MockWebServer Fake Mock

  50. Activity Fragment

  51. Activity @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun superImportantTest() { ...

    ActivityScenario.launch( LoginFragment::class.java ) ... } } Fragment @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun superImportantTest() { ... launchFragmentInContainer<LoginFragment>() ... } }
  52. End To End tests Functional tests

  53. Shared tests

  54. Local tests Instrumentation tests Shared tests

  55. android { ... sourceSets { androidTest { java.srcDirs += "src/sharedTest/java"

    } test { java.srcDirs += "src/sharedTest/java" } } } androidTest sharedTest test
  56. How to start

  57. App flow Priorities Analytical data

  58. App flow Priorities Analytical data

  59. App flow Priorities Analytical data

  60. Flaky tests

  61. Testing approaches Tools Parallel execution Test data

  62. FAILED TESTS FLAKY TESTS

  63. Android Test Orchestrator @Rule ClearPreferencesRule ClearDatabaseRule ClearFilesRule Interceptors @AllowFlaky @

  64. #ExploreMore Android UI testing alexzh.com @AlexZhukovich