Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Efficient Android UI Testing

Efficient Android UI Testing

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate automated tests to your project in an efficient way because they should be fast and stable.

This talk will cover the following topics:
– DSL (Domain Specific Language) in UI Testing
– Efficient combination of different types of UI tests
– Best practices for creating test suit for Android application
– Popular mistakes in UI Testing

Twitter: https://twitter.com/alex_zhukovich
LinkedIn: https://www.linkedin.com/in/alex-zhukovich/
Blog: https://alexzh.com

2b0404a5db1a74f01bf3bf94d142e28c?s=128

Alex Zhukovich

November 17, 2020
Tweet

Transcript

  1. Efficient Android UI Testing @alex_zhukovich https://alexzh.com/

  2. ANDROID TESTING UI & NON-UI TESTS DEVICES

  3. Local and remote data sources. DATA LAYERS Vendors produce devices

    with different OS versions. FRAGMENTATION Phones, Tablets, Watches, TVs and custom devices. DIFFERENT DEVICES Notifications, Widgets, Runtime permissions, App Shortcuts, etc. OS FEATURES
  4. APPLICATION IS GROWING Users Features

  5. LOCAL UI* Non-UI INSTRUMENTATION UI Non-UI ANDROID TESTS

  6. APPROACHES TOOLS LAYOUT INSPECTOR UI TEST RECORDERS FRAMEWORKS TYPES OF

    UI TESTS
  7. VIEW INFORMATION

  8. LAYOUT INSPECTOR DETAIL VIEW INFORMATION We can get information about

    text, color, alpha, margins, paddings, etc. PIXEL PERFECTNESS We can use built-in tools for comparing design and rendered layouts. 3D VISUALIZATION We can analyze layouts of the application in 3D to a better understanding of nesting layouts.
  9. ANALYZE ANY APP We can analyze any application without root

    permissions. UI AUTOMATOR VIEWER
  10. TEST RECORDER

  11. APPIUM RECORDER CREATE APPIUM TESTS It allows us to create

    simple Appium tests in few clicks. GENERATE CODE Appium Recorder generates a code of test case that use an Appium framework. PROBLEM WITH WAITING Generated code has a problem with waiting data from web servers, etc.
  12. GENERATED CODE BY APPIUM RECORDER public class SampleTest { private

    AndroidDriver driver; @Before public void setUp() throws MalformedURLException { DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); desiredCapabilities.setCapability("platformName", "Android"); desiredCapabilities.setCapability("ensureWebviewsHavePages", true); URL remoteUrl = new URL("http://localhost:4723/wd/hub"); driver = new AndroidDriver(remoteUrl, desiredCapabilities); } @Test public void sampleTest() { MobileElement el7 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/emailEditText"); el7.sendKeys("test@alexzh.com"); MobileElement el8 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/passwordEditText"); el8.sendKeys("test"); MobileElement el9 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/loginButton"); el9.click(); MobileElement el10 = (MobileElement) driver.findElementByAccessibilityId("Profile"); el10.click(); MobileElement el11 = (MobileElement) driver.findElementByAccessibilityId("More options"); el11.click(); MobileElement el12 = (MobileElement) driver.findElementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/ android.widget.ListView/android.widget.LinearLayout[2]/android.widget.LinearLayout/android.widget.RelativeLayout/android.widget.TextView"); el12.click(); } @After public void tearDown() { driver.quit(); } }
  13. CREATE ESPRESSO TESTS It allows us to create simple Espresso

    tests in few clicks. GENERATE CODE Espresso Test Recorder generates a code of test case that use an Espresso framework. PROBLEM WITH WAITING Generated code has a problem with waiting data from web servers, etc ESPRESSO TEST RECORDER
  14. GENERATED CODE BY ESPRESSO TEST RECORDER @RunWith(AndroidJUnit4.class) public class RecorderActivityTest

    { @Rule public ActivityTestRule<SplashActivity> mActivityTestRule = new ActivityTestRule<>(SplashActivity.class); @Test public void recorderActivityTest() { ViewInteraction textInputEditText = onView( allOf(withId(R.id.emailEditText), childAtPosition(childAtPosition(withId(R.id.emailInputLayout),0),0))); textInputEditText.perform(scrollTo(), replaceText("test@alexzh.com"), closeSoftKeyboard()); ViewInteraction textInputEditText2 = onView( allOf(withId(R.id.passwordEditText),childAtPosition(childAtPosition(withId(R.id.passwordInputLayout), 0), 0))); textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard()); ViewInteraction materialButton = onView( allOf(withId(R.id.loginButton), withText("Login"), childAtPosition(childAtPosition(withId(R.id.scrollView2), 0), 4))); materialButton.perform(scrollTo(), click()); ViewInteraction bottomNavigationItemView = onView( allOf( withId(R.id.navigation_profile), withContentDescription("Profile"), childAtPosition(childAtPosition(withId(R.id.navigation), 0), 2), isDisplayed() )); bottomNavigationItemView.perform(click()); ViewInteraction overflowMenuButton = onView( allOf(withContentDescription("More options"), childAtPosition(childAtPosition(withId(R.id.toolbar), 1), 0), isDisplayed())); overflowMenuButton.perform(click()); ViewInteraction appCompatTextView = onView( allOf(withId(R.id.title), withText("Log Out"), childAtPosition(childAtPosition(withId(R.id.content), 0), 0), isDisplayed())); appCompatTextView.perform(click()); } ... }
  15. UI TESTING FRAMEWORKS

  16. ESPRESSO APPIUM UI AUTOMATOR

  17. ESPRESSO START ANY ACTIVITY Espresso allows us to start a

    test from any Activity. We can also test any fragment in isolation with an additional library. SYNCHRONIZATION Espresso performs synchronization with an app after each onView call. IDLING RESOURCES The Idling Resources mechanism allows the Espresso to wait until the asynchronous operation is finished. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed()))
  18. INTERACTION WITH ANY APP Ui Automator framework can interact with

    any application installed on the device/emulator. INTERACTION WITH OS Ui Automator framework can interact with OS features, like notification, app shortcuts, permissions, etc. INTERACTION WITH DEVICE Ui Automator framework can interact with device buttons, emulate screen rotation, etc. UI AUTOMATOR val email = device.findObject(By.res(PACKAGE, "email")) email.text = EMAIL val password = device.findObject(By.res(PACKAGE, "password")) email.text = PASSWORD val login = device.findObject(By.res(PACKAGE, "login")) if (login != null) { ... }
  19. APPIUM SUPPORT MANY LANGUAGES We can use different programming languages

    to create tests using the Apium framework. like Java, Python, Ruby, C#, etc. INTERACTION WITH ANY APP Appium framework can interact with any application installed on the device/ emulator. INTERACTION WITH OS, DEVICE Appium framework can interact with OS features and device buttons, screen rotation, etc. val email = driver.findElement(By.id("email")) email.value = EMAIL val password = driver.findElement(By.id("password")) password.value = PASSWORD val login = driver.findElement(By.id("login")) login.isDisplayed()
  20. ESPRESSO UI AUTOMATOR APPIUM No SUPPORT OF RESOURCES TEST CASES

    TYPE EXECUTION TIME Yes Gray box 0.946 s No Black box 8.743 s Black box 12.154 s
  21. TYPES OF UI TESTS

  22. END-TO-END TESTS FUNCTIONAL TESTS COMPONENT TESTS VIEW COMPARISON TESTS

  23. END-TO-END TESTS SCENARIO: Splash Screen 1. Login to application 2.

    Analyse coffee drink screen 3. Open Profile screen 4. Open Menu 5. Logout from application Open Login screen
  24. END-TO-END TESTS INTERACTION WITH SERVER Interaction with production server. ENTRY

    POINT Similar to entry point of app.
  25. INTERACTION WITH SERVER Interaction with the non-production server. NAVIGATION Usually,

    navigation is not needed. ENTRY POINT We can start test from required Activity/ Fragment. FUNCTIONAL TESTS
  26. COMPONENT TESTS SHORT EXECUTION TIME Usually, we have no interaction

    with the server, no navigation, etc. APP INDEPENDANT TESTS Test cases verify the rendering of the view and usually use fake data.
  27. DEVICE DEPENDENT Test cases based on screenshots; different devices can

    have other screen resolutions. MIGRATION REQURE SCRIPTS We need to update all reference screenshots from one screen resolution to another one. VIEW COMPARISON TESTS
  28. END TO END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar

    to entry point of app. APP VERIFICATION Slow verification of application. NAVIGATION Navigate to required screen. SERVER INTERACTION Interaction with production server. ENTRY POINT Start required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast verification of components or screens.
  29. EFFICIENCY READABILITY TIME EFFICIENCY COMPLEXITY

  30. READABILITY DIFFERENT APPROACHES TIME EFFICIENCY COMPLEXITY

  31. READABILITY

  32. ESPRESSO // Enter email onView(withId(R.id.email)) .perform(replaceText(EMAIL)) // Enter password onView(withId(R.id.password))

    .perform(replaceText(PASSWORD)) // Press login button onView(withId(R.id.login)) .perform(click())
  33. ESPRESSO onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton)) .perform(click()) val progressBarIR

    = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
  34. ESPRESSO // Login screen onView(withId(R.id.emailEditText)) .perform(replaceText(email)) onView(withId(R.id.passwordEditText)) .perform(replaceText(password), closeSoftKeyboard()) onView(withId(R.id.loginButton))

    .perform(click()) // Home screen val progressBarIR = ViewVisibilityIdlingResource(...) IdlingRegistry.getInstance().register(progressBarIR) onView(withId(R.id.recyclerView)) .check(matches(withItemCount(13))) IdlingRegistry.getInstance().unregister(progressBarIR) onView(withId(R.id.navigation_profile)) .perform(click()) ...
  35. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed()))

  36. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD)

    }
  37. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD)

    } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } }
  38. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD)

    } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } }
  39. onView(withId(R.id.email)) .perform(replaceText(EMAIL) onView(withId(R.id.password)) .perform(replaceText(PASSWORD) onView(withId(R.id.login)) .check(matchers(isDisplayed())) loginScreen { login(EMAIL, PASSWORD)

    } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } } class LoginScreenRobot : BaseTestRobot() { fun login(email: String, password: String) { enterText(R.id.email, email) enterText(R.id.password, password) clickOnView(R.id.loginButton) } } fun loginScreen(func: LoginScreenRobot.() -> Unit) = LoginScreenRobot().apply { func() }
  40. BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest {

    @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { onView(withId(viewId)) .perform(replaceText(text)) } fun clickOnView(viewId: Int) { onView(withId(viewId)) .perform(click()) } ... } ESPRESSO
  41. BASIC OPERATIONS SCREEN ROBOTS TEST CASES @RunWith(AndroidJUnit4::class) class LoginActivityTest {

    @Test fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() { loginScreen { enterEmail(EMPTY_VALUE) emptyEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() { loginScreen { enterEmail(INCORRECT_EMAIL) incorrectEmailErrorDisplayed() } } @Test fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() { loginScreen { enterPassword(EMPTY_VALUE) emptyPasswordErrorDisplayed() } } ... } class LoginScreenRobot : BaseTestRobot() { fun enterEmail(email: String) = enterText(R.id.email, email) fun enterPassword(email: String) = enterText(R.id.password, password) ... } open class BaseTestRobot { fun enterText(viewId: Int, text: String) { val view = device.findObject(By.res(resId(viewId))) view.text = text } fun clickOnView(viewId: Int) { device.findObject(By.res(resIf(viewId))) .click() } ... } UI AUTOMATOR
  42. loginScreen { enterEmail(EMAIL) enterPassword(PASSWORD) pressLogin() } homeScreen { waitCoffeeDrinks() pressProfile()

    } profileScreen { waitUserData() hasEmail(EMAIL) hasUserName(USER_NAME) openMenu() pressLogout() } loginScreen { isOpen() } loginScreen { login(EMAIL, PASSWORD) } homeScreen { navigateToProfile() } profileScreen { hasEmail(EMAIL) hasUserName(USER_NAME) logout() } loginScreen { isOpen() }
  43. DIFFERENT APPROACHES

  44. DEBUG SERVER APPLICABLE TO E2E TESTS We can update the

    URL constant during compilation APK for instrumentation test cases. MAKE TESTS MORE STABLE We can have stable data on the debug server. DATA SYNCHRONIZATION We need to synchronize data between the production and debug server. Production Debug
  45. SHORT EXECUTION TIME Interaction with files and databases is faster

    than using UI for data changes. PRE-POPULATING DATA Server Database
  46. MOCK API RESPONSES MAKE TESTS MORE STABLE We can use

    predefined fake responses instead of using a production server. NO INTERACTION WITH SERVER Responses from the production server can differ from predefined data. DATA SYNCHRONIZATION We need to synchronize predefined responses with responses from the production server. Mock-Server
  47. FLEXIBILITY We can define what we want to test and

    which layers should be mocked. SHORT EXECUTION TIME Short execution time can be achieved if we mock top layers of the application. MOCK APP LAYERS APP DOMAIN DATA
  48. Activities Fragments

  49. Fragments @RunWith(AndroidJUnit4::class) class LoginActivityTest { @Test fun superImportantTest() { ...

    launchFragmentInContainer<LoginFragment>() ... } }
  50. TIME EFFICIENCY

  51. END TO END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar

    to entry point of app. APP VERIFICATION Slow verification of application. NAVIGATION Navigate to required screen. SERVER INTERACTION Interaction with production server. ENTRY POINT Start required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast verification of components or screens.
  52. END-TO-END TESTS FUNCTIONAL TESTS

  53. COMPLEXITY

  54. FRAMEWORKS DOMAIN SPECIFIC LANGUAGE TEST INFRASRUCTURE KNOWN APPROACHES

  55. UI TESTING FRAMEWORKS APPIUM ESPRESSO KAKAO BARISTA KASPERSSO UI AUTOMATOR

  56. APPIUM UI AUTOMATOR ESPRESSO KAKAO KASPRESSO BARISTA KAUTOMATOR

  57. BARISTA LESS BOILERPLATE CODE Barista API is removing a lot

    of boilerplate code with its own abstraction over Espresso. USEFUL TEST RULES Barista library provides JUnit rules for clearing data on device/emulation, like ClearFilesRule, ClearPreferencesRule, etc. HANDLING FLAKY TESTS Barista library provides annotations for handling flaky tests, like AllowFlaky, Repeat. writeTo(R.id.email, EMAIL) writeTo(R.id.password, PASSWORD) clickOn(R.id.login) assertDisplayed(R.id.name, USER_NAME) assertDisplayed(R.id.email, EMAIL)
  58. PREDEFINED DSL Kakao framework provides an abstraction on top of

    the Espresso framework. INTERCEPTORS MECHANISM An interceptor allows you to inject other logic between Kakao → Espresso call chain. KAKAO onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() }
  59. KASPRESSO BASED ON KAKAO We have access to all features

    from Kakao and Espresso frameworks. ADVANCED REPORTING SYSTEM We can generate human-readable reports with the Kaspresso framework. HANDLING FLAKY TESTS Kaspresso support functions for handling flaky operation based on interceptor mechanism, like flakySafely. beforeTest { activityTestRule.launchActivity(intent) }.afterTest { ... }.run { onScreen<LoginScreen> { email.replaceText(EMAIL) password { replaceText(PASSWORD) closeSoftKeyboard() } login.click() } }
  60. BEST PRACTICES NAMING QUALITY FLAKY TESTS

  61. LEARN YOUR TOOLS

  62. PAY ATTENTION TO NAMING

  63. NO "SLEEP" IN TESTS

  64. ALL TEST CASES SHOULD BE INDEPENDENT

  65. MAKE AN ABSTRACTION OVER THE FRAMEWORK

  66. FOLLOW THE "NO FLAKY TESTS" POLICY

  67. DON’T RELY ONLY ON UI TEST AUTOMATION

  68. THANK YOU FOR LISTENING! @alex_zhukovich https://alexzh.com/ alex-zhukovich