Efficient Android UI Testing

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate automated tests to your project in an efficient way because they should be fast and stable.

This talk will cover the following topics:
– DSL (Domain Specific Language) in UI Testing
– Efficient combination of different types of UI tests
– Best practices for creating test suit for Android application
– Popular mistakes in UI Testing

Twitter: https://twitter.com/alex_zhukovich
LinkedIn: https://www.linkedin.com/in/alex-zhukovich/
Blog: https://alexzh.com

Transcript

1. Efficient Android
   UI Testing
   @alex_zhukovich
   https://alexzh.com/
   

   View Slide

2. ANDROID TESTING
   UI & NON-UI TESTS
   DEVICES
   

   View Slide

3. Local and remote data sources.
   DATA LAYERS
   Vendors produce devices with
   different OS versions.
   FRAGMENTATION
   Phones, Tablets, Watches, TVs
   and custom devices.
   DIFFERENT DEVICES
   Notifications, Widgets, Runtime
   permissions, App Shortcuts, etc.
   OS FEATURES
   

   View Slide

4. APPLICATION
   IS GROWING
   Users
   Features
   

   View Slide

5. LOCAL
   UI* Non-UI
   INSTRUMENTATION
   UI Non-UI
   ANDROID TESTS
   

   View Slide

6. APPROACHES TOOLS
   LAYOUT INSPECTOR
   UI TEST RECORDERS
   FRAMEWORKS
   TYPES OF UI TESTS
   

   View Slide

7. VIEW INFORMATION
   

   View Slide

8. LAYOUT INSPECTOR
   DETAIL VIEW INFORMATION
   We can get information about text, color,
   alpha, margins, paddings, etc.
   PIXEL PERFECTNESS
   We can use built-in tools for comparing
   design and rendered layouts.
   3D VISUALIZATION
   We can analyze layouts of the
   application in 3D to a better
   understanding of nesting layouts.
   

   View Slide

9. ANALYZE ANY APP
   We can analyze any application
   without root permissions.
   UI AUTOMATOR VIEWER
   

   View Slide

10. TEST RECORDER
    

    View Slide

11. APPIUM RECORDER
    CREATE APPIUM TESTS
    It allows us to create simple Appium
    tests in few clicks.
    GENERATE CODE
    Appium Recorder generates a code of
    test case that use an Appium
    framework.
    PROBLEM WITH WAITING
    Generated code has a problem with
    waiting data from web servers, etc.
    

    View Slide

12. GENERATED CODE BY APPIUM RECORDER
    public class SampleTest {
    private AndroidDriver driver;
    @Before
    public void setUp() throws MalformedURLException {
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability("platformName", "Android");
    desiredCapabilities.setCapability("ensureWebviewsHavePages", true);
    URL remoteUrl = new URL("http://localhost:4723/wd/hub");
    driver = new AndroidDriver(remoteUrl, desiredCapabilities);
    }
    @Test
    public void sampleTest() {
    MobileElement el7 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/emailEditText");
    el7.sendKeys("test@alexzh.com");
    MobileElement el8 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/passwordEditText");
    el8.sendKeys("test");
    MobileElement el9 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/loginButton");
    el9.click();
    MobileElement el10 = (MobileElement) driver.findElementByAccessibilityId("Profile");
    el10.click();
    MobileElement el11 = (MobileElement) driver.findElementByAccessibilityId("More options");
    el11.click();
    MobileElement el12 = (MobileElement) driver.findElementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/
    android.widget.ListView/android.widget.LinearLayout[2]/android.widget.LinearLayout/android.widget.RelativeLayout/android.widget.TextView");
    el12.click();
    }
    @After
    public void tearDown() { driver.quit(); }
    }
    

    View Slide

13. CREATE ESPRESSO TESTS
    It allows us to create simple Espresso
    tests in few clicks.
    GENERATE CODE
    Espresso Test Recorder generates a
    code of test case that use an Espresso
    framework.
    PROBLEM WITH WAITING
    Generated code has a problem with
    waiting data from web servers, etc
    ESPRESSO TEST RECORDER
    

    View Slide

14. GENERATED CODE BY ESPRESSO TEST RECORDER
    @RunWith(AndroidJUnit4.class)
    public class RecorderActivityTest {
    @Rule
    public ActivityTestRule mActivityTestRule = new ActivityTestRule<>(SplashActivity.class);
    @Test
    public void recorderActivityTest() {
    ViewInteraction textInputEditText = onView(
    allOf(withId(R.id.emailEditText), childAtPosition(childAtPosition(withId(R.id.emailInputLayout),0),0)));
    textInputEditText.perform(scrollTo(), replaceText("test@alexzh.com"), closeSoftKeyboard());
    ViewInteraction textInputEditText2 = onView(
    allOf(withId(R.id.passwordEditText),childAtPosition(childAtPosition(withId(R.id.passwordInputLayout), 0), 0)));
    textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard());
    ViewInteraction materialButton = onView(
    allOf(withId(R.id.loginButton), withText("Login"), childAtPosition(childAtPosition(withId(R.id.scrollView2), 0), 4)));
    materialButton.perform(scrollTo(), click());
    ViewInteraction bottomNavigationItemView = onView(
    allOf(
    withId(R.id.navigation_profile),
    withContentDescription("Profile"),
    childAtPosition(childAtPosition(withId(R.id.navigation), 0), 2),
    isDisplayed()
    ));
    bottomNavigationItemView.perform(click());
    ViewInteraction overflowMenuButton = onView(
    allOf(withContentDescription("More options"), childAtPosition(childAtPosition(withId(R.id.toolbar), 1), 0), isDisplayed()));
    overflowMenuButton.perform(click());
    ViewInteraction appCompatTextView = onView(
    allOf(withId(R.id.title), withText("Log Out"), childAtPosition(childAtPosition(withId(R.id.content), 0), 0), isDisplayed()));
    appCompatTextView.perform(click());
    }
    ...
    }
    

    View Slide

15. UI TESTING FRAMEWORKS
    

    View Slide

16. ESPRESSO APPIUM
    UI AUTOMATOR
    

    View Slide

17. ESPRESSO
    START ANY ACTIVITY
    Espresso allows us to start a test from any
    Activity. We can also test any fragment in
    isolation with an additional library.
    SYNCHRONIZATION
    Espresso performs synchronization with
    an app after each onView call.
    IDLING RESOURCES
    The Idling Resources mechanism allows
    the Espresso to wait until the
    asynchronous operation is finished.
    onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    

    View Slide

18. INTERACTION WITH ANY APP
    Ui Automator framework can interact
    with any application installed on the
    device/emulator.
    INTERACTION WITH OS
    Ui Automator framework can interact
    with OS features, like notification, app
    shortcuts, permissions, etc.
    INTERACTION WITH DEVICE
    Ui Automator framework can interact
    with device buttons, emulate screen
    rotation, etc.
    UI AUTOMATOR
    val email =
    device.findObject(By.res(PACKAGE, "email"))
    email.text = EMAIL
    val password =
    device.findObject(By.res(PACKAGE, "password"))
    email.text = PASSWORD
    val login =
    device.findObject(By.res(PACKAGE, "login"))
    if (login != null) { ... }
    

    View Slide

19. APPIUM
    SUPPORT MANY LANGUAGES
    We can use different programming
    languages to create tests using the
    Apium framework. like Java, Python,
    Ruby, C#, etc.
    INTERACTION WITH ANY APP
    Appium framework can interact with
    any application installed on the device/
    emulator.
    INTERACTION WITH OS, DEVICE
    Appium framework can interact with OS
    features and device buttons, screen
    rotation, etc.
    val email = driver.findElement(By.id("email"))
    email.value = EMAIL
    val password =
    driver.findElement(By.id("password"))
    password.value = PASSWORD
    val login = driver.findElement(By.id("login"))
    login.isDisplayed()
    

    View Slide

20. ESPRESSO UI AUTOMATOR APPIUM
    No
    SUPPORT OF
    RESOURCES
    TEST CASES TYPE
    EXECUTION TIME
    Yes
    Gray box
    0.946 s
    No
    Black box
    8.743 s
    Black box
    12.154 s
    

    View Slide

21. TYPES OF UI TESTS
    

    View Slide

22. END-TO-END
    TESTS
    FUNCTIONAL
    TESTS
    COMPONENT
    TESTS
    VIEW COMPARISON
    TESTS
    

    View Slide

23. END-TO-END TESTS
    SCENARIO:
    Splash Screen
    1. Login to application
    2. Analyse coffee drink screen
    3. Open Profile screen
    4. Open Menu
    5. Logout from application
    Open Login screen
    

    View Slide

24. END-TO-END TESTS
    INTERACTION WITH SERVER
    Interaction with production server.
    ENTRY POINT
    Similar to entry point of app.
    

    View Slide

25. INTERACTION WITH SERVER
    Interaction with the non-production
    server.
    NAVIGATION
    Usually, navigation is not needed.
    ENTRY POINT
    We can start test from required Activity/
    Fragment.
    FUNCTIONAL TESTS
    

    View Slide

26. COMPONENT TESTS
    SHORT EXECUTION TIME
    Usually, we have no interaction with the
    server, no navigation, etc.
    APP INDEPENDANT TESTS
    Test cases verify the rendering of the
    view and usually use fake data.
    

    View Slide

27. DEVICE DEPENDENT
    Test cases based on screenshots;
    different devices can have other screen
    resolutions.
    MIGRATION REQURE SCRIPTS
    We need to update all reference
    screenshots from one screen resolution
    to another one.
    VIEW COMPARISON TESTS
    

    View Slide

28. END TO END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT
    Similar to entry point of app.
    APP VERIFICATION
    Slow verification of application.
    NAVIGATION
    Navigate to required screen.
    SERVER INTERACTION
    Interaction with production
    server.
    ENTRY POINT
    Start required screen of the
    app.
    SERVER INTERACTION
    Usually interaction with non-
    production server.
    NAVIGATION
    Usually no navigation.
    UI VERIFICATION
    Fast verification of
    components or screens.
    

    View Slide

29. EFFICIENCY
    READABILITY
    TIME EFFICIENCY
    COMPLEXITY
    

    View Slide

30. READABILITY DIFFERENT
    APPROACHES
    TIME EFFICIENCY COMPLEXITY
    

    View Slide

31. READABILITY
    

    View Slide

32. ESPRESSO
    // Enter email
    onView(withId(R.id.email))
    .perform(replaceText(EMAIL))
    // Enter password
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD))
    // Press login button
    onView(withId(R.id.login))
    .perform(click())
    

    View Slide

33. ESPRESSO
    onView(withId(R.id.emailEditText))
    .perform(replaceText(email))
    onView(withId(R.id.passwordEditText))
    .perform(replaceText(password), closeSoftKeyboard())
    onView(withId(R.id.loginButton))
    .perform(click())
    val progressBarIR = ViewVisibilityIdlingResource(...)
    IdlingRegistry.getInstance().register(progressBarIR)
    onView(withId(R.id.recyclerView))
    .check(matches(withItemCount(13)))
    IdlingRegistry.getInstance().unregister(progressBarIR)
    onView(withId(R.id.navigation_profile))
    .perform(click())
    ...
    

    View Slide

34. ESPRESSO
    // Login screen
    onView(withId(R.id.emailEditText))
    .perform(replaceText(email))
    onView(withId(R.id.passwordEditText))
    .perform(replaceText(password), closeSoftKeyboard())
    onView(withId(R.id.loginButton))
    .perform(click())
    // Home screen
    val progressBarIR = ViewVisibilityIdlingResource(...)
    IdlingRegistry.getInstance().register(progressBarIR)
    onView(withId(R.id.recyclerView))
    .check(matches(withItemCount(13)))
    IdlingRegistry.getInstance().unregister(progressBarIR)
    onView(withId(R.id.navigation_profile))
    .perform(click())
    ...
    

    View Slide

35. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    

    View Slide

36. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    

    View Slide

37. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    }
    

    View Slide

38. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun login(email: String, password: String) {
    enterText(R.id.email, email)
    enterText(R.id.password, password)
    clickOnView(R.id.loginButton)
    }
    }
    

    View Slide

39. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun login(email: String, password: String) {
    enterText(R.id.email, email)
    enterText(R.id.password, password)
    clickOnView(R.id.loginButton)
    }
    }
    fun loginScreen(func: LoginScreenRobot.() -> Unit) =
    LoginScreenRobot().apply { func() }
    

    View Slide

40. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)
    class LoginActivityTest {
    @Test
    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {
    loginScreen {
    enterEmail(EMPTY_VALUE)
    emptyEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {
    loginScreen {
    enterEmail(INCORRECT_EMAIL)
    incorrectEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {
    loginScreen {
    enterPassword(EMPTY_VALUE)
    emptyPasswordErrorDisplayed()
    }
    }
    ...
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun enterEmail(email: String) =
    enterText(R.id.email, email)
    fun enterPassword(email: String) =
    enterText(R.id.password, password)
    ...
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    ...
    }
    ESPRESSO
    

    View Slide

41. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)
    class LoginActivityTest {
    @Test
    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {
    loginScreen {
    enterEmail(EMPTY_VALUE)
    emptyEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {
    loginScreen {
    enterEmail(INCORRECT_EMAIL)
    incorrectEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {
    loginScreen {
    enterPassword(EMPTY_VALUE)
    emptyPasswordErrorDisplayed()
    }
    }
    ...
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun enterEmail(email: String) =
    enterText(R.id.email, email)
    fun enterPassword(email: String) =
    enterText(R.id.password, password)
    ...
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    val view = device.findObject(By.res(resId(viewId)))
    view.text = text
    }
    fun clickOnView(viewId: Int) {
    device.findObject(By.res(resIf(viewId)))
    .click()
    }
    ...
    }
    UI AUTOMATOR
    

    View Slide

42. loginScreen {
    enterEmail(EMAIL)
    enterPassword(PASSWORD)
    pressLogin()
    }
    homeScreen {
    waitCoffeeDrinks()
    pressProfile()
    }
    profileScreen {
    waitUserData()
    hasEmail(EMAIL)
    hasUserName(USER_NAME)
    openMenu()
    pressLogout()
    }
    loginScreen {
    isOpen()
    }
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    homeScreen {
    navigateToProfile()
    }
    profileScreen {
    hasEmail(EMAIL)
    hasUserName(USER_NAME)
    logout()
    }
    loginScreen {
    isOpen()
    }
    

    View Slide

43. DIFFERENT APPROACHES
    

    View Slide

44. DEBUG SERVER
    APPLICABLE TO E2E TESTS
    We can update the URL constant during
    compilation APK for instrumentation
    test cases.
    MAKE TESTS MORE STABLE
    We can have stable data on the debug
    server.
    DATA SYNCHRONIZATION
    We need to synchronize data between
    the production and debug server.
    Production
    Debug
    

    View Slide

45. SHORT EXECUTION TIME
    Interaction with files and databases is
    faster than using UI for data changes.
    PRE-POPULATING DATA
    Server
    Database
    

    View Slide

46. MOCK API RESPONSES
    MAKE TESTS MORE STABLE
    We can use predefined fake responses
    instead of using a production server.
    NO INTERACTION WITH SERVER
    Responses from the production server
    can differ from predefined data.
    DATA SYNCHRONIZATION
    We need to synchronize predefined
    responses with responses from the
    production server.
    Mock-Server
    

    View Slide

47. FLEXIBILITY
    We can define what we want to test and
    which layers should be mocked.
    SHORT EXECUTION TIME
    Short execution time can be achieved if
    we mock top layers of the application.
    MOCK APP LAYERS
    APP
    DOMAIN
    DATA
    

    View Slide

48. Activities
    Fragments
    

    View Slide

49. Fragments
    @RunWith(AndroidJUnit4::class)
    class LoginActivityTest {
    @Test
    fun superImportantTest() {
    ...
    launchFragmentInContainer()
    ...
    }
    }
    

    View Slide

50. TIME EFFICIENCY
    

    View Slide

51. END TO END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT
    Similar to entry point of app.
    APP VERIFICATION
    Slow verification of application.
    NAVIGATION
    Navigate to required screen.
    SERVER INTERACTION
    Interaction with production
    server.
    ENTRY POINT
    Start required screen of the
    app.
    SERVER INTERACTION
    Usually interaction with non-
    production server.
    NAVIGATION
    Usually no navigation.
    UI VERIFICATION
    Fast verification of
    components or screens.
    

    View Slide

52. END-TO-END TESTS FUNCTIONAL TESTS
    

    View Slide

53. COMPLEXITY
    

    View Slide

54. FRAMEWORKS DOMAIN
    SPECIFIC
    LANGUAGE
    TEST
    INFRASRUCTURE
    KNOWN
    APPROACHES
    

    View Slide

55. UI TESTING
    FRAMEWORKS
    APPIUM
    ESPRESSO
    KAKAO
    BARISTA
    KASPERSSO
    UI AUTOMATOR
    

    View Slide

56. APPIUM UI AUTOMATOR
    ESPRESSO
    KAKAO KASPRESSO BARISTA
    KAUTOMATOR
    

    View Slide

57. BARISTA
    LESS BOILERPLATE CODE
    Barista API is removing a lot of
    boilerplate code with its own abstraction
    over Espresso.
    USEFUL TEST RULES
    Barista library provides JUnit rules for
    clearing data on device/emulation, like
    ClearFilesRule, ClearPreferencesRule, etc.
    HANDLING FLAKY TESTS
    Barista library provides annotations for
    handling flaky tests, like AllowFlaky,
    Repeat.
    writeTo(R.id.email, EMAIL)
    writeTo(R.id.password, PASSWORD)
    clickOn(R.id.login)
    assertDisplayed(R.id.name, USER_NAME)
    assertDisplayed(R.id.email, EMAIL)
    

    View Slide

58. PREDEFINED DSL
    Kakao framework provides an
    abstraction on top of the Espresso
    framework.
    INTERCEPTORS MECHANISM
    An interceptor allows you to inject other
    logic between Kakao → Espresso call
    chain.
    KAKAO
    onScreen {
    email.replaceText(EMAIL)
    password {
    replaceText(PASSWORD)
    closeSoftKeyboard()
    }
    login.click()
    }
    

    View Slide

59. KASPRESSO
    BASED ON KAKAO
    We have access to all features from
    Kakao and Espresso frameworks.
    ADVANCED REPORTING SYSTEM
    We can generate human-readable
    reports with the Kaspresso framework.
    HANDLING FLAKY TESTS
    Kaspresso support functions for
    handling flaky operation based on
    interceptor mechanism, like flakySafely.
    beforeTest {
    activityTestRule.launchActivity(intent)
    }.afterTest {
    ...
    }.run {
    onScreen {
    email.replaceText(EMAIL)
    password {
    replaceText(PASSWORD)
    closeSoftKeyboard()
    }
    login.click()
    }
    }
    

    View Slide

60. BEST PRACTICES
    NAMING
    QUALITY
    FLAKY TESTS
    

    View Slide

61. LEARN YOUR TOOLS
    

    View Slide

62. PAY ATTENTION TO
    NAMING
    

    View Slide

63. NO "SLEEP" IN TESTS
    

    View Slide

64. ALL TEST CASES SHOULD
    BE INDEPENDENT
    

    View Slide

65. MAKE AN ABSTRACTION
    OVER THE FRAMEWORK
    

    View Slide

66. FOLLOW THE "NO FLAKY
    TESTS" POLICY
    

    View Slide

67. DON’T RELY ONLY ON UI
    TEST AUTOMATION
    

    View Slide

68. THANK YOU
    FOR LISTENING!
    @alex_zhukovich
    https://alexzh.com/
    alex-zhukovich
    

    View Slide