Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Efficient Android UI Testing

Efficient Android UI Testing

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate automated tests to your project in an efficient way because they should be fast and stable.

This talk will cover the following topics:
– DSL (Domain Specific Language) in UI Testing
– Efficient combination of different types of UI tests
– Best practices for creating test suit for Android application
– Popular mistakes in UI Testing

Twitter: https://twitter.com/alex_zhukovich
LinkedIn: https://www.linkedin.com/in/alex-zhukovich/
Blog: https://alexzh.com

Alex Zhukovich

November 17, 2020
Tweet

More Decks by Alex Zhukovich

Other Decks in Technology

Transcript

  1. Efficient Android
    UI Testing
    @alex_zhukovich
    https://alexzh.com/

    View Slide

  2. ANDROID TESTING
    UI & NON-UI TESTS
    DEVICES

    View Slide

  3. Local and remote data sources.
    DATA LAYERS
    Vendors produce devices with
    different OS versions.
    FRAGMENTATION
    Phones, Tablets, Watches, TVs
    and custom devices.
    DIFFERENT DEVICES
    Notifications, Widgets, Runtime
    permissions, App Shortcuts, etc.
    OS FEATURES

    View Slide

  4. APPLICATION
    IS GROWING
    Users
    Features

    View Slide

  5. LOCAL
    UI* Non-UI
    INSTRUMENTATION
    UI Non-UI
    ANDROID TESTS

    View Slide

  6. APPROACHES TOOLS
    LAYOUT INSPECTOR
    UI TEST RECORDERS
    FRAMEWORKS
    TYPES OF UI TESTS

    View Slide

  7. VIEW INFORMATION

    View Slide

  8. LAYOUT INSPECTOR
    DETAIL VIEW INFORMATION
    We can get information about text, color,
    alpha, margins, paddings, etc.
    PIXEL PERFECTNESS
    We can use built-in tools for comparing
    design and rendered layouts.
    3D VISUALIZATION
    We can analyze layouts of the
    application in 3D to a better
    understanding of nesting layouts.

    View Slide

  9. ANALYZE ANY APP
    We can analyze any application
    without root permissions.
    UI AUTOMATOR VIEWER

    View Slide

  10. TEST RECORDER

    View Slide

  11. APPIUM RECORDER
    CREATE APPIUM TESTS
    It allows us to create simple Appium
    tests in few clicks.
    GENERATE CODE
    Appium Recorder generates a code of
    test case that use an Appium
    framework.
    PROBLEM WITH WAITING
    Generated code has a problem with
    waiting data from web servers, etc.

    View Slide

  12. GENERATED CODE BY APPIUM RECORDER
    public class SampleTest {
    private AndroidDriver driver;
    @Before
    public void setUp() throws MalformedURLException {
    DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
    desiredCapabilities.setCapability("platformName", "Android");
    desiredCapabilities.setCapability("ensureWebviewsHavePages", true);
    URL remoteUrl = new URL("http://localhost:4723/wd/hub");
    driver = new AndroidDriver(remoteUrl, desiredCapabilities);
    }
    @Test
    public void sampleTest() {
    MobileElement el7 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/emailEditText");
    el7.sendKeys("[email protected]");
    MobileElement el8 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/passwordEditText");
    el8.sendKeys("test");
    MobileElement el9 = (MobileElement) driver.findElementById("com.alexzh.imbarista:id/loginButton");
    el9.click();
    MobileElement el10 = (MobileElement) driver.findElementByAccessibilityId("Profile");
    el10.click();
    MobileElement el11 = (MobileElement) driver.findElementByAccessibilityId("More options");
    el11.click();
    MobileElement el12 = (MobileElement) driver.findElementByXPath("/hierarchy/android.widget.FrameLayout/android.widget.FrameLayout/
    android.widget.ListView/android.widget.LinearLayout[2]/android.widget.LinearLayout/android.widget.RelativeLayout/android.widget.TextView");
    el12.click();
    }
    @After
    public void tearDown() { driver.quit(); }
    }

    View Slide

  13. CREATE ESPRESSO TESTS
    It allows us to create simple Espresso
    tests in few clicks.
    GENERATE CODE
    Espresso Test Recorder generates a
    code of test case that use an Espresso
    framework.
    PROBLEM WITH WAITING
    Generated code has a problem with
    waiting data from web servers, etc
    ESPRESSO TEST RECORDER

    View Slide

  14. GENERATED CODE BY ESPRESSO TEST RECORDER
    @RunWith(AndroidJUnit4.class)
    public class RecorderActivityTest {
    @Rule
    public ActivityTestRule mActivityTestRule = new ActivityTestRule<>(SplashActivity.class);
    @Test
    public void recorderActivityTest() {
    ViewInteraction textInputEditText = onView(
    allOf(withId(R.id.emailEditText), childAtPosition(childAtPosition(withId(R.id.emailInputLayout),0),0)));
    textInputEditText.perform(scrollTo(), replaceText("[email protected]"), closeSoftKeyboard());
    ViewInteraction textInputEditText2 = onView(
    allOf(withId(R.id.passwordEditText),childAtPosition(childAtPosition(withId(R.id.passwordInputLayout), 0), 0)));
    textInputEditText2.perform(scrollTo(), replaceText("test"), closeSoftKeyboard());
    ViewInteraction materialButton = onView(
    allOf(withId(R.id.loginButton), withText("Login"), childAtPosition(childAtPosition(withId(R.id.scrollView2), 0), 4)));
    materialButton.perform(scrollTo(), click());
    ViewInteraction bottomNavigationItemView = onView(
    allOf(
    withId(R.id.navigation_profile),
    withContentDescription("Profile"),
    childAtPosition(childAtPosition(withId(R.id.navigation), 0), 2),
    isDisplayed()
    ));
    bottomNavigationItemView.perform(click());
    ViewInteraction overflowMenuButton = onView(
    allOf(withContentDescription("More options"), childAtPosition(childAtPosition(withId(R.id.toolbar), 1), 0), isDisplayed()));
    overflowMenuButton.perform(click());
    ViewInteraction appCompatTextView = onView(
    allOf(withId(R.id.title), withText("Log Out"), childAtPosition(childAtPosition(withId(R.id.content), 0), 0), isDisplayed()));
    appCompatTextView.perform(click());
    }
    ...
    }

    View Slide

  15. UI TESTING FRAMEWORKS

    View Slide

  16. ESPRESSO APPIUM
    UI AUTOMATOR

    View Slide

  17. ESPRESSO
    START ANY ACTIVITY
    Espresso allows us to start a test from any
    Activity. We can also test any fragment in
    isolation with an additional library.
    SYNCHRONIZATION
    Espresso performs synchronization with
    an app after each onView call.
    IDLING RESOURCES
    The Idling Resources mechanism allows
    the Espresso to wait until the
    asynchronous operation is finished.
    onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))

    View Slide

  18. INTERACTION WITH ANY APP
    Ui Automator framework can interact
    with any application installed on the
    device/emulator.
    INTERACTION WITH OS
    Ui Automator framework can interact
    with OS features, like notification, app
    shortcuts, permissions, etc.
    INTERACTION WITH DEVICE
    Ui Automator framework can interact
    with device buttons, emulate screen
    rotation, etc.
    UI AUTOMATOR
    val email =
    device.findObject(By.res(PACKAGE, "email"))
    email.text = EMAIL
    val password =
    device.findObject(By.res(PACKAGE, "password"))
    email.text = PASSWORD
    val login =
    device.findObject(By.res(PACKAGE, "login"))
    if (login != null) { ... }

    View Slide

  19. APPIUM
    SUPPORT MANY LANGUAGES
    We can use different programming
    languages to create tests using the
    Apium framework. like Java, Python,
    Ruby, C#, etc.
    INTERACTION WITH ANY APP
    Appium framework can interact with
    any application installed on the device/
    emulator.
    INTERACTION WITH OS, DEVICE
    Appium framework can interact with OS
    features and device buttons, screen
    rotation, etc.
    val email = driver.findElement(By.id("email"))
    email.value = EMAIL
    val password =
    driver.findElement(By.id("password"))
    password.value = PASSWORD
    val login = driver.findElement(By.id("login"))
    login.isDisplayed()

    View Slide

  20. ESPRESSO UI AUTOMATOR APPIUM
    No
    SUPPORT OF
    RESOURCES
    TEST CASES TYPE
    EXECUTION TIME
    Yes
    Gray box
    0.946 s
    No
    Black box
    8.743 s
    Black box
    12.154 s

    View Slide

  21. TYPES OF UI TESTS

    View Slide

  22. END-TO-END
    TESTS
    FUNCTIONAL
    TESTS
    COMPONENT
    TESTS
    VIEW COMPARISON
    TESTS

    View Slide

  23. END-TO-END TESTS
    SCENARIO:
    Splash Screen
    1. Login to application
    2. Analyse coffee drink screen
    3. Open Profile screen
    4. Open Menu
    5. Logout from application
    Open Login screen

    View Slide

  24. END-TO-END TESTS
    INTERACTION WITH SERVER
    Interaction with production server.
    ENTRY POINT
    Similar to entry point of app.

    View Slide

  25. INTERACTION WITH SERVER
    Interaction with the non-production
    server.
    NAVIGATION
    Usually, navigation is not needed.
    ENTRY POINT
    We can start test from required Activity/
    Fragment.
    FUNCTIONAL TESTS

    View Slide

  26. COMPONENT TESTS
    SHORT EXECUTION TIME
    Usually, we have no interaction with the
    server, no navigation, etc.
    APP INDEPENDANT TESTS
    Test cases verify the rendering of the
    view and usually use fake data.

    View Slide

  27. DEVICE DEPENDENT
    Test cases based on screenshots;
    different devices can have other screen
    resolutions.
    MIGRATION REQURE SCRIPTS
    We need to update all reference
    screenshots from one screen resolution
    to another one.
    VIEW COMPARISON TESTS

    View Slide

  28. END TO END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT
    Similar to entry point of app.
    APP VERIFICATION
    Slow verification of application.
    NAVIGATION
    Navigate to required screen.
    SERVER INTERACTION
    Interaction with production
    server.
    ENTRY POINT
    Start required screen of the
    app.
    SERVER INTERACTION
    Usually interaction with non-
    production server.
    NAVIGATION
    Usually no navigation.
    UI VERIFICATION
    Fast verification of
    components or screens.

    View Slide

  29. EFFICIENCY
    READABILITY
    TIME EFFICIENCY
    COMPLEXITY

    View Slide

  30. READABILITY DIFFERENT
    APPROACHES
    TIME EFFICIENCY COMPLEXITY

    View Slide

  31. READABILITY

    View Slide

  32. ESPRESSO
    // Enter email
    onView(withId(R.id.email))
    .perform(replaceText(EMAIL))
    // Enter password
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD))
    // Press login button
    onView(withId(R.id.login))
    .perform(click())

    View Slide

  33. ESPRESSO
    onView(withId(R.id.emailEditText))
    .perform(replaceText(email))
    onView(withId(R.id.passwordEditText))
    .perform(replaceText(password), closeSoftKeyboard())
    onView(withId(R.id.loginButton))
    .perform(click())
    val progressBarIR = ViewVisibilityIdlingResource(...)
    IdlingRegistry.getInstance().register(progressBarIR)
    onView(withId(R.id.recyclerView))
    .check(matches(withItemCount(13)))
    IdlingRegistry.getInstance().unregister(progressBarIR)
    onView(withId(R.id.navigation_profile))
    .perform(click())
    ...

    View Slide

  34. ESPRESSO
    // Login screen
    onView(withId(R.id.emailEditText))
    .perform(replaceText(email))
    onView(withId(R.id.passwordEditText))
    .perform(replaceText(password), closeSoftKeyboard())
    onView(withId(R.id.loginButton))
    .perform(click())
    // Home screen
    val progressBarIR = ViewVisibilityIdlingResource(...)
    IdlingRegistry.getInstance().register(progressBarIR)
    onView(withId(R.id.recyclerView))
    .check(matches(withItemCount(13)))
    IdlingRegistry.getInstance().unregister(progressBarIR)
    onView(withId(R.id.navigation_profile))
    .perform(click())
    ...

    View Slide

  35. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))

    View Slide

  36. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }

    View Slide

  37. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    }

    View Slide

  38. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun login(email: String, password: String) {
    enterText(R.id.email, email)
    enterText(R.id.password, password)
    clickOnView(R.id.loginButton)
    }
    }

    View Slide

  39. onView(withId(R.id.email))
    .perform(replaceText(EMAIL)
    onView(withId(R.id.password))
    .perform(replaceText(PASSWORD)
    onView(withId(R.id.login))
    .check(matchers(isDisplayed()))
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun login(email: String, password: String) {
    enterText(R.id.email, email)
    enterText(R.id.password, password)
    clickOnView(R.id.loginButton)
    }
    }
    fun loginScreen(func: LoginScreenRobot.() -> Unit) =
    LoginScreenRobot().apply { func() }

    View Slide

  40. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)
    class LoginActivityTest {
    @Test
    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {
    loginScreen {
    enterEmail(EMPTY_VALUE)
    emptyEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {
    loginScreen {
    enterEmail(INCORRECT_EMAIL)
    incorrectEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {
    loginScreen {
    enterPassword(EMPTY_VALUE)
    emptyPasswordErrorDisplayed()
    }
    }
    ...
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun enterEmail(email: String) =
    enterText(R.id.email, email)
    fun enterPassword(email: String) =
    enterText(R.id.password, password)
    ...
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    onView(withId(viewId))
    .perform(replaceText(text))
    }
    fun clickOnView(viewId: Int) {
    onView(withId(viewId))
    .perform(click())
    }
    ...
    }
    ESPRESSO

    View Slide

  41. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)
    class LoginActivityTest {
    @Test
    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {
    loginScreen {
    enterEmail(EMPTY_VALUE)
    emptyEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {
    loginScreen {
    enterEmail(INCORRECT_EMAIL)
    incorrectEmailErrorDisplayed()
    }
    }
    @Test
    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {
    loginScreen {
    enterPassword(EMPTY_VALUE)
    emptyPasswordErrorDisplayed()
    }
    }
    ...
    }
    class LoginScreenRobot : BaseTestRobot() {
    fun enterEmail(email: String) =
    enterText(R.id.email, email)
    fun enterPassword(email: String) =
    enterText(R.id.password, password)
    ...
    }
    open class BaseTestRobot {
    fun enterText(viewId: Int, text: String) {
    val view = device.findObject(By.res(resId(viewId)))
    view.text = text
    }
    fun clickOnView(viewId: Int) {
    device.findObject(By.res(resIf(viewId)))
    .click()
    }
    ...
    }
    UI AUTOMATOR

    View Slide

  42. loginScreen {
    enterEmail(EMAIL)
    enterPassword(PASSWORD)
    pressLogin()
    }
    homeScreen {
    waitCoffeeDrinks()
    pressProfile()
    }
    profileScreen {
    waitUserData()
    hasEmail(EMAIL)
    hasUserName(USER_NAME)
    openMenu()
    pressLogout()
    }
    loginScreen {
    isOpen()
    }
    loginScreen {
    login(EMAIL, PASSWORD)
    }
    homeScreen {
    navigateToProfile()
    }
    profileScreen {
    hasEmail(EMAIL)
    hasUserName(USER_NAME)
    logout()
    }
    loginScreen {
    isOpen()
    }

    View Slide

  43. DIFFERENT APPROACHES

    View Slide

  44. DEBUG SERVER
    APPLICABLE TO E2E TESTS
    We can update the URL constant during
    compilation APK for instrumentation
    test cases.
    MAKE TESTS MORE STABLE
    We can have stable data on the debug
    server.
    DATA SYNCHRONIZATION
    We need to synchronize data between
    the production and debug server.
    Production
    Debug

    View Slide

  45. SHORT EXECUTION TIME
    Interaction with files and databases is
    faster than using UI for data changes.
    PRE-POPULATING DATA
    Server
    Database

    View Slide

  46. MOCK API RESPONSES
    MAKE TESTS MORE STABLE
    We can use predefined fake responses
    instead of using a production server.
    NO INTERACTION WITH SERVER
    Responses from the production server
    can differ from predefined data.
    DATA SYNCHRONIZATION
    We need to synchronize predefined
    responses with responses from the
    production server.
    Mock-Server

    View Slide

  47. FLEXIBILITY
    We can define what we want to test and
    which layers should be mocked.
    SHORT EXECUTION TIME
    Short execution time can be achieved if
    we mock top layers of the application.
    MOCK APP LAYERS
    APP
    DOMAIN
    DATA

    View Slide

  48. Activities
    Fragments

    View Slide

  49. Fragments
    @RunWith(AndroidJUnit4::class)
    class LoginActivityTest {
    @Test
    fun superImportantTest() {
    ...
    launchFragmentInContainer()
    ...
    }
    }

    View Slide

  50. TIME EFFICIENCY

    View Slide

  51. END TO END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT
    Similar to entry point of app.
    APP VERIFICATION
    Slow verification of application.
    NAVIGATION
    Navigate to required screen.
    SERVER INTERACTION
    Interaction with production
    server.
    ENTRY POINT
    Start required screen of the
    app.
    SERVER INTERACTION
    Usually interaction with non-
    production server.
    NAVIGATION
    Usually no navigation.
    UI VERIFICATION
    Fast verification of
    components or screens.

    View Slide

  52. END-TO-END TESTS FUNCTIONAL TESTS

    View Slide

  53. COMPLEXITY

    View Slide

  54. FRAMEWORKS DOMAIN
    SPECIFIC
    LANGUAGE
    TEST
    INFRASRUCTURE
    KNOWN
    APPROACHES

    View Slide

  55. UI TESTING
    FRAMEWORKS
    APPIUM
    ESPRESSO
    KAKAO
    BARISTA
    KASPERSSO
    UI AUTOMATOR

    View Slide

  56. APPIUM UI AUTOMATOR
    ESPRESSO
    KAKAO KASPRESSO BARISTA
    KAUTOMATOR

    View Slide

  57. BARISTA
    LESS BOILERPLATE CODE
    Barista API is removing a lot of
    boilerplate code with its own abstraction
    over Espresso.
    USEFUL TEST RULES
    Barista library provides JUnit rules for
    clearing data on device/emulation, like
    ClearFilesRule, ClearPreferencesRule, etc.
    HANDLING FLAKY TESTS
    Barista library provides annotations for
    handling flaky tests, like AllowFlaky,
    Repeat.
    writeTo(R.id.email, EMAIL)
    writeTo(R.id.password, PASSWORD)
    clickOn(R.id.login)
    assertDisplayed(R.id.name, USER_NAME)
    assertDisplayed(R.id.email, EMAIL)

    View Slide

  58. PREDEFINED DSL
    Kakao framework provides an
    abstraction on top of the Espresso
    framework.
    INTERCEPTORS MECHANISM
    An interceptor allows you to inject other
    logic between Kakao → Espresso call
    chain.
    KAKAO
    onScreen {
    email.replaceText(EMAIL)
    password {
    replaceText(PASSWORD)
    closeSoftKeyboard()
    }
    login.click()
    }

    View Slide

  59. KASPRESSO
    BASED ON KAKAO
    We have access to all features from
    Kakao and Espresso frameworks.
    ADVANCED REPORTING SYSTEM
    We can generate human-readable
    reports with the Kaspresso framework.
    HANDLING FLAKY TESTS
    Kaspresso support functions for
    handling flaky operation based on
    interceptor mechanism, like flakySafely.
    beforeTest {
    activityTestRule.launchActivity(intent)
    }.afterTest {
    ...
    }.run {
    onScreen {
    email.replaceText(EMAIL)
    password {
    replaceText(PASSWORD)
    closeSoftKeyboard()
    }
    login.click()
    }
    }

    View Slide

  60. BEST PRACTICES
    NAMING
    QUALITY
    FLAKY TESTS

    View Slide

  61. LEARN YOUR TOOLS

    View Slide

  62. PAY ATTENTION TO
    NAMING

    View Slide

  63. NO "SLEEP" IN TESTS

    View Slide

  64. ALL TEST CASES SHOULD
    BE INDEPENDENT

    View Slide

  65. MAKE AN ABSTRACTION
    OVER THE FRAMEWORK

    View Slide

  66. FOLLOW THE "NO FLAKY
    TESTS" POLICY

    View Slide

  67. DON’T RELY ONLY ON UI
    TEST AUTOMATION

    View Slide

  68. THANK YOU
    FOR LISTENING!
    @alex_zhukovich
    https://alexzh.com/
    alex-zhukovich

    View Slide