Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Efficient Android UI Testing, Droidcon Italy 2021

Efficient Android UI Testing, Droidcon Italy 2021

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate fast and stable automated tests into your project.

In this talk, we will discuss the following topics:
- How to create fast and stable UI tests
- How to avoid flaky tests
- How to cover applications which include traditional and Jetpack Compose views and screens
- How to share UI tests between local and instrumentation tests
- How DSL can speed up adding stable UI tests to the project

Alex Zhukovich

November 12, 2021
Tweet

More Decks by Alex Zhukovich

Other Decks in Technology

Transcript

  1. Ef
    fi
    cient Android


    UI Testing
    @alex_zhukovich
    https://alexzh.com/

    View Slide

  2. View Slide

  3. Multiple languages
    Light and Dark themes
    Accessibility options
    RTL and LTR support

    View Slide

  4. Let's Run The Tests

    View Slide

  5. ANDROID TESTS
    LOCAL INSTRUMENTATION

    View Slide

  6. ANDROID TESTS
    LOCAL INSTRUMENTATION
    UI* Non-UI

    View Slide

  7. ANDROID TESTS
    LOCAL INSTRUMENTATION
    UI* Non-UI UI Non-UI

    View Slide

  8. ANDROID TESTS
    LOCAL INSTRUMENTATION
    UI* Non-UI UI Non-UI
    SHARED CODE

    View Slide

  9. androidTest
    Requires device or emulator
    test
    Executes in local JVM

    View Slide

  10. androidTest
    test
    commonTest
    android {




    ...


    sourceSets {


    androidTest {


    java.srcDirs += "src/commonTest/java"


    }


    test {


    java.srcDirs += "src/commonTest/java"


    }


    }


    }


    View Slide

  11. User Interface tests Snapshot tests

    View Slide

  12. app.apk
    tests.apk
    BUILD APKs

    View Slide

  13. app.apk tests.apk
    >_ adb shell


    am instrument

    View Slide

  14. app.apk tests.apk
    >_ adb shell


    am instrument

    View Slide

  15. ACTIVITY VIEW / COMPOSABLE
    FRAGMENT

    View Slide

  16. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    ActivityScenario.launch(


    MainActivity::class.java


    )


    View Slide

  17. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val activityOptions = bundleOf(


    Pair("param-1", 1),


    Pair("param-2", 2),


    Pair("param-3", 3),


    )


    val scenario = ActivityScenario.launch(


    MainActivity::class.java,


    activityOptions


    )


    scenario.moveToState(Lifecycle.State.STARTED)


    ActivityScenario.launch(


    View Slide

  18. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val scenario = launchFragmentInContainer(


    fragmentArgs: Bundle? = null,


    @StyleRes themeResId: Int =

    R.style.FragmentScenarioEmptyFragmentActivityTheme,


    initialState: Lifecycle.State =

    Lifecycle.State.RESUMED,


    factory: FragmentFactory? = null


    )


    scenario.moveToState(Lifecycle.State.STARTED)


    debugImplementation "androidx.fragment:fragment-testing:1.4.0-rc01"

    View Slide

  19. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Activity
    Fragment

    View Slide

  20. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.fragment.app.testing.FragmentScenario$EmptyFragmentActivity
    Activity
    Fragment
    Compilation
    Added to AndroidManifest

    View Slide

  21. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    @get:Rule


    val composeTestRule = createComposeRule()


    composeTestRule.apply {


    setContent { LoginScreen() }


    onNodeWithTag("email")


    .performTextInput("[email protected]")


    onNodeWithTag("password")


    .performTextInput("password")


    onNodeWithTag("login")


    .performClick()


    ...


    }
    debugImplementation "androidx.compose.ui:ui-test-manifest:1:0:5"

    View Slide

  22. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Component
    Activity
    @Composable

    View Slide

  23. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.activity.ComponentActivity
    Component
    Activity
    @Composable
    Compilation
    Added to AndroidManifest

    View Slide

  24. View Slide

  25. What Is a Good Test Case?

    View Slide

  26. Readable
    Stable Extendable
    Fast

    View Slide

  27. Readable
    Stable Extendable
    Fast

    View Slide

  28. onView(withId(R.id.emailEditText))


    .perform(replaceText(email))


    onView(withId(R.id.passwordEditText))


    .perform(replaceText(password), closeSoftKeyboard())


    onView(withId(R.id.loginButton))


    .perform(click())


    val progressBarIR = ViewVisibilityIdlingResource(...)


    IdlingRegistry.getInstance().register(progressBarIR)


    onView(withId(R.id.recyclerView))


    .check(matches(withItemCount(13)))


    IdlingRegistry.getInstance().unregister(progressBarIR)


    onView(withId(R.id.navigation_profile))


    .perform(click())


    ...

    View Slide

  29. // Login screen


    onView(withId(R.id.emailEditText))


    .perform(replaceText(email))


    onView(withId(R.id.passwordEditText))


    .perform(replaceText(password), closeSoftKeyboard())


    onView(withId(R.id.loginButton))


    .perform(click())


    // Home screen


    val progressBarIR = ViewVisibilityIdlingResource(...)


    IdlingRegistry.getInstance().register(progressBarIR)


    onView(withId(R.id.recyclerView))


    .check(matches(withItemCount(13)))


    IdlingRegistry.getInstance().unregister(progressBarIR)


    onView(withId(R.id.navigation_profile))


    .perform(click())


    ...

    View Slide

  30. onView(withId(R.id.email))


    .perform(replaceText(EMAIL)


    onView(withId(R.id.password))


    .perform(replaceText(PASSWORD)


    onView(withId(R.id.login))


    .perform(click())

    View Slide

  31. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }

    View Slide

  32. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }
    open class BaseTestRobot {


    fun enterText(viewId: Int, text: String) {


    onView(withId(viewId))


    .perform(replaceText(text))


    }


    fun clickOnView(viewId: Int) {


    onView(withId(viewId))


    .perform(click())


    }


    }

    View Slide

  33. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }
    open class BaseTestRobot {


    fun login(email: String, password: String) {


    enterText(R.id.email, email)


    enterText(R.id.password, password)


    clickOnView(R.id.loginButton)


    }


    }

    View Slide

  34. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }
    open class BaseTestRobot {


    LoginScreenRobot().apply { func() }

    View Slide

  35. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)


    class LoginActivityTest {


    @Test


    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {


    loginScreen {


    enterEmail(EMPTY_VALUE)


    emptyEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {


    loginScreen {


    enterEmail(INCORRECT_EMAIL)


    incorrectEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {


    loginScreen {


    enterPassword(EMPTY_VALUE)


    emptyPasswordErrorDisplayed()


    }


    }


    ...


    }
    class LoginScreenRobot : BaseTestRobot() {


    fun enterEmail(email: String) =


    enterText(R.id.email, email)




    fun enterPassword(email: String) =

    enterText(R.id.password, password)


    ...


    }
    open class BaseTestRobot {


    fun enterText(viewId: Int, text: String) {


    onView(withId(viewId))


    .perform(replaceText(text))


    }


    fun clickOnView(viewId: Int) {


    onView(withId(viewId))


    .perform(click())


    }



    ...


    }
    ESPRESSO

    View Slide

  36. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)


    class LoginActivityTest {


    @Test


    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {


    loginScreen {


    enterEmail(EMPTY_VALUE)


    emptyEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {


    loginScreen {


    enterEmail(INCORRECT_EMAIL)


    incorrectEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {


    loginScreen {


    enterPassword(EMPTY_VALUE)


    emptyPasswordErrorDisplayed()


    }


    }


    ...


    }
    class LoginScreenRobot : BaseTestRobot() {


    fun enterEmail(email: String) =


    enterText(R.id.email, email)




    fun enterPassword(email: String) =

    enterText(R.id.password, password)


    ...


    }
    open class BaseTestRobot {


    fun enterText(viewId: Int, text: String) {


    val view = device.findObject(By.res(resId(viewId)))


    view.text = text


    }


    fun clickOnView(viewId: Int) {


    device.findObject(By.res(resIf(viewId)))

    .click()


    }



    ...


    }
    UI AUTOMATOR

    View Slide

  37. open class BaseTestRobot {


    @get:Rule val composeTestRule = createComposeRule()




    fun enterText(tag: String, text: String) {


    composeTestRule.onNodeWithTag(tag)


    .performTextInput(text)


    }


    fun clickOnView(tag: String) {


    composeTestRule.onNodeWithTag(tag)


    .performClick()


    }


    }
    @RunWith(AndroidJUnit4::class)


    class LoginActivityTest {


    @Test


    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {


    loginScreen {


    enterEmail(EMPTY_VALUE)


    emptyEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {


    loginScreen {


    enterEmail(INCORRECT_EMAIL)


    incorrectEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {


    loginScreen {


    enterPassword(EMPTY_VALUE)


    emptyPasswordErrorDisplayed()


    }


    }


    ...


    }
    BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    COMPOSE UI TESTS
    class LoginScreenRobot : BaseTestRobot() {


    fun enterEmail(email: String) =


    enterText("email", email)




    fun enterPassword(email: String) =

    enterText("password", password)


    ...


    }

    View Slide

  38. Challenges

    In Mobile Testing

    View Slide

  39. Light and Dark themes

    View Slide

  40. YES NO
    RTL support

    View Slide

  41. Accessibility support
    Accessibility checks
    Hello …
    FONT SIZE
    Small, Default, Large, Largest
    DISPLAY SCALING
    Small, Default, Large, Largest

    View Slide

  42. Testing permissions

    View Slide

  43. Testing permissions
    UI interaction
    GrantPermissionRule
    ADB commands

    View Slide

  44. Testing permissions
    UI interaction
    GrantPermissionRule
    ADB commands
    Android Test Orchestrator

    View Slide

  45. Test Case
    Test Case
    Test Case
    Clean up data after test execution

    View Slide

  46. /data/data/PACKAGE/
    databases
    fi
    les
    shared_prefs
    Clean up data after test execution

    View Slide

  47. databases
    fi
    les
    shared_prefs
    @get:Rul
    e

    val clearDatabaseRule = ClearDatabaseRule(
    )

    @get:Rul
    e

    val clearFileRule = ClearFilesRule(
    )

    @get:Rul
    e

    val clearPreferencesRule = ClearPreferencesRule(
    )

    https://github.com/AdevintaSpain/Barista
    Clean up data after test execution

    View Slide

  48. Server
    Database
    Pre-populating the database

    View Slide

  49. Test Case
    Database
    DATA LAYER
    Query
    Dependencies
    UI interaction
    Pre-populating the database

    View Slide

  50. Real data vs Fake data
    Prod server
    Dev server
    Mock/Fake

    View Slide

  51. Real data vs Fake data
    Prod server
    Dev server DATA SYNCHRONIZATION
    We need to synchronize data between
    the production and dev servers.
    INTERACTION WITH THE SERVER
    We make requests to the production
    server. The connection can require
    certificates, VPN, etc.
    PRODUCTION BACK-END
    We always use the latest available
    production environment.

    View Slide

  52. Real data vs Fake data
    Mock/Fake
    DATA SYNCHRONIZATION
    We need to synchronize predefined
    responses with responses from the
    production server.
    NO INTERACTION WITH THE SERVER
    Responses from the production server
    can differ from the predefined data.
    We can use predefined fake responses
    instead of calling the production
    server.
    MAKE TESTS MORE STABLE

    View Slide

  53. End-To-End vs Functional tests

    View Slide

  54. INTERACTION WITH THE SERVER
    Interaction with the production server.
    ENTRY POINT
    Similar to the entry point of an app.
    END-TO-END TESTS

    View Slide

  55. FUNCTIONAL TESTS
    INTERACTION WITH THE SERVER
    Interaction with the non-production
    server.
    NAVIGATION
    Usually, navigation is not needed.
    ENTRY POINT
    We can start the test from a required
    Activity/Fragment.

    View Slide

  56. END-TO-END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT


    Similar to the entry point of an
    app.
    APP VERIFICATION


    Slow veri
    fi
    cation of the app.
    NAVIGATION


    Navigate to the required
    screen.
    SERVER INTERACTION


    Interaction with the
    production server.
    ENTRY POINT


    Start the required screen of the
    app.
    SERVER INTERACTION


    Usually interaction with non-
    production server.
    NAVIGATION


    Usually no navigation.
    UI VERIFICATION


    Fast veri
    fi
    cation of
    components or screens.

    View Slide

  57. END-TO-END TESTS FUNCTIONAL TESTS

    View Slide

  58. Flaky Tests

    View Slide

  59. External dependencies
    Framework
    Test case execution
    Device and emulator

    View Slide

  60. External dependencies
    - Network connection (VPN)


    - Network speed


    - Back-end

    View Slide

  61. Framework
    - Framework issues


    - Toast, Snackbar, etc


    - Custom Views

    View Slide

  62. Device and emulator
    - Performance


    - Noti
    fi
    cations


    - Device memory

    View Slide

  63. Test case execution
    - Simulate User actions


    - Incorrect state before/after
    a test case


    - Toast, Snackbar, etc

    View Slide

  64. - Network connection (VPN)


    - Network speed


    - Back-end
    - Framework issues


    - Toast, Snackbar, etc


    - Custom Views
    - Simulate User actions


    - Incorrect state before/after a
    test case


    - Toast, Snackbar, etc
    - Performance


    - Noti
    fi
    cations


    - Device memory
    External dependencies
    Framework
    Test case execution
    Device and emulator

    View Slide

  65. Best Practices

    View Slide

  66. PAY ATTENTION TO NAMING

    View Slide

  67. NO "SLEEP" IN TESTS

    View Slide

  68. ALL TEST CASES SHOULD BE
    INDEPENDENT

    View Slide

  69. USE MULTIPLE SMALL TESTS
    INSTEAD OF ONE BIG TEST

    View Slide

  70. DO NOT SPEND TIME NAVIGATING
    TO THE REQUIRED SCREEN

    View Slide

  71. FOLLOW THE

    "NO FLAKY TESTS" POLICY

    View Slide

  72. DO NOT RELY ONLY ON

    UI TEST AUTOMATION

    View Slide

  73. LEARN YOUR TOOLS

    View Slide

  74. @alex_zhukovich https://alexzh.com/
    THANK YOU


    FOR LISTENING!

    View Slide