Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Efficient Android UI Testing, Droidcon Italy 2021

Efficient Android UI Testing, Droidcon Italy 2021

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate fast and stable automated tests into your project.

In this talk, we will discuss the following topics:
- How to create fast and stable UI tests
- How to avoid flaky tests
- How to cover applications which include traditional and Jetpack Compose views and screens
- How to share UI tests between local and instrumentation tests
- How DSL can speed up adding stable UI tests to the project

Alex Zhukovich

November 12, 2021
Tweet

More Decks by Alex Zhukovich

Other Decks in Technology

Transcript

  1. Ef
    fi
    cient Android


    UI Testing
    @alex_zhukovich
    https://alexzh.com/

    View full-size slide

  2. Multiple languages
    Light and Dark themes
    Accessibility options
    RTL and LTR support

    View full-size slide

  3. Let's Run The Tests

    View full-size slide

  4. ANDROID TESTS
    LOCAL INSTRUMENTATION

    View full-size slide

  5. ANDROID TESTS
    LOCAL INSTRUMENTATION
    UI* Non-UI

    View full-size slide

  6. ANDROID TESTS
    LOCAL INSTRUMENTATION
    UI* Non-UI UI Non-UI

    View full-size slide

  7. ANDROID TESTS
    LOCAL INSTRUMENTATION
    UI* Non-UI UI Non-UI
    SHARED CODE

    View full-size slide

  8. androidTest
    Requires device or emulator
    test
    Executes in local JVM

    View full-size slide

  9. androidTest
    test
    commonTest
    android {




    ...


    sourceSets {


    androidTest {


    java.srcDirs += "src/commonTest/java"


    }


    test {


    java.srcDirs += "src/commonTest/java"


    }


    }


    }


    View full-size slide

  10. User Interface tests Snapshot tests

    View full-size slide

  11. app.apk
    tests.apk
    BUILD APKs

    View full-size slide

  12. app.apk tests.apk
    >_ adb shell


    am instrument

    View full-size slide

  13. app.apk tests.apk
    >_ adb shell


    am instrument

    View full-size slide

  14. ACTIVITY VIEW / COMPOSABLE
    FRAGMENT

    View full-size slide

  15. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    ActivityScenario.launch(


    MainActivity::class.java


    )


    View full-size slide

  16. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val activityOptions = bundleOf(


    Pair("param-1", 1),


    Pair("param-2", 2),


    Pair("param-3", 3),


    )


    val scenario = ActivityScenario.launch(


    MainActivity::class.java,


    activityOptions


    )


    scenario.moveToState(Lifecycle.State.STARTED)


    ActivityScenario.launch(


    View full-size slide

  17. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val scenario = launchFragmentInContainer(


    fragmentArgs: Bundle? = null,


    @StyleRes themeResId: Int =

    R.style.FragmentScenarioEmptyFragmentActivityTheme,


    initialState: Lifecycle.State =

    Lifecycle.State.RESUMED,


    factory: FragmentFactory? = null


    )


    scenario.moveToState(Lifecycle.State.STARTED)


    debugImplementation "androidx.fragment:fragment-testing:1.4.0-rc01"

    View full-size slide

  18. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Activity
    Fragment

    View full-size slide

  19. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.fragment.app.testing.FragmentScenario$EmptyFragmentActivity
    Activity
    Fragment
    Compilation
    Added to AndroidManifest

    View full-size slide

  20. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    @get:Rule


    val composeTestRule = createComposeRule()


    composeTestRule.apply {


    setContent { LoginScreen() }


    onNodeWithTag("email")


    .performTextInput("[email protected]")


    onNodeWithTag("password")


    .performTextInput("password")


    onNodeWithTag("login")


    .performClick()


    ...


    }
    debugImplementation "androidx.compose.ui:ui-test-manifest:1:0:5"

    View full-size slide

  21. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Component
    Activity
    @Composable

    View full-size slide

  22. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.activity.ComponentActivity
    Component
    Activity
    @Composable
    Compilation
    Added to AndroidManifest

    View full-size slide

  23. What Is a Good Test Case?

    View full-size slide

  24. Readable
    Stable Extendable
    Fast

    View full-size slide

  25. Readable
    Stable Extendable
    Fast

    View full-size slide

  26. onView(withId(R.id.emailEditText))


    .perform(replaceText(email))


    onView(withId(R.id.passwordEditText))


    .perform(replaceText(password), closeSoftKeyboard())


    onView(withId(R.id.loginButton))


    .perform(click())


    val progressBarIR = ViewVisibilityIdlingResource(...)


    IdlingRegistry.getInstance().register(progressBarIR)


    onView(withId(R.id.recyclerView))


    .check(matches(withItemCount(13)))


    IdlingRegistry.getInstance().unregister(progressBarIR)


    onView(withId(R.id.navigation_profile))


    .perform(click())


    ...

    View full-size slide

  27. // Login screen


    onView(withId(R.id.emailEditText))


    .perform(replaceText(email))


    onView(withId(R.id.passwordEditText))


    .perform(replaceText(password), closeSoftKeyboard())


    onView(withId(R.id.loginButton))


    .perform(click())


    // Home screen


    val progressBarIR = ViewVisibilityIdlingResource(...)


    IdlingRegistry.getInstance().register(progressBarIR)


    onView(withId(R.id.recyclerView))


    .check(matches(withItemCount(13)))


    IdlingRegistry.getInstance().unregister(progressBarIR)


    onView(withId(R.id.navigation_profile))


    .perform(click())


    ...

    View full-size slide

  28. onView(withId(R.id.email))


    .perform(replaceText(EMAIL)


    onView(withId(R.id.password))


    .perform(replaceText(PASSWORD)


    onView(withId(R.id.login))


    .perform(click())

    View full-size slide

  29. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }

    View full-size slide

  30. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }
    open class BaseTestRobot {


    fun enterText(viewId: Int, text: String) {


    onView(withId(viewId))


    .perform(replaceText(text))


    }


    fun clickOnView(viewId: Int) {


    onView(withId(viewId))


    .perform(click())


    }


    }

    View full-size slide

  31. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }
    open class BaseTestRobot {


    fun login(email: String, password: String) {


    enterText(R.id.email, email)


    enterText(R.id.password, password)


    clickOnView(R.id.loginButton)


    }


    }

    View full-size slide

  32. onView(withId(R.id.email))


    login(EMAIL, PASSWORD)


    }
    open class BaseTestRobot {


    LoginScreenRobot().apply { func() }

    View full-size slide

  33. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)


    class LoginActivityTest {


    @Test


    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {


    loginScreen {


    enterEmail(EMPTY_VALUE)


    emptyEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {


    loginScreen {


    enterEmail(INCORRECT_EMAIL)


    incorrectEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {


    loginScreen {


    enterPassword(EMPTY_VALUE)


    emptyPasswordErrorDisplayed()


    }


    }


    ...


    }
    class LoginScreenRobot : BaseTestRobot() {


    fun enterEmail(email: String) =


    enterText(R.id.email, email)




    fun enterPassword(email: String) =

    enterText(R.id.password, password)


    ...


    }
    open class BaseTestRobot {


    fun enterText(viewId: Int, text: String) {


    onView(withId(viewId))


    .perform(replaceText(text))


    }


    fun clickOnView(viewId: Int) {


    onView(withId(viewId))


    .perform(click())


    }



    ...


    }
    ESPRESSO

    View full-size slide

  34. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)


    class LoginActivityTest {


    @Test


    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {


    loginScreen {


    enterEmail(EMPTY_VALUE)


    emptyEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {


    loginScreen {


    enterEmail(INCORRECT_EMAIL)


    incorrectEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {


    loginScreen {


    enterPassword(EMPTY_VALUE)


    emptyPasswordErrorDisplayed()


    }


    }


    ...


    }
    class LoginScreenRobot : BaseTestRobot() {


    fun enterEmail(email: String) =


    enterText(R.id.email, email)




    fun enterPassword(email: String) =

    enterText(R.id.password, password)


    ...


    }
    open class BaseTestRobot {


    fun enterText(viewId: Int, text: String) {


    val view = device.findObject(By.res(resId(viewId)))


    view.text = text


    }


    fun clickOnView(viewId: Int) {


    device.findObject(By.res(resIf(viewId)))

    .click()


    }



    ...


    }
    UI AUTOMATOR

    View full-size slide

  35. open class BaseTestRobot {


    @get:Rule val composeTestRule = createComposeRule()




    fun enterText(tag: String, text: String) {


    composeTestRule.onNodeWithTag(tag)


    .performTextInput(text)


    }


    fun clickOnView(tag: String) {


    composeTestRule.onNodeWithTag(tag)


    .performClick()


    }


    }
    @RunWith(AndroidJUnit4::class)


    class LoginActivityTest {


    @Test


    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {


    loginScreen {


    enterEmail(EMPTY_VALUE)


    emptyEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {


    loginScreen {


    enterEmail(INCORRECT_EMAIL)


    incorrectEmailErrorDisplayed()


    }


    }


    @Test


    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {


    loginScreen {


    enterPassword(EMPTY_VALUE)


    emptyPasswordErrorDisplayed()


    }


    }


    ...


    }
    BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    COMPOSE UI TESTS
    class LoginScreenRobot : BaseTestRobot() {


    fun enterEmail(email: String) =


    enterText("email", email)




    fun enterPassword(email: String) =

    enterText("password", password)


    ...


    }

    View full-size slide

  36. Challenges

    In Mobile Testing

    View full-size slide

  37. Light and Dark themes

    View full-size slide

  38. YES NO
    RTL support

    View full-size slide

  39. Accessibility support
    Accessibility checks
    Hello …
    FONT SIZE
    Small, Default, Large, Largest
    DISPLAY SCALING
    Small, Default, Large, Largest

    View full-size slide

  40. Testing permissions

    View full-size slide

  41. Testing permissions
    UI interaction
    GrantPermissionRule
    ADB commands

    View full-size slide

  42. Testing permissions
    UI interaction
    GrantPermissionRule
    ADB commands
    Android Test Orchestrator

    View full-size slide

  43. Test Case
    Test Case
    Test Case
    Clean up data after test execution

    View full-size slide

  44. /data/data/PACKAGE/
    databases
    fi
    les
    shared_prefs
    Clean up data after test execution

    View full-size slide

  45. databases
    fi
    les
    shared_prefs
    @get:Rul
    e

    val clearDatabaseRule = ClearDatabaseRule(
    )

    @get:Rul
    e

    val clearFileRule = ClearFilesRule(
    )

    @get:Rul
    e

    val clearPreferencesRule = ClearPreferencesRule(
    )

    https://github.com/AdevintaSpain/Barista
    Clean up data after test execution

    View full-size slide

  46. Server
    Database
    Pre-populating the database

    View full-size slide

  47. Test Case
    Database
    DATA LAYER
    Query
    Dependencies
    UI interaction
    Pre-populating the database

    View full-size slide

  48. Real data vs Fake data
    Prod server
    Dev server
    Mock/Fake

    View full-size slide

  49. Real data vs Fake data
    Prod server
    Dev server DATA SYNCHRONIZATION
    We need to synchronize data between
    the production and dev servers.
    INTERACTION WITH THE SERVER
    We make requests to the production
    server. The connection can require
    certificates, VPN, etc.
    PRODUCTION BACK-END
    We always use the latest available
    production environment.

    View full-size slide

  50. Real data vs Fake data
    Mock/Fake
    DATA SYNCHRONIZATION
    We need to synchronize predefined
    responses with responses from the
    production server.
    NO INTERACTION WITH THE SERVER
    Responses from the production server
    can differ from the predefined data.
    We can use predefined fake responses
    instead of calling the production
    server.
    MAKE TESTS MORE STABLE

    View full-size slide

  51. End-To-End vs Functional tests

    View full-size slide

  52. INTERACTION WITH THE SERVER
    Interaction with the production server.
    ENTRY POINT
    Similar to the entry point of an app.
    END-TO-END TESTS

    View full-size slide

  53. FUNCTIONAL TESTS
    INTERACTION WITH THE SERVER
    Interaction with the non-production
    server.
    NAVIGATION
    Usually, navigation is not needed.
    ENTRY POINT
    We can start the test from a required
    Activity/Fragment.

    View full-size slide

  54. END-TO-END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT


    Similar to the entry point of an
    app.
    APP VERIFICATION


    Slow veri
    fi
    cation of the app.
    NAVIGATION


    Navigate to the required
    screen.
    SERVER INTERACTION


    Interaction with the
    production server.
    ENTRY POINT


    Start the required screen of the
    app.
    SERVER INTERACTION


    Usually interaction with non-
    production server.
    NAVIGATION


    Usually no navigation.
    UI VERIFICATION


    Fast veri
    fi
    cation of
    components or screens.

    View full-size slide

  55. END-TO-END TESTS FUNCTIONAL TESTS

    View full-size slide

  56. External dependencies
    Framework
    Test case execution
    Device and emulator

    View full-size slide

  57. External dependencies
    - Network connection (VPN)


    - Network speed


    - Back-end

    View full-size slide

  58. Framework
    - Framework issues


    - Toast, Snackbar, etc


    - Custom Views

    View full-size slide

  59. Device and emulator
    - Performance


    - Noti
    fi
    cations


    - Device memory

    View full-size slide

  60. Test case execution
    - Simulate User actions


    - Incorrect state before/after
    a test case


    - Toast, Snackbar, etc

    View full-size slide

  61. - Network connection (VPN)


    - Network speed


    - Back-end
    - Framework issues


    - Toast, Snackbar, etc


    - Custom Views
    - Simulate User actions


    - Incorrect state before/after a
    test case


    - Toast, Snackbar, etc
    - Performance


    - Noti
    fi
    cations


    - Device memory
    External dependencies
    Framework
    Test case execution
    Device and emulator

    View full-size slide

  62. Best Practices

    View full-size slide

  63. PAY ATTENTION TO NAMING

    View full-size slide

  64. NO "SLEEP" IN TESTS

    View full-size slide

  65. ALL TEST CASES SHOULD BE
    INDEPENDENT

    View full-size slide

  66. USE MULTIPLE SMALL TESTS
    INSTEAD OF ONE BIG TEST

    View full-size slide

  67. DO NOT SPEND TIME NAVIGATING
    TO THE REQUIRED SCREEN

    View full-size slide

  68. FOLLOW THE

    "NO FLAKY TESTS" POLICY

    View full-size slide

  69. DO NOT RELY ONLY ON

    UI TEST AUTOMATION

    View full-size slide

  70. LEARN YOUR TOOLS

    View full-size slide

  71. @alex_zhukovich https://alexzh.com/
    THANK YOU


    FOR LISTENING!

    View full-size slide