Efficient Android UI Testing, Droidcon Italy 2021

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate fast and stable automated tests into your project.

In this talk, we will discuss the following topics:
- How to create fast and stable UI tests
- How to avoid flaky tests
- How to cover applications which include traditional and Jetpack Compose views and screens
- How to share UI tests between local and instrumentation tests
- How DSL can speed up adding stable UI tests to the project

Transcript

1. Ef
   fi
   cient Android
   
   
   UI Testing
   @alex_zhukovich
   https://alexzh.com/
   

   View Slide

2. View Slide

3. Multiple languages
   Light and Dark themes
   Accessibility options
   RTL and LTR support
   

   View Slide

4. Let's Run The Tests
   

   View Slide

5. ANDROID TESTS
   LOCAL INSTRUMENTATION
   

   View Slide

6. ANDROID TESTS
   LOCAL INSTRUMENTATION
   UI* Non-UI
   

   View Slide

7. ANDROID TESTS
   LOCAL INSTRUMENTATION
   UI* Non-UI UI Non-UI
   

   View Slide

8. ANDROID TESTS
   LOCAL INSTRUMENTATION
   UI* Non-UI UI Non-UI
   SHARED CODE
   

   View Slide

9. androidTest
   Requires device or emulator
   test
   Executes in local JVM
   

   View Slide

10. androidTest
    test
    commonTest
    android {
    
    
    
    
    ...
    
    
    sourceSets {
    
    
    androidTest {
    
    
    java.srcDirs += "src/commonTest/java"
    
    
    }
    
    
    test {
    
    
    java.srcDirs += "src/commonTest/java"
    
    
    }
    
    
    }
    
    
    }
    
    
    

    View Slide

11. User Interface tests Snapshot tests
    

    View Slide

12. app.apk
    tests.apk
    BUILD APKs
    

    View Slide

13. app.apk tests.apk
    >_ adb shell
    
    
    am instrument
    

    View Slide

14. app.apk tests.apk
    >_ adb shell
    
    
    am instrument
    

    View Slide

15. ACTIVITY VIEW / COMPOSABLE
    FRAGMENT
    

    View Slide

16. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    ActivityScenario.launch(
    
    
    MainActivity::class.java
    
    
    )
    
    
    

    View Slide

17. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val activityOptions = bundleOf(
    
    
    Pair("param-1", 1),
    
    
    Pair("param-2", 2),
    
    
    Pair("param-3", 3),
    
    
    )
    
    
    val scenario = ActivityScenario.launch(
    
    
    MainActivity::class.java,
    
    
    activityOptions
    
    
    )
    
    
    scenario.moveToState(Lifecycle.State.STARTED)
    
    
    ActivityScenario.launch(
    
    
    

    View Slide

18. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val scenario = launchFragmentInContainer(
    
    
    fragmentArgs: Bundle? = null,
    
    
    @StyleRes themeResId: Int =
    

    R.style.FragmentScenarioEmptyFragmentActivityTheme,
    
    
    initialState: Lifecycle.State =
    

    Lifecycle.State.RESUMED,
    
    
    factory: FragmentFactory? = null
    
    
    )
    
    
    scenario.moveToState(Lifecycle.State.STARTED)
    
    
    debugImplementation "androidx.fragment:fragment-testing:1.4.0-rc01"
    

    View Slide

19. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Activity
    Fragment
    

    View Slide

20. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.fragment.app.testing.FragmentScenario$EmptyFragmentActivity
    Activity
    Fragment
    Compilation
    Added to AndroidManifest
    

    View Slide

21. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    @get:Rule
    
    
    val composeTestRule = createComposeRule()
    
    
    composeTestRule.apply {
    
    
    setContent { LoginScreen() }
    
    
    onNodeWithTag("email")
    
    
    .performTextInput("test@test.com")
    
    
    onNodeWithTag("password")
    
    
    .performTextInput("password")
    
    
    onNodeWithTag("login")
    
    
    .performClick()
    
    
    ...
    
    
    }
    debugImplementation "androidx.compose.ui:ui-test-manifest:1:0:5"
    

    View Slide

22. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Component
    Activity
    @Composable
    

    View Slide

23. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.activity.ComponentActivity
    Component
    Activity
    @Composable
    Compilation
    Added to AndroidManifest
    

    View Slide

24. View Slide

25. What Is a Good Test Case?
    

    View Slide

26. Readable
    Stable Extendable
    Fast
    

    View Slide

27. Readable
    Stable Extendable
    Fast
    

    View Slide

28. onView(withId(R.id.emailEditText))
    
    
    .perform(replaceText(email))
    
    
    onView(withId(R.id.passwordEditText))
    
    
    .perform(replaceText(password), closeSoftKeyboard())
    
    
    onView(withId(R.id.loginButton))
    
    
    .perform(click())
    
    
    val progressBarIR = ViewVisibilityIdlingResource(...)
    
    
    IdlingRegistry.getInstance().register(progressBarIR)
    
    
    onView(withId(R.id.recyclerView))
    
    
    .check(matches(withItemCount(13)))
    
    
    IdlingRegistry.getInstance().unregister(progressBarIR)
    
    
    onView(withId(R.id.navigation_profile))
    
    
    .perform(click())
    
    
    ...
    

    View Slide

29. // Login screen
    
    
    onView(withId(R.id.emailEditText))
    
    
    .perform(replaceText(email))
    
    
    onView(withId(R.id.passwordEditText))
    
    
    .perform(replaceText(password), closeSoftKeyboard())
    
    
    onView(withId(R.id.loginButton))
    
    
    .perform(click())
    
    
    // Home screen
    
    
    val progressBarIR = ViewVisibilityIdlingResource(...)
    
    
    IdlingRegistry.getInstance().register(progressBarIR)
    
    
    onView(withId(R.id.recyclerView))
    
    
    .check(matches(withItemCount(13)))
    
    
    IdlingRegistry.getInstance().unregister(progressBarIR)
    
    
    onView(withId(R.id.navigation_profile))
    
    
    .perform(click())
    
    
    ...
    

    View Slide

30. onView(withId(R.id.email))
    
    
    .perform(replaceText(EMAIL)
    
    
    onView(withId(R.id.password))
    
    
    .perform(replaceText(PASSWORD)
    
    
    onView(withId(R.id.login))
    
    
    .perform(click())
    

    View Slide

31. onView(withId(R.id.email))
    
    
    login(EMAIL, PASSWORD)
    
    
    }
    

    View Slide

32. onView(withId(R.id.email))
    
    
    login(EMAIL, PASSWORD)
    
    
    }
    open class BaseTestRobot {
    
    
    fun enterText(viewId: Int, text: String) {
    
    
    onView(withId(viewId))
    
    
    .perform(replaceText(text))
    
    
    }
    
    
    fun clickOnView(viewId: Int) {
    
    
    onView(withId(viewId))
    
    
    .perform(click())
    
    
    }
    
    
    }
    

    View Slide

33. onView(withId(R.id.email))
    
    
    login(EMAIL, PASSWORD)
    
    
    }
    open class BaseTestRobot {
    
    
    fun login(email: String, password: String) {
    
    
    enterText(R.id.email, email)
    
    
    enterText(R.id.password, password)
    
    
    clickOnView(R.id.loginButton)
    
    
    }
    
    
    }
    

    View Slide

34. onView(withId(R.id.email))
    
    
    login(EMAIL, PASSWORD)
    
    
    }
    open class BaseTestRobot {
    
    
    LoginScreenRobot().apply { func() }
    

    View Slide

35. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)
    
    
    class LoginActivityTest {
    
    
    @Test
    
    
    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {
    
    
    loginScreen {
    
    
    enterEmail(EMPTY_VALUE)
    
    
    emptyEmailErrorDisplayed()
    
    
    }
    
    
    }
    
    
    @Test
    
    
    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {
    
    
    loginScreen {
    
    
    enterEmail(INCORRECT_EMAIL)
    
    
    incorrectEmailErrorDisplayed()
    
    
    }
    
    
    }
    
    
    @Test
    
    
    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {
    
    
    loginScreen {
    
    
    enterPassword(EMPTY_VALUE)
    
    
    emptyPasswordErrorDisplayed()
    
    
    }
    
    
    }
    
    
    ...
    
    
    }
    class LoginScreenRobot : BaseTestRobot() {
    
    
    fun enterEmail(email: String) =
    
    
    enterText(R.id.email, email)
    
    
    
    
    fun enterPassword(email: String) =
    

    enterText(R.id.password, password)
    
    
    ...
    
    
    }
    open class BaseTestRobot {
    
    
    fun enterText(viewId: Int, text: String) {
    
    
    onView(withId(viewId))
    
    
    .perform(replaceText(text))
    
    
    }
    
    
    fun clickOnView(viewId: Int) {
    
    
    onView(withId(viewId))
    
    
    .perform(click())
    
    
    }
    
    
    

    ...
    
    
    }
    ESPRESSO
    

    View Slide

36. BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    @RunWith(AndroidJUnit4::class)
    
    
    class LoginActivityTest {
    
    
    @Test
    
    
    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {
    
    
    loginScreen {
    
    
    enterEmail(EMPTY_VALUE)
    
    
    emptyEmailErrorDisplayed()
    
    
    }
    
    
    }
    
    
    @Test
    
    
    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {
    
    
    loginScreen {
    
    
    enterEmail(INCORRECT_EMAIL)
    
    
    incorrectEmailErrorDisplayed()
    
    
    }
    
    
    }
    
    
    @Test
    
    
    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {
    
    
    loginScreen {
    
    
    enterPassword(EMPTY_VALUE)
    
    
    emptyPasswordErrorDisplayed()
    
    
    }
    
    
    }
    
    
    ...
    
    
    }
    class LoginScreenRobot : BaseTestRobot() {
    
    
    fun enterEmail(email: String) =
    
    
    enterText(R.id.email, email)
    
    
    
    
    fun enterPassword(email: String) =
    

    enterText(R.id.password, password)
    
    
    ...
    
    
    }
    open class BaseTestRobot {
    
    
    fun enterText(viewId: Int, text: String) {
    
    
    val view = device.findObject(By.res(resId(viewId)))
    
    
    view.text = text
    
    
    }
    
    
    fun clickOnView(viewId: Int) {
    
    
    device.findObject(By.res(resIf(viewId)))
    

    .click()
    
    
    }
    
    
    

    ...
    
    
    }
    UI AUTOMATOR
    

    View Slide

37. open class BaseTestRobot {
    
    
    @get:Rule val composeTestRule = createComposeRule()
    
    
    
    
    fun enterText(tag: String, text: String) {
    
    
    composeTestRule.onNodeWithTag(tag)
    
    
    .performTextInput(text)
    
    
    }
    
    
    fun clickOnView(tag: String) {
    
    
    composeTestRule.onNodeWithTag(tag)
    
    
    .performClick()
    
    
    }
    
    
    }
    @RunWith(AndroidJUnit4::class)
    
    
    class LoginActivityTest {
    
    
    @Test
    
    
    fun shouldBeDisplayedEmptyEmailErrorsWhenValueIsEmpty() {
    
    
    loginScreen {
    
    
    enterEmail(EMPTY_VALUE)
    
    
    emptyEmailErrorDisplayed()
    
    
    }
    
    
    }
    
    
    @Test
    
    
    fun shouldBeDisplayedEmailErrorsWhenValueIsNotEmail() {
    
    
    loginScreen {
    
    
    enterEmail(INCORRECT_EMAIL)
    
    
    incorrectEmailErrorDisplayed()
    
    
    }
    
    
    }
    
    
    @Test
    
    
    fun shouldBeDisplayedEmptyPasswordErrorsWhenValueIsEmpty() {
    
    
    loginScreen {
    
    
    enterPassword(EMPTY_VALUE)
    
    
    emptyPasswordErrorDisplayed()
    
    
    }
    
    
    }
    
    
    ...
    
    
    }
    BASIC OPERATIONS
    SCREEN ROBOTS
    TEST CASES
    COMPOSE UI TESTS
    class LoginScreenRobot : BaseTestRobot() {
    
    
    fun enterEmail(email: String) =
    
    
    enterText("email", email)
    
    
    
    
    fun enterPassword(email: String) =
    

    enterText("password", password)
    
    
    ...
    
    
    }
    

    View Slide

38. Challenges
    

    In Mobile Testing
    

    View Slide

39. Light and Dark themes
    

    View Slide

40. YES NO
    RTL support
    

    View Slide

41. Accessibility support
    Accessibility checks
    Hello …
    FONT SIZE
    Small, Default, Large, Largest
    DISPLAY SCALING
    Small, Default, Large, Largest
    

    View Slide

42. Testing permissions
    

    View Slide

43. Testing permissions
    UI interaction
    GrantPermissionRule
    ADB commands
    

    View Slide

44. Testing permissions
    UI interaction
    GrantPermissionRule
    ADB commands
    Android Test Orchestrator
    

    View Slide

45. Test Case
    Test Case
    Test Case
    Clean up data after test execution
    

    View Slide

46. /data/data/PACKAGE/
    databases
    fi
    les
    shared_prefs
    Clean up data after test execution
    

    View Slide

47. databases
    fi
    les
    shared_prefs
    @get:Rul
    e
    
    val clearDatabaseRule = ClearDatabaseRule(
    )
    
    @get:Rul
    e
    
    val clearFileRule = ClearFilesRule(
    )
    
    @get:Rul
    e
    
    val clearPreferencesRule = ClearPreferencesRule(
    )
    
    https://github.com/AdevintaSpain/Barista
    Clean up data after test execution
    

    View Slide

48. Server
    Database
    Pre-populating the database
    

    View Slide

49. Test Case
    Database
    DATA LAYER
    Query
    Dependencies
    UI interaction
    Pre-populating the database
    

    View Slide

50. Real data vs Fake data
    Prod server
    Dev server
    Mock/Fake
    

    View Slide

51. Real data vs Fake data
    Prod server
    Dev server DATA SYNCHRONIZATION
    We need to synchronize data between
    the production and dev servers.
    INTERACTION WITH THE SERVER
    We make requests to the production
    server. The connection can require
    certificates, VPN, etc.
    PRODUCTION BACK-END
    We always use the latest available
    production environment.
    

    View Slide

52. Real data vs Fake data
    Mock/Fake
    DATA SYNCHRONIZATION
    We need to synchronize predefined
    responses with responses from the
    production server.
    NO INTERACTION WITH THE SERVER
    Responses from the production server
    can differ from the predefined data.
    We can use predefined fake responses
    instead of calling the production
    server.
    MAKE TESTS MORE STABLE
    

    View Slide

53. End-To-End vs Functional tests
    

    View Slide

54. INTERACTION WITH THE SERVER
    Interaction with the production server.
    ENTRY POINT
    Similar to the entry point of an app.
    END-TO-END TESTS
    

    View Slide

55. FUNCTIONAL TESTS
    INTERACTION WITH THE SERVER
    Interaction with the non-production
    server.
    NAVIGATION
    Usually, navigation is not needed.
    ENTRY POINT
    We can start the test from a required
    Activity/Fragment.
    

    View Slide

56. END-TO-END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT
    
    
    Similar to the entry point of an
    app.
    APP VERIFICATION
    
    
    Slow veri
    fi
    cation of the app.
    NAVIGATION
    
    
    Navigate to the required
    screen.
    SERVER INTERACTION
    
    
    Interaction with the
    production server.
    ENTRY POINT
    
    
    Start the required screen of the
    app.
    SERVER INTERACTION
    
    
    Usually interaction with non-
    production server.
    NAVIGATION
    
    
    Usually no navigation.
    UI VERIFICATION
    
    
    Fast veri
    fi
    cation of
    components or screens.
    

    View Slide

57. END-TO-END TESTS FUNCTIONAL TESTS
    

    View Slide

58. Flaky Tests
    

    View Slide

59. External dependencies
    Framework
    Test case execution
    Device and emulator
    

    View Slide

60. External dependencies
    - Network connection (VPN)
    
    
    - Network speed
    
    
    - Back-end
    

    View Slide

61. Framework
    - Framework issues
    
    
    - Toast, Snackbar, etc
    
    
    - Custom Views
    

    View Slide

62. Device and emulator
    - Performance
    
    
    - Noti
    fi
    cations
    
    
    - Device memory
    

    View Slide

63. Test case execution
    - Simulate User actions
    
    
    - Incorrect state before/after
    a test case
    
    
    - Toast, Snackbar, etc
    

    View Slide

64. - Network connection (VPN)
    
    
    - Network speed
    
    
    - Back-end
    - Framework issues
    
    
    - Toast, Snackbar, etc
    
    
    - Custom Views
    - Simulate User actions
    
    
    - Incorrect state before/after a
    test case
    
    
    - Toast, Snackbar, etc
    - Performance
    
    
    - Noti
    fi
    cations
    
    
    - Device memory
    External dependencies
    Framework
    Test case execution
    Device and emulator
    

    View Slide

65. Best Practices
    

    View Slide

66. PAY ATTENTION TO NAMING
    

    View Slide

67. NO "SLEEP" IN TESTS
    

    View Slide

68. ALL TEST CASES SHOULD BE
    INDEPENDENT
    

    View Slide

69. USE MULTIPLE SMALL TESTS
    INSTEAD OF ONE BIG TEST
    

    View Slide

70. DO NOT SPEND TIME NAVIGATING
    TO THE REQUIRED SCREEN
    

    View Slide

71. FOLLOW THE
    

    "NO FLAKY TESTS" POLICY
    

    View Slide

72. DO NOT RELY ONLY ON
    

    UI TEST AUTOMATION
    

    View Slide

73. LEARN YOUR TOOLS
    

    View Slide

74. @alex_zhukovich https://alexzh.com/
    THANK YOU
    
    
    FOR LISTENING!
    

    View Slide