Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Efficient UI testing in Android

Efficient UI testing in Android

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate fast and stable automated tests into your project.

In this talk, we will discuss the following topics:
- How to create fast and stable UI tests
- How to avoid flaky tests
- How to cover applications which include traditional and Jetpack Compose - views and screens
- How to share UI tests between local and instrumentation tests
- How DSL can speed up adding stable UI tests to the project

Alex Zhukovich

August 01, 2022
Tweet

More Decks by Alex Zhukovich

Other Decks in Programming

Transcript

  1. @alex_zhukovich
    https://alexzh.com/
    Ef
    fi
    cient

    UI Testing

    View full-size slide

  2. Exploring the app

    View full-size slide

  3. ACTIVITY FRAGMENT

    View full-size slide

  4. ACTIVITY FRAGMENT
    COMPOSABLE
    FUNCTION

    View full-size slide

  5. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    ActivityScenario.launch(


    MainActivity::class.java


    )


    View full-size slide

  6. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val activityOptions = bundleOf(


    Pair("param-1", 1),


    Pair("param-2", 2),


    Pair("param-3", 3),


    )


    val scenario = ActivityScenario.launch(


    MainActivity::class.java,


    activityOptions


    )


    scenario.moveToState(Lifecycle.State.STARTED)


    ActivityScenario.launch(


    View full-size slide

  7. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    val scenario = launchFragmentInContainer(


    fragmentArgs: Bundle? = null,


    @StyleRes themeResId: Int =

    R.style.FragmentScenarioEmptyFragmentActivityTheme,


    initialState: Lifecycle.State =

    Lifecycle.State.RESUMED,


    factory: FragmentFactory? = null


    )


    scenario.moveToState(Lifecycle.State.STARTED)


    debugImplementation “androidx.fragment:fragment-testing:1.4.1"

    View full-size slide

  8. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Activity
    Fragment

    View full-size slide

  9. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.fragment.app.testing.FragmentScenario$EmptyFragmentActivity
    Activity
    Fragment
    Compilation
    Added to AndroidManifest

    View full-size slide

  10. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    @get:Rule


    val composeTestRule = createComposeRule()


    composeTestRule.apply {


    setContent { LoginScreen() }


    onNodeWithTag("email")


    .performTextInput("[email protected]")


    onNodeWithTag("password")


    .performTextInput("password")


    onNodeWithTag("login")


    .performClick()


    ...


    }
    debugImplementation “androidx.compose.ui:ui-test-manifest:1:1:1"

    View full-size slide

  11. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    Component
    Activity
    @Composable

    View full-size slide

  12. launchActivity
    launchFragmentInContainer
    ComposeContentTestRule
    androidx.activity.ComponentActivity
    Component
    Activity
    @Composable
    Compilation
    Added to AndroidManifest

    View full-size slide

  13. End-To-End testing

    View full-size slide

  14. END-TO-END TESTS
    class ProfileE2ETest {


    @get:Rule


    val composableTestRule = createAndroidComposeRule()


    fun hasText(@StringRes resId: Int) =


    hasText(composableTestRule.activity.getString(resId))




    @Test


    fun createAccount_emptyNameErrorMessage() {


    composableTestRule.apply {


    onNode(hasText(R.string.navigation_settings_label))


    .performClick()


    onNode(hasText(R.string.settingsScreen_profile_title))


    .performClick()


    onNode(hasText(R.string.profileScreen_createAccount_button))


    .performClick()


    onNode(hasText(R.string.createAccountScreen_createAccount_button))


    .performClick()


    onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label))


    .assertIsDisplayed()


    }


    }


    }


    View full-size slide

  15. END-TO-END TESTS
    class ProfileE2ETest {


    @get:Rule


    val composableTestRule = createAndroidComposeRule()


    fun hasText(@StringRes resId: Int) =


    hasText(composableTestRule.activity.getString(resId))




    @Test


    fun createAccount_emptyNameErrorMessage() {


    composableTestRule.apply {


    onNode(hasText(R.string.navigation_settings_label))


    .performClick()


    onNode(hasText(R.string.settingsScreen_profile_title))


    .performClick()


    onNode(hasText(R.string.profileScreen_createAccount_button))


    .performClick()


    onNode(hasText(R.string.createAccountScreen_createAccount_button))


    .performClick()


    onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label))


    .assertIsDisplayed()


    }


    }


    }


    View full-size slide

  16. END-TO-END TESTS
    class ProfileE2ETest {


    @get:Rule


    val composableTestRule = createAndroidComposeRule()


    fun hasText(@StringRes resId: Int) =


    hasText(composableTestRule.activity.getString(resId))




    @Test


    fun createAccount_emptyNameErrorMessage() {


    composableTestRule.apply {


    onNode(hasText(R.string.navigation_settings_label))


    .performClick()


    onNode(hasText(R.string.settingsScreen_profile_title))


    .performClick()


    onNode(hasText(R.string.profileScreen_createAccount_button))


    .performClick()


    onNode(hasText(R.string.createAccountScreen_createAccount_button))


    .performClick()


    onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label))


    .assertIsDisplayed()


    }


    }


    }


    NAVIGATION ≈ 3 SEC

    View full-size slide

  17. END-TO-END TESTS
    NAVIGATION
    Navigate to the required screen
    INTERACTION WITH THE SERVER
    Interaction with the production
    server
    ENTRY POINT
    Similar to the entry point of an app
    APP VERIFICATION
    Slow verification of the app

    View full-size slide

  18. Functional testing

    View full-size slide

  19. FUNCTIONAL TESTS
    class LoginScreenTest : KoinTest {


    @get:Rule


    val composableTestRule = createAndroidComposeRule()


    @Before


    fun setup() {


    stopKoin()


    startKoin {


    androidContext(InstrumentationRegistry.getInstrumentation().targetContext)


    modules(dataModule, appModule)


    }


    }


    @Test


    fun createAccount_emptyNameErrorMessage() {


    composableTestRule.apply {


    setContent {


    CreateAccountScreen(


    viewModel = get(),


    onLogin = { },


    onCreateAccountSuccess = { }

    )


    }


    onNode(hasText(R.string.createAccountScreen_createAccount_button))


    .performClick()


    onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label))


    .assertIsDisplayed()


    }


    }


    fun hasText(@StringRes resId: Int) =


    hasText(composableTestRule.activity.getString(resId))


    }


    View full-size slide

  20. FUNCTIONAL TESTS
    NAVIGATION
    Usually no navigation
    INTERACTION WITH THE SERVER
    Usually interaction with non-
    production server
    ENTRY POINT
    Start the required screen of the app
    APP VERIFICATION
    Fast verification of components or
    screens

    View full-size slide

  21. Screenshot testing

    View full-size slide

  22. INTERACTION
    PIXEL PERFECTNESS

    View full-size slide

  23. INTERACTION
    PIXEL PERFECTNESS

    View full-size slide

  24. class WeekCalendarTest : ScreenshotTest {


    private val testDate = LocalDate.of(2022, 5, 5)


    @get:Rule


    val composeTestRule = createComposeRule()


    @Test


    fun weekCalendar_todayIsSelectedDate() {


    composeTestRule.setContent {


    AppTheme(darkTheme = theme == Theme.DARK) {


    WeekCalendar(


    startDate = testDate.minusDays(6),


    selectedDate = testDate.minusDays(1),


    onSelectedDateChange = {},


    todayDate = testDate.minusDays(1)


    )


    }


    }




    compareScreenshot(composeTestRule)


    }


    }


    View full-size slide

  25. Ef
    fi
    cient UI testing

    View full-size slide

  26. END-TO-END TESTS
    FUNCTIONAL TESTS
    SCREENSHOT TESTS

    View full-size slide

  27. FUNCTIONAL TESTS
    SCREENSHOT TESTS

    View full-size slide

  28. END-TO-END TESTS SCREENSHOT TESTS

    View full-size slide

  29. END-TO-END TESTS
    FUNCTIONAL TESTS

    View full-size slide

  30. END-TO-END TESTS
    FUNCTIONAL TESTS
    SCREENSHOT TESTS

    View full-size slide

  31. Lesson learned

    View full-size slide

  32. onNodeWithText("Login")


    .performClick()


    onNodeWithText("Email")


    .performTextInput(email)


    onNodeWithText("Password")


    .performTextInput(password)


    onNode(hasText("LOGIN"))


    .performClick()


    onNodeWithText(email)


    .assertIsDisplayed()


    onNodeWithText(username)


    .assertIsDisplayed()


    View full-size slide

  33. // Profile screen


    onNodeWithText("Login")


    .performClick()


    // Login screen


    onNodeWithText("Email")


    .performTextInput(email)


    onNodeWithText("Password")


    .performTextInput(password)


    onNode(hasText("LOGIN"))


    .performClick()


    // Profile screen


    onNodeWithText(email)


    .assertIsDisplayed()


    onNodeWithText(username)


    .assertIsDisplayed()


    View full-size slide

  34. profileScreen {


    launch()


    hasTitle()


    tapOnLogin()


    }


    loginScreen {


    login(email, password)


    }


    profileScreen {


    hasUserInfo(email, username)


    }


    View full-size slide

  35. FLAKY TESTS
    READABILITY

    View full-size slide

  36. External dependencies
    Framework
    Test case execution
    Device and emulator

    View full-size slide

  37. External dependencies
    - Network connection (VPN)


    - Network speed


    - Back-end

    View full-size slide

  38. Framework
    - Framework issues


    - Toast, Snackbar, etc


    - Custom Views

    View full-size slide

  39. Device and emulator
    - Performance


    - Noti
    fi
    cations


    - Device memory

    View full-size slide

  40. Test case execution
    - Simulate User actions


    - Incorrect state before/after
    a test case


    - Toast, Snackbar, etc

    View full-size slide

  41. - Network connection (VPN)


    - Network speed


    - Back-end
    - Framework issues


    - Toast, Snackbar, etc


    - Custom Views
    - Simulate User actions


    - Incorrect state before/after a
    test case


    - Toast, Snackbar, etc
    - Performance


    - Noti
    fi
    cations


    - Device memory
    External dependencies
    Framework
    Test case execution
    Device and emulator

    View full-size slide

  42. LEARN YOUR TOOLS
    FLAKY TESTS
    READABILITY

    View full-size slide

  43. https://github.com/cashapp/paparazzi
    https://github.com/pedrovgs/Shot
    SHOT PAPARAZZI

    View full-size slide

  44. SHOT PAPARAZZI
    https://github.com/cashapp/paparazzi
    https://github.com/pedrovgs/Shot
    DEVICE JVM

    View full-size slide

  45. https://github.com/cashapp/paparazzi
    https://github.com/pedrovgs/Shot
    SHOT
    DEVICE
    PAPARAZZI
    JVM

    View full-size slide

  46. LEARN YOUR TOOLS
    PARAMETERIZED TESTS
    FLAKY TESTS
    READABILITY

    View full-size slide

  47. AndroidUiTestingUtils
    https://github.com/sergio-sastre/AndroidUiTestingUtils
    Locale Light/Dark


    mode
    Display size,


    screen orientation
    Font size

    View full-size slide

  48. TestParameterInjector
    https://github.com/google/TestParameterInjector
    Strings Enums
    Primitive Types

    View full-size slide

  49. @RunWith(TestParameterInjector::class)


    class SettingsScreenParamScreenshotTest : ScreenshotTest {


    @get:Rule


    val composeTestRule = createEmptyComposeRule()


    @Test


    fun settingsScreen_customFontSizeAndUiMode(


    @TestParameter fontSize: FontSize,


    @TestParameter uiMode: UiMode


    ) {


    val activityScenario = ActivityScenarioConfigurator.ForComposable()


    .setFontSize(fontSize)


    .setUiMode(uiMode)


    .launchConfiguredActivity()


    .onActivity {


    it.setContent {


    AppTheme {


    SettingsScreen(


    onProfile = {},


    onDocs = {}


    )


    }


    }


    }


    activityScenario.waitForActivity()


    compareScreenshot(composeTestRule, "settingsScreen_${uiMode}_${fontSize}_defaultState")


    activityScenario.close()


    }


    }

    View full-size slide

  50. @RunWith(TestParameterInjector::class)


    class SettingsScreenParamScreenshotTest : ScreenshotTest {


    @get:Rule


    val composeTestRule = createEmptyComposeRule()


    @Test


    fun settingsScreen_customFontSizeAndUiMode(


    @TestParameter fontSize: FontSize,


    @TestParameter uiMode: UiMode


    ) {


    val activityScenario = ActivityScenarioConfigurator.ForComposable()


    .setFontSize(fontSize)


    .setUiMode(uiMode)


    .launchConfiguredActivity()


    .onActivity {


    it.setContent {


    AppTheme {


    SettingsScreen(


    onProfile = {},


    onDocs = {}


    )


    }


    }


    }


    activityScenario.waitForActivity()


    compareScreenshot(composeTestRule, "settingsScreen_${uiMode}_${fontSize}_defaultState")


    activityScenario.close()


    }


    }

    View full-size slide

  51. DIFFERENT APPROACHES
    LEARN YOUR TOOLS
    PARAMETERIZED TESTS
    FLAKY TESTS
    READABILITY

    View full-size slide

  52. Server
    Database
    PRE-POPULATING THE DATABASE

    View full-size slide

  53. Database
    DATA LAYER
    Query
    Dependencies
    UI interaction
    PRE-POPULATING THE DATABASE
    TEST CASE

    View full-size slide

  54. Prod
    server
    Dev server
    Mock/Fake
    REAL DATA VS FAKE DATA

    View full-size slide

  55. DATA SYNCHRONIZATION
    We need to synchronize data between
    the production and dev servers.
    INTERACTION WITH THE SERVER
    We make requests to the production
    server. The connection can require
    certificates, VPN, etc.
    PRODUCTION BACK-END
    We always use the latest available
    production environment.
    REAL DATA VS FAKE DATA
    Prod
    server
    Dev server

    View full-size slide

  56. DATA SYNCHRONIZATION
    We need to synchronize predefined
    responses with responses from the
    production server.
    We can use predefined fake responses
    instead of calling the production
    server.
    MAKE TESTS MORE STABLE
    REAL DATA VS FAKE DATA
    Mock/Fake
    NO INTERACTION WITH THE SERVER
    Responses from the production server
    can differ from the predefined data.

    View full-size slide

  57. END-TO-END (E2E) TESTS FUNCTIONAL TESTS
    ENTRY POINT


    Similar to the entry point of an
    app.
    APP VERIFICATION


    Slow veri
    fi
    cation of the app.
    NAVIGATION


    Navigate to the required
    screen.
    SERVER INTERACTION


    Interaction with the
    production server.
    ENTRY POINT


    Start the required screen of the
    app.
    SERVER INTERACTION


    Usually interaction with non-
    production server.
    NAVIGATION


    Usually no navigation.
    UI VERIFICATION


    Fast veri
    fi
    cation of
    components or screens.

    View full-size slide

  58. GROUP TEST CASES TESTING WIDGETS TESTING SHORTCUTS

    View full-size slide

  59. #ExploreMore


    UI Testing
    @alex_zhukovich https://alexzh.com/

    View full-size slide