Efficient UI testing in Android

Mobile apps are growing. They become more complex and require more testing. It means that it is time to integrate fast and stable automated tests into your project.

In this talk, we will discuss the following topics:
- How to create fast and stable UI tests
- How to avoid flaky tests
- How to cover applications which include traditional and Jetpack Compose - views and screens
- How to share UI tests between local and instrumentation tests
- How DSL can speed up adding stable UI tests to the project

Transcript

1. @alex_zhukovich https://alexzh.com/ Ef fi cient 
 UI Testing
6. Exploring the app
7. ACTIVITY FRAGMENT
8. ACTIVITY FRAGMENT COMPOSABLE FUNCTION
9. launchActivity launchFragmentInContainer ComposeContentTestRule ActivityScenario.launch( MainActivity::class.java )
10. launchActivity launchFragmentInContainer ComposeContentTestRule val activityOptions = bundleOf( Pair("param-1", 1), Pair("param-2", 2), Pair("param-3", 3), ) val scenario = ActivityScenario.launch( MainActivity::class.java, activityOptions ) scenario.moveToState(Lifecycle.State.STARTED) ActivityScenario.launch(
11. launchActivity launchFragmentInContainer ComposeContentTestRule val scenario = launchFragmentInContainer( fragmentArgs: Bundle? = null, @StyleRes themeResId: Int = 
 R.style.FragmentScenarioEmptyFragmentActivityTheme, initialState: Lifecycle.State = 
 Lifecycle.State.RESUMED, factory: FragmentFactory? = null ) scenario.moveToState(Lifecycle.State.STARTED) debugImplementation “androidx.fragment:fragment-testing:1.4.1"
12. launchActivity launchFragmentInContainer ComposeContentTestRule Activity Fragment
13. launchActivity launchFragmentInContainer ComposeContentTestRule androidx.fragment.app.testing.FragmentScenario$EmptyFragmentActivity Activity Fragment Compilation Added to AndroidManifest
14. launchActivity launchFragmentInContainer ComposeContentTestRule @get:Rule val composeTestRule = createComposeRule() composeTestRule.apply { setContent { LoginScreen() } onNodeWithTag("email") .performTextInput("test@test.com") onNodeWithTag("password") .performTextInput("password") onNodeWithTag("login") .performClick() ... } debugImplementation “androidx.compose.ui:ui-test-manifest:1:1:1"
15. launchActivity launchFragmentInContainer ComposeContentTestRule Component Activity @Composable
16. launchActivity launchFragmentInContainer ComposeContentTestRule androidx.activity.ComponentActivity Component Activity @Composable Compilation Added to AndroidManifest
17. End-To-End testing
20. END-TO-END TESTS class ProfileE2ETest { @get:Rule val composableTestRule = createAndroidComposeRule<HomeActivity>() fun hasText(@StringRes resId: Int) = hasText(composableTestRule.activity.getString(resId)) @Test fun createAccount_emptyNameErrorMessage() { composableTestRule.apply { onNode(hasText(R.string.navigation_settings_label)) .performClick() onNode(hasText(R.string.settingsScreen_profile_title)) .performClick() onNode(hasText(R.string.profileScreen_createAccount_button)) .performClick() onNode(hasText(R.string.createAccountScreen_createAccount_button)) .performClick() onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label)) .assertIsDisplayed() } } }
21. END-TO-END TESTS class ProfileE2ETest { @get:Rule val composableTestRule = createAndroidComposeRule<HomeActivity>() fun hasText(@StringRes resId: Int) = hasText(composableTestRule.activity.getString(resId)) @Test fun createAccount_emptyNameErrorMessage() { composableTestRule.apply { onNode(hasText(R.string.navigation_settings_label)) .performClick() onNode(hasText(R.string.settingsScreen_profile_title)) .performClick() onNode(hasText(R.string.profileScreen_createAccount_button)) .performClick() onNode(hasText(R.string.createAccountScreen_createAccount_button)) .performClick() onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label)) .assertIsDisplayed() } } }
22. END-TO-END TESTS class ProfileE2ETest { @get:Rule val composableTestRule = createAndroidComposeRule<HomeActivity>() fun hasText(@StringRes resId: Int) = hasText(composableTestRule.activity.getString(resId)) @Test fun createAccount_emptyNameErrorMessage() { composableTestRule.apply { onNode(hasText(R.string.navigation_settings_label)) .performClick() onNode(hasText(R.string.settingsScreen_profile_title)) .performClick() onNode(hasText(R.string.profileScreen_createAccount_button)) .performClick() onNode(hasText(R.string.createAccountScreen_createAccount_button)) .performClick() onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label)) .assertIsDisplayed() } } } NAVIGATION ≈ 3 SEC
23. END-TO-END TESTS NAVIGATION Navigate to the required screen INTERACTION WITH THE SERVER Interaction with the production server ENTRY POINT Similar to the entry point of an app APP VERIFICATION Slow verification of the app
24. Functional testing
25. FUNCTIONAL TESTS class LoginScreenTest : KoinTest { @get:Rule val composableTestRule = createAndroidComposeRule<ComponentActivity>() @Before fun setup() { stopKoin() startKoin { androidContext(InstrumentationRegistry.getInstrumentation().targetContext) modules(dataModule, appModule) } } @Test fun createAccount_emptyNameErrorMessage() { composableTestRule.apply { setContent { CreateAccountScreen( viewModel = get(), onLogin = { }, onCreateAccountSuccess = { } 
 ) } onNode(hasText(R.string.createAccountScreen_createAccount_button)) .performClick() onNode(hasText(R.string.createAccountScreen_error_nameIsTooShort_label)) .assertIsDisplayed() } } 
 
 fun hasText(@StringRes resId: Int) = hasText(composableTestRule.activity.getString(resId)) }
27. FUNCTIONAL TESTS NAVIGATION Usually no navigation INTERACTION WITH THE SERVER Usually interaction with non- production server ENTRY POINT Start the required screen of the app APP VERIFICATION Fast verification of components or screens
28. Screenshot testing
29. INTERACTION PIXEL PERFECTNESS
30. INTERACTION PIXEL PERFECTNESS
33. class WeekCalendarTest : ScreenshotTest { private val testDate = LocalDate.of(2022, 5, 5) @get:Rule val composeTestRule = createComposeRule() @Test fun weekCalendar_todayIsSelectedDate() { composeTestRule.setContent { AppTheme(darkTheme = theme == Theme.DARK) { WeekCalendar( startDate = testDate.minusDays(6), selectedDate = testDate.minusDays(1), onSelectedDateChange = {}, todayDate = testDate.minusDays(1) ) } } compareScreenshot(composeTestRule) } }
34. Ef fi cient UI testing
35. END-TO-END TESTS FUNCTIONAL TESTS SCREENSHOT TESTS
36. FUNCTIONAL TESTS SCREENSHOT TESTS
37. END-TO-END TESTS SCREENSHOT TESTS
38. END-TO-END TESTS FUNCTIONAL TESTS
39. END-TO-END TESTS FUNCTIONAL TESTS SCREENSHOT TESTS
40. Lesson learned
41. READABILITY
42. onNodeWithText("Login") .performClick() onNodeWithText("Email") .performTextInput(email) onNodeWithText("Password") .performTextInput(password) onNode(hasText("LOGIN")) .performClick() onNodeWithText(email) .assertIsDisplayed() onNodeWithText(username) .assertIsDisplayed()
43. // Profile screen onNodeWithText("Login") .performClick() // Login screen onNodeWithText("Email") .performTextInput(email) onNodeWithText("Password") .performTextInput(password) onNode(hasText("LOGIN")) .performClick() // Profile screen onNodeWithText(email) .assertIsDisplayed() onNodeWithText(username) .assertIsDisplayed()
44. profileScreen { launch() hasTitle() tapOnLogin() } loginScreen { login(email, password) } profileScreen { hasUserInfo(email, username) }
45. FLAKY TESTS READABILITY
46. External dependencies Framework Test case execution Device and emulator
47. External dependencies - Network connection (VPN) - Network speed - Back-end
48. Framework - Framework issues - Toast, Snackbar, etc - Custom Views
49. Device and emulator - Performance - Noti fi cations - Device memory
50. Test case execution - Simulate User actions - Incorrect state before/after a test case - Toast, Snackbar, etc
51. - Network connection (VPN) - Network speed - Back-end - Framework issues - Toast, Snackbar, etc - Custom Views - Simulate User actions - Incorrect state before/after a test case - Toast, Snackbar, etc - Performance - Noti fi cations - Device memory External dependencies Framework Test case execution Device and emulator
52. LEARN YOUR TOOLS FLAKY TESTS READABILITY
53. https://github.com/cashapp/paparazzi https://github.com/pedrovgs/Shot SHOT PAPARAZZI
54. SHOT PAPARAZZI https://github.com/cashapp/paparazzi https://github.com/pedrovgs/Shot DEVICE JVM
55. https://github.com/cashapp/paparazzi https://github.com/pedrovgs/Shot SHOT DEVICE PAPARAZZI JVM
56. LEARN YOUR TOOLS PARAMETERIZED TESTS FLAKY TESTS READABILITY
57. AndroidUiTestingUtils https://github.com/sergio-sastre/AndroidUiTestingUtils Locale Light/Dark mode Display size, screen orientation Font size
58. TestParameterInjector https://github.com/google/TestParameterInjector Strings Enums Primitive Types
59. @RunWith(TestParameterInjector::class) class SettingsScreenParamScreenshotTest : ScreenshotTest { @get:Rule val composeTestRule = createEmptyComposeRule() @Test fun settingsScreen_customFontSizeAndUiMode( @TestParameter fontSize: FontSize, @TestParameter uiMode: UiMode ) { val activityScenario = ActivityScenarioConfigurator.ForComposable() .setFontSize(fontSize) .setUiMode(uiMode) .launchConfiguredActivity() .onActivity { it.setContent { AppTheme { SettingsScreen( onProfile = {}, onDocs = {} ) } } } activityScenario.waitForActivity() compareScreenshot(composeTestRule, "settingsScreen_${uiMode}_${fontSize}_defaultState") activityScenario.close() } }
60. @RunWith(TestParameterInjector::class) class SettingsScreenParamScreenshotTest : ScreenshotTest { @get:Rule val composeTestRule = createEmptyComposeRule() @Test fun settingsScreen_customFontSizeAndUiMode( @TestParameter fontSize: FontSize, @TestParameter uiMode: UiMode ) { val activityScenario = ActivityScenarioConfigurator.ForComposable() .setFontSize(fontSize) .setUiMode(uiMode) .launchConfiguredActivity() .onActivity { it.setContent { AppTheme { SettingsScreen( onProfile = {}, onDocs = {} ) } } } activityScenario.waitForActivity() compareScreenshot(composeTestRule, "settingsScreen_${uiMode}_${fontSize}_defaultState") activityScenario.close() } }
62. DIFFERENT APPROACHES LEARN YOUR TOOLS PARAMETERIZED TESTS FLAKY TESTS READABILITY
63. Server Database PRE-POPULATING THE DATABASE
64. Database DATA LAYER Query Dependencies UI interaction PRE-POPULATING THE DATABASE TEST CASE
65. Prod server Dev server Mock/Fake REAL DATA VS FAKE DATA
66. DATA SYNCHRONIZATION We need to synchronize data between the production and dev servers. INTERACTION WITH THE SERVER We make requests to the production server. The connection can require certificates, VPN, etc. PRODUCTION BACK-END We always use the latest available production environment. REAL DATA VS FAKE DATA Prod server Dev server
67. DATA SYNCHRONIZATION We need to synchronize predefined responses with responses from the production server. We can use predefined fake responses instead of calling the production server. MAKE TESTS MORE STABLE REAL DATA VS FAKE DATA Mock/Fake NO INTERACTION WITH THE SERVER Responses from the production server can differ from the predefined data.
68. END-TO-END (E2E) TESTS FUNCTIONAL TESTS ENTRY POINT Similar to the entry point of an app. APP VERIFICATION Slow veri fi cation of the app. NAVIGATION Navigate to the required screen. SERVER INTERACTION Interaction with the production server. ENTRY POINT Start the required screen of the app. SERVER INTERACTION Usually interaction with non- production server. NAVIGATION Usually no navigation. UI VERIFICATION Fast veri fi cation of components or screens.
69. Next steps
70. GROUP TEST CASES TESTING WIDGETS TESTING SHORTCUTS
71. #ExploreMore UI Testing @alex_zhukovich https://alexzh.com/