Instrumentation Testing Strategies

Tips for maximizing value from Android tests

- Core principles of testing
- Write once, run everywhere: shared tests with Robolectric + AndroidX Test
- Robot pattern for UI tests

https://www.meetup.com/Android-Meetup/events/264205264/

Linked resources in slides:

- Frictionless Android testing: write once, run everywhere (Google I/O '18)
https://www.youtube.com/watch?v=wYMIadv9iF8
- Testing Android Apps at Scale with Nitrogen (Android Dev Summit '18)
https://www.youtube.com/watch?v=-_kZC29sWAo
- Instrumentation Testing Robots by Jake Wharton
https://academy.realm.io/posts/kau-jake-wharton-testing-robots/
- Demo application
https://github.com/ssaqua/WallPad

Transcript

1. Instrumentation Testing Strategies Sung-Soo Hong Tips for maximizing value from Android tests
2. Core principles Write once, run everywhere - shared tests with Robolectric + AndroidX Test Robot pattern - manageable UI tests
3. Why do we write tests? Proves your code is working as intended Ensures your code keeps working as intended Form of documentation Enables confidence to refactor Release frequently
4. We get value from tests when they are run
5. Tests are a long term commitment Treat tests as production code Fix failing tests or bugs in tests Refactor existing tests Remove obsolete tests Tests require infrastructure CI servers Influences application architecture
6. Tests slow down development?
7. Tests slow down development?
8. Unit Tests Run on local JVM No dependencies on Android framework (or mocked) Execute quickly, on the order of milliseconds Cannot accept any flakiness Focused on small units, optimized for failure
9. Instrumentation Tests Run on Android environment Higher fidelity at the expense of execution time and debugging Difficult to avoid flakiness Possible to be agnostic to app architecture "The Android robot is reproduced or modified from work created and shared by Google and used according to terms described in the Creative Commons 3.0 Attribution License."
10. class RepositoryTest { private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() lateinit var repository: Repository @Before fun setUp() { repository = TestHelper.createRepository(mockService, mockCache) }} @Test fun `getItems() requests from service when cache is empty`() { val items = repository.getItems() verify(mockService).getItems() checkItemsAreFromService(items) }} }}
11. class RepositoryTest { private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() lateinit var repository: Repository @Before fun setUp() { repository = TestHelper.createRepository(mockService, mockCache) }} @Test fun `getItems() requests from service when cache is empty`() { val items = repository.getItems() verify(mockService).getItems() checkItemsAreFromService(items) }} }}
12. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() lateinit var repository: Repository @Before fun setUp() { repository = TestHelper.createRepository(mockService, mockCache) }} @Test fun `getItems() requests from service when cache is empty`() {{ val items = repository.getItems() verify(mockService).getItems() checkItemsAreFromService(items) }} }}
13. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = TestHelper.createRepository(mockService, mockCache) val items = repository.getItems() verify(mockService).getItems() checkItemsAreFromService(items) }} }}
14. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = TestHelper.createRepository(mockService, mockCache) val items = repository.getItems() verify(mockService).getItems() checkItemsAreFromService(items) }} }}
15. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = Repository(mockService,_mockCache)) `when`(mockCache.get()).thenReturn(emptyList()) val_serviceItems = listOf(Item()) `when`(mockService.getItems()).thenReturn(serviceItems) val items = repository.getItems() verify(mockService).getItems() checkItemsAreFromService(items) }} }}
16. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = Repository(mockService,_mockCache)) `when`(mockCache.get()).thenReturn(emptyList()) val_serviceItems = listOf(Item()) `when`(mockService.getItems()).thenReturn(serviceItems) val items = repository.getItems() verify(mockService).getItems() checkItemsAreFromService(items) }} }}
17. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = Repository(mockService,_mockCache)) `when`(mockCache.get()).thenReturn(emptyList()) val_serviceItems = listOf(Item()) `when`(mockService.getItems()).thenReturn(serviceItems) val items = repository.getItems() verify(mockService).getItems() assertEquals(items, serviceItems) }} }}
18. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = Repository(mockService,_mockCache)) `when`(mockCache.get()).thenReturn(emptyList()) val_serviceItems = listOf(Item()) `when`(mockService.getItems()).thenReturn(serviceItems) val items = repository.getItems() verify(mockService).getItems() assertEquals(items, serviceItems) }} }}
19. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = Repository(mockService,_mockCache)) `when`(mockCache.get()).thenReturn(emptyList()) repository.getItems() verify(mockService).getItems() }} }}
20. class RepositoryTest {{ private val mockService: Service = mock() private val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = Repository(mockService,_mockCache)) `when`(mockCache.get()).thenReturn(emptyList()) repository.getItems() verify(mockService).getItems() }} }}
21. class RepositoryTest {{ val mockService: Service = mock() val mockCache: Cache<List<Item>> = mock() @Test fun `getItems() requests from service when cache is empty`() {{ val repository = Repository(mockService,_mockCache)) `when`(mockCache.get()).thenReturn(emptyList()) repository.getItems() verify(mockService).getItems() }} }}
22. Prioritize readability for tests Violating DRY or including magic numbers is acceptable Minimize any layers of abstraction OOP encapsulation rarely matters for testing code Consider disabling MemberVisibilityCanBePrivate inspection in tests scope
23. Write once, run everywhere shared tests with Robolectric + AndroidX Test +
24. Dependencies testImplementation 'androidx.test:runner:1.2.0' testImplementation 'androidx.test:core-ktx:1.2.0' testImplementation 'androidx.test.ext:junit:1.1.1' testImplementation 'androidx.test.espresso:espresso-core:3.2.0' testImplementation 'org.robolectric:robolectric:4.3' androidTestImplementation 'androidx.test:runner:1.2.0' androidTestImplementation 'androidx.test:core-ktx:1.2.0' androidTestImplementation 'androidx.test.ext:junit:1.1.1' androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
25. Dependencies testImplementation 'androidx.test:runner:1.2.0' testImplementation 'androidx.test:core-ktx:1.2.0' testImplementation 'androidx.test.ext:junit:1.1.1' testImplementation 'androidx.test.espresso:espresso-core:3.2.0' testImplementation 'org.robolectric:robolectric:4.3' testImplementation_'org.mockito:mockito-core:3.0.0' androidTestImplementation 'androidx.test:runner:1.2.0' androidTestImplementation 'androidx.test:core-ktx:1.2.0' androidTestImplementation 'androidx.test.ext:junit:1.1.1' androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0' androidTestImplementation_'org.mockito:mockito-android:3.0.0' // or dexmaker
26. Module build.gradle android { sourceSets { def sharedTestDir = 'src/sharedTest/java' test.java.srcDirs += sharedTestDir androidTest.java.srcDirs += sharedTestDir } testOptions { unitTests.includeAndroidResources = true } }
27. Example: app under test
29. @RunWith(AndroidJUnit4::class) class GalleryFragmentTest {{ @Test fun progressBarVisible_onLoadingStatus() {{ launchFragmentInContainer {{ //_setup GalleryFragment with bundle args and mock VM here //_mock VM exposes imagesLiveData GalleryFragment() }} imagesLiveData.postValue(Resource(Status.LOADING)) onView(withId(R.id.progress_bar)).check(matches(isDisplayed())) onView(withId(R.id.empty_text)).check(matches(not(isDisplayed()))) onView(withId(R.id.error_text)).check(matches(not(isDisplayed()))) onView(withId(R.id.recycler_view)).check(matches(not(isDisplayed()))) }} }}
30. @RunWith(AndroidJUnit4::class) // androidx.test.ext.junit.runners class GalleryFragmentTest {{ @Test fun progressBarVisible_onLoadingStatus() {{ launchFragmentInContainer {{ //_setup GalleryFragment with bundle args and mock VM here //_mock VM exposes imagesLiveData GalleryFragment() }} imagesLiveData.postValue(Resource(Status.LOADING)) onView(withId(R.id.progress_bar)).check(matches(isDisplayed())) onView(withId(R.id.empty_text)).check(matches(not(isDisplayed()))) onView(withId(R.id.error_text)).check(matches(not(isDisplayed()))) onView(withId(R.id.recycler_view)).check(matches(not(isDisplayed()))) }{ }}
31. Demo: write once, run everywhere?
32. Project Nitrogen Single entry point for Android tests Execute in any environment Virtual device management Covers the entire test lifecycle: configuration, execution, reporting Testing Android Apps at Scale with Nitrogen (Android Dev Summit '18) Frictionless Android testing: write once, run everywhere (Google I/O '18)
33. Robot pattern https://academy.realm.io/posts/kau-jake-wharton-testing-robots/
34. View
35. View VM Model
36. View VM Model
37. Test VM Model View
38. View Test VM Model
39. View Test Robot VM Model
40. View Test Robot VM Model Application architecture
41. View Test Robot VM Model UI Test architecture
42. class PhotoViewerRobot { }
43. class PhotoViewerRobot { fun save() { ... } }
44. class PhotoViewerRobot { fun save() { ... } fun hasPhotoWithId(id: String) { ... } }
45. class PhotoViewerRobot { fun save() { ... } fun hasPhotoWithId(id: String) { ... } fun navToSearch() { ... } }
46. class PhotoViewerRobot { fun save() { ... } fun hasPhotoWithId(id: String) { ... } fun navToSearch() { ... } fun navToSaved() { ... } }
47. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} fun navToSaved() {} } @Test fun saveImage() { }
48. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} fun navToSaved() {} } fun photoViewer( block: PhotoViewerRobot.() -> Unit ): PhotoViewerRobot { return PhotoViewerRobot().apply(block) } @Test fun saveImage() { }
49. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} fun navToSaved() {} } fun photoViewer( block: PhotoViewerRobot.() -> Unit ): PhotoViewerRobot { return PhotoViewerRobot().apply(block) } @Test fun saveImage() { photoViewer { } }
50. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} fun navToSaved() {} } fun photoViewer( block: PhotoViewerRobot.() -> Unit ): PhotoViewerRobot { return PhotoViewerRobot().apply(block) } @Test fun saveImage() { photoViewer { hasPhotoWithId("test_photo") save() } }
51. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} fun navToSaved(block): SavedRobot {} } fun photoViewer( block: PhotoViewerRobot.() -> Unit ): PhotoViewerRobot { return PhotoViewerRobot().apply(block) } @Test fun saveImage() { photoViewer { hasPhotoWithId("test_photo") save() } }
52. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} fun navToSaved(block): SavedRobot {} } fun photoViewer( block: PhotoViewerRobot.() -> Unit ): PhotoViewerRobot { return PhotoViewerRobot().apply(block) } @Test fun saveImage() { photoViewer { hasPhotoWithId("test_photo") save() }.navToSaved { hasPhotoWithId("test_photo") } }
53. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} infix fun navToSaved(block) ... } fun photoViewer( block: PhotoViewerRobot.() -> Unit ): PhotoViewerRobot { return PhotoViewerRobot().apply(block) } @Test fun saveImage() { photoViewer { hasPhotoWithId("test_photo") save() }.navToSaved { hasPhotoWithId("test_photo") } }
54. class PhotoViewerRobot { // actions and assertions fun save() {} fun hasPhotoWithId(id: String) {} fun navToSearch() {} infix fun navToSaved(block) ... } fun photoViewer( block: PhotoViewerRobot.() -> Unit ): PhotoViewerRobot { return PhotoViewerRobot().apply(block) } @Test fun saveImage() { photoViewer { hasPhotoWithId("test_photo") save() } navToSaved { hasPhotoWithId("test_photo") } }
55. class PhotoViewerRobot( block: PhotoViewerRobot.() -> Unit ) { init { block() } } @Test fun saveImage() { PhotoViewerRobot { hasPhotoWithId("test_photo") save() navToSaved() } SavedRobot { hasPhotoWithId("test_photo") } }
56. Demo: Robot UI test
57. github.com/ssaqua/Wallpad
58. Thanks!