Nitrogen in your (test) pyramid.

Unit, integration, UI tests are part of the famous test pyramid, different tools were used in order to full fill it, but still today there are lot of issues about how to execute it in a proper manner and how to sync all environments (local, ci, etc) in order to satisfy all the testing criteria, and having a good reports. From IO 2018 with the announcement of project Nitrogen, Google'd like to "nitrogenize" our tests creating a single entry point and solve all issues we had in the past. With October release of Robolectric 4.0 and androidx.test 1.0.0, both testing environments are converging on a set of common test APIs. Robolectric now supports the AndroidJUnit4 test runner, ActivityTestRule, and Espresso for interacting with UI components. Let's see togheter how to organise different levels of our 'test pyramid', with a focus to the last news/releases in order ot be ready for the Nitrogen release.

Transcript

1. Nitrogen in your (test) pyramid @lupsyn
2. Who I am ?
3. Who I am ? Enrico Bruno Del Zotto
4. Android stuff in London Who I am ? Enrico Bruno Del Zotto
5. Android stuff in London Who I am ? Enrico Bruno Del Zotto
6. Android stuff in London Who I am ? Enrico Bruno Del Zotto
7. The test pyramid
8. The test pyramid
9. Unit tests
10. Unit tests Low level of the pyramid Coverage small set of our code (a unit) Validate our app behaviour one class a time.
11. Integration tests
12. Integration tests Medium level of the pyramid Validate either interactions between levels of the stack within a module, or interactions between related modules. Presence of Android Context
13. End to end tests
14. End to end tests Higher level of the pyramid Validate the user journey in the app and are the slowest and most expensive kind of test. You have to run normally in a device.
16. Failing UI test
17. Failing UI test Refactor
18. Failing UI test Failing unit test Refactor
19. Failing UI test Failing unit test Passing unit test Refactor
20. Failing UI test Failing unit test Passing unit test Passing UI test Refactor
21. Failing UI test Failing unit test Passing unit test Passing UI test Refactor Refactor
22. UI tests should be agnostic.
23. Coverage set
24. Test for all these frameworks use the same generic format :
25. Test for all these frameworks use the same generic format : “Given, when, then”
27. Why learn different API to do the same thing ?
38. AndroidX test library JUnit : runners & rules Core : ApplicationProvider, Activity/Fragment scenario Espresso : Yes you know it Truth : Fluent assertions, it’s not AssertJ ! AssertEquals(View.visibilty,View.Visible) -> assertThat(view).isVisible()
39. AndroidX test library androidx.test.runner.screenshot androidx.test.platform.ui androidx.test.filters androidx.test.uiautomator
40. AndroidX test library Provides common test APIs across test environments including instrumentation tests.
41. AndroidX test library
42. AndroidX test library We can now use the new Scenario API to launch the Activity or Fragment Use Espresso to make some view interactions Use Truth to make view assertions Run instrumented tests on JVM instead on the device using the new AndroidX test runner
43. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable { [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
44. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable { [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
45. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable { [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
46. Robolectric 4.x
47. Robolectric 4.x Provides a JVM compliant version of the android.jar file, it handles inflation of views, resource loading, and lots of other stuff that’s implemented in native C code on Android devices. This enables you to run your Android tests in your continuous integration environment without any additional setup
48. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
49. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
50. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
51. testOptions { unitTests { includeAndroidResources = true } unitTests.all { include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
52. testOptions { unitTests { includeAndroidResources = true } unitTests.all { include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
53. testOptions { unitTests { includeAndroidResources = true } unitTests.all { include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
54. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
55. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
56. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
57. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
58. mainScreen { setReturnResposListPageOneSuccessfully() launchTheApp() repositoriesListScreenIsCorrectlyLoaded }
59. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs += files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
60. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs += files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
61. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs += files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
62. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs += files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
63. Code examples
64. Some tips…
65. Some tips… Move instrumented test into unitTest folder…
66. Using the shared test folder :
67. Using the shared test folder : sourceSets { String sharedTestDir = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
68. Using the shared test folder : sourceSets { String sharedTestDir = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
69. Using the shared test folder : sourceSets { String sharedTestDir = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
70. Using the shared test folder : sourceSets { String sharedTestDir = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
71. If you have lot of build variants, in which manner specify which UI test should run into JVM? Mixing everything into one shared folder isn’t a good solution!
72. If you have lot of build variants, in which manner specify which UI test should run into JVM? Mixing everything into one shared folder isn’t a good solution!
73. Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
74. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find()) { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
75. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find()) { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
76. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find()) { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
77. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find()) { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
78. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find()) { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
79. You can use Junit 4 ‘categories’ to exclude tests which for some reason has some incompatibilities with JVM:
80. You can use Junit 4 ‘categories’ to exclude tests which for some reason has some incompatibilities with JVM: testOptions { [...] unitTests.all { [...] useJUnit { //With this category we can annotate tests will not run on JVM //due conflicts or other Robolectric Issue, but can run without any issue on Espresso or //normal device. excludeCategories 'com.playground.utils.DoNotRunOnJvm' } [...] } }
81. You can use Junit 4 ‘categories’ to exclude tests which for some reason has some incompatibilities with JVM: testOptions { [...] unitTests.all { [...] useJUnit { //With this category we can annotate tests will not run on JVM //due conflicts or other Robolectric Issue, but can run without any issue on Espresso or //normal device. excludeCategories 'com.playground.utils.DoNotRunOnJvm' } [...] } }
82. @Category(DoNotRunOnJvm::class) class AirportPickerUiTest : BaseHolidaySearchUiTest() { [...] @Before override fun setUp() { super.setUp() willOpenHolidaySearchScreenCorrectly() } @Test fun willOpenAirportPickerScreenCorrectly() { `when`.user.apply { onBottomNavigation.clicksSearch() onHolidaySearchActions.clickAirportPicker() } then.user.sees.apply { airportPickerAssertions.withFields() airportPickerAssertions.withSuccessfullyLoadedFlightsFrom() } } [...] }
83. @Category(DoNotRunOnJvm::class) class AirportPickerUiTest : BaseHolidaySearchUiTest() { [...] @Before override fun setUp() { super.setUp() willOpenHolidaySearchScreenCorrectly() } @Test fun willOpenAirportPickerScreenCorrectly() { `when`.user.apply { onBottomNavigation.clicksSearch() onHolidaySearchActions.clickAirportPicker() } then.user.sees.apply { airportPickerAssertions.withFields() airportPickerAssertions.withSuccessfullyLoadedFlightsFrom() } } [...] }
84. https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
85. Create a custom filter to check if our test has custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no.
86. Create a custom filter to check if our test has custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... }
87. Create a custom filter to check if our test has custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
88. Create a custom filter to check if our test has custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
89. Create a custom filter to check if our test has custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
90. Create a custom filter to check if our test has custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
91. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
92. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
93. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
94. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
95. Using @TestFilter(flavors = [FLAVOR_1]) annotation 1- It is easy to figure out which test gets run on which flavors as that information is right above the test method. 2- Adding new tests does not require updating any external configuration files. 3- All tests are in androidTest folder so it’s easier to navigate through them.
96. How to debug a JVM test which runs by command line ?
97. How to debug a JVM test which runs by command line ? ./gradlew test'$buildVariantName'Test - Dorg.gradle.daemon=false --tests '$package.className' -- debug-jvm 'Listening for transport dt_socket at address: 5005' export GRADLE_OPTS="- agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=500 5"
98. How to debug a JVM test which runs by command line ? ./gradlew test'$buildVariantName'Test - Dorg.gradle.daemon=false --tests '$package.className' -- debug-jvm 'Listening for transport dt_socket at address: 5005' export GRADLE_OPTS="- agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=500 5"
99. Use Robolectric.properties
100. Use Robolectric.properties application = com.playground.base.MvRxTestApplication qualifiers = en-rGB-w360dp-h640dp-xhdpi sdk=28
101. Use Robolectric.properties application = com.playground.base.MvRxTestApplication qualifiers = en-rGB-w360dp-h640dp-xhdpi sdk=28
102. And shadow classes !
103. And shadow classes ! Why “Shadow?” Shadow objects are not quite Proxies, not quite Fakes, not quite Mocks or Stubs. Shadows are sometimes hidden, sometimes seen, and can lead you to the real object. At least we didn’t call them “sheep”, which we were considering.
104. …to avoid lot of JVM incompatibilities !
105. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
106. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
107. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
108. What to expect :
109. What to expect :
110. References : Testing Rebooted (with AndroidX Test) (Android Dev Summit '18) Frictionless Android testing: write once, run everywhere (Google I/O '18) Do Android devs dream of Robolectric Espresso? by Sarah Sharp (The Telegraph) Testing Android Apps at Scale with Nitrogen (Android Dev Summit '18) Android Test documentation How to remote debug a Robolectric test by command line using Gradle Code examples : https://github.com/lupsyn/mvRx_playground
111. Let’s keep in touch ! @lupsyn @lupsyn Enrico Bruno Del Zotto delzotto.enricoATgmail.com
112. Thanks!