Nitrogen in your (test) pyramid.

Nitrogen in your (test) pyramid.

Unit, integration, UI tests are part of the famous test pyramid, different tools were used in order to full fill it, but still today there are lot of issues about how to execute it in a proper manner and how to sync all environments (local, ci, etc) in order to satisfy all the testing criteria, and having a good reports. From IO 2018 with the announcement of project Nitrogen, Google'd like to "nitrogenize" our tests creating a single entry point and solve all issues we had in the past. With October release of Robolectric 4.0 and androidx.test 1.0.0, both testing environments are converging on a set of common test APIs. Robolectric now supports the AndroidJUnit4 test runner, ActivityTestRule, and Espresso for interacting with UI components. Let's see togheter how to organise different levels of our 'test pyramid', with a focus to the last news/releases in order ot be ready for the Nitrogen release.

82502a841e2a637061c294399d6c5ef3?s=128

Enrico Bruno Del Zotto

September 09, 2019
Tweet

Transcript

  1. Nitrogen in your (test) pyramid @lupsyn

  2. Who I am ?

  3. Who I am ? Enrico Bruno Del Zotto

  4. Android stuff in London Who I am ? Enrico Bruno

    Del Zotto
  5. Android stuff in London Who I am ? Enrico Bruno

    Del Zotto
  6. Android stuff in London Who I am ? Enrico Bruno

    Del Zotto
  7. The test pyramid

  8. The test pyramid

  9. Unit tests

  10. Unit tests Low level of the pyramid Coverage small set

    of our code (a unit) Validate our app behaviour one class a time.
  11. Integration tests

  12. Integration tests Medium level of the pyramid Validate either interactions

    between levels of the stack within a module, or interactions between related modules. Presence of Android Context
  13. End to end tests

  14. End to end tests Higher level of the pyramid Validate

    the user journey in the app and are the slowest and most expensive kind of test. You have to run normally in a device.
  15. None
  16. Failing UI test

  17. Failing UI test Refactor

  18. Failing UI test Failing unit test Refactor

  19. Failing UI test Failing unit test Passing unit test Refactor

  20. Failing UI test Failing unit test Passing unit test Passing

    UI test Refactor
  21. Failing UI test Failing unit test Passing unit test Passing

    UI test Refactor Refactor
  22. UI tests should be agnostic.

  23. Coverage set

  24. Test for all these frameworks use the same generic format

    :
  25. Test for all these frameworks use the same generic format

    : “Given, when, then”
  26. None
  27. Why learn different API to do the same thing ?

  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. AndroidX test library JUnit : runners & rules Core :

    ApplicationProvider, Activity/Fragment scenario Espresso : Yes you know it Truth : Fluent assertions, it’s not AssertJ ! AssertEquals(View.visibilty,View.Visible) -> assertThat(view).isVisible()
  39. AndroidX test library androidx.test.runner.screenshot androidx.test.platform.ui androidx.test.filters androidx.test.uiautomator

  40. AndroidX test library Provides common test APIs across test environments

    including instrumentation tests.
  41. AndroidX test library

  42. AndroidX test library We can now use the new Scenario

    API to launch the Activity or Fragment Use Espresso to make some view interactions Use Truth to make view assertions Run instrumented tests on JVM instead on the device using the new AndroidX test runner
  43. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable {

    [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
  44. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable {

    [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
  45. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable {

    [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
  46. Robolectric 4.x

  47. Robolectric 4.x Provides a JVM compliant version of the android.jar

    file, it handles inflation of views, resource loading, and lots of other stuff that’s implemented in native C code on Android devices. This enables you to run your Android tests in your continuous integration environment without any additional setup
  48. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore

    testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
  49. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore

    testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
  50. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore

    testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
  51. testOptions { unitTests { includeAndroidResources = true } unitTests.all {

    include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
  52. testOptions { unitTests { includeAndroidResources = true } unitTests.all {

    include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
  53. testOptions { unitTests { includeAndroidResources = true } unitTests.all {

    include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
  54. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  55. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  56. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  57. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  58. mainScreen { setReturnResposListPageOneSuccessfully() launchTheApp() repositoriesListScreenIsCorrectlyLoaded }

  59. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  60. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  61. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  62. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  63. Code examples

  64. Some tips…

  65. Some tips… Move instrumented test into unitTest folder…

  66. Using the shared test folder :

  67. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  68. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  69. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  70. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  71. If you have lot of build variants, in which manner

    specify which UI test should run into JVM? Mixing everything into one shared folder isn’t a good solution!
  72. If you have lot of build variants, in which manner

    specify which UI test should run into JVM? Mixing everything into one shared folder isn’t a good solution!
  73. Adding dynamically using a Gradle task could be a good

    option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  74. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  75. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  76. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  77. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  78. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  79. You can use Junit 4 ‘categories’ to exclude tests which

    for some reason has some incompatibilities with JVM:
  80. You can use Junit 4 ‘categories’ to exclude tests which

    for some reason has some incompatibilities with JVM: testOptions { [...] unitTests.all { [...] useJUnit { //With this category we can annotate tests will not run on JVM //due conflicts or other Robolectric Issue, but can run without any issue on Espresso or //normal device. excludeCategories 'com.playground.utils.DoNotRunOnJvm' } [...] } }
  81. You can use Junit 4 ‘categories’ to exclude tests which

    for some reason has some incompatibilities with JVM: testOptions { [...] unitTests.all { [...] useJUnit { //With this category we can annotate tests will not run on JVM //due conflicts or other Robolectric Issue, but can run without any issue on Espresso or //normal device. excludeCategories 'com.playground.utils.DoNotRunOnJvm' } [...] } }
  82. @Category(DoNotRunOnJvm::class) class AirportPickerUiTest : BaseHolidaySearchUiTest() { [...] @Before override fun

    setUp() { super.setUp() willOpenHolidaySearchScreenCorrectly() } @Test fun willOpenAirportPickerScreenCorrectly() { `when`.user.apply { onBottomNavigation.clicksSearch() onHolidaySearchActions.clickAirportPicker() } then.user.sees.apply { airportPickerAssertions.withFields() airportPickerAssertions.withSuccessfullyLoadedFlightsFrom() } } [...] }
  83. @Category(DoNotRunOnJvm::class) class AirportPickerUiTest : BaseHolidaySearchUiTest() { [...] @Before override fun

    setUp() { super.setUp() willOpenHolidaySearchScreenCorrectly() } @Test fun willOpenAirportPickerScreenCorrectly() { `when`.user.apply { onBottomNavigation.clicksSearch() onHolidaySearchActions.clickAirportPicker() } then.user.sees.apply { airportPickerAssertions.withFields() airportPickerAssertions.withSuccessfullyLoadedFlightsFrom() } } [...] }
  84. https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491

  85. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no.
  86. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... }
  87. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  88. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  89. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  90. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  91. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  92. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  93. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  94. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  95. Using @TestFilter(flavors = [FLAVOR_1]) annotation 1- It is easy to

    figure out which test gets run on which flavors as that information is right above the test method. 2- Adding new tests does not require updating any external configuration files. 3- All tests are in androidTest folder so it’s easier to navigate through them.
  96. How to debug a JVM test which runs by command

    line ?
  97. How to debug a JVM test which runs by command

    line ? ./gradlew test'$buildVariantName'Test - Dorg.gradle.daemon=false --tests '$package.className' -- debug-jvm 'Listening for transport dt_socket at address: 5005' export GRADLE_OPTS="- agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=500 5"
  98. How to debug a JVM test which runs by command

    line ? ./gradlew test'$buildVariantName'Test - Dorg.gradle.daemon=false --tests '$package.className' -- debug-jvm 'Listening for transport dt_socket at address: 5005' export GRADLE_OPTS="- agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=500 5"
  99. Use Robolectric.properties

  100. Use Robolectric.properties application = com.playground.base.MvRxTestApplication qualifiers = en-rGB-w360dp-h640dp-xhdpi sdk=28

  101. Use Robolectric.properties application = com.playground.base.MvRxTestApplication qualifiers = en-rGB-w360dp-h640dp-xhdpi sdk=28

  102. And shadow classes !

  103. And shadow classes ! Why “Shadow?” Shadow objects are not

    quite Proxies, not quite Fakes, not quite Mocks or Stubs. Shadows are sometimes hidden, sometimes seen, and can lead you to the real object. At least we didn’t call them “sheep”, which we were considering.
  104. …to avoid lot of JVM incompatibilities !

  105. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import

    android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
  106. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import

    android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
  107. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import

    android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
  108. What to expect :

  109. What to expect :

  110. References : Testing Rebooted (with AndroidX Test) (Android Dev Summit

    '18) Frictionless Android testing: write once, run everywhere (Google I/O '18) Do Android devs dream of Robolectric Espresso? by Sarah Sharp (The Telegraph) Testing Android Apps at Scale with Nitrogen (Android Dev Summit '18) Android Test documentation How to remote debug a Robolectric test by command line using Gradle Code examples : https://github.com/lupsyn/mvRx_playground
  111. Let’s keep in touch ! @lupsyn @lupsyn Enrico Bruno Del

    Zotto delzotto.enricoATgmail.com
  112. Thanks!