Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Nitrogen in your (test) pyramid 2

Nitrogen in your (test) pyramid 2

Unit, integration, UI tests are part of the famous test pyramid, different tools were used in order to full fill it, but still today there are lot of issues about how to execute it in a proper manner and how to sync all environments (local, ci, etc) in order to satisfy all the testing criteria, and having a good reports.

From IO 2018 with the announcement of project Nitrogen, Google’d like to “nitrogenize” our tests creating a single entry point and solve all issues
we had in the past. With October release of Robolectric 4.0 and androidx.test 1.0.0, both testing environments are converging on a set of common test APIs. Robolectric now supports the AndroidJUnit4 test runner, ActivityTestRule, and Espresso for interacting with UI components.

Let’s see togheter how to organise different levels of our ‘test pyramid’, with a focus to the last news/releases in order ot be ready for the Nitrogen release.

Enrico Bruno Del Zotto

September 25, 2019
Tweet

More Decks by Enrico Bruno Del Zotto

Other Decks in Technology

Transcript

  1. Unit tests Low level of the pyramid Coverage small set

    of our code (a unit) Validate our app behaviour one class a time.
  2. Integration tests Medium level of the pyramid Validate either interactions

    between levels of the stack within a module, or interactions between related modules. Presence of Android Context
  3. End to end tests Higher level of the pyramid Validate

    the user journey in the app and are the slowest and most expensive kind of test. You have to run normally in a device.
  4. @RunWith(AndroidJUnit4::class) class MainUiTest { @get:Rule val activityRule = ActivityTestRule(MainActivity::class.java) @Before

    fun setup() { activityRule.launchActivity(Intent().putExtra("fragment", MainFragment::class.java)) } @Test fun testAssertHelloText() { onView(withId(R.id.subtitle)).check(matches(withText("Hello World!"))) } ActivityRule
  5. @RunWith(AndroidJUnit4::class) class MainUiTest { @get:Rule val activityRule = ActivityTestRule(MainActivity::class.java) @Before

    fun setup() { activityRule.launchActivity(Intent().putExtra("fragment", MainFragment::class.java)) } @Test fun testAssertHelloText() { onView(withId(R.id.subtitle)).check(matches(withText("Hello World!"))) } ActivityRule
  6. @RunWith(AndroidJUnit4::class) class MainUiTest { @get:Rule val activityRule = ActivityTestRule(MainActivity::class.java) @Before

    fun setup() { activityRule.launchActivity(Intent().putExtra("fragment", MainFragment::class.java)) } @Test fun testAssertHelloText() { onView(withId(R.id.subtitle)).check(matches(withText("Hello World!"))) } ActivityRule
  7. @RunWith(AndroidJUnit4::class) class MainUiTest { @get:Rule val activityRule = ActivityTestRule(MainActivity::class.java) @Before

    fun setup() { activityRule.launchActivity(Intent().putExtra("fragment", MainFragment::class.java)) } @Test fun testAssertHelloText() { onView(withId(R.id.subtitle)).check(matches(withText("Hello World!"))) } ActivityRule
  8. AndroidX test library Core ApplicationProvider : Provides ability to retrieve

    the current application Context in tests ActivityScenario
  9. AndroidX test library Core ApplicationProvider : Provides ability to retrieve

    the current application Context in tests ActivityScenario FragmentScenario
  10. @Test fun should_clear_locations_when_activity_in_STARTED_state() { //Given val activityScenario = ActivityScenario.launch(LocationActivity::class.java) //When

    activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then assertThat(activityScenario.getState()).isEqualTo(State.STARTED) activityScenario.onActivity { assertThat(it.noLocationSelected()).isEqualTo(true) } activityScenario.close() } Activity Scenario
  11. @Test fun should_clear_locations_when_activity_in_STARTED_state() { //Given val activityScenario = ActivityScenario.launch(LocationActivity::class.java) //When

    activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then assertThat(activityScenario.getState()).isEqualTo(State.STARTED) activityScenario.onActivity { assertThat(it.noLocationSelected()).isEqualTo(true) } activityScenario.close() } Activity Scenario
  12. @Test fun should_clear_locations_when_activity_in_STARTED_state() { //Given val activityScenario = ActivityScenario.launch(LocationActivity::class.java) //When

    activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then assertThat(activityScenario.getState()).isEqualTo(State.STARTED) activityScenario.onActivity { assertThat(it.noLocationSelected()).isEqualTo(true) } activityScenario.close() } Activity Scenario
  13. @Test fun should_clear_locations_when_activity_in_STARTED_state() { //Given val activityScenario = ActivityScenario.launch(LocationActivity::class.java) //When

    activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then assertThat(activityScenario.getState()).isEqualTo(State.STARTED) activityScenario.onActivity { assertThat(it.noLocationSelected()).isEqualTo(true) } activityScenario.close() } Activity Scenario
  14. @Test fun should_clear_locations_when_activity_in_STARTED_state() { //Given val activityScenario = ActivityScenario.launch(LocationActivity::class.java) //When

    activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then assertThat(activityScenario.getState()).isEqualTo(State.STARTED) activityScenario.onActivity { assertThat(it.noLocationSelected()).isEqualTo(true) } activityScenario.close() } Activity Scenario
  15. LaunchFragmentInContainer @Test fun locationFragmentIsDisplayed() { launchFragmentInContainer<LocationFragment>() onView(withId(R.id.location_fragment)) .check(matches(isDisplayed())) } val

    factory = LocationFragmentFactory() val args = Bundle().apply { putString(FRAGMENT_ARG, FRAGMENT_ARG_KEY) } launchFragmentInContainer<LocationFragment>(args, factory)
  16. LaunchFragmentInContainer @Test fun locationFragmentIsDisplayed() { launchFragmentInContainer<LocationFragment>() onView(withId(R.id.location_fragment)) .check(matches(isDisplayed())) } val

    factory = LocationFragmentFactory() val args = Bundle().apply { putString(FRAGMENT_ARG, FRAGMENT_ARG_KEY) } launchFragmentInContainer<LocationFragment>(args, factory)
  17. @Test fun should_start_selectLocationActivity_when_location_is_not_selected() { //Given Intents.init() val activityScenario = ActivityScenario.launch(LocationActivity::class.java)

    //When activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then intended(hasComponent(SelectLocationActivity::class.java.name)) Intents.release() activityScenario.close() } espresso.intent
  18. @Test fun should_start_selectLocationActivity_when_location_is_not_selected() { //Given Intents.init() val activityScenario = ActivityScenario.launch(LocationActivity::class.java)

    //When activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then intended(hasComponent(SelectLocationActivity::class.java.name)) Intents.release() activityScenario.close() } espresso.intent
  19. @Test fun should_start_selectLocationActivity_when_location_is_not_selected() { //Given Intents.init() val activityScenario = ActivityScenario.launch(LocationActivity::class.java)

    //When activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then intended(hasComponent(SelectLocationActivity::class.java.name)) Intents.release() activityScenario.close() } espresso.intent
  20. @Test fun should_start_selectLocationActivity_when_location_is_not_selected() { //Given Intents.init() val activityScenario = ActivityScenario.launch(LocationActivity::class.java)

    //When activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then intended(hasComponent(SelectLocationActivity::class.java.name)) Intents.release() activityScenario.close() } espresso.intent
  21. @Test fun should_start_selectLocationActivity_when_location_is_not_selected() { //Given Intents.init() val activityScenario = ActivityScenario.launch(LocationActivity::class.java)

    //When activityScenario.onActivity { it.clearLocation() } activityScenario.moveToState(Lifecycle.State.STARTED) //Then intended(hasComponent(SelectLocationActivity::class.java.name)) Intents.release() activityScenario.close() } espresso.intent
  22. Readable assertThat(middleName).isAbsent() assertFalse(middleName.isPresent()) assertThat(googleColors).contains(PINK); <[BLUE, RED, YELLOW, BLUE, GREEN, RED]>

    should have contained <PINK> assertTrue(googleColors.contains(PINK)); AssertionFailedError AssertJ Truth
  23. Readable assertThat(middleName).isAbsent() assertFalse(middleName.isPresent()) assertFalse(googleColors.contains(PINK)); assertFalse(googleColors.contains(BLACK)); ... assertThat(googleColors) .containsNoneOf(PINK, BLACK, WHITE,

    ORANGE); assertThat(googleColors).contains(PINK); <[BLUE, RED, YELLOW, BLUE, GREEN, RED]> should have contained <PINK> assertTrue(googleColors.contains(PINK)); AssertionFailedError AssertJ Truth
  24. Subject class CarSubject(metadata: FailureMetadata, subject: Car) : Subject<CarSubject, Car>(metadata, subject)

    { companion object { private val FACTORY = Factory<CarSubject, Car> { metadata, actual -> CarSubject(metadata, actual) } fun car(): Factory<CarSubject, Car> { return FACTORY } } fun isElectrical(): CarSubject { if (!actual().isElectrical) { failWithActual(simpleFact("expected to be electrical")) } return this } fun hasChargingLevel(chargingLevel: Int): CarSubject { if (actual().chargingLevel != chargingLevel) { failWithActual(simpleFact("expected to be $chargingLevel")) } return this }
  25. Subject class CarSubject(metadata: FailureMetadata, subject: Car) : Subject<CarSubject, Car>(metadata, subject)

    { companion object { private val FACTORY = Factory<CarSubject, Car> { metadata, actual -> CarSubject(metadata, actual) } fun car(): Factory<CarSubject, Car> { return FACTORY } } fun isElectrical(): CarSubject { if (!actual().isElectrical) { failWithActual(simpleFact("expected to be electrical")) } return this } fun hasChargingLevel(chargingLevel: Int): CarSubject { if (actual().chargingLevel != chargingLevel) { failWithActual(simpleFact("expected to be $chargingLevel")) } return this }
  26. Subject class CarSubject(metadata: FailureMetadata, subject: Car) : Subject<CarSubject, Car>(metadata, subject)

    { companion object { private val FACTORY = Factory<CarSubject, Car> { metadata, actual -> CarSubject(metadata, actual) } fun car(): Factory<CarSubject, Car> { return FACTORY } } fun isElectrical(): CarSubject { if (!actual().isElectrical) { failWithActual(simpleFact("expected to be electrical")) } return this } fun hasChargingLevel(chargingLevel: Int): CarSubject { if (actual().chargingLevel != chargingLevel) { failWithActual(simpleFact("expected to be $chargingLevel")) } return this }
  27. Subject class CarSubject(metadata: FailureMetadata, subject: Car) : Subject<CarSubject, Car>(metadata, subject)

    { companion object { private val FACTORY = Factory<CarSubject, Car> { metadata, actual -> CarSubject(metadata, actual) } fun car(): Factory<CarSubject, Car> { return FACTORY } } fun isElectrical(): CarSubject { if (!actual().isElectrical) { failWithActual(simpleFact("expected to be electrical")) } return this } fun hasChargingLevel(chargingLevel: Int): CarSubject { if (actual().chargingLevel != chargingLevel) { failWithActual(simpleFact("expected to be $chargingLevel")) } return this }
  28. Subject @Test fun `When buy a new car, then it

    should be electrical`() { val car = Car() assert_().about(CarSubject.car()).that(car).isElectrical() }
  29. Subject @Test fun `When buy a new car, then it

    should be electrical`() { val car = Car() assert_().about(CarSubject.car()).that(car).isElectrical() } @Test fun `When buy a new car, then it should be electrical, charged, and fast`() { val car = Car() assert_().about(CarSubject.car()).that(car).isElectrical().hasChargingLevel(100).isFast() }
  30. Subject @Test fun `When buy a new car, then it

    should be electrical`() { val car = Car() assert_().about(CarSubject.car()).that(car).isElectrical() } @Test fun `When buy a new car, then it should be electrical, charged, and fast`() { val car = Car() assert_().about(CarSubject.car()).that(car).isElectrical().hasChargingLevel(100).isFast() }
  31. Subject @Test fun `When buy a new car, then it

    should be electrical`() { val car = Car() assert_().about(CarSubject.car()).that(car).isElectrical() } @Test fun `When buy a new car, then it should be electrical, charged, and fast`() { val car = Car() assert_().about(CarSubject.car()).that(car).isElectrical().hasChargingLevel(100).isFast() }
  32. AndroidX test library 1- JUnit runners & rules 2- Core

    ApplicationProvider, Activity/Fragment scenario 3- Espresso yes you know it 4- Truth Fluent assertions, it’s not AssertJ ! Recap
  33. AndroidX test library We can now use the new Scenario

    API to launch the Activity or Fragment Use Espresso to make some view interactions Use Truth to make view assertions Run instrumented tests on JVM instead on the device using the new AndroidX test runner
  34. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable {

    [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
  35. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable {

    [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
  36. public final class AndroidJUnit4 extends Runner implements Filterable, Sortable {

    [...] private static String getRunnerClassName() { String runnerClassName = System.getProperty("android.junit.runner", null); if (runnerClassName == null) { // TODO: remove this logic when nitrogen is hooked up to always pass this property if (System.getProperty("java.runtime.name").toLowerCase().contains("android")) { return "androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner"; } else { return "org.robolectric.RobolectricTestRunner"; } } return runnerClassName; } [...] }
  37. Robolectric 4.x Provides a JVM compliant version of the android.jar

    file, it handles inflation of views, resource loading, and lots of other stuff that’s implemented in native C code on Android devices. This enables you to run your Android tests in your continuous integration environment without any additional setup
  38. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore

    testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
  39. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore

    testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
  40. testImplementation Libs.junit testImplementation Libs.robolectric testImplementation Libs.Mockito.core testImplementation Libs.AndroidX.Test.archCore testImplementation Libs.AndroidX.Test.espressoCore

    testImplementation Libs.AndroidX.Test.espressoContrib testImplementation Libs.AndroidX.Test.espressoIdlingResources testImplementation Libs.AndroidX.Test.junitAssertions testImplementation Libs.AndroidX.Test.core testImplementation Libs.AndroidX.Test.runner testImplementation Libs.AndroidX.Test.rules testImplementation Libs.AndroidX.Test.espressoCore testImplementation Libs.AndroidX.Test.thruth
  41. testOptions { unitTests { includeAndroidResources = true } unitTests.all {

    include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
  42. testOptions { unitTests { includeAndroidResources = true } unitTests.all {

    include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
  43. testOptions { unitTests { includeAndroidResources = true } unitTests.all {

    include '**/*Tests.class' include '**/*Test.class' useJUnit { } ignoreFailures false // use afterTest to listen to the test execution results afterTest { descriptor, result -> println "Executing test for ${descriptor.name} with result: $ {result.resultType}" } systemProperty 'robolectric.logging.enabled', 'true' jvmArgs '-noverify' maxHeapSize = "2g" } }
  44. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  45. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  46. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  47. class RepositoryListUiTest : BaseInstrumentationTestCase() { @Test fun willDisplayFirstPaginatedListCorrectly() { given.gitHubServer.returnsReposListPageOneSuccessfully()

    `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withSuccessfullyLoadedFirstPageItem() } @Test fun willDisplayErrorCorrectly() { given.gitHubServer.returnErrorOnFetchingRepositories() `when`.user.launchesTheApp() then.user.sees.repositoriesListScreenAssertion.withErrorState() } }
  48. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  49. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  50. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  51. sourceSets { String sharedTestDir = 'src/sharedTest/java' test { debug.assets.srcDirs +=

    files("$projectDir/schemas".toString()) debug.assets.srcDirs += files("$projectDir/assets".toString()) debug.assets.srcDirs += files("$projectDir/raw".toString()) java.srcDirs += 'src/androidTest/java' kotlin.srcDirs += 'src/androidTest/java' java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { assets.srcDirs += files("$projectDir/schemas".toString()) test.resources.srcDirs += 'src/test/resources' resources.srcDirs += ['src/test/resources'] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  52. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  53. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  54. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  55. Using the shared test folder : sourceSets { String sharedTestDir

    = 'src/sharedTest/java' test { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir test.resources.srcDirs += 'src/test/resources' } androidTest { [...] java.srcDirs += sharedTestDir kotlin.srcDirs += sharedTestDir } }
  56. If you have lot of build variants, in which manner

    specify which UI test should run into JVM? Mixing everything into one shared folder isn’t a good solution!
  57. If you have lot of build variants, in which manner

    specify which UI test should run into JVM? Mixing everything into one shared folder isn’t a good solution!
  58. Adding dynamically using a Gradle task could be a good

    option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  59. Adding dynamically using a Gradle task could be a good

    option : ./gradlew sourceSets | grep src/androidTest'$variantName'
  60. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option :
  61. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option :
  62. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option :
  63. sourceSets.all { sourceSet -> Matcher testTaskMatcher = patternOfUnitTestTask.matcher(sourceSet.name) if (testTaskMatcher.find())

    { def variantName = testTaskMatcher.group(1) println "Added androidTest sourceSets to $sourceSet.name (variant name = $ {variantName}" sourceSet.java.srcDirs += "src/androidTest${variantName}/java" sourceSet.kotlin.srcDirs += "src/androidTest${variantName}/java" } } Adding dynamically using a Gradle task could be a good option :
  64. You can use Junit 4 ‘categories’ to exclude tests which

    for some reason has some incompatibilities with JVM:
  65. You can use Junit 4 ‘categories’ to exclude tests which

    for some reason has some incompatibilities with JVM: testOptions { [...] unitTests.all { [...] useJUnit { //With this category we can annotate tests will not run on JVM //due conflicts or other Robolectric Issue, but can run without any issue on Espresso or //normal device. excludeCategories 'com.playground.utils.DoNotRunOnJvm' } [...] } }
  66. You can use Junit 4 ‘categories’ to exclude tests which

    for some reason has some incompatibilities with JVM: testOptions { [...] unitTests.all { [...] useJUnit { //With this category we can annotate tests will not run on JVM //due conflicts or other Robolectric Issue, but can run without any issue on Espresso or //normal device. excludeCategories 'com.playground.utils.DoNotRunOnJvm' } [...] } }
  67. /** * Marks a test class or test method as

    belonging to one or more categories of tests. * The value is an array of arbitrary classes. * * This annotation is only interpreted by the Categories runner (at present). * * For example: * <pre> * public interface FastTests {} * public interface SlowTests {} * * public static class A { * &#064;Test * public void a() { * fail(); * } * * &#064;Category(SlowTests.class) * &#064;Test * public void b() { * } * } * * &#064;Category({SlowTests.class, FastTests.class}) * public static class B { * &#064;Test * public void c() { * * } * } * </pre> * * For more usage, see code example on {@link Categories}. */ @Retention(RetentionPolicy.RUNTIME) @Inherited @ValidateWith(CategoryValidator.class) public @interface Category { Class<?>[] value(); }
  68. @Category(DoNotRunOnJvm::class) class AirportPickerUiTest : BaseHolidaySearchUiTest() { [...] @Before override fun

    setUp() { super.setUp() willOpenHolidaySearchScreenCorrectly() } @Test fun willOpenAirportPickerScreenCorrectly() { `when`.user.apply { onBottomNavigation.clicksSearch() onHolidaySearchActions.clickAirportPicker() } then.user.sees.apply { airportPickerAssertions.withFields() airportPickerAssertions.withSuccessfullyLoadedFlightsFrom() } } [...] }
  69. @Category(DoNotRunOnJvm::class) class AirportPickerUiTest : BaseHolidaySearchUiTest() { [...] @Before override fun

    setUp() { super.setUp() willOpenHolidaySearchScreenCorrectly() } @Test fun willOpenAirportPickerScreenCorrectly() { `when`.user.apply { onBottomNavigation.clicksSearch() onHolidaySearchActions.clickAirportPicker() } then.user.sees.apply { airportPickerAssertions.withFields() airportPickerAssertions.withSuccessfullyLoadedFlightsFrom() } } [...] }
  70. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no.
  71. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... }
  72. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  73. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  74. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  75. Create a custom filter to check if our test has

    custom @TestFilter(flavors = [FLAVOR_1]) annotation. Based on that and the flavour name it check if we need to run it or no. @Test @TestFilter(flavors = [FLAVOR_1]) fun test1() { //... } @Test fun test2() { //... } @Test @TestFilter(flavors = [FLAVOR_2]) fun test3() { //... } https://medium.com/stepstone-tech/taming-ui-tests-when-using-multiple-gradle-flavors-6190b3692491
  76. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  77. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  78. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  79. class FlavorFilter(bundle: Bundle) : ParentFilter() { companion object { private

    const val CLASS_BUNDLE_KEY = "class" } private val shouldFilterTests: Boolean init { Timber.i("FlavorFilter bundle: %s", bundle) val listTestsForOrchestrator = java.lang.Boolean.parseBoolean(bundle.getString("listTestsForOrchestrator", "false")) val runsSingleClass = bundle.containsKey(CLASS_BUNDLE_KEY) shouldFilterTests = listTestsForOrchestrator && !runsSingleClass } override fun evaluateTest(description: Description): Boolean { if (!shouldFilterTests) { return true } val classTestFilter = description.testClass.getAnnotation(TestFilter::class.java) val testFilter = description.getAnnotation(TestFilter::class.java) if (testFilter != null) { return evaluateTestWithFilter(testFilter) } else if (classTestFilter != null) { return evaluateTestWithFilter(classTestFilter) } return true } private fun evaluateTestWithFilter(testFilter: TestFilter): Boolean { val filters = testFilter.filters val notFilters = testFilter.notFilters return (filters.contains(BuildConfig.FLAVOR) && !notFilters.contains(BuildConfig.FLAVOR)) } override fun describe(): String { return "Flavor filter" } }
  80. Using @TestFilter(flavors = [FLAVOR_1]) annotation 1- It is easy to

    figure out which test gets run on which flavors as that information is right above the test method. 2- Adding new tests does not require updating any external configuration files. 3- All tests are in androidTest folder so it’s easier to navigate through them.
  81. How to debug a JVM test which runs by command

    line ? ./gradlew test'$buildVariantName'Test - Dorg.gradle.daemon=false --tests '$package.className' -- debug-jvm 'Listening for transport dt_socket at address: 5005' export GRADLE_OPTS="- agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=500 5"
  82. How to debug a JVM test which runs by command

    line ? ./gradlew test'$buildVariantName'Test - Dorg.gradle.daemon=false --tests '$package.className' -- debug-jvm 'Listening for transport dt_socket at address: 5005' export GRADLE_OPTS="- agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=500 5"
  83. Use rules to clean your env! /** * Clear all

    application data here. * Done to ensure all UI tests start from the same clean state. */ class ClearAppDataRule : ExternalResource() { private val db: MvRxDb by ApplicationProvider.getApplicationContext<MvRxTestApplication>().inject() override fun before() { if (SystemUtils.isART()) { db.clearAllTables() } super.before() } }
  84. @RunWith(RobolectricTestRunner::class) @Config(sdk = [21], application = TestTdaApplication::class) abstract class RobolectricTestBase

    { private var app: Application? = null companion object { @JvmStatic @BeforeClass fun beforeClass() { // This forces the usage of com.almworks.sqlite4java:1.0.392, which internally uses SQLite 3.8.7. // This version of SQLite is new enough to support all SQLite features we use SQLiteLibraryLoader.load() } } @Before @CallSuper open fun setUp() { MockitoAnnotations.initMocks(this) app = ApplicationProvider.getApplicationContext() } } …are still a lot…
  85. @RunWith(RobolectricTestRunner::class) @Config(sdk = [21], application = TestTdaApplication::class) abstract class RobolectricTestBase

    { private var app: Application? = null companion object { @JvmStatic @BeforeClass fun beforeClass() { // This forces the usage of com.almworks.sqlite4java:1.0.392, which internally uses SQLite 3.8.7. // This version of SQLite is new enough to support all SQLite features we use SQLiteLibraryLoader.load() } } @Before @CallSuper open fun setUp() { MockitoAnnotations.initMocks(this) app = ApplicationProvider.getApplicationContext() } } …are still a lot…
  86. Use shadow classes ! Why “Shadow?” Shadow objects are not

    quite Proxies, not quite Fakes, not quite Mocks or Stubs. Shadows are sometimes hidden, sometimes seen, and can lead you to the real object. At least we didn’t call them “sheep”, which we were considering.
  87. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import

    android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
  88. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import

    android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
  89. …to avoid lot of JVM incompatibilities ! package com.tui.tda.shadows import

    android.util.Log import com.tui.tda.components.messages.fragments.MessagesOnlineDetai lsFragment import org.robolectric.annotation.Implementation import org.robolectric.annotation.Implements @Implements(MessagesOnlineDetailsFragment::class) class MessagesOnlineDetailsFragmentShadow { /** * Run SoomthScrollPosition isn't working in Roboelectric, so we need * to shadow the class in this case. */ @Implementation fun smoothScrollToPosition() { Log.e(MessagesOnlineDetailsFragmentShadow::class.java.name, "smoothScrollToPosition() shadow method due https:// github.com/robolectric/robolectric/issues/3552") } }
  90. References : Testing Rebooted (with AndroidX Test) (Android Dev Summit

    '18) Frictionless Android testing: write once, run everywhere (Google I/O '18) Do Android devs dream of Robolectric Espresso? by Sarah Sharp (The Telegraph) Testing Android Apps at Scale with Nitrogen (Android Dev Summit '18) Android Test documentation How to remote debug a Robolectric test by command line using Gradle Code examples : https://github.com/lupsyn/mvRx_playground