Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Guide to the jungle of testing frameworks

Guide to the jungle of testing frameworks

There are many tools, libraries and frameworks available for Android developers to test their applications. The jungle is huge and it's not easy to find the right ones. Some frameworks are good for unit testing, some are good for instrumentation testing, and some can be used for both. Some have great capabilities but annoying weaknesses. Some are good for testing UI, other allow you to make good mocks. We will look at many frameworks, the popular ones like Mockito, Robolectric, Espresso, and some other.
Presented at GDG DevFest Minsk 2016.

Tomáš Kypta

November 26, 2016
Tweet

More Decks by Tomáš Kypta

Other Decks in Programming

Transcript

  1. Android test code • project sources • ${module}/src/main/java • instrumentation

    tests • ${module}/src/androidTest/java • unit tests • ${module}/src/test/java • full Gradle and Android Studio support
  2. • the essential piece of both instrumentation and unit tests

    • alone can be used only for pure Java code • doesn’t provide any mocks or Android APIs
  3. Instrumentation Tests • running on physical device or emulator •

    gradle connectedAndroidTest • ${module}/build/reports/androidTests/connected/ index.html
  4. Legacy Instrumentation Tests • JUnit3 • Tests extend from TestCase

    • AndroidTestCase • ActivityInstrumentationTestCase2 • ServiceTestCase • … deprecated since API level 24
  5. Testing Support Library • JUnit4 compatible • AndroidJUnitRunner android {

    defaultConfig { testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" } } dependencies { androidTestCompile 'com.android.support.test:runner:0.5' }
  6. Testing Support Library • JUnit test rules • AndroidTestRule •

    ServiceTestRule • DisableOnAndroidDebug • LogLogcatRule • … androidTestCompile 'com.android.support.test:rules:0.5'
  7. • framework for functional UI tests • part of Android

    Testing Support Library androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.2'
  8. @Test public void sayHello() { onView(withId(R.id.edit_text)) .perform(typeText(STRING_TO_BE_TYPED), closeSoftKeyboard()); onView(withText("Say hello!"))

    .perform(click()); String expectedText = "Hello, " + STRING_TO_BE_TYPED + "!"; onView(withId(R.id.textView)) .check(matches(withText(expectedText))); }
  9. Problems • testing on device is not isolated • device

    state affects the result • e.g. screen on/off might affect test result onView(withId(R.id.my_view)) .check(matches(isDisplayed()));
  10. UI Automator • APIs for building UI tests • interaction

    with both your apps and system apps • Android 4.3+ androidTestCompile 'com.android.support.test.uiautomator:uiautomator-v18:2.1.2'
  11. UI Automator • several parts • API for information retrieval

    and performing operations • API for cross-app testing • uiautomatorviewer tool
  12. UI Automator @RunWith(AndroidJUnit4.class) public class AnActivityTest { private static final

    String MY_PACKAGE = "com.example.helloworld"; private static final int LAUNCH_TIMEOUT = 5000; UiDevice mDevice; @Before public void setUP() { mDevice = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation()); mDevice.pressHome(); Context context = InstrumentationRegistry.getContext(); Intent intent = context.getPackageManager().getLaunchIntentForPackage(MY_PACKAGE); intent.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TASK); context.startActivity(intent); // Wait for the app to appear mDevice.wait(Until.hasObject(By.pkg(MY_PACKAGE).depth(0)), LAUNCH_TIMEOUT); } // do the testing }
  13. UI Automator @RunWith(AndroidJUnit4.class) public class AnActivityTest { // a setup

    @Test public void testText() throws Exception { UiObject textLabel = mDevice.findObject( new UiSelector().packageName(MY_PACKAGE).text("Hello World!")); UiObject button = mDevice.findObject( new UiSelector().packageName(MY_PACKAGE).text("Change text")); button.click(); UiObject2 textResult = mDevice.findObject(By.res(MY_PACKAGE, "txt_label")); Assert.assertEquals("Hello Minsk!", textResult.getText()); } }
  14. • use with UI Automator • direct manipulation with sensor

    values on Genymotion devices • allows to omit mocking sensor values androidTestCompile 'com.genymotion.api:genymotion-api:1.0.2'
  15. if (!GenymotionManager.isGenymotionDevice()) { return; //don't execute this test } GenymotionManager

    genymotion = Genymotion.getGenymotionManager( getInstrumentation().getContext());
  16. if (!GenymotionManager.isGenymotionDevice()) { return; //don't execute this test } GenymotionManager

    genymotion = Genymotion.getGenymotionManager( getInstrumentation().getContext()); genymotion .getRadio().call("555123456");
  17. if (!GenymotionManager.isGenymotionDevice()) { return; //don't execute this test } GenymotionManager

    genymotion = Genymotion.getGenymotionManager( getInstrumentation().getContext()); genymotion .getRadio().call("555123456"); genymotion .getNetwork().setProfile(Network.Profile.EDGE);
  18. • for iOS, Android, Windows apps • based on WebDriver

    protocol • alternative to UI Automator • aims to automate apps from any language and any test framework • …with access to back-end APIs and DBs • Android 4.1+
  19. Unit Tests • run on JVM • mockable android.jar •

    gradle test • ${module}/build/reports/tests/${variant}/index.html
  20. ...

  21. • Helps rarely • returns 0, false, null, … Method

    ... not mocked. android {
 testOptions {
 unitTests.returnDefaultValues = true }
 }
  22. • mocking framework • easy to use • compatible with

    Android unit testing testCompile 'org.mockito:mockito-core:2.2.11'
  23. • can be used also in instrumentation tests • needs

    dexmaker androidTestCompile 'org.mockito:mockito-core:2.2.11' androidTestCompile 'com.google.dexmaker:dexmaker:1.2' androidTestCompile 'com.google.dexmaker:dexmaker-mockito:1.2'
  24. @RunWith(JUnit4.class) public class ContextManagerTest { @Mock Context mAppContext; @Before public

    void setUp() { MockitoAnnotations.initMocks(this); } @Test public void testWithContext() { … } }
  25. @RunWith(JUnit4.class) public class ContextManagerTest { @Test public void testWithContext() {

    Context appContext = mock(Context.class); Mockito.when(appContext.getPackageName()) .thenReturn(“com.example.app”); … } }
  26. • Mockito.spy() • wrapping a real object • Mockito.verify() •

    verify that special condition are met • e.g. method called, method called twice, …
  27. Limitations • final classes • opt-in incubating support in Mockito

    2 • anonymous classes • primitive types • static methods
  28. • functional testing framework • runs on JVM • at

    first, might be difficult to use • the ultimate mock of Android APIs • provides mocks of system managers • allows custom shadows
  29. • possible to use for UI testing • better to

    use for business logic @RunWith(RobolectricTestRunner.class) public class MyTest { … }
  30. Potential problems • difficult to search for solutions • long

    history of bigger API changes • many obsolete posts
  31. • cross-platform BDD framework • human-like test definitions testCompile ‘junit:junit:4.12'

    testCompile ‘info.cukes:cucumber-java:1.2.5' testCompile 'info.cukes:cucumber-junit:1.2.5'
  32. • describe the desired behaviour Feature: CoffeeMaker
 
 Scenario: a

    few coffees
 Given I previously had 3 coffees
 When I add one coffee
 Then I had 4 coffees
  33. • create the mapping public class CoffeeMakerDefs {
 CoffeeMaker mCoffeeMaker

    = new CoffeeMaker();
 
 @Given("^I previously had (\\d+) coffees$")
 public void hadCoffeesPreviously(int coffees) {
 mCoffeeMaker.setCoffeeCount(coffees);
 }
 
 
 }
  34. • create the mapping public class CoffeeMakerDefs {
 CoffeeMaker mCoffeeMaker

    = new CoffeeMaker();
 
 @Given("^I previously had (\\d+) coffees$")
 public void hadCoffeesPreviously(int coffees) {
 mCoffeeMaker.setCoffeeCount(coffees);
 }
 
 @When("^I add one coffee$")
 public void addCoffee() {
 mCoffeeMaker.addCoffee();
 }
 
 
 }
  35. • create the mapping public class CoffeeMakerDefs {
 CoffeeMaker mCoffeeMaker

    = new CoffeeMaker();
 
 @Given("^I previously had (\\d+) coffees$")
 public void hadCoffeesPreviously(int coffees) {
 mCoffeeMaker.setCoffeeCount(coffees);
 }
 
 @When("^I add one coffee$")
 public void addCoffee() {
 mCoffeeMaker.addCoffee();
 }
 
 @Then("^I had (\\d+) coffees$")
 public void hadCoffees(int coffees) {
 Assert.assertEquals(coffees, mCoffeeMaker.getCoffeeCount());
 }
 }
  36. • place definition and mapping at the same paths! •

    ${module}/src/test/java/com/example/MyMapping.java • ${module}/src/test/resources/com/example/ MyDefinition.feature @RunWith(Cucumber.class)
 public class RunCucumberTest {
 }
  37. Instrumentation Tests & Code Coverage • has to be explicitly

    enabled • gradle createDebugCoverageReport • ${module}/build/reports/coverage/debug/index.html • ${module}/build/outputs/code-coverage/connected/$ {deviceName}-coverage.ec • doesn’t work on some devices!!! buildTypes {
 debug {
 testCoverageEnabled true
 } }
  38. JaCoCo • enabled by default for unit tests • gradle

    test • generates binary report in build/jacoco • ${module}/build/jacoco/testDebugUnitTest.exec • it’s necessary to create a JacocoReport task to obtain a readable report
  39. Good tests • run in any order • run in

    isolation • run consistently • run fast • are orthogonal
  40. Rules of thumb • prefer pure Java • abstract away

    from Android APIs • separate business logic and UI • don’t write business logic into activities and fragments • MVP, MVVM is a way to go • try avoid static and final • use dependency injection