Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Test-Driven Android Development

AppFoundry
February 02, 2015

Test-Driven Android Development

Android & Testing talk given at
- jDays 2015 in Göteborg, Sweden
- GDG Devfest Istanbul 2014
- Karel de Grote University College (KdG) in Antwerp on the 2nd of February 2015.

Everybody knows testing is important, so let’s focus on test-driven development, testing best practices and the most useful Android testing libraries in our quest to improve the user experience and developer happiness. In this talk you'll get an overview of how several types of testing (unit, integration, UI testing) fit into an Android project. The Android team has improved the testing story for Android during the years, but the Android testing framework can be complemented with several other tools and libraries to help you build better quality applications. And let's not forget that the ease and thoroughness of your testing is also determined by the architecture of your application. We'll show you how to get the most out of Android's testing infrastructure, in order to enable you to build great quality apps.

AppFoundry

February 02, 2015
Tweet

More Decks by AppFoundry

Other Decks in Programming

Transcript

  1. “Computers are designed to do simple repetitive tasks. The second

    you have humans doing repetitive tasks, all the computers get together late at night and laugh at you...” Neil Ford
  2. EXCUSES We never make mistakes! The functionality is trivial Tests

    slow us down Management won’t let us WE = HERO TRIVIAL SLOW MANAGEMENT ǩ
  3. Repeatability (no matter what development stack he uses) in which

    they are being run Tests need to be run by every developer ♂ Tests must not rely on the environment
  4. Self-documenting Testable code = clear Testable code = easy to

    follow we can just look at the test No need to explain how a component works No need to write documentation tests = usage examples
  5. Test types Unit tests Ŷ Functional tests Ŷ Acceptance tests

    Ŷ Integration tests Ŷ Regression tests Ŷ
  6. Black and white Black box tests Ŷ Without knowledge Ŷ

    Testing software knows Ŷ White box tests Ŷ and tests the internal structures / workings of an application of the application
  7. Black and white Black box tests Ŷ White box tests

    Ŷ • tests for precision / correctness • tester knows + is testing internal structure of the application • unit / integration testing • application internal structure is not known to the tester
 • acceptance / system testing
  8. Android testing start with the test prefix called setUp() called

    tearDown() Uses JUnit 3 Ŷ test methods Ŷ setup method Ŷ clean up method Ŷ
  9. JUnit 3 JUnit 3 test import junit.framework.*; 
 public class

    MyTestClass extends TestCase {
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 ...
 }
 public void testMethodName() throws Exception {
 ...
 }
 }
  10. Android testing to test Android components > junit.framework.TestCase to test

    a class that does not call the Android API > junit.framework.TestCase Uses JUnit 3 Ŷ plain JUnit Ŷ Android JUnit extensions Ŷ AndroidTestCase Ŷ InstrumentationTestCase Ŷ
  11. Instrumentation component / application lifecycle the system has with the

    app independently from normal lifecycle loaded into the same process Set of hooks into Android Ŷ hooks control Android component Ŷ Monitor interaction Ŷ Test package + app under test Ŷ invoke methods Ŷ modify and examine fields Ŷ
  12. Instrumentation touches
 taps
 scrolling Runs on device/emulator Ŷ Can be

    started from IDE/shell Ŷ Allows direct control of UI Ŷ
  13. Instrumentation Test Activity state saving and restoration // Start the

    main activity of the application under test
 mActivity = getActivity();
 // Get a handle to the Activity object's main UI widget, a Spinner
 mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner);
 // Set the Spinner to a known position
 mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION);
 // Stop the activity - The onDestroy() method should save the state of the Spinner
 mActivity.finish();
 // Re-start the Activity - the onResume() method should restore the state of the Spinner
 mActivity = getActivity();
 // Get the Spinner's current position
 int currentPosition = mActivity.getSpinnerPosition();
 // Assert that the current position is the same as the starting position
 assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition);
  14. Isolate tests from the system MockApplication Ŷ MockContext Ŷ MockResources

    Ŷ MockContentProvider Ŷ MockContentResolver Ŷ MockPackageManager Ŷ
  15. Test runner against and Android application Instrumentation runs TestCases Ŷ

    default config Ŷ test runner android { defaultConfig { testInstrumentationRunner "android.test.InstrumentationTestRunner" } }
  16. Your first instrumentation test Activity test import android.test.ActivityInstrumentationTestCase2; public class

    ActivityTest extends ActivityInstrumentationTestCase2<Activity> { public ActivityTest() { super(Activity.class); } public void testSomething() { // test code } }
  17. Your first instrumentation test Activity test private ConverterActivity activity;
 private

    EditText celsiusTextField;
 private TextView fahrenheitTextField;
 private TextView celsiusTextLabel;
 private TextView fahrenheitTextLabel;
 private Button convertButton;
 
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 activity = getActivity();
 celsiusTextField = (EditText) activity.findViewById(R.id.celsius_textview);
 fahrenheitTextField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 celsiusTextLabel = (TextView) activity.findViewById(R.id.celsius_label);
 fahrenheitTextLabel = (TextView) activity.findViewById(R.id.fahrenheit_label);
 convertButton = (Button) activity.findViewById(R.id.conversion_button);
 }
  18. Your first instrumentation test Activity test public void testConversion() {


    // instrumentation commands
 TouchUtils.tapView(this, celsiusTextField);
 sendKeys(KeyEvent.KEYCODE_1, KeyEvent.KEYCODE_2);
 TouchUtils.tapView(this, convertButton);
 
 // check result against expected value
 String actualFahrenheit = fahrenheitTextField.getText().toString();
 String expected = "53.6";
 assertEquals("Temperature conversion incorrect", expected, actualFahrenheit);
 }
  19. Testing in an app in application project for testing using

    the Android testing framework Testing = integrated Ŷ No separate test project needed Ŷ main sourceSet Ŷ androidTest sourceSet Ŷ test apk is build Ŷ deploy to device Ŷ
  20. Testing in an app assembleTest build tests task Ŷ test

    sourceSet dependencies Ŷ test dependencies dependencies { androidTestCompile 'com.google.guava:guava:18.0' }
  21. Running tests ensure both your app and the test app

    are built! run in parallel on all devices both apps Build Ŷ Install Ŷ Run the tests Ŷ Uninstall both apps Ŷ Multiple devices Ŷ connectedCheck Ŷ
  22. Drawbacks / limitations of implementation details you often have to

    add things like Thread.sleep(1000) to make tests always work Required knowledge Ŷ Flakiness Ŷ Complex Ŷ Tests run slow Ŷ
  23. Drawbacks / limitations • widget IDs • widget properties •

    what has focus • … • menus • … when the UI changes -> tests often need dramatic changes too Deep knowledge required Ŷ Deep Android knowledge Ŷ Makes for brittle tests Ŷ
  24. Robotium Easier to write Ŷ Automatic timing / delays Ŷ

    Test execution is faster Ŷ Find views Ŷ
  25. Robotium Write tests with Solo Ŷ Robotium main class public

    void setUp() { solo = new Solo(getInstrumentation(), getActivity()); }
  26. Robotium API get views getView()
 getButton()
 getImage() … clicks clickOnButton()


    clickOnEditText()
 … scroll scrollListToTop()
 scrollDownList(index)
 … take screenshot takeScreenshot() send keys sendKey()
 … checks isCheckBoxChecked()
 isRadioButtonChecked()
 … wait waitForActivity()
 waitForDialogToOpen()
 … type typeText(index, text)
 typeText(id, text)
 … Solo
  27. Plain instrumentation test Activity test private ConverterActivity activity;
 private EditText

    celsiusTextField;
 private TextView fahrenheitTextField;
 private TextView celsiusTextLabel;
 private TextView fahrenheitTextLabel;
 private Button convertButton;
 
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 activity = getActivity();
 celsiusTextField = (EditText) activity.findViewById(R.id.celsius_textview);
 fahrenheitTextField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 celsiusTextLabel = (TextView) activity.findViewById(R.id.celsius_label);
 fahrenheitTextLabel = (TextView) activity.findViewById(R.id.fahrenheit_label);
 convertButton = (Button) activity.findViewById(R.id.conversion_button);
 }
  28. Your first instrumentation test Activity test public void testConversion() {


    // instrumentation commands
 TouchUtils.tapView(this, celsiusTextField);
 sendKeys(KeyEvent.KEYCODE_1, KeyEvent.KEYCODE_2);
 TouchUtils.tapView(this, convertButton);
 
 // check result against expected value
 String actualFahrenheit = fahrenheitTextField.getText().toString();
 String expected = "53.6";
 assertEquals("Temperature conversion incorrect", expected, actualFahrenheit);
 }
  29. Robotium test Robotium test public void testConversion() {
 solo.enterText(0, "12");


    solo.clickOnButton("Convert!");
 assertTrue(solo.searchText("53.6"));
 }
  30. Robotium setup Robotium dependency Ŷ Uses SDK test runner Ŷ

    Robotium dependency dependencies { androidTestCompile ‘com.jayway.android.robotium:robotium-solo:5.2.1’ }
  31. Robotium test base class Robotium base /**
 * Base class

    for Activity Instrumentation tests that utilise Robotium.
 */
 public abstract class RobotiumActivityInstrumentationTestCase<T extends Activity> extends
 ActivityInstrumentationTestCase2<T> {
 
 protected Solo solo;
 
 public RobotiumActivityInstrumentationTestCase(Class<T> activityClass) {
 super(activityClass);
 }
 
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 solo = new Solo(getInstrumentation(), getActivity());
 }
 
 @Override
 protected void tearDown() throws Exception {
 super.tearDown();
 solo.finishOpenedActivities();
 }
 }
  32. Robotium: ListView Robotium list public void testClickOnHowardInlist() {
 solo.clickOnText("Howard Wolowitz");


    }
 
 public void testClickOnSecondItemInList() {
 solo.clickInList(2);
 }
  33. Robotium: menu item Robotium menu item public void testClickInfoMenuItem() {


    // open the ActionBar overflow
 solo.sendKey(KeyEvent.KEYCODE_MENU);
 solo.clickOnText("Info");
 }
  34. Robotium: intra-activity Robotium wait & see public void testClickingOnPersonShouldShowDetails() {


    int ACTIVITY_WAITING_TIMEOUT = 10000;
 // first activity should start
 assertThat(solo.waitForActivity(PersonListActivity.class, ACTIVITY_WAITING_TIMEOUT), is(true));
 // click on a person
 solo.clickOnText("Howard Wolowitz");
 
 assertThat(solo.waitForActivity(PersonDetailActivity.class, ACTIVITY_WAITING_TIMEOUT), is(true));
 }
  35. Robotium errors Robotium error junit.framework.AssertionFailedError: 
 Button with text: 'test

    button' is not found!
 at com.robotium.solo.Getter.getView(Getter.java:67) at com.robotium.solo.Solo.getButton(Solo.java:1957) at com.devoxx.android.bigbang.test.RobotiumTests.testErrorMessages(RobotiumTests.java:24) public void testErrorMessages() {
 solo.getButton("test button");
 }
  36. Robotium webview getWebElement(By by)
 clickOnWebElement(By by)
 enterTextInWebElement(By by) based on:


    id - tagName - textContent - className
 
 e.g. By.className(String className) Can interact with WebView Ŷ Select web content Ŷ Interact with web content Ŷ
  37. Hamcrest Library of matchers Ŷ Syntactic sugar Ŷ Assert assertThat(someString,

    is(equalTo(“Expected”))); actual value expectation on the value (Matcher)
  38. Hamcrest matchers Core anything
 is
 … Logical allOf
 anyOf
 not

    Object equalTo
 instanceOf
 (not)nullValue Custom Beans hasProperty Text equalToIgnoringCase
 startsWith
 … Number greaterThan(OrEqualTo)
 closeTo Collections hasItem, hasItems
 hasItemInArray
  39. Hamcrest Lots of useful matchers Ŷ Custom matchers Ŷ not

    a number public void testSquareRootOfMinusOneIsNotANumber() {
 assertThat(Math.sqrt(-1), is(notANumber()));
 }
  40. Hamcrest custom matcher Hamcrest matcher /**
 * Hamcrest matcher: test

    if a double has the value NaN (not a number)
 */
 public class IsNotANumber extends TypeSafeMatcher<Double> {
 
 @Override
 public boolean matchesSafely(Double number) {
 return number.isNaN();
 }
 
 public void describeTo(Description description) {
 description.appendText("not a number");
 }
 
 @Factory
 public static <T> Matcher<Double> notANumber() {
 return new IsNotANumber();
 }
 }
  41. Espresso on top of Instrumentation APIs Google’s new approach Ŷ

    Abstraction layer Ŷ * GTAC 10/2013 Ŷ Android API level ≥ 8 Ŷ Open source Ŷ https://code.google.com/p/android-test-kit/
  42. Espresso readable tests fast based on Matchers Fluent API Ŷ

    Extensible Ŷ Better debugging / logging Ŷ Performance Ŷ
  43. Espresso Easy API for test authors Find view Do stuff

    with view Check some state Without • waitUntil(timeout) • boilerplate
  44. Espresso API onView(Matcher<View>) onData(Matcher<Object>) onView(Matcher<View>) onView(Matcher<View>) withId() / withText() is()

    / instanceOf() click() / enterText() / scrollTo() matches(Matcher<View>) isDisplayed / isEnabled / hasFocus withText / … Find stuff Do stuff Check
  45. Espresso Adapter views onData Ŷ Espresso ViewAction // search for

    a row and perform action
 onData(hasToString(startsWith("Sheldon"))).perform(click()); onData(is(instanceOf(DummyData.BigBangCharacter.class)))
 .atPosition(5)
 .perform(click()); // search for a row, and perform an action on a child view
 onData(hasToString(startsWith("Amy")))
 .onChildView(withId(R.id.list_item_photo))
 .perform(click());
  46. Espresso swiping Espresso swiping public void testSwipingBackAndForward() {
 // Should

    be on position 0 to start with.
 onView(withText("Position #0")).check(matches(isDisplayed()));
 
 // Swipe left once.
 onView(withId(R.id.pager_layout)).perform(swipeLeft());
 
 // Now position 1 should be visible.
 onView(withText("Position #1")).check(matches(isDisplayed()));
 
 // Swipe back to the right.
 onView(withId(R.id.pager_layout)).perform(swipeRight());
 
 // Now position 0 should be visible again.
 onView(withText("Position #0")).check(matches(isDisplayed()));
 
 // Swipe right again.
 onView(withId(R.id.pager_layout)).perform(swipeRight());
 
 // Position 0 should still be visible as this is the first view in the pager.
 onView(withText("Position #0")).check(matches(isDisplayed()));
 }
  47. Espresso scrolling Espresso scrolling // You can pass more than

    one action to perform. This is useful if you are performing two actions
 // back-to-back on the same view.
 public void testScrollToInScrollView() {
 onView(withId(is(R.id.bottom_view)))
 .perform(scrollTo(), click());
 }
  48. Espresso API onView(Matcher<View>) onData(Matcher<Object>) Espresso withId() withText() ViewMatchers perform() check()

    ViewInteraction / DataInteraction click() enterText() scrollTo() ViewActions matches(Matcher<View>) doesNotExist() ViewAssertions find view to operate on composed of © Stephan Linzner (Google)
  49. View Matchers Espresso uses Hamcrest Matchers Espresso Matchers = Hamcrest

    Ŷ Make assertions on views Ŷ Filter views Ŷ Reuse existing Matchers Ŷ Create your own Matchers Ŷ from hamcrest-core and hamcrest-library
  50. View Matchers Hamcrest Ŷ allOf() / anyOf() ➪ give more

    power Ŷ TestRunner // assert that textview does not start with with "XYZ" and contains "ABC" anywhere
 onView(withId(R.id.celsius_textview))
 .check(matches(allOf(withText(not(startsWith("XYZ"))), withText(containsString("ABC")))));
 // assert that textview ends with "XYZ" or contains "ABC" anywhere
 onView(withId(R.id.celsius_textview))
 .check(matches(anyOf(withText(endsWith("XYZ")), withText(containsString("ABC")))));
  51. View Actions • click() • scrollTo() • writeText() Perform actions

    on Views Ŷ Guaranteed to run on UI thread Ŷ Create your custom Actions Ŷ Basic view actions Ŷ
  52. Custom ViewAction Espresso ViewAction /**
 * Clears view text by

    setting {@link EditText}s text property to "".
 */
 public class ClearTextAction implements ViewAction {
 
 @Override
 public Matcher<View> getConstraints() {
 return allOf(isDisplayed(), isAssignableFrom(EditText.class));
 }
 
 @Override
 public void perform(UiController uiController, View view) {
 ((EditText) view).setText("");
 }
 
 @Override
 public String getDescription() {
 return "Clear the EditText component";
 }
 }
  53. Custom ViewAction Espresso ViewAction /**
 * Your own custom View

    Actions.
 */
 public class CustomViewActions {
 
 private CustomViewActions() {
 }
 
 /**
 * Returns an action that clears text on the view. Extra thoroughly!
 */
 public static ViewAction clearTextExtraThoroughly() {
 return new ClearTextAction();
 }
 }
  54. Reliability through synchronization between Instrumentation thread and UI thread for

    background resources Espresso synchronizes AsyncTasks default thread pool • wait until app is idle • perform operation on the UI thread • wait until completion • check result for custom implementations
 IntentService
 HandlerThread
 ThreadPool Synchronization Ŷ Synchronization Ŷ Synchronization Ŷ AsyncTask Ŷ IdlingResource interface Ŷ
  55. Espresso synchronization Espresso IdlingResource public class IdleMonitor implements IdlingResource {


    
 @Override
 public String getName() {
 return IdleMonitor.class.getSimpleName();
 }
 
 @Override
 public boolean isIdleNow() {
 // return true if resource is idle
 return false;
 }
 
 @Override
 public void registerIdleTransitionCallback(ResourceCallback resourceCallback) {
 // store a reference to the resourceCallback
 // notify resourceCallback when idle
 }
 } Espresso.registerIdlingResources(idlingResource);
  56. Custom test runner Application.onCreate() always returns before 
 Instrumentation.onStart() before

    Instrumentation exits Lifecycle control Ŷ Ensure all activities finish Ŷ Activity lifecycle monitoring Ŷ Improved error logging Ŷ Improved stability / reliability Ŷ Can be used with other libraries Ŷ
  57. android.support.test.* for all Google testing frameworks Unbundled static testing library

    Ŷ Android support library Ŷ Full Gradle support Ŷ Open source / AOSP Ŷ Container Ŷ
  58. JUnit 4 support @Parameters Test annotations Ŷ Test suite annotations

    Ŷ Parameterized tests Ŷ @Suite.SuiteClasses @Test @Before @After @BeforeClass @AfterClass
  59. Support testing library usage Espresso dependencies dependencies {
 androidTestCompile ‘com.android.support.test:testing-support-lib:0.1’


    androidTestCompile ‘com.android.support.test.espresso:espresso-lib:2.0’ androidTestCompile ‘com.android.support.test.espresso:espresso-contrib:2.0’ androidTestCompile ‘com.android.support.test.espresso:idling-resource:2.0’
 }
  60. Support testing library usage Test runner Ŷ TestRunner android {


    defaultConfig {
 testInstrumentationRunner "android.support.test.runner.AndroidTestRunner "
 }
 }
  61. Espresso errors Espresso error com.google.android.apps.common.testing.ui.espresso.NoMatchingViewException: No views in hierarchy found

    matching: with text: is "Celcius" View Hierarchy: +>DecorView{id=-1, visibility=VISIBLE, width=768, height=1184, has-focus=true, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=1} | +->ActionBarOverlayLayout{id=16909075, res-name=action_bar_overlay_layout, visibility=VISIBLE, width=768, height=1184, has-focus=true, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=3} | … +---->ActionMenuView{id=-1, visibility=VISIBLE, width=112, height=96, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=656.0, y=0.0, child-count=1} | +----->OverflowMenuButton{id=-1, desc=More options, visibility=VISIBLE, width=112, height=96, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=true, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0} | +--->ActionBarContextView{id=16909078, res-name=action_context_bar, visibility=GONE, width=0, height=0, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=true, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=0} | +-->ActionBarContainer{id=16909079, res-name=split_action_bar, visibility=GONE, width=0, height=0, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=true, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=0} |
  62. Intento: inter-application testing through custom Instrumentation verify that a given

    Intent has been sent stub outgoing Intents stub the result from a broadcasted Intent Intercept Intents Ŷ Records all outgoing Intents Ŷ Intent stubbing in tests Ŷ Intent validation Ŷ
  63. Android testing AndroidTestCase needs an instance of the emulator to

    run • spin up emulator • deploy the APK • run the actual tests Big time sink: Ŷ
  64. Robolectric that would otherwise need an emulator / device (mock

    implementations of Android core libraries) too slow / hard for unit testing without the baggage of a device Replaces the behavior of code Ŷ Write JUnit tests Ŷ When? Ŷ Does this by using Shadow classes Ŷ InstrumentationTestCase Ŷ vs real device / speedy emulator Ŷ http://robolectric.org/
  65. Robolectric • unit testing fast and easy • make your

    own Shadow objects • JUnit 4 supported • does not cover all functionality (sensors, OpenGL, …) ➪ device needed • integration testing (interaction Activities - Services, camera app, …) Advantages Ŷ Disadvantages Ŷ
  66. Robolectric Robolectric 
 public class MyActivityTest {
 
 @Test
 public

    void clickingButton_shouldChangeResultsViewText() throws Exception {
 MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();
 
 Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);
 TextView results = (TextView) activity.findViewById(R.id.results_text_view);
 
 pressMeButton.performClick();
 String resultsText = results.getText().toString();
 assertThat(resultsText, equalTo("Testing Android Rocks!"));
 }
  67. Robolectric setup • Robolectric • Novoda gradle-android-test-plugin see Paul Blundell

    Tests run inside the JVM Ŷ Gradle - Maven Ŷ Configure it yourself Ŷ Gradle plugin Ŷ http://blog.blundell-apps.com/android-gradle-app-with-robolectric-junit-tests/
  68. Robolectric setup Robolectric setup apply plugin: 'java'
 
 dependencies {


    def androidModule = project(':appfoundry-app')
 compile androidModule
 
 testCompile rootProject.ext.libraries.junit
 testCompile rootProject.ext.libraries.robolectric
 
 testCompile androidModule.android.applicationVariants.toList().first().javaCompile.classpath
 testCompile androidModule.android.applicationVariants.toList().first().javaCompile.outputs.files
 testCompile files(androidModule.plugins.findPlugin("com.android.application").getBootClasspath())
 
 
 }
  69. Robolectric setup Robolectric test runner public class RobolectricGradleTestRunner extends RobolectricTestRunner

    {
 
 private static final int MAX_SDK_SUPPORTED_BY_ROBOLECTRIC = 18;
 
 public RobolectricGradleTestRunner(Class<?> testClass) throws InitializationError {
 super(testClass);
 }
 
 @Override
 protected AndroidManifest getAppManifest(Config config) {
 String manifestProperty = "../appfoundry-app/src/main/AndroidManifest.xml";
 String resProperty = "../appfoundry-app/src/main/res";
 return new AndroidManifest(Fs.fileFromPath(manifestProperty), Fs.fileFromPath(resProperty)) {
 @Override
 public int getTargetSdkVersion() {
 return MAX_SDK_SUPPORTED_BY_ROBOLECTRIC;
 }
 };
 }
 }
  70. Robolectric Annotate your class @RunWith(RobolectricTestRunner.class)
 public class MyActivityTest {
 


    @Test
 public void clickingButton_shouldChangeResultsViewText() throws Exception {
 MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();
 
 Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);
 TextView results = (TextView) activity.findViewById(R.id.results_text_view);
 
 pressMeButton.performClick();
 String resultsText = results.getText().toString();
 assertThat(resultsText, equalTo("Testing Android Rocks!"));
 }
  71. Robolectric Robolectric @RunWith(RobolectricTestRunner.class)
 public class MyActivityTest {
 
 @Test
 public

    void pressingSubmitButtonShouldStartSecondActivity() throws Exception {
 pressMeButton.performClick();
 
 ShadowActivity shadowActivity = shadowOf(activity);
 Intent startedIntent = shadowActivity.getNextStartedActivity();
 ShadowIntent shadowIntent = shadowOf(startedIntent);
 assertThat(shadowIntent.getComponent().getClassName(), equalTo(MyActivity.class.getName()));
 }
  72. Android testing JUnit 4 Support JUnit 4 test @RunWith(JUnit4.class)
 public

    class AndroidJUnit4Test {
 
 Conference devoxx;
 
 @Before
 public void setUp() {
 devoxx = Conferences.newInstance(Conference.DEVOXX);
 devoxx.start();
 }
 
 @After
 public void tearDown() {
 devoxx.stop();
 }
 
 @Test
 public void testPreconditions() {
 assertThat(devoxx, is(notNullValue()));
 }
 
 @Test
 public void devoxxShouldBeGreat() {
 assertThat(devoxx, is(awesome()));
 }
  73. Mockito in isolation Change runtime implementation Ŷ That can be

    predictably tested Ŷ Verify behaviour Ŷ Mockito Ŷ
  74. Mockito the behaviour of your components during a test Java

    mocking framework Ŷ Mock dependencies Ŷ Inject Mocks to validate Ŷ Mock function return values Ŷ https://code.google.com/p/mockito/
  75. Mockito Mockito @Test
 public void testMockedConverter() throws Exception {
 //

    expectations
 final float expected_result = 32.0f;
 
 // given
 ConverterActivity activity = Robolectric.buildActivity(ConverterActivity.class).create().get();
 activity.onCreate(new Bundle());
 EditText celsiusField = (EditText) activity.findViewById(R.id.celsius_textview);
 celsiusField.setText("0");
 
 CelsiusFahrenheitConverter mockConverter = Mockito.mock(CelsiusFahrenheitConverter.class);
 Mockito.when(mockConverter.convertCelsiusToFahrenheit(Mockito.anyFloat())).thenReturn(expected_result);
 activity.setConverter(mockConverter);
 
 // when
 Button convertButton = (Button) activity.findViewById(R.id.conversion_button);
 convertButton.performClick();
 
 // then
 Mockito.verify(mockConverter, Mockito.times(1)).convertCelsiusToFahrenheit(Mockito.anyFloat());
 TextView fahrenheitField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 String fahrenheitString = fahrenheitField.getText().toString();
 assertThat(fahrenheitString, equalTo(String.valueOf(expected_result)));
 }
  76. Monkey pseudo-random stream of user events Generates events Ŷ clicks

    Ŷ touches Ŷ gestures Ŷ system-level events Ŷ Stress test your app Ŷ
  77. Monkey number of events to attempt operational constraints, such as

    restricting the test to a single package Event types and frequencies command-line tool Ŷ basic configuration Ŷ $ adb shell monkey -p your.package.name -v 500 Ŷ
  78. AssertJ Android for checking assertions to read / write http://square.github.io/assertj-android/

    Syntactic sugar Ŷ Extension of the AssertJ library Ŷ Makes tests easier Ŷ Fluent syntax Ŷ
  79. AssertJ Android Regular JUnit assertEquals(View.VISIBLE, layout.getVisibility()); assertEquals(VERTICAL, layout.getOrientation()); assertEquals(4, layout.getChildCount());

    assertEquals(SHOW_DIVIDERS_MIDDLE, layout.getShowDividers()); AssertJ Android assertThat(layout).isVisible() .isVertical() .hasChildCount(4) .hasShowDividers(SHOW_DIVIDERS_MIDDLE);
  80. Testing network resources how to trigger edge cases like API

    rate limit exceeded? HTTP calls should be asynchronous … Hit a real server? Ŷ Brittle tests! Ŷ Slow tests Ŷ Network or server is down Ŷ Incomplete tests Ŷ Further complicated in Android Ŷ
  81. Using a test server • virtually no changes or code

    required in your app codebase • can be shared across multiple platforms (Android, iOS, web, …) • another dependency (that can fail) • something new to figure out • not always easy to trigger errors or edge cases • what if ‘state’ is needed? • ‘slow’ tests execution, still HTTP calls needed PROs Ŷ CONs Ŷ
  82. Mocking the Retrofit interface • robust: no network errors, timeouts,

    … • easy to generate errors / edge cases • easy to control server-side state • write your dummy data in Java • extra test code needed • you don’t test the JSON deserialization (but you can add unit tests) • mock a part of your application ➪ not a true integration test PROs Ŷ CONs Ŷ You trust the test suites of Retrofit, Gson, … and mock out the data
  83. Mocking the Retrofit interface Retrofit mock public interface GitHub {


    @GET("/repos/{owner}/{repo}/contributors")
 List<Contributor> contributors(@Path("owner") String owner, @Path("repo") String repo);
 }
  84. Mocking the Retrofit interface Canned data /** A mock implementation

    of the {@link GitHub} API interface. */
 static class MockGitHub implements GitHub {
 private final Map<String, Map<String, List<Contributor>>> ownerRepoContributors;
 
 public MockGitHub() {
 ownerRepoContributors = new LinkedHashMap<String, Map<String, List<Contributor>>>();
 
 // Seed some mock data.
 addContributor("square", "retrofit", "John Doe", 12);
 addContributor("square", "retrofit", "Bob Smith", 2);
 addContributor("square", "retrofit", "Big Bird", 40);
 addContributor("square", "picasso", "Proposition Joe", 39);
 addContributor("square", "picasso", "Keiser Soze", 152);
 }
  85. Mocking the Retrofit interface Mock @Override public List<Contributor> contributors(String owner,

    String repo) {
 Map<String, List<Contributor>> repoContributors = ownerRepoContributors.get(owner);
 if (repoContributors == null) {
 return Collections.emptyList();
 }
 List<Contributor> contributors = repoContributors.get(repo);
 if (contributors == null) {
 return Collections.emptyList();
 }
 return contributors;
 }
  86. Mocking the Retrofit interface Mock // Create a very simple

    REST adapter which points the GitHub API endpoint.
 RestAdapter restAdapter = new RestAdapter.Builder()
 .setEndpoint(API_URL)
 .build();
 
 // Wrap our REST adapter to allow mock implementations and fake network delay.
 MockRestAdapter mockRestAdapter = MockRestAdapter.from(restAdapter);
 
 // Instantiate a mock object so we can interact with it later.
 MockGitHub mockGitHub = new MockGitHub();
 // Use the mock REST adapter and our mock object to create the API interface.
 GitHub gitHub = mockRestAdapter.create(GitHub.class, mockGitHub);
 
 // Query for some contributors for a few repositories.
 printContributors(gitHub, "square", "retrofit");
 printContributors(gitHub, "square", "picasso");
  87. Wiremock predictable requests-responses wiremock.org Ŷ Tom Akehurst Ŷ Reproduce real-world

    scenarios Ŷ Increase reliability Ŷ Stateful scenarios Ŷ Non-Android specific Ŷ
  88. Wiremock via HTTP/JSON API • _admin/mappings/new • mappings folder HTTP

    server Ŷ configure responses Ŷ use with Java API Ŷ
  89. Wiremock Wiremock JSON API { "request": { "method": "GET", "url":

    "/api/mytest" }, "response": { "status": 200, "body": "More content\n" } } wiremock.org documentation
  90. Wiremock Wiremock Java test @Test public void exampleTest() { //

    make your app do requests to http://localhost:8080 stubFor(get(urlEqualTo("/my/resource")) .willReturn(aResponse() .withStatus(200) .withBody("<response>Some content</response>"))); onView(withId(R.id.submit_button)).perform(click()); onView(withText(“error message”)).check(matches(isDisplayed())); verify(postRequestedFor(urlMatching(“/my/resource/[a-z0-9]+”))); } wiremock.org documentation
  91. Wiremock … out-of-the-box Android support is coming… Stateful Ŷ Errors

    / delays Ŷ Wiremock in the test APK Ŷ Wiremock as server on network Ŷ
  92. Suggested talks Jake Wharton https://speakerdeck.com/jakewharton/android-apps- with-dagger-devoxx-2013 https://www.parleys.com/play/ 529bde2ce4b0e619540cc3ae Jake Wharton

    https://speakerdeck.com/jakewharton/dependency- injection-with-dagger-2-devoxx-2014 https://www.parleys.com/play/ 5471cdd1e4b065ebcfa1d557 Android Apps with Dagger Ŷ Dependency Injection with Dagger 2 Ŷ
  93. Automated distributed testing Automated distributed instrumentation testing Screenshots share with

    the designers…? in parallel Run tests on multiple devices Reports / logs captured for each test
  94. Spoon: device view Results of each test on one device

    Ŷ Useful for tracking down Ŷ device-specific failures of individual tests
  95. Spoon: screenshots Visual inspection of test execution Ŷ screenshots Spoon.screenshot(activity,

    "initial_state"); /* Normal test code... */ Spoon.screenshot(activity, "after_login"); http://square.github.io/spoon/sample/
  96. Build pipeline Commit stage Automated test/QA stage Manual QA testing

    UAT test Production Capacity / load testing
  97. Build pipeline Checkout / compile Unit tests Test coverage Code

    analysis Create deployable artifact Deploy for automatic QA test Trigger automated QA stage
  98. Build pipeline tools Build (maven - gradle) Dependency repo (nexus

    - artifactory) Testing framework (JUnit - ...) Test coverage (Cobertura - Emma - Jacoco) Code analysis (Checkstyle, findbugs, pmd, Android Lint) Creation of deployable artifact (buildtool, artifact repo) Trigger next stage
  99. Continuous integration benefits without additional effort Fast feedback - fewer

    errors Ŷ Test everything on every (nightly) build Ŷ Regression tests Ŷ Less manual testing Ŷ
  100. Test coverage open-source toolkit for measuring and reporting Java code

    coverage Metrics Ŷ Code safely Ŷ JaCoCo Ŷ Find risky code Ŷ
  101. Suggested reading Android Application Testing Guide Diego Torres Milano (9781849513500)

    Robotium Automated Testing for Android Hrushikesh Zadgaonkar (9781782168010) The Busy Coder’s Guide to Android Development Mark Murphy - http://commonsware.com/Android/
  102. Suggested reading Test Driven Development: By Example Beck, Kent (978-0321146533)

    Continuous Integration: Improving Software Quality and Reducing Risk Duvall, Paul M. et al. (978-0321336385) Working Effectively with Legacy Code Feathers, Michael (978-0131177055)