Slide 1

Slide 1 text

@fmaelbrancke #DV14 #AndroidTesting Testing your Android app Filip Maelbrancke AppFoundry

Slide 2

Slide 2 text

Your host Filip Maelbrancke Consultant @ AppFoundry ! fi[email protected] @fmaelbrancke

Slide 3

Slide 3 text

Android & Testing 01 02 03 Why? What? How?

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Testing Avoid problems Android testing

Slide 9

Slide 9 text

TESTING Manual testing is tedious Humans are not very good at testing But computers are...

Slide 10

Slide 10 text

“Computers are designed to do simple repetitive tasks. The second you have humans doing repetitive tasks, all the computers get together late at night and laugh at you...” Neil Ford

Slide 11

Slide 11 text

Manual vs Automated

Slide 12

Slide 12 text

Manual vs Automated

Slide 13

Slide 13 text

Manual vs Automated

Slide 14

Slide 14 text

Mitigate issues? Be pro-active… and test.

Slide 15

Slide 15 text

EXCUSES We never make mistakes! The functionality is trivial Tests slow us down Management won’t let us WE = HERO TRIVIAL SLOW MANAGEMENT ǩ

Slide 16

Slide 16 text

TEST TRAITS (UNIT) TESTS ISOLATED REPEATABLE FAST SELF-DOCUMENTING ž

Slide 17

Slide 17 text

Repeatability (no matter what development stack he uses) in which they are being run Tests need to be run by every developer ♂ Tests must not rely on the environment

Slide 18

Slide 18 text

Speed The shorter, the less distraction less of an interruption to our workflow

Slide 19

Slide 19 text

Speed The shorter, the less distraction less of an interruption to our workflow

Slide 20

Slide 20 text

Self-documenting Testable code = clear Testable code = easy to follow we can just look at the test No need to explain how a component works No need to write documentation tests = usage examples

Slide 21

Slide 21 text

Tests should also be … Easy to execute Easy to debug Realistic Fluent API

Slide 22

Slide 22 text

Unit tests are real measure of project health and code quality

Slide 23

Slide 23 text

Android dev

Slide 24

Slide 24 text

Test types Unit tests Ŷ Functional tests Ŷ Acceptance tests Ŷ Integration tests Ŷ Regression tests Ŷ

Slide 25

Slide 25 text

Black and white Black box tests Ŷ Without knowledge Ŷ Testing software knows Ŷ White box tests Ŷ and tests the internal structures / workings of an application of the application

Slide 26

Slide 26 text

Black and white Black box tests Ŷ White box tests Ŷ • tests for precision / correctness • tester knows + is testing internal structure of the application • unit / integration testing • application internal structure is not known to the tester
 • acceptance / system testing

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

d.android.com

Slide 29

Slide 29 text

developer.android.com testing JUnit Ŷ Instrumentation tests Ŷ Monkey / MonkeyRunner Ŷ

Slide 30

Slide 30 text

Android testing start with the test prefix called setUp() called tearDown() Uses JUnit 3 Ŷ test methods Ŷ setup method Ŷ clean up method Ŷ

Slide 31

Slide 31 text

JUnit 3 JUnit 3 test import junit.framework.*; 
 public class MyTestClass extends TestCase {
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 ...
 }
 public void testMethodName() throws Exception {
 ...
 }
 }

Slide 32

Slide 32 text

Android testing to test Android components > junit.framework.TestCase to test a class that does not call the Android API > junit.framework.TestCase Uses JUnit 3 Ŷ plain JUnit Ŷ Android JUnit extensions Ŷ AndroidTestCase Ŷ InstrumentationTestCase Ŷ

Slide 33

Slide 33 text

Instrumentation component / application lifecycle the system has with the app independently from normal lifecycle loaded into the same process Set of hooks into Android Ŷ hooks control Android component Ŷ Monitor interaction Ŷ Test package + app under test Ŷ invoke methods Ŷ modify and examine fields Ŷ

Slide 34

Slide 34 text

Instrumentation Process Test package InstrumentationTestRunner Application package Test case classes JUnit

Slide 35

Slide 35 text

Instrumentation touches
 taps
 scrolling Runs on device/emulator Ŷ Can be started from IDE/shell Ŷ Allows direct control of UI Ŷ

Slide 36

Slide 36 text

Instrumentation Test Activity state saving and restoration // Start the main activity of the application under test
 mActivity = getActivity();
 // Get a handle to the Activity object's main UI widget, a Spinner
 mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner);
 // Set the Spinner to a known position
 mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION);
 // Stop the activity - The onDestroy() method should save the state of the Spinner
 mActivity.finish();
 // Re-start the Activity - the onResume() method should restore the state of the Spinner
 mActivity = getActivity();
 // Get the Spinner's current position
 int currentPosition = mActivity.getSpinnerPosition();
 // Assert that the current position is the same as the starting position
 assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition);

Slide 37

Slide 37 text

Android test case AndroidTestCase Ŷ InstrumentationTestCase Ŷ junit.framework.Assert junit.framework.TestCase Android test case classes

Slide 38

Slide 38 text

Assert assertEquals assertTrue assertFalse fail assertNull assertNotSame assertSame assertNotNull Assert

Slide 39

Slide 39 text

ViewAssertions

Slide 40

Slide 40 text

AndroidTestCase ApplicationTestCase Ŷ ProviderTestCase2 Ŷ ServiceTestCase Ŷ

Slide 41

Slide 41 text

InstrumentationTestCase ActivityInstrumentationTestCase2 Ŷ … Ŷ

Slide 42

Slide 42 text

Android instrumentation tests Application Ŷ Activity Ŷ Service Ŷ Content provider Ŷ

Slide 43

Slide 43 text

Isolate tests from the system MockApplication Ŷ MockContext Ŷ MockResources Ŷ MockContentProvider Ŷ MockContentResolver Ŷ MockPackageManager Ŷ

Slide 44

Slide 44 text

Test runner against and Android application Instrumentation runs TestCases Ŷ default config Ŷ test runner android { defaultConfig { testInstrumentationRunner "android.test.InstrumentationTestRunner" } }

Slide 45

Slide 45 text

Your first instrumentation test Activity test ! import android.test.ActivityInstrumentationTestCase2; ! public class ActivityTest extends ActivityInstrumentationTestCase2 { ! public ActivityTest() { super(Activity.class); } ! public void testSomething() { // test code ! } }

Slide 46

Slide 46 text

Testing demo Testing a temperature converter ☼

Slide 47

Slide 47 text

Testing demo Simple master-detail app ☼

Slide 48

Slide 48 text

Your first instrumentation test Activity test private ConverterActivity activity;
 private EditText celsiusTextField;
 private TextView fahrenheitTextField;
 private TextView celsiusTextLabel;
 private TextView fahrenheitTextLabel;
 private Button convertButton;
 
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 activity = getActivity();
 celsiusTextField = (EditText) activity.findViewById(R.id.celsius_textview);
 fahrenheitTextField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 celsiusTextLabel = (TextView) activity.findViewById(R.id.celsius_label);
 fahrenheitTextLabel = (TextView) activity.findViewById(R.id.fahrenheit_label);
 convertButton = (Button) activity.findViewById(R.id.conversion_button);
 }

Slide 49

Slide 49 text

Your first instrumentation test Activity test public void testConversion() {
 // instrumentation commands
 TouchUtils.tapView(this, celsiusTextField);
 sendKeys(KeyEvent.KEYCODE_1, KeyEvent.KEYCODE_2);
 TouchUtils.tapView(this, convertButton);
 
 // check result against expected value
 String actualFahrenheit = fahrenheitTextField.getText().toString();
 String expected = "53.6";
 assertEquals("Temperature conversion incorrect", expected, actualFahrenheit);
 }!

Slide 50

Slide 50 text

Testing in an app in application project for testing using the Android testing framework Testing = integrated Ŷ No separate test project needed Ŷ main sourceSet Ŷ androidTest sourceSet Ŷ test apk is build Ŷ deploy to device Ŷ

Slide 51

Slide 51 text

Testing in an app assembleTest build tests task Ŷ test sourceSet dependencies Ŷ test dependencies dependencies { androidTestCompile 'com.google.guava:guava:18.0' }

Slide 52

Slide 52 text

Running tests ensure both your app and the test app are built! run in parallel on all devices both apps Build Ŷ Install Ŷ Run the tests Ŷ Uninstall both apps Ŷ Multiple devices Ŷ connectedCheck Ŷ

Slide 53

Slide 53 text

Test report build/outputs/androidTest-results build/reports/androidTests xml test results Ŷ html test report Ŷ

Slide 54

Slide 54 text

Drawbacks / limitations of implementation details you often have to add things like Thread.sleep(1000) to make tests always work Required knowledge Ŷ Flakiness Ŷ Complex Ŷ Tests run slow Ŷ

Slide 55

Slide 55 text

Drawbacks / limitations • widget IDs • widget properties • what has focus • … • menus • … when the UI changes -> tests often need dramatic changes too Deep knowledge required Ŷ Deep Android knowledge Ŷ Makes for brittle tests Ŷ

Slide 56

Slide 56 text

Instrumentation testing with Robotium Instrumentation testing framework Simplify tests making

Slide 57

Slide 57 text

Robotium Easier to write Ŷ Automatic timing / delays Ŷ Test execution is faster Ŷ Find views Ŷ

Slide 58

Slide 58 text

Robotium Write tests with Solo Ŷ Robotium main class public void setUp() { solo = new Solo(getInstrumentation(), getActivity()); ! }

Slide 59

Slide 59 text

Robotium API get views getView()
 getButton()
 getImage() … clicks clickOnButton()
 clickOnEditText()
 … scroll scrollListToTop()
 scrollDownList(index)
 … take screenshot takeScreenshot() send keys sendKey()
 … checks isCheckBoxChecked()
 isRadioButtonChecked()
 … wait waitForActivity()
 waitForDialogToOpen()
 … type typeText(index, text)
 typeText(id, text)
 … Solo

Slide 60

Slide 60 text

Instrumentation testing Testing a temperature converter ☼

Slide 61

Slide 61 text

Plain instrumentation test Activity test private ConverterActivity activity;
 private EditText celsiusTextField;
 private TextView fahrenheitTextField;
 private TextView celsiusTextLabel;
 private TextView fahrenheitTextLabel;
 private Button convertButton;
 
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 activity = getActivity();
 celsiusTextField = (EditText) activity.findViewById(R.id.celsius_textview);
 fahrenheitTextField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 celsiusTextLabel = (TextView) activity.findViewById(R.id.celsius_label);
 fahrenheitTextLabel = (TextView) activity.findViewById(R.id.fahrenheit_label);
 convertButton = (Button) activity.findViewById(R.id.conversion_button);
 }

Slide 62

Slide 62 text

Your first instrumentation test Activity test public void testConversion() {
 // instrumentation commands
 TouchUtils.tapView(this, celsiusTextField);
 sendKeys(KeyEvent.KEYCODE_1, KeyEvent.KEYCODE_2);
 TouchUtils.tapView(this, convertButton);
 
 // check result against expected value
 String actualFahrenheit = fahrenheitTextField.getText().toString();
 String expected = "53.6";
 assertEquals("Temperature conversion incorrect", expected, actualFahrenheit);
 }!

Slide 63

Slide 63 text

Robotium test Robotium test public void testConversion() {
 solo.enterText(0, "12");
 solo.clickOnButton("Convert!");
 assertTrue(solo.searchText("53.6"));
 }

Slide 64

Slide 64 text

Robotium setup Robotium dependency Ŷ Uses SDK test runner Ŷ Robotium dependency dependencies { ! androidTestCompile ‘com.jayway.android.robotium:robotium-solo:5.2.1’ ! }

Slide 65

Slide 65 text

Robotium test base class Robotium base /**
 * Base class for Activity Instrumentation tests that utilise Robotium.
 */
 public abstract class RobotiumActivityInstrumentationTestCase extends
 ActivityInstrumentationTestCase2 {
 
 protected Solo solo;
 
 public RobotiumActivityInstrumentationTestCase(Class activityClass) {
 super(activityClass);
 }
 
 @Override
 protected void setUp() throws Exception {
 super.setUp();
 solo = new Solo(getInstrumentation(), getActivity());
 }
 
 @Override
 protected void tearDown() throws Exception {
 super.tearDown();
 solo.finishOpenedActivities();
 }
 }

Slide 66

Slide 66 text

Robotium: ListView Robotium list public void testClickOnHowardInlist() {
 solo.clickOnText("Howard Wolowitz");
 }
 
 public void testClickOnSecondItemInList() {
 solo.clickInList(2);
 }

Slide 67

Slide 67 text

Robotium: menu item Robotium menu item public void testClickInfoMenuItem() {
 // open the ActionBar overflow
 solo.sendKey(KeyEvent.KEYCODE_MENU);
 solo.clickOnText("Info");
 }

Slide 68

Slide 68 text

Robotium: intra-activity Robotium wait & see public void testClickingOnPersonShouldShowDetails() {
 int ACTIVITY_WAITING_TIMEOUT = 10000;
 // first activity should start
 assertThat(solo.waitForActivity(PersonListActivity.class, ACTIVITY_WAITING_TIMEOUT), is(true));
 // click on a person
 solo.clickOnText("Howard Wolowitz");
 
 assertThat(solo.waitForActivity(PersonDetailActivity.class, ACTIVITY_WAITING_TIMEOUT), is(true));
 }

Slide 69

Slide 69 text

Robotium errors Robotium error junit.framework.AssertionFailedError: 
 Button with text: 'test button' is not found!
 ! at com.robotium.solo.Getter.getView(Getter.java:67) at com.robotium.solo.Solo.getButton(Solo.java:1957) at com.devoxx.android.bigbang.test.RobotiumTests.testErrorMessages(RobotiumTests.java:24) public void testErrorMessages() {
 solo.getButton("test button");
 }

Slide 70

Slide 70 text

Robotium webview getWebElement(By by)
 clickOnWebElement(By by)
 enterTextInWebElement(By by) based on:
 id - tagName - textContent - className
 
 e.g. By.className(String className) Can interact with WebView Ŷ Select web content Ŷ Interact with web content Ŷ

Slide 71

Slide 71 text

Assert! Don’t assume Ŷ Assert Ŷ

Slide 72

Slide 72 text

Expressiveness 

Slide 73

Slide 73 text

Hamcrest assertThat(audience, is(payingAttention())) Verification Mocking Assertions

Slide 74

Slide 74 text

Hamcrest Library of matchers Ŷ Syntactic sugar Ŷ Assert assertThat(someString, is(equalTo(“Expected”))); actual value expectation on the value (Matcher)

Slide 75

Slide 75 text

Hamcrest Assert Hamcrest assertEquals(“Conversion failed”, “Expected”, someString); assertThat(someString, is(equalTo(“Expected”)));

Slide 76

Slide 76 text

Hamcrest: collections Assert assertEquals(person, bigbangCharacters.getPersons().iterator().next()); Hamcrest assertThat(bigbangCharacters.getPersons(), hasItem(person));

Slide 77

Slide 77 text

Hamcrest matchers Core anything
 is
 … Logical allOf
 anyOf
 not Object equalTo
 instanceOf
 (not)nullValue Custom Beans hasProperty Text equalToIgnoringCase
 startsWith
 … Number greaterThan(OrEqualTo)
 closeTo Collections hasItem, hasItems
 hasItemInArray

Slide 78

Slide 78 text

Hamcrest Lots of useful matchers Ŷ Custom matchers Ŷ not a number public void testSquareRootOfMinusOneIsNotANumber() {
 assertThat(Math.sqrt(-1), is(notANumber()));
 }

Slide 79

Slide 79 text

Hamcrest custom matcher Hamcrest matcher /**
 * Hamcrest matcher: test if a double has the value NaN (not a number)
 */
 public class IsNotANumber extends TypeSafeMatcher {
 
 @Override
 public boolean matchesSafely(Double number) {
 return number.isNaN();
 }
 
 public void describeTo(Description description) {
 description.appendText("not a number");
 }
 
 @Factory
 public static Matcher notANumber() {
 return new IsNotANumber();
 }
 }

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

Espresso

Slide 82

Slide 82 text

Espresso on top of Instrumentation APIs Google’s new approach Ŷ Abstraction layer Ŷ * GTAC 10/2013 Ŷ Android API level ≥ 8 Ŷ Open source Ŷ https://code.google.com/p/android-test-kit/

Slide 83

Slide 83 text

Espresso readable tests fast based on Matchers Fluent API Ŷ Extensible Ŷ Better debugging / logging Ŷ Performance Ŷ

Slide 84

Slide 84 text

Espresso Easy API for test authors Find view Do stuff with view Check some state Without • waitUntil(timeout) • boilerplate

Slide 85

Slide 85 text

Espresso API onView(Matcher) onData(Matcher) onView(Matcher) onView(Matcher) withId() / withText() is() / instanceOf() click() / enterText() / scrollTo() matches(Matcher) isDisplayed / isEnabled / hasFocus withText / … Find stuff Do stuff Check

Slide 86

Slide 86 text

Espresso Espresso onView(withId(R.id.some_id)).perform(click()); 
 
 onView(withText("Hi there”)).check(matches(isDisplayed())); 
 
 onView(withId(R.id.some_id)).check(matches(withText(containsString(“someString”)))); 
 
 onData(is(someObject)).perform(click());

Slide 87

Slide 87 text

Espresso Adapter views onData Ŷ Espresso ViewAction // search for a row and perform action
 onData(hasToString(startsWith("Sheldon"))).perform(click()); ! onData(is(instanceOf(DummyData.BigBangCharacter.class)))
 .atPosition(5)
 .perform(click()); ! // search for a row, and perform an action on a child view
 onData(hasToString(startsWith("Amy")))
 .onChildView(withId(R.id.list_item_photo))
 .perform(click());

Slide 88

Slide 88 text

Espresso swiping Espresso swiping public void testSwipingBackAndForward() {
 // Should be on position 0 to start with.
 onView(withText("Position #0")).check(matches(isDisplayed()));
 
 // Swipe left once.
 onView(withId(R.id.pager_layout)).perform(swipeLeft());
 
 // Now position 1 should be visible.
 onView(withText("Position #1")).check(matches(isDisplayed()));
 
 // Swipe back to the right.
 onView(withId(R.id.pager_layout)).perform(swipeRight());
 
 // Now position 0 should be visible again.
 onView(withText("Position #0")).check(matches(isDisplayed()));
 
 // Swipe right again.
 onView(withId(R.id.pager_layout)).perform(swipeRight());
 
 // Position 0 should still be visible as this is the first view in the pager.
 onView(withText("Position #0")).check(matches(isDisplayed()));
 }

Slide 89

Slide 89 text

Espresso scrolling Espresso scrolling // You can pass more than one action to perform. This is useful if you are performing two actions
 // back-to-back on the same view.
 public void testScrollToInScrollView() {
 onView(withId(is(R.id.bottom_view)))
 .perform(scrollTo(), click());
 }

Slide 90

Slide 90 text

Espresso API onView(Matcher) onData(Matcher) Espresso withId() withText() ViewMatchers perform() check() ViewInteraction / DataInteraction click() enterText() scrollTo() ViewActions matches(Matcher) doesNotExist() ViewAssertions find view to operate on composed of © Stephan Linzner (Google)

Slide 91

Slide 91 text

View Matchers Espresso uses Hamcrest Matchers Espresso Matchers = Hamcrest Ŷ Make assertions on views Ŷ Filter views Ŷ Reuse existing Matchers Ŷ Create your own Matchers Ŷ from hamcrest-core and hamcrest-library

Slide 92

Slide 92 text

View Matchers provides Android specific View Matchers ViewMatchers Ŷ

Slide 93

Slide 93 text

View Matchers Hamcrest Ŷ allOf() / anyOf() ➪ give more power Ŷ TestRunner // assert that textview does not start with with "XYZ" and contains "ABC" anywhere
 onView(withId(R.id.celsius_textview))
 .check(matches(allOf(withText(not(startsWith("XYZ"))), withText(containsString("ABC")))));
 // assert that textview ends with "XYZ" or contains "ABC" anywhere
 onView(withId(R.id.celsius_textview))
 .check(matches(anyOf(withText(endsWith("XYZ")), withText(containsString("ABC")))));

Slide 94

Slide 94 text

View Actions • click() • scrollTo() • writeText() Perform actions on Views Ŷ Guaranteed to run on UI thread Ŷ Create your custom Actions Ŷ Basic view actions Ŷ

Slide 95

Slide 95 text

Custom ViewAction Espresso ViewAction /**
 * Clears view text by setting {@link EditText}s text property to "".
 */
 public class ClearTextAction implements ViewAction {
 
 @Override
 public Matcher getConstraints() {
 return allOf(isDisplayed(), isAssignableFrom(EditText.class));
 }
 
 @Override
 public void perform(UiController uiController, View view) {
 ((EditText) view).setText("");
 }
 
 @Override
 public String getDescription() {
 return "Clear the EditText component";
 }
 }

Slide 96

Slide 96 text

Custom ViewAction Espresso ViewAction /**
 * Your own custom View Actions.
 */
 public class CustomViewActions {
 
 private CustomViewActions() {
 }
 
 /**
 * Returns an action that clears text on the view. Extra thoroughly!
 */
 public static ViewAction clearTextExtraThoroughly() {
 return new ClearTextAction();
 }
 }

Slide 97

Slide 97 text

Use Custom ViewAction Espresso ViewAction public void testClearCelsiusTextField() {
 onView(withText("Celsius")).perform(clearTextExtraThoroughly());
 }

Slide 98

Slide 98 text

Reliability through synchronization between Instrumentation thread and UI thread for background resources Espresso synchronizes AsyncTasks default thread pool • wait until app is idle • perform operation on the UI thread • wait until completion • check result for custom implementations
 IntentService
 HandlerThread
 ThreadPool Synchronization Ŷ Synchronization Ŷ Synchronization Ŷ AsyncTask Ŷ IdlingResource interface Ŷ

Slide 99

Slide 99 text

Espresso synchronization Espresso IdlingResource public class IdleMonitor implements IdlingResource {
 
 @Override
 public String getName() {
 return IdleMonitor.class.getSimpleName();
 }
 
 @Override
 public boolean isIdleNow() {
 // return true if resource is idle
 return false;
 }
 
 @Override
 public void registerIdleTransitionCallback(ResourceCallback resourceCallback) {
 // store a reference to the resourceCallback
 // notify resourceCallback when idle
 }
 } ! ! Espresso.registerIdlingResources(idlingResource);

Slide 100

Slide 100 text

Espresso setup + collection of dependencies bundled = Espresso and all of its dependencies Espresso dependencies Ŷ Standalone Ŷ Separate library Ŷ

Slide 101

Slide 101 text

Espresso setup Espresso dependencies dependencies {
 androidTestCompile files('libs/espresso-1.1.jar',
 'libs/testrunner-1.1.jar',
 'libs/testrunner-runtime-1.1.jar')
 
 androidTestCompile 'com.google.guava:guava:14.0.1',
 'com.squareup.dagger:dagger:1.1.0',
 'org.hamcrest:hamcrest-core:1.1',
 'org.hamcrest:hamcrest-integration:1.1',
 'org.hamcrest:hamcrest-library:1.1'
 }
 android {
 packagingOptions {
 // exclude duplicated dependency files from being packaged in the apk
 exclude 'LICENSE.txt'
 }
 }

Slide 102

Slide 102 text

Double Espresso setup Espresso dependencies dependencies {
 androidTestCompile 'com.jakewharton.espresso:espresso:1.1-r3'
 androidTestCompile('com.jakewharton.espresso:espresso:1.1-r3') {
 exclude group: 'com.squareup.dagger'
 }
 androidTestCompile 'com.jakewharton.espresso:espresso-support-v4:1.1-r3'
 }
 android {
 packagingOptions {
 // exclude duplicated dependency files from being packaged in the apk
 exclude 'LICENSE.txt'
 }
 }

Slide 103

Slide 103 text

Test runner Custom test runner Ŷ GoogleInstrumentationTestRunner Ŷ TestRunner android {
 defaultConfig {
 testInstrumentationRunner "com.google.android.apps.common.testing.testrunner.GoogleInstrumentationTestRunner"
 }
 }

Slide 104

Slide 104 text

Custom test runner Application.onCreate() always returns before 
 Instrumentation.onStart() before Instrumentation exits Lifecycle control Ŷ Ensure all activities finish Ŷ Activity lifecycle monitoring Ŷ Improved error logging Ŷ Improved stability / reliability Ŷ Can be used with other libraries Ŷ

Slide 105

Slide 105 text

TESTING ANNOUNCEMENT

Slide 106

Slide 106 text

android.support.test.* for all Google testing frameworks Unbundled static testing library Ŷ Android support library Ŷ Full Gradle support Ŷ Open source / AOSP Ŷ Container Ŷ

Slide 107

Slide 107 text

Testing support library AndroidJUnitRunner Ŷ Intento Ŷ UiAutomator Ŷ Espresso Ŷ

Slide 108

Slide 108 text

JUnit 4 support @Parameters Test annotations Ŷ Test suite annotations Ŷ Parameterized tests Ŷ @Suite.SuiteClasses @Test @Before @After @BeforeClass @AfterClass

Slide 109

Slide 109 text

Support testing library usage Espresso dependencies dependencies {
 androidTestCompile ‘android.test.support:support-lib:?’
 androidTestCompile ‘android.test.support:espresso-lib:1.2’ androidTestCompile ‘android.test.support:espresso-contrib:1.2’ androidTestCompile ‘android.test.support:idling-resource:1.2’
 }

Slide 110

Slide 110 text

Support testing library usage Test runner Ŷ TestRunner android {
 defaultConfig {
 testInstrumentationRunner "android.support.test.runner.AndroidTestRunner "
 }
 }

Slide 111

Slide 111 text

Espresso errors Espresso error com.google.android.apps.common.testing.ui.espresso.NoMatchingViewException: No views in hierarchy found matching: with text: is "Celcius" ! View Hierarchy: +>DecorView{id=-1, visibility=VISIBLE, width=768, height=1184, has-focus=true, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=1} | +->ActionBarOverlayLayout{id=16909075, res-name=action_bar_overlay_layout, visibility=VISIBLE, width=768, height=1184, has-focus=true, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=3} | … +---->ActionMenuView{id=-1, visibility=VISIBLE, width=112, height=96, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=656.0, y=0.0, child-count=1} | +----->OverflowMenuButton{id=-1, desc=More options, visibility=VISIBLE, width=112, height=96, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=true, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0} | +--->ActionBarContextView{id=16909078, res-name=action_context_bar, visibility=GONE, width=0, height=0, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=true, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=0} | +-->ActionBarContainer{id=16909079, res-name=split_action_bar, visibility=GONE, width=0, height=0, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=true, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=0} |

Slide 112

Slide 112 text

Intento: inter-application testing through custom Instrumentation verify that a given Intent has been sent stub outgoing Intents stub the result from a broadcasted Intent Intercept Intents Ŷ Records all outgoing Intents Ŷ Intent stubbing in tests Ŷ Intent validation Ŷ

Slide 113

Slide 113 text

Robolectric

Slide 114

Slide 114 text

Android testing Android core libraries depend upon the actual Android operating system

Slide 115

Slide 115 text

Android testing Android core libraries depend upon the actual Android operating system

Slide 116

Slide 116 text

Android testing AndroidTestCase needs an instance of the emulator to run • spin up emulator • deploy the APK • run the actual tests Big time sink: Ŷ

Slide 117

Slide 117 text

Robolectric that would otherwise need an emulator / device (mock implementations of Android core libraries) too slow / hard for unit testing without the baggage of a device Replaces the behavior of code Ŷ Write JUnit tests Ŷ When? Ŷ Does this by using Shadow classes Ŷ InstrumentationTestCase Ŷ vs real device / speedy emulator Ŷ http://robolectric.org/

Slide 118

Slide 118 text

Robolectric • unit testing fast and easy • make your own Shadow objects • JUnit 4 supported • does not cover all functionality (sensors, OpenGL, …) ➪ device needed • integration testing (interaction Activities - Services, camera app, …) Advantages Ŷ Disadvantages Ŷ

Slide 119

Slide 119 text

Robolectric Robolectric 
 public class MyActivityTest {
 
 @Test
 public void clickingButton_shouldChangeResultsViewText() throws Exception {
 MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();
 
 Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);
 TextView results = (TextView) activity.findViewById(R.id.results_text_view);
 
 pressMeButton.performClick();
 String resultsText = results.getText().toString();
 assertThat(resultsText, equalTo("Testing Android Rocks!"));
 }

Slide 120

Slide 120 text

Robolectric setup • Robolectric • Novoda gradle-android-test-plugin see Paul Blundell Tests run inside the JVM Ŷ Gradle - Maven Ŷ Configure it yourself Ŷ Gradle plugin Ŷ http://blog.blundell-apps.com/android-gradle-app-with-robolectric-junit-tests/

Slide 121

Slide 121 text

Robolectric setup

Slide 122

Slide 122 text

Robolectric setup Robolectric setup apply plugin: 'java'
 
 dependencies {
 def androidModule = project(':appfoundry-app')
 compile androidModule
 
 testCompile rootProject.ext.libraries.junit
 testCompile rootProject.ext.libraries.robolectric
 
 testCompile androidModule.android.applicationVariants.toList().first().javaCompile.classpath
 testCompile androidModule.android.applicationVariants.toList().first().javaCompile.outputs.files
 testCompile files(androidModule.plugins.findPlugin("com.android.application").getBootClasspath())
 
 
 }

Slide 123

Slide 123 text

Robolectric setup where your AndroidManifest and resources are Custom test runner Ŷ tell Robolectric Ŷ

Slide 124

Slide 124 text

Robolectric setup Robolectric test runner public class RobolectricGradleTestRunner extends RobolectricTestRunner {
 
 private static final int MAX_SDK_SUPPORTED_BY_ROBOLECTRIC = 18;
 
 public RobolectricGradleTestRunner(Class> testClass) throws InitializationError {
 super(testClass);
 }
 
 @Override
 protected AndroidManifest getAppManifest(Config config) {
 String manifestProperty = "../appfoundry-app/src/main/AndroidManifest.xml";
 String resProperty = "../appfoundry-app/src/main/res";
 return new AndroidManifest(Fs.fileFromPath(manifestProperty), Fs.fileFromPath(resProperty)) {
 @Override
 public int getTargetSdkVersion() {
 return MAX_SDK_SUPPORTED_BY_ROBOLECTRIC;
 }
 };
 }
 }

Slide 125

Slide 125 text

Robolectric Annotate your class @RunWith(RobolectricTestRunner.class)
 public class MyActivityTest {
 
 @Test
 public void clickingButton_shouldChangeResultsViewText() throws Exception {
 MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();
 
 Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);
 TextView results = (TextView) activity.findViewById(R.id.results_text_view);
 
 pressMeButton.performClick();
 String resultsText = results.getText().toString();
 assertThat(resultsText, equalTo("Testing Android Rocks!"));
 }

Slide 126

Slide 126 text

Robolectric Robolectric @RunWith(RobolectricTestRunner.class)
 public class MyActivityTest {
 
 @Test
 public void pressingSubmitButtonShouldStartSecondActivity() throws Exception {
 pressMeButton.performClick();
 
 ShadowActivity shadowActivity = shadowOf(activity);
 Intent startedIntent = shadowActivity.getNextStartedActivity();
 ShadowIntent shadowIntent = shadowOf(startedIntent);
 assertThat(shadowIntent.getComponent().getClassName(), equalTo(MyActivity.class.getName()));
 }

Slide 127

Slide 127 text

Robolectric

Slide 128

Slide 128 text

Android testing JUnit 4 Support JUnit 4 test @RunWith(JUnit4.class)
 public class AndroidJUnit4Test {
 
 Conference devoxx;
 
 @Before
 public void setUp() {
 devoxx = Conferences.newInstance(Conference.DEVOXX);
 devoxx.start();
 }
 
 @After
 public void tearDown() {
 devoxx.stop();
 }
 
 @Test
 public void testPreconditions() {
 assertThat(devoxx, is(notNullValue()));
 }
 
 @Test
 public void devoxxShouldBeGreat() {
 assertThat(devoxx, is(awesome()));
 }

Slide 129

Slide 129 text

Mocking & Stubbing

Slide 130

Slide 130 text

Mockito in isolation Change runtime implementation Ŷ That can be predictably tested Ŷ Verify behaviour Ŷ Mockito Ŷ

Slide 131

Slide 131 text

Mockito the behaviour of your components during a test Java mocking framework Ŷ Mock dependencies Ŷ Inject Mocks to validate Ŷ Mock function return values Ŷ https://code.google.com/p/mockito/

Slide 132

Slide 132 text

Mockito Mockito @Test
 public void testMockedConverter() throws Exception {
 // expectations
 final float expected_result = 32.0f;
 
 // given
 ConverterActivity activity = Robolectric.buildActivity(ConverterActivity.class).create().get();
 activity.onCreate(new Bundle());
 EditText celsiusField = (EditText) activity.findViewById(R.id.celsius_textview);
 celsiusField.setText("0");
 
 CelsiusFahrenheitConverter mockConverter = Mockito.mock(CelsiusFahrenheitConverter.class);
 Mockito.when(mockConverter.convertCelsiusToFahrenheit(Mockito.anyFloat())).thenReturn(expected_result);
 activity.setConverter(mockConverter);
 
 // when
 Button convertButton = (Button) activity.findViewById(R.id.conversion_button);
 convertButton.performClick();
 
 // then
 Mockito.verify(mockConverter, Mockito.times(1)).convertCelsiusToFahrenheit(Mockito.anyFloat());
 TextView fahrenheitField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 String fahrenheitString = fahrenheitField.getText().toString();
 assertThat(fahrenheitString, equalTo(String.valueOf(expected_result)));
 }

Slide 133

Slide 133 text

The Monkey

Slide 134

Slide 134 text

evil monkey…

Slide 135

Slide 135 text

Monkey pseudo-random stream of user events Generates events Ŷ clicks Ŷ touches Ŷ gestures Ŷ system-level events Ŷ Stress test your app Ŷ

Slide 136

Slide 136 text

Monkey number of events to attempt operational constraints, such as restricting the test to a single package Event types and frequencies command-line tool Ŷ basic configuration Ŷ $ adb shell monkey -p your.package.name -v 500 Ŷ

Slide 137

Slide 137 text

AssertJ Android for checking assertions to read / write http://square.github.io/assertj-android/ Syntactic sugar Ŷ Extension of the AssertJ library Ŷ Makes tests easier Ŷ Fluent syntax Ŷ

Slide 138

Slide 138 text

AssertJ Android Assertions for Android objects Ŷ + support libraries Ŷ

Slide 139

Slide 139 text

AssertJ Android Regular JUnit AssertJ Android assertEquals(View.GONE, view.getVisibility()); assertThat(view).isGone(); expected: <8> but was: <4> Expected visibility but was .

Slide 140

Slide 140 text

AssertJ Android Regular JUnit assertEquals(View.VISIBLE, layout.getVisibility()); assertEquals(VERTICAL, layout.getOrientation()); assertEquals(4, layout.getChildCount()); assertEquals(SHOW_DIVIDERS_MIDDLE, layout.getShowDividers()); AssertJ Android assertThat(layout).isVisible() .isVertical() .hasChildCount(4) .hasShowDividers(SHOW_DIVIDERS_MIDDLE);

Slide 141

Slide 141 text

Testing network resources

Slide 142

Slide 142 text

Testing network resources Unit tests Ŷ Integration tests Ŷ Retrofit Ŷ

Slide 143

Slide 143 text

Testing network resources how to trigger edge cases like API rate limit exceeded? HTTP calls should be asynchronous … Hit a real server? Ŷ Brittle tests! Ŷ Slow tests Ŷ Network or server is down Ŷ Incomplete tests Ŷ Further complicated in Android Ŷ

Slide 144

Slide 144 text

Testing network resources Use a test server Ŷ Mock the Retrofit interface Ŷ

Slide 145

Slide 145 text

Using a test server • virtually no changes or code required in your app codebase • can be shared across multiple platforms (Android, iOS, web, …) • another dependency (that can fail) • something new to figure out • not always easy to trigger errors or edge cases • what if ‘state’ is needed? • ‘slow’ tests execution, still HTTP calls needed PROs Ŷ CONs Ŷ

Slide 146

Slide 146 text

Mocking the Retrofit interface • robust: no network errors, timeouts, … • easy to generate errors / edge cases • easy to control server-side state • write your dummy data in Java • extra test code needed • you don’t test the JSON deserialization (but you can add unit tests) • mock a part of your application ➪ not a true integration test PROs Ŷ CONs Ŷ You trust the test suites of Retrofit, Gson, … and mock out the data

Slide 147

Slide 147 text

Network testing Mock network interface ‘Fake’-‘real’ server

Slide 148

Slide 148 text

Mocking the Retrofit interface MockRestAdapter sample

Slide 149

Slide 149 text

Mocking the Retrofit interface Retrofit mock public interface GitHub {
 @GET("/repos/{owner}/{repo}/contributors")
 List contributors(@Path("owner") String owner, @Path("repo") String repo);
 }

Slide 150

Slide 150 text

Mocking the Retrofit interface Canned data /** A mock implementation of the {@link GitHub} API interface. */
 static class MockGitHub implements GitHub {
 private final Map>> ownerRepoContributors;
 
 public MockGitHub() {
 ownerRepoContributors = new LinkedHashMap>>();
 
 // Seed some mock data.
 addContributor("square", "retrofit", "John Doe", 12);
 addContributor("square", "retrofit", "Bob Smith", 2);
 addContributor("square", "retrofit", "Big Bird", 40);
 addContributor("square", "picasso", "Proposition Joe", 39);
 addContributor("square", "picasso", "Keiser Soze", 152);
 }

Slide 151

Slide 151 text

Mocking the Retrofit interface Mock @Override public List contributors(String owner, String repo) {
 Map> repoContributors = ownerRepoContributors.get(owner);
 if (repoContributors == null) {
 return Collections.emptyList();
 }
 List contributors = repoContributors.get(repo);
 if (contributors == null) {
 return Collections.emptyList();
 }
 return contributors;
 }

Slide 152

Slide 152 text

Mocking the Retrofit interface Mock // Create a very simple REST adapter which points the GitHub API endpoint.
 RestAdapter restAdapter = new RestAdapter.Builder()
 .setEndpoint(API_URL)
 .build();
 
 // Wrap our REST adapter to allow mock implementations and fake network delay.
 MockRestAdapter mockRestAdapter = MockRestAdapter.from(restAdapter);
 
 // Instantiate a mock object so we can interact with it later.
 MockGitHub mockGitHub = new MockGitHub();
 // Use the mock REST adapter and our mock object to create the API interface.
 GitHub gitHub = mockRestAdapter.create(GitHub.class, mockGitHub);
 
 // Query for some contributors for a few repositories.
 printContributors(gitHub, "square", "retrofit");
 printContributors(gitHub, "square", "picasso");

Slide 153

Slide 153 text

Network testing Mock network interface ‘Fake’-‘real’ server

Slide 154

Slide 154 text

Wiremock predictable requests-responses wiremock.org Ŷ Tom Akehurst Ŷ Reproduce real-world scenarios Ŷ Increase reliability Ŷ Stateful scenarios Ŷ Non-Android specific Ŷ

Slide 155

Slide 155 text

Wiremock via HTTP/JSON API • _admin/mappings/new • mappings folder HTTP server Ŷ configure responses Ŷ use with Java API Ŷ

Slide 156

Slide 156 text

Wiremock Wiremock JSON API { "request": { "method": "GET", "url": "/api/mytest" }, "response": { "status": 200, "body": "More content\n" } } wiremock.org documentation

Slide 157

Slide 157 text

Wiremock Wiremock Java API stubFor(get(urlEqualTo("/some/thing")) .willReturn(aResponse() .withStatus(404) .withBody(“No!!”))); wiremock.org documentation

Slide 158

Slide 158 text

Wiremock Wiremock Java test @Test public void exampleTest() { // make your app do requests to http://localhost:8080 stubFor(get(urlEqualTo("/my/resource")) .willReturn(aResponse() .withStatus(200) .withBody("Some content"))); ! onView(withId(R.id.submit_button)).perform(click()); onView(withText(“error message”)).check(matches(isDisplayed())); ! verify(postRequestedFor(urlMatching(“/my/resource/[a-z0-9]+”))); } wiremock.org documentation

Slide 159

Slide 159 text

Wiremock … out-of-the-box Android support is coming… Stateful Ŷ Errors / delays Ŷ Wiremock in the test APK Ŷ Wiremock as server on network Ŷ

Slide 160

Slide 160 text

Application ARCHITECTURE

Slide 161

Slide 161 text

No content

Slide 162

Slide 162 text

Software design patterns Good code = testable code Ŷ Dependency injection Ŷ

Slide 163

Slide 163 text

Software design patterns Architecting Android Applications with Dagger Dependency injection Ŷ Dagger Ŷ Avoid reflection at runtime Ŷ Define dependencies Ŷ Jake Wharton Ŷ

Slide 164

Slide 164 text

AUTOMATION &
 REPORTING

Slide 165

Slide 165 text

Automated distributed testing Automated distributed instrumentation testing Screenshots share with the designers…? in parallel Run tests on multiple devices Reports / logs captured for each test

Slide 166

Slide 166 text

Spoon https://square.github.io/spoon/ Automate test execution Ŷ across multiple devices Ŷ Aggregation of screenshots Ŷ Aggregate the results Ŷ

Slide 167

Slide 167 text

Spoon High level tests overview Ŷ Test failure specific Ŷ to a single device or all devices

Slide 168

Slide 168 text

Spoon: device view Results of each test on one device Ŷ Useful for tracking down Ŷ device-specific failures of individual tests

Slide 169

Slide 169 text

Spoon: test view Results of single test on all devices Ŷ

Slide 170

Slide 170 text

Spoon: screenshots Visual inspection of test execution Ŷ screenshots Spoon.screenshot(activity, "initial_state"); /* Normal test code... */ Spoon.screenshot(activity, "after_login"); http://square.github.io/spoon/sample/

Slide 171

Slide 171 text

Build pipeline Commit stage Automated test/QA stage Manual QA testing UAT test Production Capacity / load testing

Slide 172

Slide 172 text

Build pipeline Checkout / compile Unit tests Test coverage Code analysis Create deployable artifact Deploy for automatic QA test Trigger automated QA stage

Slide 173

Slide 173 text

Build pipeline tools Build (maven - gradle) Dependency repo (nexus - artifactory) Testing framework (JUnit - ...) Test coverage (Cobertura - Emma - Jacoco) Code analysis (Checkstyle, findbugs, pmd, Android Lint) Creation of deployable artifact (buildtool, artifact repo) Trigger next stage

Slide 174

Slide 174 text

Continuous integration benefits without additional effort Fast feedback - fewer errors Ŷ Test everything on every (nightly) build Ŷ Regression tests Ŷ Less manual testing Ŷ

Slide 175

Slide 175 text

Jenkins

Slide 176

Slide 176 text

Jenkins

Slide 177

Slide 177 text

Continuous integration • Jenkins • Bamboo Travis … Server Ŷ Hosted Ŷ https://travis-ci.org

Slide 178

Slide 178 text

Test coverage open-source toolkit for measuring and reporting Java code coverage Metrics Ŷ Code safely Ŷ JaCoCo Ŷ Find risky code Ŷ

Slide 179

Slide 179 text

Android test coverage

Slide 180

Slide 180 text

Coverage 88%

Slide 181

Slide 181 text

References

Slide 182

Slide 182 text

Suggested reading Android Application Testing Guide Diego Torres Milano (9781849513500) Robotium Automated Testing for Android Hrushikesh Zadgaonkar (9781782168010) The Busy Coder’s Guide to Android Development Mark Murphy - http://commonsware.com/Android/

Slide 183

Slide 183 text

Suggested reading Test Driven Development: By Example Beck, Kent (978-0321146533) Continuous Integration: Improving Software Quality and Reducing Risk Duvall, Paul M. et al. (978-0321336385) Working Effectively with Legacy Code Feathers, Michael (978-0131177055)

Slide 184

Slide 184 text

Questions? Filip Maelbrancke Consultant @ AppFoundry ! fi[email protected] @fmaelbrancke