Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing your Android application at jDays 2015

Testing your Android application at jDays 2015

Slides from the 'Testing your Android app' session at jDays 2015 in Göteborg, Sweden

Filip Maelbrancke

March 18, 2015

More Decks by Filip Maelbrancke

Other Decks in Programming


  1. “Computers are designed to do simple repetitive tasks. The second

    you have humans doing repetitive tasks, all the computers get together late at night and laugh at you...” Neil Ford
  2. EXCUSES We never make mistakes! The functionality is trivial Tests

    slow us down Management won’t let us WE = HERO TRIVIAL SLOW MANAGEMENT ǩ
  3. Repeatability (no matter what development stack he uses) in which

    they are being run Tests need to be run by every developer ♂ Tests must not rely on the environment
  4. Instrumentation Test Activity state saving and restoration // Start the

    main activity of the application under test
 mActivity = getActivity();
 // Get a handle to the Activity object's main UI widget, a Spinner
 mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner);
 // Set the Spinner to a known position
 // Stop the activity - The onDestroy() method should save the state of the Spinner
 // Re-start the Activity - the onResume() method should restore the state of the Spinner
 mActivity = getActivity();
 // Get the Spinner's current position
 int currentPosition = mActivity.getSpinnerPosition();
 // Assert that the current position is the same as the starting position
 assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition);
  5. Self-documenting Testable code = clear Testable code = easy to

    follow we can just look at the test No need to explain how a component works No need to write documentation tests = usage examples
  6. android.support.test.* for all Google testing frameworks Unbundled static testing library

    Ŷ Android support library Ŷ Full Gradle support Ŷ Open source / AOSP Ŷ Container Ŷ
  7. JUnit 4 support @Parameters Test annotations Ŷ Test suite annotations

    Ŷ Parameterized tests Ŷ @Suite.SuiteClasses @Test @Before @After @BeforeClass @AfterClass
  8. Espresso Easy API for test authors Find view Do stuff

    with view Check some state Without • waitUntil(timeout) • boilerplate
  9. Espresso API onView(Matcher<View>) onData(Matcher<Object>) onView(Matcher<View>) onView(Matcher<View>) withId() / withText() is()

    / instanceOf() click() / enterText() / scrollTo() matches(Matcher<View>) isDisplayed / isEnabled / hasFocus withText / … Find stuff Do stuff Check
  10. Espresso API onView(Matcher<View>) onData(Matcher<Object>) Espresso withId() withText() ViewMatchers perform() check()

    ViewInteraction / DataInteraction click() enterText() scrollTo() ViewActions matches(Matcher<View>) doesNotExist() ViewAssertions find view to operate on composed of © Stephan Linzner (Google)
  11. View Matchers Espresso uses Hamcrest Matchers Espresso Matchers = Hamcrest

    Ŷ Make assertions on views Ŷ Filter views Ŷ Reuse existing Matchers Ŷ Create your own Matchers Ŷ from hamcrest-core and hamcrest-library
  12. View Matchers provides Android specific View Matchers ViewMatchers Ŷ •

    click() • scrollTo() • writeText() Basic view actions Ŷ
  13. Hamcrest Library of matchers Ŷ Syntactic sugar Ŷ Assert assertThat(someString,

    is(equalTo(“Expected”))); actual value expectation on the value (Matcher)
  14. Hamcrest matchers Core anything
 … Logical allOf

    Object equalTo
 (not)nullValue Custom Beans hasProperty Text equalToIgnoringCase
 … Number greaterThan(OrEqualTo)
 closeTo Collections hasItem, hasItems
  15. View Matchers Hamcrest Ŷ allOf() / anyOf() ➪ give more

    power Ŷ TestRunner // assert that textview does not start with with "XYZ" and contains "ABC" anywhere
 .check(matches(allOf(withText(not(startsWith("XYZ"))), withText(containsString("ABC")))));
 // assert that textview ends with "XYZ" or contains "ABC" anywhere
 .check(matches(anyOf(withText(endsWith("XYZ")), withText(containsString("ABC")))));
  16. Hamcrest Lots of useful matchers Ŷ Custom matchers Ŷ not

    a number public void testSquareRootOfMinusOneIsNotANumber() {
 assertThat(Math.sqrt(-1), is(notANumber()));
  17. Hamcrest custom matcher Hamcrest matcher /**
 * Hamcrest matcher: test

    if a double has the value NaN (not a number)
 public class IsNotANumber extends TypeSafeMatcher<Double> {
 public boolean matchesSafely(Double number) {
 return number.isNaN();
 public void describeTo(Description description) {
 description.appendText("not a number");
 public static <T> Matcher<Double> notANumber() {
 return new IsNotANumber();
  18. Custom ViewAction Espresso ViewAction /**
 * Clears view text by

    setting {@link EditText}s text property to "".
 public class ClearTextAction implements ViewAction {
 public Matcher<View> getConstraints() {
 return allOf(isDisplayed(), isAssignableFrom(EditText.class));
 public void perform(UiController uiController, View view) {
 ((EditText) view).setText("");
 public String getDescription() {
 return "Clear the EditText component";
  19. Custom ViewAction Espresso ViewAction /**
 * Your own custom View

 public class CustomViewActions {
 private CustomViewActions() {
 * Returns an action that clears text on the view. Extra thoroughly!
 public static ViewAction clearTextExtraThoroughly() {
 return new ClearTextAction();
  20. Reliability through synchronization between Instrumentation thread and UI thread for

    background resources Espresso synchronizes AsyncTasks default thread pool • wait until app is idle • perform operation on the UI thread • wait until completion • check result for custom implementations
 ThreadPool Synchronization Ŷ Synchronization Ŷ Synchronization Ŷ AsyncTask Ŷ IdlingResource interface Ŷ
  21. Espresso synchronization Espresso IdlingResource public class IdleMonitor implements IdlingResource {

 public String getName() {
 return IdleMonitor.class.getSimpleName();
 public boolean isIdleNow() {
 // return true if resource is idle
 return false;
 public void registerIdleTransitionCallback(ResourceCallback resourceCallback) {
 // store a reference to the resourceCallback
 // notify resourceCallback when idle
 } Espresso.registerIdlingResources(idlingResource);
  22. Android testing AndroidTestCase needs an instance of the emulator to

    run • spin up emulator • deploy the APK • run the actual tests Big time sink: Ŷ
  23. Robolectric that would otherwise need an emulator / device (mock

    implementations of Android core libraries) too slow / hard for unit testing without the baggage of a device Replaces the behavior of code Ŷ Write JUnit tests Ŷ When? Ŷ Does this by using Shadow classes Ŷ InstrumentationTestCase Ŷ vs real device / speedy emulator Ŷ http://robolectric.org/
  24. Robolectric Annotate your class @RunWith(RobolectricTestRunner.class)
 public class MyActivityTest {

 public void clickingButton_shouldChangeResultsViewText() throws Exception {
 MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();
 Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);
 TextView results = (TextView) activity.findViewById(R.id.results_text_view);
 String resultsText = results.getText().toString();
 assertThat(resultsText, equalTo("Testing Android Rocks!"));
  25. Robolectric setup • Robolectric • Novoda gradle-android-test-plugin see Paul Blundell

    Tests run inside the JVM Ŷ Gradle - Maven Ŷ Configure it yourself Ŷ Gradle plugin Ŷ
  26. Android testing JUnit 4 Support JUnit 4 test @RunWith(AndroidJUnit4.class)

    class AndroidJUnit4Test {
 Conference jDays;
 public void setUp() {
 jDays = Conferences.newInstance(Conference.JDAYS);
 public void tearDown() {
 public void testPreconditions() {
 assertThat(jDays, is(notNullValue()));
 public void jDaysSouldBeGreat() {
 assertThat(jDays, is(awesome()));
  27. Mockito in isolation Change runtime implementation Ŷ That can be

    predictably tested Ŷ Verify behaviour Ŷ Mockito Ŷ
  28. Mockito the behaviour of your components during a test Java

    mocking framework Ŷ Mock dependencies Ŷ Inject Mocks to validate Ŷ Mock function return values Ŷ https://code.google.com/p/mockito/
  29. Mockito Mockito @Test
 public void testMockedConverter() throws Exception {

 final float expected_result = 32.0f;
 // given
 ConverterActivity activity = Robolectric.buildActivity(ConverterActivity.class).create().get();
 activity.onCreate(new Bundle());
 EditText celsiusField = (EditText) activity.findViewById(R.id.celsius_textview);
 CelsiusFahrenheitConverter mockConverter = Mockito.mock(CelsiusFahrenheitConverter.class);
 // when
 Button convertButton = (Button) activity.findViewById(R.id.conversion_button);
 // then
 Mockito.verify(mockConverter, Mockito.times(1)).convertCelsiusToFahrenheit(Mockito.anyFloat());
 TextView fahrenheitField = (TextView) activity.findViewById(R.id.fahrenheit_textview);
 String fahrenheitString = fahrenheitField.getText().toString();
 assertThat(fahrenheitString, equalTo(String.valueOf(expected_result)));
  30. Monkey pseudo-random stream of user events Generates events Ŷ clicks

    Ŷ touches Ŷ gestures Ŷ system-level events Ŷ Stress test your app Ŷ
  31. Monkey number of events to attempt operational constraints, such as

    restricting the test to a single package Event types and frequencies command-line tool Ŷ basic configuration Ŷ $ adb shell monkey -p your.package.name -v 500 Ŷ
  32. AssertJ Android for checking assertions to read / write http://square.github.io/assertj-android/

    Syntactic sugar Ŷ Extension of the AssertJ library Ŷ Makes tests easier Ŷ Fluent syntax Ŷ
  33. AssertJ Android Regular JUnit assertEquals(View.VISIBLE, layout.getVisibility()); assertEquals(VERTICAL, layout.getOrientation()); assertEquals(4, layout.getChildCount());

    assertEquals(SHOW_DIVIDERS_MIDDLE, layout.getShowDividers()); AssertJ Android assertThat(layout).isVisible() .isVertical() .hasChildCount(4) .hasShowDividers(SHOW_DIVIDERS_MIDDLE);
  34. Testing network resources how to trigger edge cases like API

    rate limit exceeded? HTTP calls should be asynchronous … Hit a real server? Ŷ Brittle tests! Ŷ Slow tests Ŷ Network or server is down Ŷ Incomplete tests Ŷ Further complicated in Android Ŷ
  35. Using a test server • virtually no changes or code

    required in your app codebase • can be shared across multiple platforms (Android, iOS, web, …) • another dependency (that can fail) • something new to figure out • not always easy to trigger errors or edge cases • what if ‘state’ is needed? • ‘slow’ tests execution, still HTTP calls needed PROs Ŷ CONs Ŷ
  36. Mocking the Retrofit interface • robust: no network errors, timeouts,

    … • easy to generate errors / edge cases • easy to control server-side state • write your dummy data in Java • extra test code needed • you don’t test the JSON deserialization (but you can add unit tests) • mock a part of your application ➪ not a true integration test PROs Ŷ CONs Ŷ You trust the test suites of Retrofit, Gson, … and mock out the data
  37. Mocking the Retrofit interface Retrofit mock public interface GitHub {

 List<Contributor> contributors(@Path("owner") String owner, @Path("repo") String repo);
  38. Mocking the Retrofit interface Canned data /** A mock implementation

    of the {@link GitHub} API interface. */
 static class MockGitHub implements GitHub {
 private final Map<String, Map<String, List<Contributor>>> ownerRepoContributors;
 public MockGitHub() {
 ownerRepoContributors = new LinkedHashMap<String, Map<String, List<Contributor>>>();
 // Seed some mock data.
 addContributor("square", "retrofit", "John Doe", 12);
 addContributor("square", "retrofit", "Bob Smith", 2);
 addContributor("square", "retrofit", "Big Bird", 40);
 addContributor("square", "picasso", "Proposition Joe", 39);
 addContributor("square", "picasso", "Keiser Soze", 152);
  39. Mocking the Retrofit interface Mock @Override public List<Contributor> contributors(String owner,

    String repo) {
 Map<String, List<Contributor>> repoContributors = ownerRepoContributors.get(owner);
 if (repoContributors == null) {
 return Collections.emptyList();
 List<Contributor> contributors = repoContributors.get(repo);
 if (contributors == null) {
 return Collections.emptyList();
 return contributors;
  40. Mocking the Retrofit interface Mock // Create a very simple

    REST adapter which points to the GitHub API endpoint.
 RestAdapter restAdapter = new RestAdapter.Builder()
 // Wrap our REST adapter to allow mock implementations and fake network delay.
 MockRestAdapter mockRestAdapter = MockRestAdapter.from(restAdapter);
 // Instantiate a mock object so we can interact with it later.
 MockGitHub mockGitHub = new MockGitHub();
 // Use the mock REST adapter and our mock object to create the API interface.
 GitHub gitHub = mockRestAdapter.create(GitHub.class, mockGitHub);
 // Query for some contributors for a few repositories.
 printContributors(gitHub, "square", "retrofit");
 printContributors(gitHub, "square", "picasso");
  41. MockWebServer predictable requests-responses Scriptable web server Ŷ okhttp (square) Ŷ

    Specify responses Ŷ Test HTTP clients Ŷ Verify requests Ŷ Like Mockito (for HTTP stack) Ŷ • script your mocks • run app code • verify that expected requests were made
  42. MockWebServer MockWebServer // Create a MockWebServer. These are lean enough

    that you can create // a new instance for every unit test. MockWebServer mockWebServer = new MockWebServer();
 // Setup the MockWebServer & schedule a response
 MockResponse mockResponse = new MockResponse();
 // Setup your networking lib. Ask the server for its url.
 … mockWebServer.getUrl(“/“);
  43. MockWebServer Java test @Test public void exampleTest() { // make

    your app do requests to http://localhost:8080 // setup MockWebServer onView(withId(R.id.submit_button)).perform(click()); onView(withText(“error message”)).check(matches(isDisplayed())); verify(postRequestedFor(urlMatching(“/my/resource/[a-z0-9]+”))); }
  44. Software design patterns Architecting Android Applications with Dagger Dependency injection

    Ŷ Dagger Ŷ Avoid reflection at runtime Ŷ Define dependencies Ŷ Jake Wharton Ŷ https://parleys.com/play/529bde2ce4b0e619540cc3ae
  45. Automated distributed testing Automated distributed instrumentation testing Screenshots share with

    the designers…? in parallel Run tests on multiple devices Reports / logs captured for each test
  46. Spoon: device view Results of each test on one device

    Ŷ Useful for tracking down Ŷ device-specific failures of individual tests
  47. Spoon: screenshots Visual inspection of test execution Ŷ screenshots Spoon.screenshot(activity,

    "initial_state"); /* Normal test code... */ Spoon.screenshot(activity, "after_login"); http://square.github.io/spoon/sample/
  48. Build pipeline Checkout / compile Unit tests Test coverage Code

    analysis Create deployable artifact Deploy for automatic QA test Trigger automated QA stage
  49. Build pipeline tools Build (maven - gradle) Dependency repo (nexus

    - artifactory) Testing framework (JUnit - ...) Test coverage (Cobertura - Emma - Jacoco) Code analysis (Checkstyle, findbugs, pmd, Android Lint) Creation of deployable artifact (buildtool, artifact repo) Trigger next stage
  50. Continuous integration benefits without additional effort Fast feedback - fewer

    errors Ŷ Test everything on every (nightly) build Ŷ Regression tests Ŷ Less manual testing Ŷ
  51. Test coverage open-source toolkit for measuring and reporting Java code

    coverage Metrics Ŷ Code safely Ŷ JaCoCo Ŷ Find risky code Ŷ
  52. Suggested reading Android Application Testing Guide Diego Torres Milano (9781849513500)

    Robotium Automated Testing for Android Hrushikesh Zadgaonkar (9781782168010) The Busy Coder’s Guide to Android Development Mark Murphy - http://commonsware.com/Android/
  53. Suggested reading Test Driven Development: By Example Beck, Kent (978-0321146533)

    Continuous Integration: Improving Software Quality and Reducing Risk Duvall, Paul M. et al. (978-0321336385) Working Effectively with Legacy Code Feathers, Michael (978-0131177055)