Testing your Android application at jDays 2015

Slide 1

Slide 1 text

Test-Driven Android Development

Slide 2

Slide 2 text

Android & Testing 01 02 03 Why? What? How?

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Your host Filip Maelbrancke Consultant @ AppFoundry ﬁ[email protected] @fmaelbrancke

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Testing Ship with confidence

Slide 8

Slide 8 text

TESTING Manual testing is tedious Humans are not very good at testing But computers are...

Slide 9

Slide 9 text

“Computers are designed to do simple repetitive tasks. The second you have humans doing repetitive tasks, all the computers get together late at night and laugh at you...” Neil Ford

Slide 10

Slide 10 text

Manual vs Automated

Slide 11

Slide 11 text

Mitigate issues? Be pro-active… and test.

Slide 12

Slide 12 text

EXCUSES We never make mistakes! The functionality is trivial Tests slow us down Management won’t let us WE = HERO TRIVIAL SLOW MANAGEMENT ǩ

Slide 13

Slide 13 text

TEST TRAITS (UNIT) TESTS ISOLATED REPEATABLE FAST SELF-DOCUMENTING ž

Slide 14

Slide 14 text

Repeatability (no matter what development stack he uses) in which they are being run Tests need to be run by every developer ♂ Tests must not rely on the environment

Slide 15

Slide 15 text

Android dev

Slide 16

Slide 16 text

d.android.com

Slide 17

Slide 17 text

Instrumentation Process Test package InstrumentationTestRunner Application package Test case classes JUnit

Slide 18

Slide 18 text

Instrumentation Test Activity state saving and restoration // Start the main activity of the application under test  mActivity = getActivity();  // Get a handle to the Activity object's main UI widget, a Spinner  mSpinner = (Spinner)mActivity.ﬁndViewById(com.android.example.spinner.R.id.Spinner);  // Set the Spinner to a known position  mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION);  // Stop the activity - The onDestroy() method should save the state of the Spinner  mActivity.ﬁnish();  // Re-start the Activity - the onResume() method should restore the state of the Spinner  mActivity = getActivity();  // Get the Spinner's current position  int currentPosition = mActivity.getSpinnerPosition();  // Assert that the current position is the same as the starting position  assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition);

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

TEST TRAITS (UNIT) TESTS ISOLATED REPEATABLE FAST SELF-DOCUMENTING ž

Slide 21

Slide 21 text

Self-documenting Testable code = clear Testable code = easy to follow we can just look at the test No need to explain how a component works No need to write documentation tests = usage examples

Slide 22

Slide 22 text

Tests should also be … Easy to execute Easy to debug Realistic Fluent API

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

TESTING ANNOUNCEMENT

Slide 25

Slide 25 text

android.support.test.* for all Google testing frameworks Unbundled static testing library Ŷ Android support library Ŷ Full Gradle support Ŷ Open source / AOSP Ŷ Container Ŷ

Slide 26

Slide 26 text

Testing support library AndroidJUnitRunner Ŷ Intento Ŷ UiAutomator Ŷ Espresso Ŷ

Slide 27

Slide 27 text

JUnit 4 support @Parameters Test annotations Ŷ Test suite annotations Ŷ Parameterized tests Ŷ @Suite.SuiteClasses @Test @Before @After @BeforeClass @AfterClass

Slide 28

Slide 28 text

Testing demo Simple master-detail app ☼

Slide 29

Slide 29 text

Espresso

Slide 30

Slide 30 text

Espresso Easy API for test authors Find view Do stuff with view Check some state Without • waitUntil(timeout) • boilerplate

Slide 31

Slide 31 text

Espresso API onView(Matcher) onData(Matcher) onView(Matcher) onView(Matcher) withId() / withText() is() / instanceOf() click() / enterText() / scrollTo() matches(Matcher) isDisplayed / isEnabled / hasFocus withText / … Find stuff Do stuff Check

Slide 32

Slide 32 text

Espresso Espresso onView(withId(R.id.some_id)).perform(click());     onView(withText("Hi there”)).check(matches(isDisplayed()));     onView(withId(R.id.some_id)).check(matches(withText(containsString(“someString”))));     onData(is(someObject)).perform(click());

Slide 33

Slide 33 text

Espresso API onView(Matcher) onData(Matcher) Espresso withId() withText() ViewMatchers perform() check() ViewInteraction / DataInteraction click() enterText() scrollTo() ViewActions matches(Matcher) doesNotExist() ViewAssertions find view to operate on composed of © Stephan Linzner (Google)

Slide 34

Slide 34 text

View Matchers Espresso uses Hamcrest Matchers Espresso Matchers = Hamcrest Ŷ Make assertions on views Ŷ Filter views Ŷ Reuse existing Matchers Ŷ Create your own Matchers Ŷ from hamcrest-core and hamcrest-library

Slide 35

Slide 35 text

View Matchers provides Android speciﬁc View Matchers ViewMatchers Ŷ • click() • scrollTo() • writeText() Basic view actions Ŷ

Slide 36

Slide 36 text

Expressiveness 

Slide 37

Slide 37 text

Hamcrest assertThat(audience, is(payingAttention())) Verification Mocking Assertions

Slide 38

Slide 38 text

Hamcrest Library of matchers Ŷ Syntactic sugar Ŷ Assert assertThat(someString, is(equalTo(“Expected”))); actual value expectation on the value (Matcher)

Slide 39

Slide 39 text

Hamcrest Assert Hamcrest assertEquals(“Conversion failed”, “Expected”, someString); assertThat(someString, is(equalTo(“Expected”)));

Slide 40

Slide 40 text

Hamcrest: collections Assert assertEquals(person, bigbangCharacters.getPersons().iterator().next()); Hamcrest assertThat(bigbangCharacters.getPersons(), hasItem(person));

Slide 41

Slide 41 text

Hamcrest matchers Core anything  is  … Logical allOf  anyOf  not Object equalTo  instanceOf  (not)nullValue Custom Beans hasProperty Text equalToIgnoringCase  startsWith  … Number greaterThan(OrEqualTo)  closeTo Collections hasItem, hasItems  hasItemInArray

Slide 42

Slide 42 text

View Matchers Hamcrest Ŷ allOf() / anyOf() ➪ give more power Ŷ TestRunner // assert that textview does not start with with "XYZ" and contains "ABC" anywhere  onView(withId(R.id.celsius_textview))  .check(matches(allOf(withText(not(startsWith("XYZ"))), withText(containsString("ABC")))));  // assert that textview ends with "XYZ" or contains "ABC" anywhere  onView(withId(R.id.celsius_textview))  .check(matches(anyOf(withText(endsWith("XYZ")), withText(containsString("ABC")))));

Slide 43

Slide 43 text

Hamcrest Lots of useful matchers Ŷ Custom matchers Ŷ not a number public void testSquareRootOfMinusOneIsNotANumber() {  assertThat(Math.sqrt(-1), is(notANumber()));  }

Slide 44

Slide 44 text

Hamcrest custom matcher Hamcrest matcher /**  * Hamcrest matcher: test if a double has the value NaN (not a number)  */  public class IsNotANumber extends TypeSafeMatcher {    @Override  public boolean matchesSafely(Double number) {  return number.isNaN();  }    public void describeTo(Description description) {  description.appendText("not a number");  }    @Factory  public static Matcher notANumber() {  return new IsNotANumber();  }  }

Slide 45

Slide 45 text

Testing demo Testing a temperature converter ☼

Slide 46

Slide 46 text

Custom ViewAction Espresso ViewAction /**  * Clears view text by setting {@link EditText}s text property to "".  */  public class ClearTextAction implements ViewAction {    @Override  public Matcher getConstraints() {  return allOf(isDisplayed(), isAssignableFrom(EditText.class));  }    @Override  public void perform(UiController uiController, View view) {  ((EditText) view).setText("");  }    @Override  public String getDescription() {  return "Clear the EditText component";  }  }

Slide 47

Slide 47 text

Custom ViewAction Espresso ViewAction /**  * Your own custom View Actions.  */  public class CustomViewActions {    private CustomViewActions() {  }    /**  * Returns an action that clears text on the view. Extra thoroughly!  */  public static ViewAction clearTextExtraThoroughly() {  return new ClearTextAction();  }  }

Slide 48

Slide 48 text

Use Custom ViewAction Espresso ViewAction public void testClearCelsiusTextField() {  onView(withText("Celsius")).perform(clearTextExtraThoroughly());  }

Slide 49

Slide 49 text

Reliability through synchronization between Instrumentation thread and UI thread for background resources Espresso synchronizes AsyncTasks default thread pool • wait until app is idle • perform operation on the UI thread • wait until completion • check result for custom implementations  IntentService  HandlerThread  ThreadPool Synchronization Ŷ Synchronization Ŷ Synchronization Ŷ AsyncTask Ŷ IdlingResource interface Ŷ

Slide 50

Slide 50 text

Espresso synchronization Espresso IdlingResource public class IdleMonitor implements IdlingResource {    @Override  public String getName() {  return IdleMonitor.class.getSimpleName();  }    @Override  public boolean isIdleNow() {  // return true if resource is idle  return false;  }    @Override  public void registerIdleTransitionCallback(ResourceCallback resourceCallback) {  // store a reference to the resourceCallback  // notify resourceCallback when idle  }  } Espresso.registerIdlingResources(idlingResource);

Slide 51

Slide 51 text

TEST TRAITS (UNIT) TESTS ISOLATED REPEATABLE FAST SELF-DOCUMENTING ž

Slide 52

Slide 52 text

Speed The shorter, the less distraction less of an interruption to our workflow

Slide 53

Slide 53 text

Speed The shorter, the less distraction less of an interruption to our workflow

Slide 54

Slide 54 text

Android testing AndroidTestCase needs an instance of the emulator to run • spin up emulator • deploy the APK • run the actual tests Big time sink: Ŷ

Slide 55

Slide 55 text

Android testing Android core libraries depend upon the actual Android operating system

Slide 56

Slide 56 text

Robolectric

Slide 57

Slide 57 text

Robolectric that would otherwise need an emulator / device (mock implementations of Android core libraries) too slow / hard for unit testing without the baggage of a device Replaces the behavior of code Ŷ Write JUnit tests Ŷ When? Ŷ Does this by using Shadow classes Ŷ InstrumentationTestCase Ŷ vs real device / speedy emulator Ŷ http://robolectric.org/

Slide 58

Slide 58 text

Robolectric Annotate your class @RunWith(RobolectricTestRunner.class)  public class MyActivityTest {    @Test  public void clickingButton_shouldChangeResultsViewText() throws Exception {  MyActivity activity = Robolectric.buildActivity(MyActivity.class).create().get();    Button pressMeButton = (Button) activity.findViewById(R.id.press_me_button);  TextView results = (TextView) activity.findViewById(R.id.results_text_view);    pressMeButton.performClick();  String resultsText = results.getText().toString();  assertThat(resultsText, equalTo("Testing Android Rocks!"));  }

Slide 59

Slide 59 text

Robolectric setup • Robolectric • Novoda gradle-android-test-plugin see Paul Blundell Tests run inside the JVM Ŷ Gradle - Maven Ŷ Configure it yourself Ŷ Gradle plugin Ŷ

Slide 60

Slide 60 text

Android testing JUnit 4 Support JUnit 4 test @RunWith(AndroidJUnit4.class)  public class AndroidJUnit4Test {    Conference jDays;    @Before  public void setUp() {  jDays = Conferences.newInstance(Conference.JDAYS);  jDays.start();  }    @After  public void tearDown() {  jDays.stop();  }    @Test  public void testPreconditions() {  assertThat(jDays, is(notNullValue()));  }    @Test  public void jDaysSouldBeGreat() {  assertThat(jDays, is(awesome()));  }

Slide 61

Slide 61 text

Testing support JUnit 4 Ŷ Mockable android.jar Ŷ First-party unit test support Ŷ Espresso 2 Ŷ

Slide 62

Slide 62 text

Mocking & Stubbing

Slide 63

Slide 63 text

Mockito in isolation Change runtime implementation Ŷ That can be predictably tested Ŷ Verify behaviour Ŷ Mockito Ŷ

Slide 64

Slide 64 text

Mockito the behaviour of your components during a test Java mocking framework Ŷ Mock dependencies Ŷ Inject Mocks to validate Ŷ Mock function return values Ŷ https://code.google.com/p/mockito/

Slide 65

Slide 65 text

Mockito Mockito @Test  public void testMockedConverter() throws Exception {  // expectations  final float expected_result = 32.0f;    // given  ConverterActivity activity = Robolectric.buildActivity(ConverterActivity.class).create().get();  activity.onCreate(new Bundle());  EditText celsiusField = (EditText) activity.findViewById(R.id.celsius_textview);  celsiusField.setText("0");    CelsiusFahrenheitConverter mockConverter = Mockito.mock(CelsiusFahrenheitConverter.class);  Mockito.when(mockConverter.convertCelsiusToFahrenheit(Mockito.anyFloat())).thenReturn(expected_result);  activity.setConverter(mockConverter);    // when  Button convertButton = (Button) activity.findViewById(R.id.conversion_button);  convertButton.performClick();    // then  Mockito.verify(mockConverter, Mockito.times(1)).convertCelsiusToFahrenheit(Mockito.anyFloat());  TextView fahrenheitField = (TextView) activity.findViewById(R.id.fahrenheit_textview);  String fahrenheitString = fahrenheitField.getText().toString();  assertThat(fahrenheitString, equalTo(String.valueOf(expected_result)));  }

Slide 66

Slide 66 text

The Monkey

Slide 67

Slide 67 text

evil monkey…

Slide 68

Slide 68 text

Monkey pseudo-random stream of user events Generates events Ŷ clicks Ŷ touches Ŷ gestures Ŷ system-level events Ŷ Stress test your app Ŷ

Slide 69

Slide 69 text

Monkey number of events to attempt operational constraints, such as restricting the test to a single package Event types and frequencies command-line tool Ŷ basic configuration Ŷ $ adb shell monkey -p your.package.name -v 500 Ŷ

Slide 70

Slide 70 text

AssertJ Android for checking assertions to read / write http://square.github.io/assertj-android/ Syntactic sugar Ŷ Extension of the AssertJ library Ŷ Makes tests easier Ŷ Fluent syntax Ŷ

Slide 71

Slide 71 text

AssertJ Android Assertions for Android objects Ŷ + support libraries Ŷ

Slide 72

Slide 72 text

AssertJ Android Regular JUnit AssertJ Android assertEquals(View.GONE, view.getVisibility()); assertThat(view).isGone(); expected: <8> but was: <4> Expected visibility but was .

Slide 73

Slide 73 text

AssertJ Android Regular JUnit assertEquals(View.VISIBLE, layout.getVisibility()); assertEquals(VERTICAL, layout.getOrientation()); assertEquals(4, layout.getChildCount()); assertEquals(SHOW_DIVIDERS_MIDDLE, layout.getShowDividers()); AssertJ Android assertThat(layout).isVisible() .isVertical() .hasChildCount(4) .hasShowDividers(SHOW_DIVIDERS_MIDDLE);

Slide 74

Slide 74 text

Testing network resources

Slide 75

Slide 75 text

Testing network resources Unit tests Ŷ Integration tests Ŷ Retrofit Ŷ

Slide 76

Slide 76 text

Testing network resources how to trigger edge cases like API rate limit exceeded? HTTP calls should be asynchronous … Hit a real server? Ŷ Brittle tests! Ŷ Slow tests Ŷ Network or server is down Ŷ Incomplete tests Ŷ Further complicated in Android Ŷ

Slide 77

Slide 77 text

Testing network resources Use a test server Ŷ Mock the Retrofit interface Ŷ

Slide 78

Slide 78 text

Using a test server • virtually no changes or code required in your app codebase • can be shared across multiple platforms (Android, iOS, web, …) • another dependency (that can fail) • something new to ﬁgure out • not always easy to trigger errors or edge cases • what if ‘state’ is needed? • ‘slow’ tests execution, still HTTP calls needed PROs Ŷ CONs Ŷ

Slide 79

Slide 79 text

Mocking the Retrofit interface • robust: no network errors, timeouts, … • easy to generate errors / edge cases • easy to control server-side state • write your dummy data in Java • extra test code needed • you don’t test the JSON deserialization (but you can add unit tests) • mock a part of your application ➪ not a true integration test PROs Ŷ CONs Ŷ You trust the test suites of Retrofit, Gson, … and mock out the data

Slide 80

Slide 80 text

Network testing Mock network interface ‘Fake’-‘real’ server

Slide 81

Slide 81 text

Mocking the Retrofit interface MockRestAdapter sample

Slide 82

Slide 82 text

Mocking the Retrofit interface Retrofit mock public interface GitHub {  @GET("/repos/{owner}/{repo}/contributors")  List contributors(@Path("owner") String owner, @Path("repo") String repo);  }

Slide 83

Slide 83 text

Mocking the Retrofit interface Canned data /** A mock implementation of the {@link GitHub} API interface. */  static class MockGitHub implements GitHub {  private final Map>> ownerRepoContributors;    public MockGitHub() {  ownerRepoContributors = new LinkedHashMap>>();    // Seed some mock data.  addContributor("square", "retrofit", "John Doe", 12);  addContributor("square", "retrofit", "Bob Smith", 2);  addContributor("square", "retrofit", "Big Bird", 40);  addContributor("square", "picasso", "Proposition Joe", 39);  addContributor("square", "picasso", "Keiser Soze", 152);  }

Slide 84

Slide 84 text

Mocking the Retrofit interface Mock @Override public List contributors(String owner, String repo) {  Map> repoContributors = ownerRepoContributors.get(owner);  if (repoContributors == null) {  return Collections.emptyList();  }  List contributors = repoContributors.get(repo);  if (contributors == null) {  return Collections.emptyList();  }  return contributors;  }

Slide 85

Slide 85 text

Mocking the Retrofit interface Mock // Create a very simple REST adapter which points to the GitHub API endpoint.  RestAdapter restAdapter = new RestAdapter.Builder()  .setEndpoint(API_URL)  .build();    // Wrap our REST adapter to allow mock implementations and fake network delay.  MockRestAdapter mockRestAdapter = MockRestAdapter.from(restAdapter);    // Instantiate a mock object so we can interact with it later.  MockGitHub mockGitHub = new MockGitHub();  // Use the mock REST adapter and our mock object to create the API interface.  GitHub gitHub = mockRestAdapter.create(GitHub.class, mockGitHub);    // Query for some contributors for a few repositories.  printContributors(gitHub, "square", "retrofit");  printContributors(gitHub, "square", "picasso");

Slide 86

Slide 86 text

Network testing Mock network interface ‘Fake’-‘real’ server

Slide 87

Slide 87 text

MockWebServer predictable requests-responses Scriptable web server Ŷ okhttp (square) Ŷ Specify responses Ŷ Test HTTP clients Ŷ Verify requests Ŷ Like Mockito (for HTTP stack) Ŷ • script your mocks • run app code • verify that expected requests were made

Slide 88

Slide 88 text

MockWebServer MockWebServer // Create a MockWebServer. These are lean enough that you can create // a new instance for every unit test. MockWebServer mockWebServer = new MockWebServer();  mockWebServer.play();    // Setup the MockWebServer & schedule a response  MockResponse mockResponse = new MockResponse();  mockResponse.setResponseCode(500);  mockWebServer().enqueue(mockResponse);    // Setup your networking lib. Ask the server for its url.  … mockWebServer.getUrl(“/“);    DO YOUR TEST

Slide 89

Slide 89 text

MockWebServer Java test @Test public void exampleTest() { // make your app do requests to http://localhost:8080 // setup MockWebServer onView(withId(R.id.submit_button)).perform(click()); onView(withText(“error message”)).check(matches(isDisplayed())); verify(postRequestedFor(urlMatching(“/my/resource/[a-z0-9]+”))); }

Slide 90

Slide 90 text

Android testing tools

Slide 91

Slide 91 text

Application ARCHITECTURE

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

Software design patterns Good code = testable code Ŷ Dependency injection Ŷ

Slide 94

Slide 94 text

Software design patterns Architecting Android Applications with Dagger Dependency injection Ŷ Dagger Ŷ Avoid reflection at runtime Ŷ Define dependencies Ŷ Jake Wharton Ŷ https://parleys.com/play/529bde2ce4b0e619540cc3ae

Slide 95

Slide 95 text

Dependency injection Decoupling Ŷ Swap mocks in for tests Ŷ

Slide 96

Slide 96 text

Android lifecycle

Slide 97

Slide 97 text

Model View Presenter Model Presenter View

Slide 98

Slide 98 text

Model View Presenter Model Presenter View Android UI

Slide 99

Slide 99 text

Model View Presenter Testable presentation logic Ŷ No Android dependencies Ŷ

Slide 100

Slide 100 text

AUTOMATION &  REPORTING

Slide 101

Slide 101 text

Automated distributed testing Automated distributed instrumentation testing Screenshots share with the designers…? in parallel Run tests on multiple devices Reports / logs captured for each test

Slide 102

Slide 102 text

Spoon https://square.github.io/spoon/ Automate test execution Ŷ across multiple devices Ŷ Aggregation of screenshots Ŷ Aggregate the results Ŷ

Slide 103

Slide 103 text

Spoon High level tests overview Ŷ Test failure specific Ŷ to a single device or all devices

Slide 104

Slide 104 text

Spoon: device view Results of each test on one device Ŷ Useful for tracking down Ŷ device-speciﬁc failures of individual tests

Slide 105

Slide 105 text

Spoon: test view Results of single test on all devices Ŷ

Slide 106

Slide 106 text

Spoon: screenshots Visual inspection of test execution Ŷ screenshots Spoon.screenshot(activity, "initial_state"); /* Normal test code... */ Spoon.screenshot(activity, "after_login"); http://square.github.io/spoon/sample/

Slide 107

Slide 107 text

Build pipeline Checkout / compile Unit tests Test coverage Code analysis Create deployable artifact Deploy for automatic QA test Trigger automated QA stage

Slide 108

Slide 108 text

Build pipeline tools Build (maven - gradle) Dependency repo (nexus - artifactory) Testing framework (JUnit - ...) Test coverage (Cobertura - Emma - Jacoco) Code analysis (Checkstyle, findbugs, pmd, Android Lint) Creation of deployable artifact (buildtool, artifact repo) Trigger next stage

Slide 109

Slide 109 text

Continuous integration benefits without additional eﬀort Fast feedback - fewer errors Ŷ Test everything on every (nightly) build Ŷ Regression tests Ŷ Less manual testing Ŷ

Slide 110

Slide 110 text

Unit tests are real measure of project health and code quality

Slide 111

Slide 111 text

Jenkins

Slide 112

Slide 112 text

Continuous integration • Jenkins • Bamboo Travis … Server Ŷ Hosted Ŷ https://travis-ci.org

Slide 113

Slide 113 text

Test coverage open-source toolkit for measuring and reporting Java code coverage Metrics Ŷ Code safely Ŷ JaCoCo Ŷ Find risky code Ŷ

Slide 114

Slide 114 text

Android test coverage

Slide 115

Slide 115 text

Coverage 88%

Slide 116

Slide 116 text

References

Slide 117

Slide 117 text

Suggested reading Android Application Testing Guide Diego Torres Milano (9781849513500) Robotium Automated Testing for Android Hrushikesh Zadgaonkar (9781782168010) The Busy Coder’s Guide to Android Development Mark Murphy - http://commonsware.com/Android/

Slide 118

Slide 118 text

Suggested reading Test Driven Development: By Example Beck, Kent (978-0321146533) Continuous Integration: Improving Software Quality and Reducing Risk Duvall, Paul M. et al. (978-0321336385) Working Effectively with Legacy Code Feathers, Michael (978-0131177055)

Slide 119

Slide 119 text

Questions? Filip Maelbrancke Consultant @ AppFoundry ﬁ[email protected] @fmaelbrancke