“Computers are designed to do simple repetitive tasks. The second you have humans doing repetitive tasks, all the computers get together late at night and laugh at you...” Neil Ford
Repeatability (no matter what development stack he uses) in which they are being run Tests need to be run by every developer ♂ Tests must not rely on the environment
Instrumentation Test Activity state saving and restoration // Start the main activity of the application under test mActivity = getActivity(); // Get a handle to the Activity object's main UI widget, a Spinner mSpinner = (Spinner)mActivity.findViewById(com.android.example.spinner.R.id.Spinner); // Set the Spinner to a known position mActivity.setSpinnerPosition(TEST_STATE_DESTROY_POSITION); // Stop the activity - The onDestroy() method should save the state of the Spinner mActivity.finish(); // Re-start the Activity - the onResume() method should restore the state of the Spinner mActivity = getActivity(); // Get the Spinner's current position int currentPosition = mActivity.getSpinnerPosition(); // Assert that the current position is the same as the starting position assertEquals(TEST_STATE_DESTROY_POSITION, currentPosition);
Self-documenting Testable code = clear Testable code = easy to follow we can just look at the test No need to explain how a component works No need to write documentation tests = usage examples
android.support.test.* for all Google testing frameworks Unbundled static testing library Ŷ Android support library Ŷ Full Gradle support Ŷ Open source / AOSP Ŷ Container Ŷ
JUnit 4 support @Parameters Test annotations Ŷ Test suite annotations Ŷ Parameterized tests Ŷ @Suite.SuiteClasses @Test @Before @After @BeforeClass @AfterClass
Hamcrest Library of matchers Ŷ Syntactic sugar Ŷ Assert assertThat(someString, is(equalTo(“Expected”))); actual value expectation on the value (Matcher)
View Matchers Hamcrest Ŷ allOf() / anyOf() ➪ give more power Ŷ TestRunner // assert that textview does not start with with "XYZ" and contains "ABC" anywhere onView(withId(R.id.celsius_textview)) .check(matches(allOf(withText(not(startsWith("XYZ"))), withText(containsString("ABC"))))); // assert that textview ends with "XYZ" or contains "ABC" anywhere onView(withId(R.id.celsius_textview)) .check(matches(anyOf(withText(endsWith("XYZ")), withText(containsString("ABC")))));
Hamcrest Lots of useful matchers Ŷ Custom matchers Ŷ not a number public void testSquareRootOfMinusOneIsNotANumber() { assertThat(Math.sqrt(-1), is(notANumber())); }
Hamcrest custom matcher Hamcrest matcher /** * Hamcrest matcher: test if a double has the value NaN (not a number) */ public class IsNotANumber extends TypeSafeMatcher {
@Override public boolean matchesSafely(Double number) { return number.isNaN(); }
public void describeTo(Description description) { description.appendText("not a number"); }
@Factory public static Matcher notANumber() { return new IsNotANumber(); } }
Custom ViewAction Espresso ViewAction /** * Clears view text by setting {@link EditText}s text property to "". */ public class ClearTextAction implements ViewAction {
@Override public Matcher getConstraints() { return allOf(isDisplayed(), isAssignableFrom(EditText.class)); }
Custom ViewAction Espresso ViewAction /** * Your own custom View Actions. */ public class CustomViewActions {
private CustomViewActions() { }
/** * Returns an action that clears text on the view. Extra thoroughly! */ public static ViewAction clearTextExtraThoroughly() { return new ClearTextAction(); } }
Reliability through synchronization between Instrumentation thread and UI thread for background resources Espresso synchronizes AsyncTasks default thread pool • wait until app is idle • perform operation on the UI thread • wait until completion • check result for custom implementations IntentService HandlerThread ThreadPool Synchronization Ŷ Synchronization Ŷ Synchronization Ŷ AsyncTask Ŷ IdlingResource interface Ŷ
Espresso synchronization Espresso IdlingResource public class IdleMonitor implements IdlingResource {
@Override public String getName() { return IdleMonitor.class.getSimpleName(); }
@Override public boolean isIdleNow() { // return true if resource is idle return false; }
@Override public void registerIdleTransitionCallback(ResourceCallback resourceCallback) { // store a reference to the resourceCallback // notify resourceCallback when idle } } Espresso.registerIdlingResources(idlingResource);
Robolectric that would otherwise need an emulator / device (mock implementations of Android core libraries) too slow / hard for unit testing without the baggage of a device Replaces the behavior of code Ŷ Write JUnit tests Ŷ When? Ŷ Does this by using Shadow classes Ŷ InstrumentationTestCase Ŷ vs real device / speedy emulator Ŷ http://robolectric.org/
Mockito the behaviour of your components during a test Java mocking framework Ŷ Mock dependencies Ŷ Inject Mocks to validate Ŷ Mock function return values Ŷ https://code.google.com/p/mockito/
Monkey number of events to attempt operational constraints, such as restricting the test to a single package Event types and frequencies command-line tool Ŷ basic configuration Ŷ $ adb shell monkey -p your.package.name -v 500 Ŷ
Testing network resources how to trigger edge cases like API rate limit exceeded? HTTP calls should be asynchronous … Hit a real server? Ŷ Brittle tests! Ŷ Slow tests Ŷ Network or server is down Ŷ Incomplete tests Ŷ Further complicated in Android Ŷ
Using a test server • virtually no changes or code required in your app codebase • can be shared across multiple platforms (Android, iOS, web, …) • another dependency (that can fail) • something new to figure out • not always easy to trigger errors or edge cases • what if ‘state’ is needed? • ‘slow’ tests execution, still HTTP calls needed PROs Ŷ CONs Ŷ
Mocking the Retrofit interface • robust: no network errors, timeouts, … • easy to generate errors / edge cases • easy to control server-side state • write your dummy data in Java • extra test code needed • you don’t test the JSON deserialization (but you can add unit tests) • mock a part of your application ➪ not a true integration test PROs Ŷ CONs Ŷ You trust the test suites of Retrofit, Gson, … and mock out the data
Mocking the Retrofit interface Canned data /** A mock implementation of the {@link GitHub} API interface. */ static class MockGitHub implements GitHub { private final Map>> ownerRepoContributors;
public MockGitHub() { ownerRepoContributors = new LinkedHashMap>>();
Mocking the Retrofit interface Mock // Create a very simple REST adapter which points to the GitHub API endpoint. RestAdapter restAdapter = new RestAdapter.Builder() .setEndpoint(API_URL) .build();
// Wrap our REST adapter to allow mock implementations and fake network delay. MockRestAdapter mockRestAdapter = MockRestAdapter.from(restAdapter);
// Instantiate a mock object so we can interact with it later. MockGitHub mockGitHub = new MockGitHub(); // Use the mock REST adapter and our mock object to create the API interface. GitHub gitHub = mockRestAdapter.create(GitHub.class, mockGitHub);
// Query for some contributors for a few repositories. printContributors(gitHub, "square", "retrofit"); printContributors(gitHub, "square", "picasso");
MockWebServer predictable requests-responses Scriptable web server Ŷ okhttp (square) Ŷ Specify responses Ŷ Test HTTP clients Ŷ Verify requests Ŷ Like Mockito (for HTTP stack) Ŷ • script your mocks • run app code • verify that expected requests were made
MockWebServer MockWebServer // Create a MockWebServer. These are lean enough that you can create // a new instance for every unit test. MockWebServer mockWebServer = new MockWebServer(); mockWebServer.play();
// Setup the MockWebServer & schedule a response MockResponse mockResponse = new MockResponse(); mockResponse.setResponseCode(500); mockWebServer().enqueue(mockResponse);
// Setup your networking lib. Ask the server for its url. … mockWebServer.getUrl(“/“);
MockWebServer Java test @Test public void exampleTest() { // make your app do requests to http://localhost:8080 // setup MockWebServer onView(withId(R.id.submit_button)).perform(click()); onView(withText(“error message”)).check(matches(isDisplayed())); verify(postRequestedFor(urlMatching(“/my/resource/[a-z0-9]+”))); }
Automated distributed testing Automated distributed instrumentation testing Screenshots share with the designers…? in parallel Run tests on multiple devices Reports / logs captured for each test
Spoon: screenshots Visual inspection of test execution Ŷ screenshots Spoon.screenshot(activity, "initial_state"); /* Normal test code... */ Spoon.screenshot(activity, "after_login"); http://square.github.io/spoon/sample/
Continuous integration benefits without additional effort Fast feedback - fewer errors Ŷ Test everything on every (nightly) build Ŷ Regression tests Ŷ Less manual testing Ŷ
Suggested reading Android Application Testing Guide Diego Torres Milano (9781849513500) Robotium Automated Testing for Android Hrushikesh Zadgaonkar (9781782168010) The Busy Coder’s Guide to Android Development Mark Murphy - http://commonsware.com/Android/
Suggested reading Test Driven Development: By Example Beck, Kent (978-0321146533) Continuous Integration: Improving Software Quality and Reducing Risk Duvall, Paul M. et al. (978-0321336385) Working Effectively with Legacy Code Feathers, Michael (978-0131177055)
filipmaelbrancke
miyabigouji
star_zero
pospome
murajun1978
sr2mg4
mkeeda
ivargrimstad
tomokusaba
maroon1st
ogiwarat
kin29
devoc
holman
sugarenia
bryan
frogandcode
caitiem20
tanoku
akmur
jonrohan
dougneiner
eileencodes
sachag
moore