Android Testing Support Library - the Nitty Gritty

The talk was delivered at Droidcon Vienna 2016 and covers the more interesting parts of the Testing Support Lib

Transcript

1. Android Testing Support Library The Nitty Gritty © Zan Markan 2016 - @zmarkan
2. Testing 101 • on JVM vs on device • unit / integration / functional / end to end • Robolectric, Calabash, Instrumentation, Robotium, Appium © Zan Markan 2016 - @zmarkan
3. "Support" • Android framework vs Support libraries • Trend to unbundle • support-v4, appcompat-v7, recyclerview, ... © Zan Markan 2016 - @zmarkan
4. Android Testing Support Library © Zan Markan 2016 - @zmarkan
5. Good Old Times... • jUnit3 syntax • Remember ActivityInstrumentationTestCase2? public class OldSchoolTest extends ActivityInstrumentationTestCase2<ResourceListActivity> { public OldSchoolTest() { super(ResourceListActivity.class); } //... } © Zan Markan 2016 - @zmarkan
6. More jUnit3 goodness • overriding setUp() and tearDown() • testPrefixedMethods() & test_prefixedMethods() • Ignorance Inheritance is bliss © Zan Markan 2016 - @zmarkan
7. public class OldSchoolTest extends ActivityInstrumentationTestCase2<ResourceListActivity> { public OldSchoolTest() { super(ResourceListActivity.class); } @Override protected void setUp() throws Exception { super.setUp(); //Omg. } @Override protected void tearDown() throws Exception { super.tearDown(); //Srsly?? } @SmallTest public void testMustHaveTheTestPrefix(){ fail("No."); } @SmallTest public void test_canHaveUnderscoreInPrefixAsWell(){ fail("Just no."); } } © Zan Markan 2016 - @zmarkan
8. Welcome to the present • jUnit4 syntax • No more extending • @Test, @Before, @After, @AfterClass,... • ActivityTestRule, InstrumentationRegistry © Zan Markan 2016 - @zmarkan
9. @RunWith(AndroidJUnitRunner.java) public class ModernTest{ @Rule ActivityTestRule<ResourceListActivity> activityRule = new ActivityTestRule<>(ResourceListActivity.class); @Before public void setupMocks{ } @After public void clearMocks(){ } @Test public void anyNameYouWish(){ //Yay } } © Zan Markan 2016 - @zmarkan
10. What else? • Espresso • More Espresso (there's a lot to it) • UIAutomator • ... © Zan Markan 2016 - @zmarkan
11. But First... The setup (note: AS does that on it's own if you create a new project - these instructions will mostly apply for legacy projects) © Zan Markan 2016 - @zmarkan
12. Gradle • Set your test runner to be AndroidJUnitRunner © Zan Markan 2016 - @zmarkan
13. Gradle • Set your test runner to be AndroidJUnitRunner testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" © Zan Markan 2016 - @zmarkan
14. defaultConfig { applicationId "com.zmarkan.testingresources" minSdkVersion 23 targetSdkVersion 23 versionCode 1 versionName "1.0" testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" } buildTypes { release { minifyEnabled false proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro' } } © Zan Markan 2016 - @zmarkan
15. Gradle • Set your test runner to be AndroidJUnitRunner • Add dependencies © Zan Markan 2016 - @zmarkan
16. Gradle • Set your test runner to be AndroidJUnitRunner • Add dependencies // JUnit4 Rules androidTestCompile 'com.android.support.test:rules:0.5' // Android JUnit Runner androidTestCompile 'com.android.support.test:runner:0.5' // Espresso core androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.1' // Espresso-contrib for DatePicker, RecyclerView, Drawer actions, Accessibility checks, CountingIdlingResource androidTestCompile 'com.android.support.test.espresso:espresso-contrib:2.2.1' © Zan Markan 2016 - @zmarkan
17. dependencies { compile 'com.android.support:appcompat-v7:23.2.1' compile 'com.android.support:design:23.2.1' //Unit testing testCompile 'junit:junit:4.12' // JUnit4 Rules androidTestCompile 'com.android.support.test:rules:0.5' // Android JUnit Runner androidTestCompile 'com.android.support.test:runner:0.5' // Espresso core androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.1' // Espresso-contrib for DatePicker, RecyclerView, Drawer actions, Accessibility checks, CountingIdlingResource androidTestCompile 'com.android.support.test.espresso:espresso-contrib:2.2.1' © Zan Markan 2016 - @zmarkan
18. Gradle • Set your test runner to be AndroidJUnitRunner • Add dependencies • Voila! © Zan Markan 2016 - @zmarkan
19. Error:Conflict with dependency 'com.android.support:support-annotations'. Resolved versions for app (23.2.1) and test app (23.1.1) differ. See http://g.co/androidstudio/app-test-app-conflict for details. © Zan Markan 2016 - @zmarkan
20. Gradle • Set your test runner to be AndroidJUnitRunner • Add dependencies • Voila! • Resolve dependencies © Zan Markan 2016 - @zmarkan
21. Dependency resolutions • App and Test app depend on different lib versions • Run ./gradlew :app:dependencies © Zan Markan 2016 - @zmarkan
22. compile - Classpath for compiling the main sources. +--- com.android.support:appcompat-v7:24.2.1 | +--- com.android.support:support-v4:24.2.1 | | +--- com.android.support:support-compat:24.2.1 | | | \--- com.android.support:support-annotations:24.2.1 | | +--- com.android.support:support-media-compat:24.2.1 | | | \--- com.android.support:support-compat:24.2.1 (*) | | +--- com.android.support:support-core-utils:24.2.1 | | | \--- com.android.support:support-compat:24.2.1 (*) | | +--- com.android.support:support-core-ui:24.2.1 | | | \--- com.android.support:support-compat:24.2.1 (*) | | \--- com.android.support:support-fragment:24.2.1 | | +--- com.android.support:support-compat:24.2.1 (*) | | +--- com.android.support:support-media-compat:24.2.1 (*) | | +--- com.android.support:support-core-ui:24.2.1 (*) | | \--- com.android.support:support-core-utils:24.2.1 (*) | +--- com.android.support:support-vector-drawable:24.2.1 | | \--- com.android.support:support-compat:24.2.1 (*) | \--- com.android.support:animated-vector-drawable:24.2.1 | \--- com.android.support:support-vector-drawable:24.2.1 (*) \--- com.android.support:design:24.2.1 +--- com.android.support:support-v4:24.2.1 (*) +--- com.android.support:appcompat-v7:24.2.1 (*) \--- com.android.support:recyclerview-v7:24.2.1 +--- com.android.support:support-annotations:24.2.1 +--- com.android.support:support-compat:24.2.1 (*) \--- com.android.support:support-core-ui:24.2.1 (*) © Zan Markan 2016 - @zmarkan
23. androidTestCompile - Classpath for compiling the androidTest sources. +--- com.android.support.test.espresso:espresso-core:2.2.2 | +--- com.squareup:javawriter:2.1.1 | +--- com.android.support.test:rules:0.5 | | \--- com.android.support.test:runner:0.5 | | +--- com.android.support:support-annotations:23.1.1 | | +--- junit:junit:4.12 | | | \--- org.hamcrest:hamcrest-core:1.3 | | \--- com.android.support.test:exposed-instrumentation-api-publish:0.5 | +--- com.android.support.test:runner:0.5 (*) | +--- javax.inject:javax.inject:1 | +--- org.hamcrest:hamcrest-library:1.3 | | \--- org.hamcrest:hamcrest-core:1.3 | +--- com.android.support.test.espresso:espresso-idling-resource:2.2.2 | +--- org.hamcrest:hamcrest-integration:1.3 | | \--- org.hamcrest:hamcrest-library:1.3 (*) | +--- com.google.code.findbugs:jsr305:2.0.1 | \--- javax.annotation:javax.annotation-api:1.2 \--- com.android.support.test.espresso:espresso-contrib:2.2.2 +--- com.android.support:design:23.1.1 | +--- com.android.support:appcompat-v7:23.1.1 | | \--- com.android.support:support-v4:23.1.1 | | \--- com.android.support:support-annotations:23.1.1 | +--- com.android.support:recyclerview-v7:23.1.1 | | +--- com.android.support:support-annotations:23.1.1 | | \--- com.android.support:support-v4:23.1.1 (*) | \--- com.android.support:support-v4:23.1.1 (*) +--- com.google.android.apps.common.testing.accessibility.framework:accessibility-test-framework:2.0 | \--- org.hamcrest:hamcrest-core:1.3 +--- com.android.support:recyclerview-v7:23.1.1 (*) +--- com.android.support.test.espresso:espresso-core:2.2.2 (*) \--- com.android.support:support-v4:23.1.1 (*) © Zan Markan 2016 - @zmarkan
24. Compile task \--- com.android.support:design:24.2.1 +--- com.android.support:support-v4:24.2.1 (*) +--- com.android.support:appcompat-v7:24.2.1 (*) \--- com.android.support:recyclerview-v7:24.2.1 +--- com.android.support:support-annotations:24.2.1 +--- com.android.support:support-compat:24.2.1 (*) \--- com.android.support:support-core-ui:24.2.1 (*) © Zan Markan 2016 - @zmarkan
25. androidTestCompile \--- com.android.support.test.espresso:espresso-contrib:2.2.2 +--- com.android.support:design:23.1.1 | +--- com.android.support:appcompat-v7:23.1.1 | | \--- com.android.support:support-v4:23.1.1 | | \--- com.android.support:support-annotations:23.1.1 | +--- com.android.support:recyclerview-v7:23.1.1 | | +--- com.android.support:support-annotations:23.1.1 ... © Zan Markan 2016 - @zmarkan
26. Dependency resolutions • App and Test app depend on different lib versions • Run ./gradlew :app:dependencies • Resolve with: • exclude dependecy (everywhere applicable) • use ResolutionStrategy © Zan Markan 2016 - @zmarkan
27. Exclude Dependency androidTestCompile('com.android.support.test.espresso:espresso-core:2.2.1'){ exclude module: 'support-annotations' } © Zan Markan 2016 - @zmarkan
28. Set ResolutionStrategy configurations.all() { resolutionStrategy.force 'com.android.support:support-annotations:22.1.0' } © Zan Markan 2016 - @zmarkan
29. Gradle • Set your test runner to be AndroidJUnitRunner • Add dependencies • Resolve dependencies • Voila! © Zan Markan 2016 - @zmarkan
30. Espresso © Zan Markan 2016 - @zmarkan
31. Espresso - components • View interactions & assertions • Hamcrest syntax • No (unnecessary) waits © Zan Markan 2016 - @zmarkan
32. Espresso - API Cheat sheet: google.github.io/android-testing-support-library/docs/espresso/ cheatsheet © Zan Markan 2016 - @zmarkan
33. Poking the screen onView(withId(R.id.button)).perform(click()); © Zan Markan 2016 - @zmarkan
34. Poking the screen onView(withId(R.id.button)).perform(click()); onView( allOf( withParent(withId(R.id.container)), withText("Hello World!") ) ).check(matches(isDisplayed())); © Zan Markan 2016 - @zmarkan
35. Espresso - contrib • RecyclerView • Drawers • Pickers • Accessibility © Zan Markan 2016 - @zmarkan
36. Espresso++ • Custom matchers • Custom ViewActions • Idling resources • Page objects • Intent mocking © Zan Markan 2016 - @zmarkan
37. Custom matchers • Find a view in the hierarchy • Good for custom views & components • Override: • describeTo • matchesSafely © Zan Markan 2016 - @zmarkan
38. Custom matchers interface MyView{ boolean hasCustomProperty(); } static BoundedMatcher withCustomPropery() { return new BoundedMatcher<Object, MyView>(MyView.class){ @Override public void describeTo(Description description) { description.appendText("Custom property is enabled"); } @Override protected boolean matchesSafely(MyView item) { return item.hasCustomProperty(); } }; } © Zan Markan 2016 - @zmarkan
39. Custom ViewActions • Same story as matchers, just a bit more extensive • example allows us to scroll in NestedScrollView • github.com/zmarkan/Android-Espresso-ScrollableScroll © Zan Markan 2016 - @zmarkan
40. Custom ViewActions - API public class ScrollableUtils { public static ViewAction scrollableScrollTo() { return actionWithAssertions(new ScrollableScrollToAction()); } } © Zan Markan 2016 - @zmarkan
41. Implementation... public class ScrollableScrollToAction implements ViewAction{ private static final String TAG = com.zmarkan.nestedscroll.action.ScrollableScrollToAction.class.getSimpleName(); @SuppressWarnings("unchecked") @Override public Matcher<View> getConstraints() { return allOf(withEffectiveVisibility(ViewMatchers.Visibility.VISIBLE), isDescendantOfA(anyOf( isAssignableFromClassOrInterface(ScrollingView.class)))); } @Override public void perform(UiController uiController, View view) { if (isDisplayingAtLeast(90).matches(view)) { Log.i(TAG, "View is already displayed. Returning."); return; } Rect rect = new Rect(); view.getDrawingRect(rect); if (!view.requestRectangleOnScreen(rect, true /* immediate */)) { Log.w(TAG, "Scrolling to view was requested, but none of the parents scrolled."); } uiController.loopMainThreadUntilIdle(); if (!isDisplayingAtLeast(90).matches(view)) { throw new PerformException.Builder() .withActionDescription(this.getDescription()) .withViewDescription(HumanReadables.describe(view)) .withCause(new RuntimeException( "Scrolling to view was attempted, but the view is not displayed")) .build(); } } @Override public String getDescription() { return "scroll to"; } } © Zan Markan 2016 - @zmarkan
42. Don't panic. • Take something that works • ...like scrollTo() in regular Espresso • modify it • profit © Zan Markan 2016 - @zmarkan
43. Custom IdlingResource • A better way to wait • Use when you have background stuff going on • Override: • getName • isIdleNow • registerIdleTransitionCallback © Zan Markan 2016 - @zmarkan
44. Custom IdlingResource public class CustomIdlingResource implements IdlingResource{ private ResourceCallback resourceCallback; private EventBus bus; private boolean loadingCompleted = false; public CustomIdlingResource(EventBus bus){ bus.register(this); } public void onEvent(LoadingCompletedEvent event){ loadingCompleted = true; isIdleNow(); } } © Zan Markan 2016 - @zmarkan
45. Custom IdlingResource @Override public String getName() { return CustomIdlingResource.class.getName(); } @Override public void registerIdleTransitionCallback( ResourceCallback resourceCallback) { this.resourceCallback = resourceCallback; } @Override public boolean isIdleNow() { boolean idle = loadingCompleted; if (idle && resourceCallback != null) { resourceCallback.onTransitionToIdle(); bus.unregister(this); } return idle; } } © Zan Markan 2016 - @zmarkan
46. Page Object Pattern • Build your own DSL • Every screen is a Page • Page-specific actions and verifications • (as seen in Calabash, etc...) © Zan Markan 2016 - @zmarkan
47. Page Class public class MainViewTestUtils { public void enterUserName(String text){ /* espresso goes here */ } public void enterPassword(String text){ /* ... */ } public void pressContinue(){ /* ... */ } public void assertErrorShown(boolean shown){ /* ... */ } } © Zan Markan 2016 - @zmarkan
48. Page Class in test @Test public void errorShownWhenPasswordIncorrect(){ MainViewTestUtils view = new MainViewTestUtils(); view.enterUsername(username); view.enterPassword(incorrectPassword); view.pressContinue(); view.assertErrorShown(true); } © Zan Markan 2016 - @zmarkan
49. To boldly go... © Zan Markan 2016 - @zmarkan
50. UI Automator • Interact with any installed app • Use to create full end-to-end tests • Can coexist with Espresso in the same app • Use uiautomatorviewer command to find items in the hierarchy © Zan Markan 2016 - @zmarkan
51. UI Automator viewer © Zan Markan 2016 - @zmarkan
52. UI Automator syntax UiDevice device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation(); device.pressHome(); device.findObject(new UiSelector().descriptionContains("Google search")).clickAndWaitForNewWindow(); © Zan Markan 2016 - @zmarkan
53. Test Runner © Zan Markan 2016 - @zmarkan
54. Test Runner • Spins up tests... • and runs them • Customise to prepare mocks • Easier run/debug via command line © Zan Markan 2016 - @zmarkan
55. Extending the Runner • Use custom Application class • Provide mocked dependencies • Specify this new runner in build.gradle © Zan Markan 2016 - @zmarkan
56. Extending the Runner public class CustomRunner extends AndroidJUnitRunner{ @Override public Application newApplication(ClassLoader cl, String className, Context context) throws InstantiationException, IllegalAccessException, ClassNotFoundException { return Instrumentation.newApplication(TestApplication.class, context); } } © Zan Markan 2016 - @zmarkan
57. Running on the Command line • Run module/package/class/method • Run Large/Medium/Small test only • Shard to run in parallel • Debug without reinstalling © Zan Markan 2016 - @zmarkan
58. Simple API • send commands to runner via ADB (adb shell [commands]) • am instrument -w -e class your.full.classname#methodName your.test.package.name/your.test.Runner.class • d.android.com/tools/testing/testing_otheride.html © Zan Markan 2016 - @zmarkan
59. Test Rules © Zan Markan 2016 - @zmarkan
60. Test Rules • Set starting activity / Service • Replace ActivityInstrumentationTestCase2 • (in most cases) • Add / Extend to create more components © Zan Markan 2016 - @zmarkan
61. Test Rules - examples • MockWebServerRule - sets up MockWebServer when required • Source: github.com/artem-zinnatullin/qualitymatters © Zan Markan 2016 - @zmarkan
62. Firebase test lab • Simple setup • CI support (via gcloud) • Support for VMs • firebase.google.com/docs/test-lab © Zan Markan 2016 - @zmarkan
63. Espresso Test Recorder • Since AS 2.2 preview 3 • Generates test code after clicking on screen • (Not necessarily nice code) • tools.android.com/tech-docs/test-recorder © Zan Markan 2016 - @zmarkan
64. Above & Beyond? • Espresso Web for WebViews • JankTestHelper • Stuff is being added. © Zan Markan 2016 - @zmarkan
65. Best of all? © Zan Markan 2016 - @zmarkan
66. Best of all? It's all open source! © Zan Markan 2016 - @zmarkan
67. fin • ! www.spacecowboyrocketcompany.com • " @zmarkan • # zan at markan dot me • $ androidchat.co (zmarkan) • %& @zmarkan © Zan Markan 2016 - @zmarkan
68. • google.github.io/android-testing-support-library/contribute • Quality Matters - Artem Zinnatulin • d.android.com/tools/testing-support-library • github.com/googlesamples/android-testing • chiuki.github.io/advanced-android-espresso © Zan Markan 2016 - @zmarkan