MAINTAINABLE ESPRESSO TESTS WITH ROBOTS AND SCREENSHOTS SAM EDWARDS - @HANDSTANDSAM 

The Dream Espresso Test Suite Easy to write Low maintenance UI tests added during the development cycle Hermetic, deterministic, targeted Run on every PR in Continuous Integration Reliable test execution environment Automated screenshots 

Audience Poll: Testing? Espresso? Screenshots? Continuous Integration? 

Why Espresso? Created by Google Access to app internals Easy to write Blazing fast 

Using Espresso 

Using Espresso 

Using Espresso 

Using Espresso 

Espresso Example @Test public void testLogin() { onView(withText(“Shopping App”))
 .check(matches(isDisplayed())); onView(withId(R.id.username))
 .perform(typeText("sam")); onView(withText(“LOG IN"))
 .perform(click()); } 

Espresso Example @Test public void testLogin() { onView(withText(“Shopping App”))
 .check(matches(isDisplayed())); onView(withId(R.id.username))
 .perform(typeText("sam")); onView(withText(“LOG IN"))
 .perform(click()); } 

Espresso Example @Test public void testLogin() { onView(withText(“Shopping App”))
 .check(matches(isDisplayed())); onView(withId(R.id.username))
 .perform(typeText("sam")); onView(withText(“LOG IN"))
 .perform(click()); } 

Espresso Test Recorder (Android Studio 2.2+) PROS: Great for beginners Great for projects without existing tests CONS: Only works on simple tests Tool is slow Generated code 

Espresso Test Recorder (Android Studio 2.2+) 

Espresso Test Recorder (Android Studio 2.2+) 

Espresso Test Recorder (Android Studio 2.2+) 

Espresso Test Recorder (Android Studio 2.2+) 

Layout Inspector 

No content

Who should write Espresso tests? “Espresso is targeted at developers, who believe that automated testing is an integral part of the development lifecycle. While it can be used for black-box testing, Espresso’s full power is unlocked by those who are familiar with the codebase under test.” 

When Should I write an Espresso test? Avoid unless necessary Only testing the UI Small and targeted 

How is Espresso Fast? Aware of Android’s Main Thread Aware of Android’s AsyncTask Thread Pool Idling Resources Never have to Thread.sleep(5000); 

No content

No content

No content

No content

Configuration for Success Disabled Animations Use Idling Resources for Background work Utilize AsyncTask Thread pool 

Where can I learn more? 

ROBOT TESTING PATTERN FOR ANDROID INSTRUMENTATION TESTS TEST TEST TEST TEST TEST 

ROBOT TESTING PATTERN FOR ANDROID INSTRUMENTATION TESTS TEST TEST TEST TEST TEST 

ROBOT TESTING PATTERN FOR ANDROID INSTRUMENTATION TESTS TEST TEST TEST TEST TEST X X X X X 

ROBOT TESTING PATTERN FOR ANDROID INSTRUMENTATION TESTS TEST TEST TEST TEST TEST ROBOT 

ROBOT TESTING PATTERN FOR ANDROID INSTRUMENTATION TESTS TEST TEST TEST TEST TEST ROBOT X 

ROBOT TESTING PATTERN FOR ANDROID INSTRUMENTATION TESTS TEST TEST TEST TEST TEST ROBOT WHAT HOW 

LOGIN ROBOT new LoginRobot() .username("sam") .password("password") .toggleRememberMe() .login(); 

new LoginRobot().username("sam").password("password").toggleRememberMe().login(); public class LoginRobot { public LoginRobot username(String username) { ViewInteraction usernameEditText = onView(allOf(withId(R.id.username), isDisplayed())); usernameEditText.perform(replaceText(username), closeSoftKeyboard()); return this; } public LoginRobot password(String password) { ViewInteraction passwordEditText = onView(allOf(withId(R.id.password), isDisplayed())); passwordEditText.perform(replaceText(password), closeSoftKeyboard()); return this; } public LoginRobot toggleRememberMe() { ViewInteraction rememberMeCheckbox = onView(allOf(withId(R.id.remember_me), isDisplayed())); rememberMeCheckbox.perform(scrollTo(), click()); return this; } public LoginRobot login() { ViewInteraction submitButton = onView(allOf(withId(R.id.submit), isDisplayed())); submitButton.perform(click()); return this; } } 

new LoginRobot().username("sam").password("password").toggleRememberMe().login(); public class LoginRobot { public LoginRobot username(String username) { ViewInteraction usernameEditText = onView(allOf(withId(R.id.username), isDisplayed())); usernameEditText.perform(replaceText(username), closeSoftKeyboard()); return this; } public LoginRobot password(String password) { ViewInteraction passwordEditText = onView(allOf(withId(R.id.password), isDisplayed())); passwordEditText.perform(replaceText(password), closeSoftKeyboard()); return this; } public LoginRobot toggleRememberMe() { ViewInteraction rememberMeCheckbox = onView(allOf(withId(R.id.remember_me), isDisplayed())); rememberMeCheckbox.perform(scrollTo(), click()); return this; } public LoginRobot login() { ViewInteraction submitButton = onView(allOf(withId(R.id.submit), isDisplayed())); submitButton.perform(click()); return this; } } 

new LoginRobot().username("sam").password("password").toggleRememberMe().login(); public class LoginRobot { public LoginRobot username(String username) { ViewInteraction usernameEditText = onView(allOf(withId(R.id.username), isDisplayed())); usernameEditText.perform(replaceText(username), closeSoftKeyboard()); return this; } public LoginRobot password(String password) { ViewInteraction passwordEditText = onView(allOf(withId(R.id.password), isDisplayed())); passwordEditText.perform(replaceText(password), closeSoftKeyboard()); return this; } public LoginRobot toggleRememberMe() { ViewInteraction rememberMeCheckbox = onView(allOf(withId(R.id.remember_me), isDisplayed())); rememberMeCheckbox.perform(scrollTo(), click()); return this; } public LoginRobot login() { ViewInteraction submitButton = onView(allOf(withId(R.id.submit), isDisplayed())); submitButton.perform(click()); return this; } } 

new LoginRobot().username("sam").password("password").toggleRememberMe().login(); public class LoginRobot { public LoginRobot username(String username) { ViewInteraction usernameEditText = onView(allOf(withId(R.id.username), isDisplayed())); usernameEditText.perform(replaceText(username), closeSoftKeyboard()); return this; } public LoginRobot password(String password) { ViewInteraction passwordEditText = onView(allOf(withId(R.id.password), isDisplayed())); passwordEditText.perform(replaceText(password), closeSoftKeyboard()); return this; } public LoginRobot toggleRememberMe() { ViewInteraction rememberMeCheckbox = onView(allOf(withId(R.id.remember_me), isDisplayed())); rememberMeCheckbox.perform(scrollTo(), click()); return this; } public LoginRobot login() { ViewInteraction submitButton = onView(allOf(withId(R.id.submit), isDisplayed())); submitButton.perform(click()); return this; } } 

HOME ROBOT ITEM
 ROBOT LOGIN ROBOT CATEGORY ROBOT new LoginRobot().username(“sam").password("password").toggleRememberMe().login(); new HomeRobot().category(“Fruits”); new CategoryRobot().item(“Pineapple”); new ItemRobot().addToCart(); MULTI-SCREEN TESTS 

HOME ROBOT ITEM
 ROBOT LOGIN ROBOT CATEGORY ROBOT new LoginRobot().username(“sam").password("password").toggleRememberMe().login(); new HomeRobot().category(“Fruits”); new CategoryRobot().item(“Pineapple”); new ItemRobot().addToCart(); MULTI-SCREEN TESTS 

HOME ROBOT ITEM
 ROBOT LOGIN ROBOT CATEGORY ROBOT new LoginRobot().username(“sam").password("password").toggleRememberMe().login(); new HomeRobot().category(“Fruits”); new CategoryRobot().item(“Pineapple”); new ItemRobot().addToCart(); MULTI-SCREEN TESTS 

HOME ROBOT ITEM
 ROBOT LOGIN ROBOT CATEGORY ROBOT new LoginRobot().username(“sam").password("password").toggleRememberMe().login(); new HomeRobot().category(“Fruits”); new CategoryRobot().item(“Pineapple”); new ItemRobot().addToCart(); MULTI-SCREEN TESTS 

HOME ROBOT ITEM
 ROBOT LOGIN ROBOT CATEGORY ROBOT new LoginRobot().username(“sam").password("password").toggleRememberMe().login(); new HomeRobot().category(“Fruits”); new CategoryRobot().item(“Pineapple”); new ItemRobot().addToCart(); MULTI-SCREEN TESTS 

public class LoginRobot { public LoginRobot username(String username) { ViewInteraction usernameEditText = onView(allOf(withId(R.id.username), isDisplayed())); usernameEditText
 .perform(replaceText(username), closeSoftKeyboard()); takeScreenshot(“entered_username”); return this; } } SCREENSHOTS FOR EACH ROBOT METHOD 

public class LoginRobot { public LoginRobot login() { ViewInteraction submitButton = onView(allOf(withId(R.id.submit), isDisplayed())); takeScreenshot(“logging_in”); submitButton.perform(click()); return this; } } SCREENSHOTS FOR EACH ROBOT METHOD 

Screenshots - WHY? 1. See What’s Being Tested 

1. See What’s Being Tested 

1. See What’s Being Tested 

1. See What’s Being Tested 

1. See What’s Being Tested 

Screenshots - WHY? 1. See What’s Being Tested 2. Diagnose Failures 

2. Diagnose Failures android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: with text: “DONE!” View Hierarchy: +>DecorView{id=-1, visibility=VISIBLE, width=720, height=1280, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is- layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=3} | +->ActionBarOverlayLayout{id=16909230, res-name=decor_content_parent, visibility=VISIBLE, width=720, height=1184, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2} | +-->FrameLayout{id=16908290, res-name=content, visibility=VISIBLE, width=720, height=1024, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is- focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=160.0, child-count=1} | +--->TextView{id=2131230720, res-name=greeting, visibility=VISIBLE, width=656, height=960, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is- focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=32.0, y=32.0, text=Hello world!, input-type=0, ime-target=false, has-links=false} | +-->ActionBarContainer{id=16909231, res-name=action_bar_container, visibility=VISIBLE, width=720, height=112, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is- enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=48.0, child-count=2} | +--->Toolbar{id=16909232, res-name=action_bar, visibility=VISIBLE, width=720, height=112, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is- focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2} | +---->TextView{id=-1, visibility=VISIBLE, width=206, height=54, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is- layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=32.0, y=29.0, text=Hello World, input-type=0, ime-target=false, has-links=false} | +---->ActionMenuView{id=-1, visibility=VISIBLE, width=80, height=112, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is- focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=640.0, y=0.0, child-count=1} | +----->OverflowMenuButton{id=-1, desc=More options, visibility=VISIBLE, width=80, height=96, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=true, is-enabled=true, is- focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=8.0} 

2. Diagnose Failures android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: with text: “DONE!” View Hierarchy: +>DecorView{id=-1, visibility=VISIBLE, width=720, height=1280, has-focus=false, has-focusable=false, has- window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout- requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child- count=3} | +->ActionBarOverlayLayout{id=16909230, res-name=decor_content_parent, visibility=VISIBLE, width=720, height=1184, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2} | +-->FrameLayout{id=16908290, res-name=content, visibility=VISIBLE, width=720, height=1024, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is- focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input- connection=false, x=0.0, y=160.0, child-count=1} | +--->TextView{id=2131230720, res-name=greeting, visibility=VISIBLE, width=656, height=960, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is- focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input- connection=false, x=32.0, y=32.0, text=Hello world!, input-type=0, ime-target=false, has-links=false} | +-->ActionBarContainer{id=16909231, res-name=action_bar_container, visibility=VISIBLE, width=720, height=112, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is- focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=48.0, child-count=2} | +--->Toolbar{id=16909232, res-name=action_bar, visibility=VISIBLE, width=720, height=112, has-focus=false, has- focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2} | +---->TextView{id=-1, visibility=VISIBLE, width=206, height=54, has-focus=false, has-focusable=false, has-window- focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is- selected=false, root-is-layout-requested=false, has-input-connection=false, x=32.0, y=29.0, text=Hello World, input-type=0, ime-target=false, has-links=false} 

Screenshots on Failures 

Screenshots on Failures Espresso.setFailureHandler(new FailureHandler() { @Override public void handle(Throwable throwable, Matcher matcher) { ScreenshotHelper.takeScreenshotForcefully("test_failed"); try { new DefaultFailureHandler(applicationContext).handle(throwable, matcher); } catch (Exception e) { logger.error(e.getMessage(), e); throw new RuntimeException(e); } } }); 

Screenshots - WHY? 1. See What’s Being Tested 2. Diagnose Failures 3. Share Your Tests 

NO ONE Sees The Value of Your Tests… YET Your Bosses Product Owners Designers Manual Testers 

To them it’s just console output. 

Let EVERYONE see
 what’s being tested. 

User Interfaces are VISUAL A User Interface is something very hard to represent in a non-visual way. Imagine you need to tell someone who is visually impaired what the Mona Lisa painting looks like. 

Same Test, But Looks Different on Different Devices 

DEVICE SIZE OS VERSION DEVICE MODEL LANGUAGE 

Test Reports Spoon Screenshot Reports Isolated Logs for each test execution 

Spoon http://square.github.io/spoon/ 

Spoon Test Runner Run instrumentation tests in parallel Shard tests across multiple devices Beautiful HTML reports java -jar spoon-runner-1.7.0-jar-with-dependencies.jar \
 --apk example-app.apk \
 --test-apk example-tests.apk 

Spoon HTML Reports 

No content

How Taking Screenshots Work Traverse View Hierarchy Generate Image Save to disk in
 a specific location adb pull screenshot report finds images 

Screenshot Libraries Spoon Falcon UiAutomator 

Other Screenshot Libraries Firebase Test Lab - Screen Shotter Fastlane - Screengrab 

Should I use Devices or Emulators? Emulator for Development and Continuous Integration Use devices for release regressions 

Use The Android Emulator Run real versions of Android Configurable Faster than real devices Available on demand and scalable Cheaper and lower maintenance For screenshots, run low density (mdpi - 384x640) 

No content

Combating Shared State Clearing preferences Clearing databases Clearing files Resetting singletons Resetting static variables Re-injecting Dependency Injections Killing any Background Work 

How AndroidJUnitRunner Works The more you know… 

No content

No content

No content

Isolated Test Execution Run each test completely separated adb shell pm clear com.mypackage (Total data wipe) 

No content

No content

To Achieve This Run Test Dex Parser to dump all methods https://github.com/linkedin/dex-test-parser Execute each test method individually Combine test results CONS: Need custom scripts Can’t use Spoon Won’t work on Firebase Test Lab 

Android Test Orchestrator Announced at Google I/O 2017. Available in the upcoming version of the Android Testing Support Library. Runs "pm clear” before each run. Service APK running in the background. Will be integrated with Android Studio. 

No content

No content

No content

No content

No content

Small, Targeted Tests Start in the activity you are testing with ActivityTestRule. If your login page breaks, none of your tests will pass. As with any test, isolating it and testing it with the expected inputs is the best way to understand if it’s passing. 

Skip to the important part Jump directly to what you are testing. Setup state without doing it through your UI. 

Hermetic and Deterministic Tests Mock everything external to what you are testing on the UI. Mock all API requests WireMock MockWebServer and RESTMock Mockito 

No content

No content

Intent Boundaries for Targeted Tests 

No content

No content

No content

Continuous Integration Tests must be run to stay up to date Run on every PR Start with Local machine - Have to write and maintain scripts End with cloud hosted - Firebase Test Lab, Amazon Device Farm, Circle CI, Travis CI, BuddyBuild, etc. 

Speed - Parallelization & Sharding Sharding Infinite availability with cloud based services Shard 1 Shard 2 Shard 3 Shard 4 Shard 5 Run Tests 

Words of Wisdom 

WITH ROBOTS AND SCREENSHOTS MAINTAINABLE ESPRESSO TESTS THANK YOU! SAM EDWARDS - @HANDSTANDSAM