Screenshot your Entire App

Screenshot your Entire App

Wouldn't it be great if your testing procedure was just a matter of looking through a folder of screenshots to see if everything looked as expected. No more manual testing or following complicated scripts to reproduce all the edge cases! It's possible, but it'll take some work.

In this presentation, I will give an overview of the steps necessary to implement this testing utopia:

Dependency injection
Mocked dependencies
Espresso testing
Fastlane Screengrab

Come find out how to make your testing as easy as watching a slideshow.

730df0227780e818df8ce1e19c9a6c48?s=128

Edward Dale

June 16, 2016
Tweet

Transcript

  1. Screenshot your Entire App Edward Dale Freeletics © Edward Dale,

    2016 1
  2. Find the bug(s) © Edward Dale, 2016 2

  3. © Edward Dale, 2016 3

  4. © Edward Dale, 2016 4

  5. Some things are best verified visually ! Screenshot © Edward

    Dale, 2016 5
  6. Screenshots are useful for • Acceptance testing • Edge case

    • Localization • Multiple devices • Regression testing © Edward Dale, 2016 6
  7. Demo App1 • Fetch location of user • Fetch weather

    forecast at that location • Show it in a nice way 1 https://github.com/scompt/ScreenshotDemo © Edward Dale, 2016 7
  8. Screenshot cases • How does it look when loading? ©

    Edward Dale, 2016 8
  9. Screenshot cases • How does it look when loading? •

    How does the error screen look? © Edward Dale, 2016 9
  10. Screenshot cases • How does it look when loading? •

    How does the error screen look? • How do all weather icons look? © Edward Dale, 2016 10
  11. Screenshot cases • How does it look when loading? •

    How does the error screen look? • How do all weather icons look? • How does it look in German? © Edward Dale, 2016 11
  12. Screenshot testing tools • Espresso tests • Dependency injection •

    Dependency mocking • Screengrab © Edward Dale, 2016 12
  13. Loading screen Espresso test @Rule public ActivityTestRule<MainActivity> mActivityRule = new

    ActivityTestRule<>(MainActivity.class); @Test public void testProgressIsShownWhileLoading() throws Exception { // The activity is already started here // More about Screengrab later! Screengrab.screenshot("progress"); // No assertions } © Edward Dale, 2016 13
  14. Loading screen test problems • What if the app is

    really fast? • What if the network is disconnected? © Edward Dale, 2016 14
  15. Loading screen test problems • What if the app is

    really fast? • What if the network is disconnected? Answers • Dependency injection • Mocked dependencies © Edward Dale, 2016 15
  16. Without dependency injection public class MainActivity extends AppCompatActivity { WeatherService

    weatherService = new ForecastIoWeatherService(); ... } • MainActivity will always use forecast.io to load weather. © Edward Dale, 2016 16
  17. With dependency injection public class MainActivity extends AppCompatActivity { @Inject

    WeatherService weatherService; ... } • MainActivity doesn't care which WeatherService it uses • Can be different for production and tests © Edward Dale, 2016 17
  18. Dagger dependency injection @Singleton @Component(modules = {ForecastIoModule.class, ... }) public

    interface ScreenshotDemoComponent { void inject(MainActivity target); } @Module @Singleton public class ForecastIoModule { @Provides WeatherService provideWeatherService(ForecastIoWeatherService impl) { return impl; } } © Edward Dale, 2016 18
  19. Dagger dependency injection public class MainActivity extends AppCompatActivity { @Inject

    WeatherService weatherService; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // Fancy Dagger code to do injection ScreenshotDemoApplication.get(this).component().inject(this); } ... } © Edward Dale, 2016 19
  20. Loading screen test pseudocode • Configure app to use WeatherService

    that loads forever • Start activity • Take screenshot © Edward Dale, 2016 20
  21. Mock WeatherService with Mockito WeatherService mockWeatherService = mock(WeatherService.class); Single<LocationWeather> neverending

    = Observable.<LocationWeather>never().toSingle(); doReturn(neverending) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); © Edward Dale, 2016 21
  22. Loading screen mocked test 1 @Rule public ActivityTestRule<MainActivity> mActivityRule =

    new ActivityTestRule<>(MainActivity.class, false, false); @Inject WeatherService mockWeatherService; @Before public void injectTest() { // Inject test dependencies ... } © Edward Dale, 2016 22
  23. Loading screen mocked test 2 @Test public void testProgressIsShownWhileLoading() throws

    Exception { // Configure app doReturn(Observable.<LocationWeather>never().toSingle()) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshot Screengrab.screenshot("progress"); } © Edward Dale, 2016 23
  24. Screenshot cases • How does it look when loading? ©

    Edward Dale, 2016 24
  25. Screenshot cases • How does it look when loading? •

    How does the error screen look? © Edward Dale, 2016 25
  26. Error screen test @Test public void testErrorShown() throws Exception {

    // Configure app doReturn(Observable.error(new Exception("message")).toSingle()) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshot Screengrab.screenshot("error"); } © Edward Dale, 2016 26
  27. Error screen test @Test public void testErrorShown() throws Exception {

    // Configure app doReturn(Observable.error(new Exception("message")).toSingle()) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshot Screengrab.screenshot("progress"); } © Edward Dale, 2016 27
  28. Screenshot cases • How does it look when loading? •

    How does the error screen look? © Edward Dale, 2016 28
  29. Screenshot cases • How does it look when loading? •

    How does the error screen look? • How do all weather icons look? © Edward Dale, 2016 29
  30. All weather test pseudocode • Configure app to use WeatherService

    that has every type of weather • Start activity • Take screenshot • Swipe • Repeat © Edward Dale, 2016 30
  31. All weather test @Test public void testErrorShown() throws Exception {

    // Configure app doReturn(Single.just(parseWeather("all_icons.json"))) .when(weatherService) .weatherForLocation(Matchers.any(Location.class)); .... © Edward Dale, 2016 31
  32. // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshots

    for (int i = 0; i < ICON_COUNT; i++) { Screengrab.screenshot("icon_" + i); onView(withId(R.id.view_pager)).perform(swipeLeft()); } // Still no assertions } © Edward Dale, 2016 32
  33. Screenshot cases • How does it look when loading? •

    How does the error screen look? • How do all weather icons look? © Edward Dale, 2016 33
  34. Screenshot cases • How does it look when loading? •

    How does the error screen look? • How do all weather icons look? • How does it look in German? © Edward Dale, 2016 34
  35. Automated localized screenshots of your Android app on every device

    Screengrab ▶ Fastlane ▶ Fabric ▶ Twitter Combination of command-line tool and classes used in tests © Edward Dale, 2016 35
  36. • Add LocaleTestRule to test classes • Enables locale switching

    @RunWith(AndroidJUnit4.class) @LargeTest public class TestMainActivity { @ClassRule public static final LocaleTestRule localeTestRule = new LocaleTestRule(); ... © Edward Dale, 2016 36
  37. • Install screengrab • Run screengrab init to generate configuration

    file • Adjust configuration © Edward Dale, 2016 37
  38. app_package_name 'com.scompt.screenshotdemo' use_tests_in_packages ['com.scompt.screenshotdemo'] app_apk_path 'app-debug.apk' tests_apk_path 'app-debug-androidTest-unaligned.apk' locales ['en-US',

    'de-DE', 'fr-FR'] ending_locale 'en-US' © Edward Dale, 2016 38
  39. Run tests $ ./gradlew clean assembleDebug assembleDebugTest $ screengrab ©

    Edward Dale, 2016 39
  40. © Edward Dale, 2016 40

  41. © Edward Dale, 2016 41

  42. Issues • Naming and organization of files • Misusing screengrab

    a bit © Edward Dale, 2016 42
  43. Alternative/related solutions • Spoon • Distributes instrumentation tests to all

    devices • Spoon + Screengrab = Amazing? • Firebase Test Lab for Android • Cloud solution that run by Google • Can also run on different locales © Edward Dale, 2016 43
  44. Thanks! Edward Dale (@scompt) Freeletics (We're hiring) © Edward Dale,

    2016 44