$30 off During Our Annual Pro Sale. View Details »

Screenshot your Entire App

Screenshot your Entire App

Wouldn't it be great if your testing procedure was just a matter of looking through a folder of screenshots to see if everything looked as expected. No more manual testing or following complicated scripts to reproduce all the edge cases! It's possible, but it'll take some work.

In this presentation, I will give an overview of the steps necessary to implement this testing utopia:

Dependency injection
Mocked dependencies
Espresso testing
Fastlane Screengrab

Come find out how to make your testing as easy as watching a slideshow.

Edward Dale

June 16, 2016
Tweet

More Decks by Edward Dale

Other Decks in Programming

Transcript

  1. Screenshot
    your Entire App
    Edward Dale
    Freeletics
    © Edward Dale, 2016 1

    View Slide

  2. Find the bug(s)
    © Edward Dale, 2016 2

    View Slide

  3. © Edward Dale, 2016 3

    View Slide

  4. © Edward Dale, 2016 4

    View Slide

  5. Some things
    are best
    verified visually
    !
    Screenshot
    © Edward Dale, 2016 5

    View Slide

  6. Screenshots are useful for
    • Acceptance testing
    • Edge case
    • Localization
    • Multiple devices
    • Regression testing
    © Edward Dale, 2016 6

    View Slide

  7. Demo App1
    • Fetch location of user
    • Fetch weather forecast at that
    location
    • Show it in a nice way
    1 https://github.com/scompt/ScreenshotDemo
    © Edward Dale, 2016 7

    View Slide

  8. Screenshot cases
    • How does it look when loading?
    © Edward Dale, 2016 8

    View Slide

  9. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    © Edward Dale, 2016 9

    View Slide

  10. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    © Edward Dale, 2016 10

    View Slide

  11. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    • How does it look in German?
    © Edward Dale, 2016 11

    View Slide

  12. Screenshot testing
    tools
    • Espresso tests
    • Dependency injection
    • Dependency mocking
    • Screengrab
    © Edward Dale, 2016 12

    View Slide

  13. Loading screen
    Espresso test
    @Rule
    public ActivityTestRule mActivityRule
    = new ActivityTestRule<>(MainActivity.class);
    @Test
    public void testProgressIsShownWhileLoading()
    throws Exception {
    // The activity is already started here
    // More about Screengrab later!
    Screengrab.screenshot("progress");
    // No assertions
    }
    © Edward Dale, 2016 13

    View Slide

  14. Loading screen test
    problems
    • What if the app is really fast?
    • What if the network is disconnected?
    © Edward Dale, 2016 14

    View Slide

  15. Loading screen test
    problems
    • What if the app is really fast?
    • What if the network is disconnected?
    Answers
    • Dependency injection
    • Mocked dependencies
    © Edward Dale, 2016 15

    View Slide

  16. Without dependency injection
    public class MainActivity extends AppCompatActivity {
    WeatherService weatherService = new ForecastIoWeatherService();
    ...
    }
    • MainActivity will always use forecast.io to load weather.
    © Edward Dale, 2016 16

    View Slide

  17. With dependency injection
    public class MainActivity extends AppCompatActivity {
    @Inject
    WeatherService weatherService;
    ...
    }
    • MainActivity doesn't care which WeatherService it uses
    • Can be different for production and tests
    © Edward Dale, 2016 17

    View Slide

  18. Dagger dependency injection
    @Singleton
    @Component(modules = {ForecastIoModule.class, ... })
    public interface ScreenshotDemoComponent {
    void inject(MainActivity target);
    }
    @Module
    @Singleton
    public class ForecastIoModule {
    @Provides
    WeatherService provideWeatherService(ForecastIoWeatherService impl) {
    return impl;
    }
    }
    © Edward Dale, 2016 18

    View Slide

  19. Dagger dependency injection
    public class MainActivity extends AppCompatActivity {
    @Inject
    WeatherService weatherService;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    // Fancy Dagger code to do injection
    ScreenshotDemoApplication.get(this).component().inject(this);
    }
    ...
    }
    © Edward Dale, 2016 19

    View Slide

  20. Loading screen test
    pseudocode
    • Configure app to use
    WeatherService that loads forever
    • Start activity
    • Take screenshot
    © Edward Dale, 2016 20

    View Slide

  21. Mock WeatherService with
    Mockito
    WeatherService mockWeatherService
    = mock(WeatherService.class);
    Single neverending
    = Observable.never().toSingle();
    doReturn(neverending)
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    © Edward Dale, 2016 21

    View Slide

  22. Loading screen mocked test 1
    @Rule
    public ActivityTestRule mActivityRule
    = new ActivityTestRule<>(MainActivity.class, false, false);
    @Inject
    WeatherService mockWeatherService;
    @Before
    public void injectTest() {
    // Inject test dependencies
    ...
    }
    © Edward Dale, 2016 22

    View Slide

  23. Loading screen mocked test 2
    @Test
    public void testProgressIsShownWhileLoading()
    throws Exception {
    // Configure app
    doReturn(Observable.never().toSingle())
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshot
    Screengrab.screenshot("progress");
    }
    © Edward Dale, 2016 23

    View Slide

  24. Screenshot cases
    • How does it look when loading?
    © Edward Dale, 2016 24

    View Slide

  25. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    © Edward Dale, 2016 25

    View Slide

  26. Error screen test
    @Test
    public void testErrorShown()
    throws Exception {
    // Configure app
    doReturn(Observable.error(new Exception("message")).toSingle())
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshot
    Screengrab.screenshot("error");
    }
    © Edward Dale, 2016 26

    View Slide

  27. Error screen test
    @Test
    public void testErrorShown()
    throws Exception {
    // Configure app
    doReturn(Observable.error(new Exception("message")).toSingle())
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshot
    Screengrab.screenshot("progress");
    }
    © Edward Dale, 2016 27

    View Slide

  28. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    © Edward Dale, 2016 28

    View Slide

  29. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    © Edward Dale, 2016 29

    View Slide

  30. All weather test
    pseudocode
    • Configure app to use
    WeatherService that has every type
    of weather
    • Start activity
    • Take screenshot
    • Swipe
    • Repeat
    © Edward Dale, 2016 30

    View Slide

  31. All weather test
    @Test
    public void testErrorShown()
    throws Exception {
    // Configure app
    doReturn(Single.just(parseWeather("all_icons.json")))
    .when(weatherService)
    .weatherForLocation(Matchers.any(Location.class));
    ....
    © Edward Dale, 2016 31

    View Slide

  32. // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshots
    for (int i = 0; i < ICON_COUNT; i++) {
    Screengrab.screenshot("icon_" + i);
    onView(withId(R.id.view_pager)).perform(swipeLeft());
    }
    // Still no assertions
    }
    © Edward Dale, 2016 32

    View Slide

  33. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    © Edward Dale, 2016 33

    View Slide

  34. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    • How does it look in German?
    © Edward Dale, 2016 34

    View Slide

  35. Automated localized screenshots of your Android app on
    every device
    Screengrab ▶ Fastlane ▶ Fabric ▶ Twitter
    Combination of command-line tool and classes used in tests
    © Edward Dale, 2016 35

    View Slide

  36. • Add LocaleTestRule to test classes
    • Enables locale switching
    @RunWith(AndroidJUnit4.class)
    @LargeTest
    public class TestMainActivity {
    @ClassRule
    public static final LocaleTestRule localeTestRule
    = new LocaleTestRule();
    ...
    © Edward Dale, 2016 36

    View Slide

  37. • Install screengrab
    • Run screengrab init to generate configuration file
    • Adjust configuration
    © Edward Dale, 2016 37

    View Slide

  38. app_package_name 'com.scompt.screenshotdemo'
    use_tests_in_packages ['com.scompt.screenshotdemo']
    app_apk_path 'app-debug.apk'
    tests_apk_path 'app-debug-androidTest-unaligned.apk'
    locales ['en-US', 'de-DE', 'fr-FR']
    ending_locale 'en-US'
    © Edward Dale, 2016 38

    View Slide

  39. Run tests
    $ ./gradlew clean assembleDebug assembleDebugTest
    $ screengrab
    © Edward Dale, 2016 39

    View Slide

  40. © Edward Dale, 2016 40

    View Slide

  41. © Edward Dale, 2016 41

    View Slide

  42. Issues
    • Naming and organization of files
    • Misusing screengrab a bit
    © Edward Dale, 2016 42

    View Slide

  43. Alternative/related solutions
    • Spoon
    • Distributes instrumentation tests to all devices
    • Spoon + Screengrab = Amazing?
    • Firebase Test Lab for Android
    • Cloud solution that run by Google
    • Can also run on different locales
    © Edward Dale, 2016 43

    View Slide

  44. Thanks!
    Edward Dale (@scompt)
    Freeletics (We're hiring)
    © Edward Dale, 2016 44

    View Slide