Screenshot your Entire App

Wouldn't it be great if your testing procedure was just a matter of looking through a folder of screenshots to see if everything looked as expected. No more manual testing or following complicated scripts to reproduce all the edge cases! It's possible, but it'll take some work.

In this presentation, I will give an overview of the steps necessary to implement this testing utopia:

Dependency injection
Mocked dependencies
Espresso testing
Fastlane Screengrab

Come find out how to make your testing as easy as watching a slideshow.

Transcript

1. Screenshot
   your Entire App
   Edward Dale
   Freeletics
   © Edward Dale, 2016 1
   

   View Slide

2. Find the bug(s)
   © Edward Dale, 2016 2
   

   View Slide

3. © Edward Dale, 2016 3
   

   View Slide

4. © Edward Dale, 2016 4
   

   View Slide

5. Some things
   are best
   verified visually
   !
   Screenshot
   © Edward Dale, 2016 5
   

   View Slide

6. Screenshots are useful for
   • Acceptance testing
   • Edge case
   • Localization
   • Multiple devices
   • Regression testing
   © Edward Dale, 2016 6
   

   View Slide

7. Demo App1
   • Fetch location of user
   • Fetch weather forecast at that
   location
   • Show it in a nice way
   1 https://github.com/scompt/ScreenshotDemo
   © Edward Dale, 2016 7
   

   View Slide

8. Screenshot cases
   • How does it look when loading?
   © Edward Dale, 2016 8
   

   View Slide

9. Screenshot cases
   • How does it look when loading?
   • How does the error screen look?
   © Edward Dale, 2016 9
   

   View Slide

10. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    © Edward Dale, 2016 10
    

    View Slide

11. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    • How does it look in German?
    © Edward Dale, 2016 11
    

    View Slide

12. Screenshot testing
    tools
    • Espresso tests
    • Dependency injection
    • Dependency mocking
    • Screengrab
    © Edward Dale, 2016 12
    

    View Slide

13. Loading screen
    Espresso test
    @Rule
    public ActivityTestRule mActivityRule
    = new ActivityTestRule<>(MainActivity.class);
    @Test
    public void testProgressIsShownWhileLoading()
    throws Exception {
    // The activity is already started here
    // More about Screengrab later!
    Screengrab.screenshot("progress");
    // No assertions
    }
    © Edward Dale, 2016 13
    

    View Slide

14. Loading screen test
    problems
    • What if the app is really fast?
    • What if the network is disconnected?
    © Edward Dale, 2016 14
    

    View Slide

15. Loading screen test
    problems
    • What if the app is really fast?
    • What if the network is disconnected?
    Answers
    • Dependency injection
    • Mocked dependencies
    © Edward Dale, 2016 15
    

    View Slide

16. Without dependency injection
    public class MainActivity extends AppCompatActivity {
    WeatherService weatherService = new ForecastIoWeatherService();
    ...
    }
    • MainActivity will always use forecast.io to load weather.
    © Edward Dale, 2016 16
    

    View Slide

17. With dependency injection
    public class MainActivity extends AppCompatActivity {
    @Inject
    WeatherService weatherService;
    ...
    }
    • MainActivity doesn't care which WeatherService it uses
    • Can be different for production and tests
    © Edward Dale, 2016 17
    

    View Slide

18. Dagger dependency injection
    @Singleton
    @Component(modules = {ForecastIoModule.class, ... })
    public interface ScreenshotDemoComponent {
    void inject(MainActivity target);
    }
    @Module
    @Singleton
    public class ForecastIoModule {
    @Provides
    WeatherService provideWeatherService(ForecastIoWeatherService impl) {
    return impl;
    }
    }
    © Edward Dale, 2016 18
    

    View Slide

19. Dagger dependency injection
    public class MainActivity extends AppCompatActivity {
    @Inject
    WeatherService weatherService;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    // Fancy Dagger code to do injection
    ScreenshotDemoApplication.get(this).component().inject(this);
    }
    ...
    }
    © Edward Dale, 2016 19
    

    View Slide

20. Loading screen test
    pseudocode
    • Configure app to use
    WeatherService that loads forever
    • Start activity
    • Take screenshot
    © Edward Dale, 2016 20
    

    View Slide

21. Mock WeatherService with
    Mockito
    WeatherService mockWeatherService
    = mock(WeatherService.class);
    Single neverending
    = Observable.never().toSingle();
    doReturn(neverending)
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    © Edward Dale, 2016 21
    

    View Slide

22. Loading screen mocked test 1
    @Rule
    public ActivityTestRule mActivityRule
    = new ActivityTestRule<>(MainActivity.class, false, false);
    @Inject
    WeatherService mockWeatherService;
    @Before
    public void injectTest() {
    // Inject test dependencies
    ...
    }
    © Edward Dale, 2016 22
    

    View Slide

23. Loading screen mocked test 2
    @Test
    public void testProgressIsShownWhileLoading()
    throws Exception {
    // Configure app
    doReturn(Observable.never().toSingle())
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshot
    Screengrab.screenshot("progress");
    }
    © Edward Dale, 2016 23
    

    View Slide

24. Screenshot cases
    • How does it look when loading?
    © Edward Dale, 2016 24
    

    View Slide

25. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    © Edward Dale, 2016 25
    

    View Slide

26. Error screen test
    @Test
    public void testErrorShown()
    throws Exception {
    // Configure app
    doReturn(Observable.error(new Exception("message")).toSingle())
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshot
    Screengrab.screenshot("error");
    }
    © Edward Dale, 2016 26
    

    View Slide

27. Error screen test
    @Test
    public void testErrorShown()
    throws Exception {
    // Configure app
    doReturn(Observable.error(new Exception("message")).toSingle())
    .when(mockWeatherService)
    .weatherForLocation(Matchers.any(Location.class));
    // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshot
    Screengrab.screenshot("progress");
    }
    © Edward Dale, 2016 27
    

    View Slide

28. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    © Edward Dale, 2016 28
    

    View Slide

29. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    © Edward Dale, 2016 29
    

    View Slide

30. All weather test
    pseudocode
    • Configure app to use
    WeatherService that has every type
    of weather
    • Start activity
    • Take screenshot
    • Swipe
    • Repeat
    © Edward Dale, 2016 30
    

    View Slide

31. All weather test
    @Test
    public void testErrorShown()
    throws Exception {
    // Configure app
    doReturn(Single.just(parseWeather("all_icons.json")))
    .when(weatherService)
    .weatherForLocation(Matchers.any(Location.class));
    ....
    © Edward Dale, 2016 31
    

    View Slide

32. // Start Activity
    mActivityRule.launchActivity(null /* Intent */);
    // Take screenshots
    for (int i = 0; i < ICON_COUNT; i++) {
    Screengrab.screenshot("icon_" + i);
    onView(withId(R.id.view_pager)).perform(swipeLeft());
    }
    // Still no assertions
    }
    © Edward Dale, 2016 32
    

    View Slide

33. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    © Edward Dale, 2016 33
    

    View Slide

34. Screenshot cases
    • How does it look when loading?
    • How does the error screen look?
    • How do all weather icons look?
    • How does it look in German?
    © Edward Dale, 2016 34
    

    View Slide

35. Automated localized screenshots of your Android app on
    every device
    Screengrab ▶ Fastlane ▶ Fabric ▶ Twitter
    Combination of command-line tool and classes used in tests
    © Edward Dale, 2016 35
    

    View Slide

36. • Add LocaleTestRule to test classes
    • Enables locale switching
    @RunWith(AndroidJUnit4.class)
    @LargeTest
    public class TestMainActivity {
    @ClassRule
    public static final LocaleTestRule localeTestRule
    = new LocaleTestRule();
    ...
    © Edward Dale, 2016 36
    

    View Slide

37. • Install screengrab
    • Run screengrab init to generate configuration file
    • Adjust configuration
    © Edward Dale, 2016 37
    

    View Slide

38. app_package_name 'com.scompt.screenshotdemo'
    use_tests_in_packages ['com.scompt.screenshotdemo']
    app_apk_path 'app-debug.apk'
    tests_apk_path 'app-debug-androidTest-unaligned.apk'
    locales ['en-US', 'de-DE', 'fr-FR']
    ending_locale 'en-US'
    © Edward Dale, 2016 38
    

    View Slide

39. Run tests
    $ ./gradlew clean assembleDebug assembleDebugTest
    $ screengrab
    © Edward Dale, 2016 39
    

    View Slide

40. © Edward Dale, 2016 40
    

    View Slide

41. © Edward Dale, 2016 41
    

    View Slide

42. Issues
    • Naming and organization of files
    • Misusing screengrab a bit
    © Edward Dale, 2016 42
    

    View Slide

43. Alternative/related solutions
    • Spoon
    • Distributes instrumentation tests to all devices
    • Spoon + Screengrab = Amazing?
    • Firebase Test Lab for Android
    • Cloud solution that run by Google
    • Can also run on different locales
    © Edward Dale, 2016 43
    

    View Slide

44. Thanks!
    Edward Dale (@scompt)
    Freeletics (We're hiring)
    © Edward Dale, 2016 44
    

    View Slide