Screenshot your Entire App

Wouldn't it be great if your testing procedure was just a matter of looking through a folder of screenshots to see if everything looked as expected. No more manual testing or following complicated scripts to reproduce all the edge cases! It's possible, but it'll take some work.

In this presentation, I will give an overview of the steps necessary to implement this testing utopia:

Dependency injection
Mocked dependencies
Espresso testing
Fastlane Screengrab

Come find out how to make your testing as easy as watching a slideshow.

Transcript

1. Screenshot your Entire App Edward Dale Freeletics © Edward Dale, 2016 1
2. Find the bug(s) © Edward Dale, 2016 2
3. © Edward Dale, 2016 3
4. © Edward Dale, 2016 4
5. Some things are best verified visually ! Screenshot © Edward Dale, 2016 5
6. Screenshots are useful for • Acceptance testing • Edge case • Localization • Multiple devices • Regression testing © Edward Dale, 2016 6
7. Demo App1 • Fetch location of user • Fetch weather forecast at that location • Show it in a nice way 1 https://github.com/scompt/ScreenshotDemo © Edward Dale, 2016 7
8. Screenshot cases • How does it look when loading? © Edward Dale, 2016 8
9. Screenshot cases • How does it look when loading? • How does the error screen look? © Edward Dale, 2016 9
10. Screenshot cases • How does it look when loading? • How does the error screen look? • How do all weather icons look? © Edward Dale, 2016 10
11. Screenshot cases • How does it look when loading? • How does the error screen look? • How do all weather icons look? • How does it look in German? © Edward Dale, 2016 11
12. Screenshot testing tools • Espresso tests • Dependency injection • Dependency mocking • Screengrab © Edward Dale, 2016 12
13. Loading screen Espresso test @Rule public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<>(MainActivity.class); @Test public void testProgressIsShownWhileLoading() throws Exception { // The activity is already started here // More about Screengrab later! Screengrab.screenshot("progress"); // No assertions } © Edward Dale, 2016 13
14. Loading screen test problems • What if the app is really fast? • What if the network is disconnected? © Edward Dale, 2016 14
15. Loading screen test problems • What if the app is really fast? • What if the network is disconnected? Answers • Dependency injection • Mocked dependencies © Edward Dale, 2016 15
16. Without dependency injection public class MainActivity extends AppCompatActivity { WeatherService weatherService = new ForecastIoWeatherService(); ... } • MainActivity will always use forecast.io to load weather. © Edward Dale, 2016 16
17. With dependency injection public class MainActivity extends AppCompatActivity { @Inject WeatherService weatherService; ... } • MainActivity doesn't care which WeatherService it uses • Can be different for production and tests © Edward Dale, 2016 17
18. Dagger dependency injection @Singleton @Component(modules = {ForecastIoModule.class, ... }) public interface ScreenshotDemoComponent { void inject(MainActivity target); } @Module @Singleton public class ForecastIoModule { @Provides WeatherService provideWeatherService(ForecastIoWeatherService impl) { return impl; } } © Edward Dale, 2016 18
19. Dagger dependency injection public class MainActivity extends AppCompatActivity { @Inject WeatherService weatherService; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); // Fancy Dagger code to do injection ScreenshotDemoApplication.get(this).component().inject(this); } ... } © Edward Dale, 2016 19
20. Loading screen test pseudocode • Configure app to use WeatherService that loads forever • Start activity • Take screenshot © Edward Dale, 2016 20
21. Mock WeatherService with Mockito WeatherService mockWeatherService = mock(WeatherService.class); Single<LocationWeather> neverending = Observable.<LocationWeather>never().toSingle(); doReturn(neverending) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); © Edward Dale, 2016 21
22. Loading screen mocked test 1 @Rule public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<>(MainActivity.class, false, false); @Inject WeatherService mockWeatherService; @Before public void injectTest() { // Inject test dependencies ... } © Edward Dale, 2016 22
23. Loading screen mocked test 2 @Test public void testProgressIsShownWhileLoading() throws Exception { // Configure app doReturn(Observable.<LocationWeather>never().toSingle()) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshot Screengrab.screenshot("progress"); } © Edward Dale, 2016 23
24. Screenshot cases • How does it look when loading? © Edward Dale, 2016 24
25. Screenshot cases • How does it look when loading? • How does the error screen look? © Edward Dale, 2016 25
26. Error screen test @Test public void testErrorShown() throws Exception { // Configure app doReturn(Observable.error(new Exception("message")).toSingle()) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshot Screengrab.screenshot("error"); } © Edward Dale, 2016 26
27. Error screen test @Test public void testErrorShown() throws Exception { // Configure app doReturn(Observable.error(new Exception("message")).toSingle()) .when(mockWeatherService) .weatherForLocation(Matchers.any(Location.class)); // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshot Screengrab.screenshot("progress"); } © Edward Dale, 2016 27
28. Screenshot cases • How does it look when loading? • How does the error screen look? © Edward Dale, 2016 28
29. Screenshot cases • How does it look when loading? • How does the error screen look? • How do all weather icons look? © Edward Dale, 2016 29
30. All weather test pseudocode • Configure app to use WeatherService that has every type of weather • Start activity • Take screenshot • Swipe • Repeat © Edward Dale, 2016 30
31. All weather test @Test public void testErrorShown() throws Exception { // Configure app doReturn(Single.just(parseWeather("all_icons.json"))) .when(weatherService) .weatherForLocation(Matchers.any(Location.class)); .... © Edward Dale, 2016 31
32. // Start Activity mActivityRule.launchActivity(null /* Intent */); // Take screenshots for (int i = 0; i < ICON_COUNT; i++) { Screengrab.screenshot("icon_" + i); onView(withId(R.id.view_pager)).perform(swipeLeft()); } // Still no assertions } © Edward Dale, 2016 32
33. Screenshot cases • How does it look when loading? • How does the error screen look? • How do all weather icons look? © Edward Dale, 2016 33
34. Screenshot cases • How does it look when loading? • How does the error screen look? • How do all weather icons look? • How does it look in German? © Edward Dale, 2016 34
35. Automated localized screenshots of your Android app on every device Screengrab ▶ Fastlane ▶ Fabric ▶ Twitter Combination of command-line tool and classes used in tests © Edward Dale, 2016 35
36. • Add LocaleTestRule to test classes • Enables locale switching @RunWith(AndroidJUnit4.class) @LargeTest public class TestMainActivity { @ClassRule public static final LocaleTestRule localeTestRule = new LocaleTestRule(); ... © Edward Dale, 2016 36
37. • Install screengrab • Run screengrab init to generate configuration file • Adjust configuration © Edward Dale, 2016 37
38. app_package_name 'com.scompt.screenshotdemo' use_tests_in_packages ['com.scompt.screenshotdemo'] app_apk_path 'app-debug.apk' tests_apk_path 'app-debug-androidTest-unaligned.apk' locales ['en-US', 'de-DE', 'fr-FR'] ending_locale 'en-US' © Edward Dale, 2016 38
39. Run tests $ ./gradlew clean assembleDebug assembleDebugTest $ screengrab © Edward Dale, 2016 39
40. © Edward Dale, 2016 40
41. © Edward Dale, 2016 41
42. Issues • Naming and organization of files • Misusing screengrab a bit © Edward Dale, 2016 42
43. Alternative/related solutions • Spoon • Distributes instrumentation tests to all devices • Spoon + Screengrab = Amazing? • Firebase Test Lab for Android • Cloud solution that run by Google • Can also run on different locales © Edward Dale, 2016 43
44. Thanks! Edward Dale (@scompt) Freeletics (We're hiring) © Edward Dale, 2016 44