Mobile Testing in the cloud

Android has a fragmented ecosystem. There are devices out there with square screens or even no 3G antenna. It’s impossible to test on all of them. But if we write automated tests, then maybe we could run these tests on hundreds of real devices in the cloud. Let’s compare Espresso to Appium and Amazon Device Farm to Firebase Device Lab and see which of these we like more.

Presented at:
- Codecamp Iasi Autumn 2016
- MBLTdev Moscow 2016
- Codecamp Cluj Autumn 2016
- DevFest Romania 2016

Transcript

1. Mobile testing
   in the cloud
   Andrei Diaconu
   

   View Slide

2. http://www.andreidiaconu.com http://www.androidiasi.ro
   Andrei Diaconu Android Iasi
   Me
   

   View Slide

3. 1. Mobile testing
   2. in the cloud
   

   View Slide

4. 2 Options
   Simple Java Unit Tests
   UI tests
   

   View Slide

5. 2 Options
   Simple Java Unit Tests
   UI tests
   Test components in isolation
   Super fast
   

   View Slide

6. 2 Options
   Simple Java Unit Tests
   UI tests
   Test components in isolation
   Super fast
   Simulate user interaction
   Slow and need Android
   

   View Slide

7. +
   

   View Slide

8. Inside man Outside man
   

   View Slide

9. Inside man Outside man
   • Access to code
   • Can change anything
   • Can simulate situations
   • Uses accessibility
   • Simulates a user
   • Can run on any app
   

   View Slide

10. Inside man Outside man
    

    View Slide

11. Inside man Outside man
    Instrumentation
    Framework
    UiAutomator
    

    View Slide

12. Inside man Outside man
    

    View Slide

13. How they work
    

    View Slide

14. View Slide

15. UiAutomator
    AppiumDriver
    JUnit
    

    View Slide

16. Instrumentation
    Framework
    JUnit
    

    View Slide

17. Instrumentation
    Framework
    UiAutomator
    JUnit
    

    View Slide

18. Let's validate
    this scenario
    

    View Slide

19. onView(withText("Favorites"))

    .perform(click());
    

    onView(withId(R.id.swipe_to_refresh))

    .perform(swipeDown());

    

    onView(withText(containsString("There are no Sessions")))

    .check(matches(isDisplayed()));
    

    View Slide

20. onView(withText("Favorites"))

    .perform(click());
    

    onView(withId(R.id.swipe_to_refresh))

    .perform(swipeDown());

    

    onView(withText(containsString("There are no Sessions")))

    .check(matches(isDisplayed()));
    

    View Slide

21. onView(withText("Favorites"))

    .perform(click());
    

    onView(withId(R.id.swipe_to_refresh))

    .perform(swipeDown());

    

    onView(withText(containsString("There are no Sessions")))

    .check(matches(isDisplayed()));
    

    View Slide

22. Let's validate
    this scenario
    

    View Slide

23. Record from Android Studio
    

    View Slide

24. driver
    .findElement(containsText("Favorites"))

    .click();
    

    driver.swipe(300, 300, 300, 600, 1500);

    

    assertTrue(
    driver
    .findElement(containsText("There are no Sessions"))

    .isDisplayed());
    ...
    private By containsText(String text){

    return By.xpath("//*[contains(@text,'"+text+"')]");

    }
    

    View Slide

25. driver
    .findElement(containsText("Favorites"))

    .click();
    

    driver.swipe(300, 300, 300, 600, 1500);

    

    assertTrue(
    driver
    .findElement(containsText("There are no Sessions"))

    .isDisplayed());
    ...
    private By containsText(String text){

    return By.xpath("//*[contains(@text,'"+text+"')]");

    }
    

    View Slide

26. driver
    .findElement(containsText("Favorites"))

    .click();
    

    driver.swipe(300, 300, 300, 600, 1500);

    

    assertTrue(
    driver
    .findElement(containsText("There are no Sessions"))

    .isDisplayed());
    ...
    private By containsText(String text){

    return By.xpath("//*[contains(@text,'"+text+"')]");

    }
    

    View Slide

27. driver
    .findElement(containsText("Favorites"))

    .click();
    

    driver.swipe(300, 300, 300, 600, 1500);

    

    assertTrue(
    driver
    .findElement(containsText("There are no Sessions"))

    .isDisplayed());
    ...
    private By containsText(String text){

    return By.xpath("//*[contains(@text,'"+text+"')]");

    }
    

    View Slide

28. Mobile testing
    in the cloud
    

    View Slide

29. in the cloud
    

    View Slide

30. Firebase Test Lab
    

    View Slide

31. Upload app and instrumentation apk
    

    View Slide

32. Select devices and configurations
    

    View Slide

33. Wait for tests to complete
    

    View Slide

34. View results, including Screenshots + Video
    

    View Slide

35. Galaxy Tab 3
    Nexus 7
    Nexus 5
    Nexus 4
    Lg G3
    

    View Slide

36. View Slide

37. Select devices and configurations
    

    View Slide

38. Configure from Android Studio
    

    View Slide

39. Run from Android Studio
    

    View Slide

40. Amazon Device Farm
    

    View Slide

41. More options,
    including Appium
    

    View Slide

42. More devices
    

    View Slide

43. Knows a few other tricks
    

    View Slide

44. Less stable
    

    View Slide

45. On Amazon you can also manual test
    

    View Slide

46. View Slide

47. So what should you pick?
    OR
    OR
    

    View Slide

48. Record from Android Studio
    

    View Slide

49. Firebase codelab (including Jenkins integration):
    https://codelabs.developers.google.com/codelabs/firebase-test-lab/index.html
    Testing codelab (including Espresso):
    https://codelabs.developers.google.com/codelabs/android-testing/#0
    Codecamp App:
    https://play.google.com/store/apps/details?id=ro.androidiasi.codecamp.iasi.live
    Android Iasi:
    http://androidiasi.ro
    Android Iasi Discussion Group:
    http://bit.ly/android-iasi
    Resources
    

    View Slide

50. http://www.androidiasi.ro
    http://bit.ly/android-iasi
    

    View Slide