Mobile automation state 2019

Slides from my talk @ SeleniumCamp 2019, Kiev, Ukraine

Useful resources:
http://appium.io/docs/en/drivers/android-espresso/
https://github.com/testdotai/appium-classifier-plugin
https://github.com/wix/Detox
https://github.com/android/android-test
http://robolectric.org/blog/2018/10/25/robolectric-4-0/
https://developer.apple.com/videos/play/wwdc2018/403/

Transcript

1. Mobile automation state 2019
2. Hello! I am Dmitry Lemeshko Developer in Test @ Elastic dmle 2
3. Agenda ▣ Available testing tools ▣ New driver and plugin from Appium ▣ AndroidX reboots native testing ▣ Tests distribution and RDC in Native iOS ▣ Blitz: Appium vs Native tools ▣ Points to consider on tool selection 3
4. Testing Tools 2019 4 Appium Espresso XCUITest Detox Xamarin UI Test
5. Platform Application type Language Require source Dev Appium Android,iOS Native, Web, Hybrid Any* No OSS Espresso Android Native,Hybrid Kotlin,Java Yes OSS* XCUITest iOS Native Swift,ObjC Yes Closed Detox Android,iOS Native Javascript No OSS UI Test Android,iOS Native,Hybrid C# No Closed
6. Community recommendation 6 Ionic Appium+Protractor React Native Detox Xamarin Xamarin UI Test Framework Test tool
7. ‘’ 7 Appium becomes a driver management interface
8. 8 Synchronization Source code Appium in 2018
9. Android drivers history 9 ▣ February 2013 □ Appium-android-driver ▣ April 2013 □ Appium-selendroid-driver ▣ July 2013 □ Appium-chromedriver ▣ November 2016 □ Appium-uiautomator2-driver ▣ October 2018 □ Appium-espresso-driver
10. Espresso driver Push the limits 10
11. How it works? 11 settings Espresso server AUT NanoHttpD Still the same black-box tool automationName: espresso
12. Features & improvements ▣ Check Idle state 12 ▣ New locator strategy □ AndroidViewTag ▣ New mobile commands □ backdoor {target, methods, elementId} □ scrollToPage {element, scrollToPage, smoothScroll} □ uiautomator {strategy, locator, action} ▣ 20% faster vs UiAutomator2
13. Element Find Plugin Extending search 13
14. 14 Powerful driver extension EF plugin
15. Adding your plugin 1. Install plugin as Node module npm i -g my-chart-test-plugin 15 2. Start session driver = wd.promiseChainRemote({ ... customFindModules: {my-plugin: my-chart-test-plugin} }); 3. Find element driver.findElement('-custom', 'my-plugin:barChart');
16. 16 Appium classifier plugin ML-based search with image procession findElement('-custom', 'ai:settings')
17. ‘’ Native tools is a fast-growing trend: Developers take ownership 17
18. 18 Android testing stack Different APIs Different Libraries Incompatibility
19. AndroidX Test Rebooting 19
20. Unified Test API 20 Core JUnit Truth Espresso
21. It’s Open Source! 21 You are welcome to contribute
22. 22 Run tests everywhere JVM EMULATOR DEVICE
23. 23 @RunWith(AndroidJUnit4::class) class LoginTest { @Test fun loginWithValidCredentials() { val scenario = ActivityScenario.launch(LoginActivity::class.java) scenario.onActivity { activity -> onView(withId(R.id.email)).perform(typeText(email)) onView(withId(R.id.password)).perform(typeText(password)) onView(withId(R.id.login)).perform(click()) } assertThat(getIntents().first()) .hasComponentClass(HomeActivity::class.java) } }
24. iOS Distributed Testing 24
25. Parallel Testing testA testB … testZ Xcode 9 testA testB … testZ testA testB … testZ
26. Random distributed Testing testM testZ … testP Xcode 10 testB testF … testX testA testC … testE
27. 27 Real device How it works? Each test runner creates a clone of the original simulator
28. XCUITest is finally on RDC 2. Build ipa files: main app and tests 28 4. Download cloud test runner (JAR file) 3. Upload files to cloud provider 5. Run JAR to start testing java -jar runner.jar --apikey 123ABC --device iPhoneXS 1. Decide on iOS version to test
29. 29 class RecipesListTest { private lazy var cells = XCUIApplication().cells func testRecipesList() { for i in 0..<cells.count { let cell = cells.element(boundBy: i) let texts = cell.staticTexts XCTAssertEqual(texts.element(boundBy: 0).label, exp[i].title) XCTAssertEqual(texts.element(boundBy: 1).label, exp[i].desc) if i % 2 != 0 { cell.swipeUp() } ...
30. Appium vs Native Tools 30
31. Blitz: Tool Advantages ▣ Any language ▣ Speed 31 ▣ Flexibility ▣ Community ▣ Selenoid ▣ Extensibility ▣ Easy start ▣ Stability ▣ Documentation ▣ Mocking back-end ▣ Platform standard ▣ Easy integration Appium Espresso/XCUITest
32. Clear points for your choice ▣ Single platform ▣ 2 Platforms + Web 32 ▣ Component testing ▣ Dev team does QA ▣ Part of CD pipeline ▣ Visual Testing ▣ Source restriction ▣ Several AUTs Appium Espresso/XCUITest
33. Appium weakness shows up in 33 ▣ Custom UI components ▣ Views with heavy structure ▣ Animations ▣ Scaling / Resources usage
34. Native tools still not good at ▣ Runners/Matchers customization ▣ 34 ▣ New release frequency ▣ Community ▣ Integrations (e.g. XCode server)
35. ‘’ 35 No winner because there is no competition. Both easy to start and integrate, you need to think about which tool fits better your app, process and testing goals.
36. Create POC Define goals and work on pilot project Make decision Build automation project on top of your POC Pros vs Cons Stability, Limitations, Integration, Support 36
37. Thanks! Any questions? 37 dmle automateMeNow