Building the automated Android UI testing in Continuous integration at LINE MAN Wongnai

The storytelling of how we use GitLab CI and the Genymotion Device Image for automated UI testing in Android at LINE MAN Wongnai.

Transcript

1. Building the automated Android UI testing in Continuous Integration Somkiat Khitwongwattana Android GDE @akexorcist At LINE MAN Wongnai
2. LINE MAN Wongnai LINE MAN Rider LINE MAN Driver Wongnai Merchant Wongnai POS Consumer Local Business Rider & Driver Wongnai POS Manager
3. LINE MAN Wongnai LINE MAN Rider LINE MAN Driver Wongnai Merchant Wongnai POS React Native Native Android (Kotlin) Wongnai POS Manager
4. Running Android UI testing in continuous integration is challenging for developers, but it's necessary for future scalability of Android apps.
5. Our previous infrastructure Which isn’t good enough for future requirements • Not scalable • Not maintainable • Unstable 🥲 😢 😭
6. Native Android React Native Marathon Detox Allure Jest Yes No UI Testing Integration End-to-end Test Reporting Yes Yes Test Executor Continuous Integration GitLab GitLab
7. Android Device Provider Marathon Detox Allure Jest Yes No UI Testing Integration End-to-end Test Reporting Yes Yes Test Executor tinuous gration tLab tLab
8. Candidates Smartphone Test Farm Genymotion Device Image Firebase Test Lab
9. Firebase Test Lab Key Features • Test both Android and iOS apps • Real and virtual devices • Google Play Service included • Wide range of supported devices • Test matrix execution is supported Note: Test reports are stored on Google Cloud Storage https://firebase.google.com/products/test-lab
10. Firebase Test Lab Pricing Virtual device • No-cost up to 60 min / day • Then $1 / device / hour Physical device • No-cost up to 30 min / day • Then $5 / device / hour
11. Genymotion Device Image Key Features • Available on well-known cloud service providers • Virtual devices without GPU • Remotely control via web browser and HTTP API • Run app for ARM natively with ARM64 instances • Provide SSH and ADB access https://www.genymotion.com
12. Genymotion Device Image Pricing Depending on the instance size t4g.medium • 2 vCPU • 4GB of memory • $0.295 / hr
13. Smartphone Test Farm Key Features • Open-source project • Remote control any device from your browser • Monitor your device inventory • Reserve and release any device with STF API https://github.com/DeviceFarmer/stf
14. Price Maintainability Custom Test Executor VPN Stability Configurability Ease of use
15. Price Ease of use C ustom Test Executor VPN Stability M aintainability C onfigurability Firebase Test Lab Genymotion Device Image Smartphone Test Farm
16. Swollen battery problem on a physical device
17. Build App APK & Test APK Run UI Test Generate Test Report
18. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance
19. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance
20. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Terraform
21. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Connect ADB & Setup device Shell Script
22. Why initialization? • ADB disabled by default • No Google Mobile Services • Undesired screen size • No WebView engine • English language by default • Reboot required to apply some changes
23. Genymotion HTTP API • ADB • Clipboard • Screen resolution • Screen orientation • Download, upload, or delete file • Send key, mouse, or tap event • Screenshot • Battery • GPS • And more
24. OpenGApps An open-source initiative that provides Google Apps package Full package • Google Chrome • Google Maps • Google Play services https://opengapps.org
25. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB
26. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB POST /api/v1/configure/adb { "active": true, "active_on_reboot": true }
27. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb connect <ip_address>
28. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB POST /api/v1/android/resolution { "width": 720, "height": 1440, "dpi": 300 }
29. Screen resolution configuration Mobile Tablet • 720 x 1,440 px • 384 x 768 dp • 300 DPI • 1,335 x 751 px • 1,335 x 751 dp • 160 DPI
30. Screen resolution configuration Mobile Tablet • 720 x 1,440 px • 384 x 768 dp • 300 DPI • 1,335 x 751 px • 1,335 x 751 dp • 160 DPI https://developer.android.com/reference/android/util/DisplayMetrics
31. Screen resolution configuration Mobile Tablet • 720 x 1,440 px • 384 x 768 dp • 300 DPI • 1,335 x 751 px • 1,335 x 751 dp • 160 DPI https://developer.android.com/reference/android/util/DisplayMetrics DP x (DPI / 160) = PX
32. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB POST /api/v1/sensors/gps { "latitude": 13.7223497, "longitude": 100.5804235, "altitude": 0, "accuracy": 10, "bearing": 0, "speed": 0 }
33. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell setprop persist.sys.local "th-TH"
34. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell wget <download_url> -O /sdcard/Download/opengapps.zip adb shell system/bin/flash-archive.sh /sdcard/Download/opengapps.zip
35. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell wget <download_url> -O /sdcard/Download/chrome.zip adb shell pm install -r /sdcard/Download/chrome.apk
36. Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps All initialize operations HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb reboot adb disconnect
37. Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps All initialize operations HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB
38. Allow Google’s Location Service Using a shell script and ADB to enable Google's Location Service from the Google Maps app
39. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell pm grant com.google.android.apps.maps android.permission.ACCESS_COARSE_LOCATION adb shell pm grant com.google.android.apps.maps android.permission.ACCESS_FINE_LOCATION
40. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell uiautomator dump adb pull /sdcard/window_dump.xml
41. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell uiautomator dump adb pull /sdcard/window_dump.xml <?xml version='1.0' encoding='UTF-8' standalone='yes' ?> <hierarchy rotation="0"> <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.android.launcher3" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1080,2072]"> <node index="0" text="" resource-id="" class="android.widget.LinearLayout" package="com.android.launcher3" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false"
42. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell uiautomator dump adb pull /sdcard/window_dump.xml <?xml version='1.0' encoding='UTF-8' standalone='yes' ?> <hierarchy rotation="0"> <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.android.launcher3" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][1080,2072]"> <node index="0" text="" resource-id="" class="android.widget.LinearLayout" package="com.android.launcher3" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false"
43. All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB PATTERN='string(//node[@text="SKIP"][1]/@bounds)' POSITION=$( xmllint --xpath $PATTERN window_dump.xml | awk -F '[\\[\\],]' '{printf("%d %d",($2+$5)/2,($3+$6)/2)}' ) adb shell input tab $POSITION
44. Allow Google’s Location Service • Launch Google Maps app • Tap on “SKIP” button • Tap on current location button • Tap on “OK” button • Back to home screen • Start the UI testing
45. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Connect ADB & Setup device Shell Script
46. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Install APKs & Run UI test Collect test result & device log
47. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Terraform
48. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance
49. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance
50. Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Cron Job Destroy instance
51. Summary Now • More scalable and stable CI system for Android UI testing • Compatible with any mobile framework for Android app • All of our Android apps have been successfully migrated to this new CI system • Provide teams with a seamless opportunity to implement automated end-to-end testing • Genymotion's monthly costs are highly cost-effective for our CI/CD system's results
52. Summary On the other side • Easy maintenance requirements introduce an overhead during Genymotion's operation (10 minutes for instance initialization and destruction) • Running Genymotion with public IP address will be vulnerable to bot script embedding by hackers
53. Thank you Somkiat Khitwongwattana Android GDE @akexorcist