Building the automated Android UI testing in Continuous integration at LINE MAN Wongnai

Slide 1

Slide 1 text

Building the automated Android UI testing in Continuous Integration Somkiat Khitwongwattana Android GDE @akexorcist At LINE MAN Wongnai

Slide 2

Slide 2 text

LINE MAN Wongnai LINE MAN Rider LINE MAN Driver Wongnai Merchant Wongnai POS Consumer Local Business Rider & Driver Wongnai POS Manager

Slide 3

Slide 3 text

LINE MAN Wongnai LINE MAN Rider LINE MAN Driver Wongnai Merchant Wongnai POS React Native Native Android (Kotlin) Wongnai POS Manager

Slide 4

Slide 4 text

Running Android UI testing in continuous integration is challenging for developers, but it's necessary for future scalability of Android apps.

Slide 5

Slide 5 text

Our previous infrastructure Which isn’t good enough for future requirements ● Not scalable ● Not maintainable ● Unstable 🥲 😢 😭

Slide 6

Slide 6 text

Native Android React Native Marathon Detox Allure Jest Yes No UI Testing Integration End-to-end Test Reporting Yes Yes Test Executor Continuous Integration GitLab GitLab

Slide 7

Slide 7 text

Android Device Provider Marathon Detox Allure Jest Yes No UI Testing Integration End-to-end Test Reporting Yes Yes Test Executor tinuous gration tLab tLab

Slide 8

Slide 8 text

Candidates Smartphone Test Farm Genymotion Device Image Firebase Test Lab

Slide 9

Slide 9 text

Firebase Test Lab Key Features ● Test both Android and iOS apps ● Real and virtual devices ● Google Play Service included ● Wide range of supported devices ● Test matrix execution is supported Note: Test reports are stored on Google Cloud Storage https://firebase.google.com/products/test-lab

Slide 10

Slide 10 text

Firebase Test Lab Pricing Virtual device ● No-cost up to 60 min / day ● Then $1 / device / hour Physical device ● No-cost up to 30 min / day ● Then $5 / device / hour

Slide 11

Slide 11 text

Genymotion Device Image Key Features ● Available on well-known cloud service providers ● Virtual devices without GPU ● Remotely control via web browser and HTTP API ● Run app for ARM natively with ARM64 instances ● Provide SSH and ADB access https://www.genymotion.com

Slide 12

Slide 12 text

Genymotion Device Image Pricing Depending on the instance size t4g.medium ● 2 vCPU ● 4GB of memory ● $0.295 / hr

Slide 13

Slide 13 text

Smartphone Test Farm Key Features ● Open-source project ● Remote control any device from your browser ● Monitor your device inventory ● Reserve and release any device with STF API https://github.com/DeviceFarmer/stf

Slide 14

Slide 14 text

Price Maintainability Custom Test Executor VPN Stability Configurability Ease of use

Slide 15

Slide 15 text

Price Ease of use C ustom Test Executor VPN Stability M aintainability C onfigurability Firebase Test Lab Genymotion Device Image Smartphone Test Farm

Slide 16

Slide 16 text

Swollen battery problem on a physical device

Slide 17

Slide 17 text

Build App APK & Test APK Run UI Test Generate Test Report

Slide 18

Slide 18 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance

Slide 19

Slide 19 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance

Slide 20

Slide 20 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Terraform

Slide 21

Slide 21 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Connect ADB & Setup device Shell Script

Slide 22

Slide 22 text

Why initialization? ● ADB disabled by default ● No Google Mobile Services ● Undesired screen size ● No WebView engine ● English language by default ● Reboot required to apply some changes

Slide 23

Slide 23 text

Genymotion HTTP API ● ADB ● Clipboard ● Screen resolution ● Screen orientation ● Download, upload, or delete file ● Send key, mouse, or tap event ● Screenshot ● Battery ● GPS ● And more

Slide 24

Slide 24 text

OpenGApps An open-source initiative that provides Google Apps package Full package ● Google Chrome ● Google Maps ● Google Play services https://opengapps.org

Slide 25

Slide 25 text

Slide 26

Slide 26 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB POST /api/v1/configure/adb { "active": true, "active_on_reboot": true }

Slide 27

Slide 27 text

Slide 28

Slide 28 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB POST /api/v1/android/resolution { "width": 720, "height": 1440, "dpi": 300 }

Slide 29

Slide 29 text

Screen resolution configuration Mobile Tablet ● 720 x 1,440 px ● 384 x 768 dp ● 300 DPI ● 1,335 x 751 px ● 1,335 x 751 dp ● 160 DPI

Slide 30

Slide 30 text

Screen resolution configuration Mobile Tablet ● 720 x 1,440 px ● 384 x 768 dp ● 300 DPI ● 1,335 x 751 px ● 1,335 x 751 dp ● 160 DPI https://developer.android.com/reference/android/util/DisplayMetrics

Slide 31

Slide 31 text

Slide 32

Slide 32 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB POST /api/v1/sensors/gps { "latitude": 13.7223497, "longitude": 100.5804235, "altitude": 0, "accuracy": 10, "bearing": 0, "speed": 0 }

Slide 33

Slide 33 text

Slide 34

Slide 34 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell wget -O /sdcard/Download/opengapps.zip adb shell system/bin/flash-archive.sh /sdcard/Download/opengapps.zip

Slide 35

Slide 35 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell wget -O /sdcard/Download/chrome.zip adb shell pm install -r /sdcard/Download/chrome.apk

Slide 36

Slide 36 text

Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps All initialize operations HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb reboot adb disconnect

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Allow Google’s Location Service Using a shell script and ADB to enable Google's Location Service from the Google Maps app

Slide 39

Slide 39 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell pm grant com.google.android.apps.maps android.permission.ACCESS_COARSE_LOCATION adb shell pm grant com.google.android.apps.maps android.permission.ACCESS_FINE_LOCATION

Slide 40

Slide 40 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install a newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell uiautomator dump adb pull /sdcard/window_dump.xml

Slide 41

Slide 41 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB adb shell uiautomator dump adb pull /sdcard/window_dump.xml

Slide 42

Slide 42 text

Slide 43

Slide 43 text

All initialize operations Enable ADB connection Connect device Change screen resolution Change device location Change device language Install OpenGApps Install newer version of Chrome for Android Reboot the device Reconnect the ADB after device rebooted Grant location permission for Google Maps Allow Google’s Location Service from Google Maps HTTP ADB HTTP HTTP ADB ADB ADB ADB ADB ADB ADB PATTERN='string(//node[@text="SKIP"][1]/@bounds)' POSITION=$( xmllint --xpath $PATTERN window_dump.xml | awk -F '[\\[\\],]' '{printf("%d %d",($2+$5)/2,($3+$6)/2)}' ) adb shell input tab $POSITION

Slide 44

Slide 44 text

Allow Google’s Location Service ● Launch Google Maps app ● Tap on “SKIP” button ● Tap on current location button ● Tap on “OK” button ● Back to home screen ● Start the UI testing

Slide 45

Slide 45 text

Slide 46

Slide 46 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Install APKs & Run UI test Collect test result & device log

Slide 47

Slide 47 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Terraform

Slide 48

Slide 48 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance

Slide 49

Slide 49 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance

Slide 50

Slide 50 text

Run UI Test GitLab Runner Genymotion Device Image on AWS EC2 Start Job Create Instance Initialization Test Execution End Job Test Execution Initialization Destroy Instance Cron Job Destroy instance

Slide 51

Slide 51 text

Summary Now ● More scalable and stable CI system for Android UI testing ● Compatible with any mobile framework for Android app ● All of our Android apps have been successfully migrated to this new CI system ● Provide teams with a seamless opportunity to implement automated end-to-end testing ● Genymotion's monthly costs are highly cost-effective for our CI/CD system's results

Slide 52

Slide 52 text

Summary On the other side ● Easy maintenance requirements introduce an overhead during Genymotion's operation (10 minutes for instance initialization and destruction) ● Running Genymotion with public IP address will be vulnerable to bot script embedding by hackers

Slide 53

Slide 53 text

Thank you Somkiat Khitwongwattana Android GDE @akexorcist