Android DevOps: Espresso on Genymotion in AWS

Talk given at AndroidMakers.fr 2018

1. Michael Aubert Android DevOps Espresso on Genymotion in AWS
2. Starling Bank 100% cloud based, Mobile only Personal & Business current accounts • Mastercard debit card • Direct debits • Faster Payments • Current Account Switching Service Card Control Spending Insights Savings Goals Public APIs, Marketplace
3. The automated QA system for our Android application CI system is in AWS cloud Genymotion-on-Demand provides Android in AWS. Push Code CI Trigger Run Tests Build APK Publish Results
4. On push to git: • Multiple Androids run tests in parallel: Sharding. • Keep all Logcat. • Record all screens into videos. • Extract JUnit test report. • Measure total code coverage. Quarterly: • Use new version of Genymotion-on-Demand
5. Scalable Maintained Supported Movable Influenceable Efficient Oracle HVX ✓ ✓ ✕ ✓ ✕ ✕ Amazon farm ✓ ✓ ✓ ✓ ✕ ✕ Firebase lab ✓ ✓ ✓ ✓ ✕ ✓ Dedicated ✕ ✕ ✕ ✕ ✓ ✕ Anbox.io ✓ ✕ ✕ ✓ ✓ ✓ Genymotion ✓ ✓ ✓ ✓ ✓ ✓ Why:
6. Why we built so much: AGPv3 broke Spoon for a while Starling engineering culture
7. How to automate Espresso: Push Code CI Trigger Run Tests Build APK Publish Results
8. Hot to run tests from Gradle: Don’t use connectedAndroidTest task in CI Your task depends on assembleAndroidTest Your task spawns several processes Task type Exec has commandLine for shell
9. Test Sharding: Gradle Prepare Android Prepare Android Application Manager Application Manager Prepare Android Application Manager Check Timeout
10. Prepare 1 Android: AWS CLI Start Genymotion Instance Capture Logcat ADB connect Record Screen Ready
11. Company AWS infrastructure ... Virtual Private Cloud 1 Virtual Private Cloud n
12. VPC Subnet x (location) Security Group y (firewall) EC2 Instance
13. AWS CLI: Find the Subnet. Where the instance will be. Find the Security Group. The instance firewall. Both need to belong to the same VPC. Find the Amazon Machine Image. The instance operating system. Genymotion-on-Demand is 1 AMI for each Android version
14. Useful AWS CLI: aws ec2 describe-vpcs aws ec2 describe-subnets aws ec2 describe-security-groups aws ec2 describe-images (takes several seconds) aws ec2 run-instances aws ec2 wait instance-running aws ec2 terminate-instances
15. How to start using Genymotion-on-Demand: Watch genymotion.com/help/cloud-PaaS/tutorial Talk to DevOps and SRE teams in your company AWS CLI documentation Don’t fight sed, embrace jq JSON processor.
16. Need your own security group: CloudFormation .yml GenymotionSecurityGroup: Type: AWS::EC2::SecurityGroup SecurityGroupIngress: - SourceSecurityGroupId: !Ref CISecurityGroup FromPort: 5555 ToPort: 5555 IpProtocol: tcp
17. Wait for Android to boot: adb connect to port 5555 May need to be retried adb wait-for-device shell getprop sys.boot_completed Loop until it returns 1
18. Prepare 1 Android: AWS CLI Start Genymotion Instance Capture Logcat ADB connect Record Screen Ready
19. Capture Logcat: adb -s IPAddress:5555 logcat 1> build/myfolder/myfile.txt 2>&1 Does not return until you kill it. Build must continue. Needs background process Once for every Android you prepare.
20. Record Screen: adb -s IPAddress:5555 shell screenrecord /sdcard/myfile.mp4 Returns after 3 minutes. Needs loop. Build must continue. Needs background process. Stop with kill -SIGTERM or video is truncated.
21. Useful Gradle: ProcessBuilder pb = new ProcessBuilder(command.split(' ')) Process process = pb.start() process.text.eachLine {println it} process.waitFor() def pid = process.class.getDeclaredField('pid').getInt(process) pid can be stored in temporary file.
22. Test Sharding: Gradle Prepare Android Prepare Android Application Manager Application Manager Prepare Android Application Manager Check Timeout
23. Android application manager runs Espresso: ADB install application ADB install test APK developer.android.com/studio/test/command-line.html Once for every Android you prepared.
24. Instrumentation command line: adb -s IPAddress:5555 shell am instrument -w -e numShards (number of Androids started) -e shardIndex (which one of the Androids) -e listener (generate JUnit report) -e coverage true -e coverageFile (measure test coverage) Useful: de.schroepf.androidxmlrunlistener.XmlRunListener
25. How: Push Code CI Trigger Run Tests Build APK Publish Results
26. Publish Build Results: CI system configured to retrieve artifacts. adb pull from each Android: - ALL screen record videos - Code coverage files .ec - JUnit test report .xml NOW YOU CAN TERMINATE EACH ANDROID INSTANCE
27. Quarterly: Use new version of Genymotion-on-Demand AMI changes needed: ➔ Enable ADB ➔ Disable animations ➔ Enable virtual keyboard Optional: Change resolution
28. Useful Android commands setprop persist.sys.usb.config adb su root settings put global window_animation_scale 0 su root settings put global transition_animation_scale 0 su root settings put global animator_duration_scale 0 su root settings put secure show_ime_with_hard_keyboard 1
29. Use Packer to make new AMI JSON configuration to execute Shell on EC2 instance. Will spawn instance from Genymotion AMI. Runs scripts to configure instance. Saves a snapshot as a new AMI. Launch instances of that new AMI to run tests. AMI snapshot will be 6GB
30. packer.json builder type: amazon-ebs ssh_username: shell (as per Genymotion release notes) subnet does not need to be same as used in CI security group does not need ADB 5555. Does need SSH 22 provisioner type shell, with .sh scripts Reboot instance after resolution change
31. Remember: ➔ AWS Basics: AMI, EC2, VPC, Security Group ➔ Build your own AMI: Packer, Shell ➔ Rebuild infrastructure at will: CloudFormation ➔ Need permissions in AWS and CI tool. ➔ Gradle: spawn several processes. ➔ Espresso: Launch instance, wait for it, adb, record everything ➔ TERMINATE INSTANCES
32. More: starlingbank.com/careers starlingbank.com/blog starling bank developer podcast community.starlingbank.com developer.starlingbank.com
33. Thanks: Deborah Munsi @ Genymotion Jean-Charles Leneveu @ Genymotion Nicolas Delhaye @ Genymotion Engineering group @ Starling AndroidMakers organisers and sponsors Blame: Teresa Ng @ Starling
34. Questions