About me
• Android developer since before Uni
• New to Berlin - Started working in September
• Formerly at Square in San Francisco
• Worked on Reader Experience, Square Register
and build systems: Buck
Slide 3
Slide 3 text
Every build is a special snowflake
Slide 4
Slide 4 text
In the beginning
there were no tests,
no CI…
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
Why Travis?
• Other engineering teams at Zenjob teams were
already using it
• Integrates easily with GitHub
• Good Android examples and documentation
• There’s a million options, this isn’t really the point.
before_install:
# Install the system image.
- sdkmanager “system-images;android-19;default;armeabi-v7a"
# Create and start emulator for the script.
# Meant to race the install task.
- echo no | avdmanager create avd --force -n test -k "system-
images;android-19;default;armeabi-v7a"
- $ANDROID_HOME/emulator/emulator -avd test -no-audio -no-window &
Slide 12
Slide 12 text
Travis takes: 15 min 13 sec
Slide 13
Slide 13 text
Next: Run UI tests
Slide 14
Slide 14 text
Hard to debug
• Button isn’t visible? Why?
• No screenshots
• Spoon doesn’t support AGP 3.0 (yet)
• Screenshot on failure is tricky
• Unsolved problem
Slide 15
Slide 15 text
After enabling our first UI test
Slide 16
Slide 16 text
No content
Slide 17
Slide 17 text
2 tests later
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
Soon we had a problem
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
Make the tests faster
• Replace SystemClock.sleep() in tests with
Espresso idling resources
• Replace real API calls with mock API calls
• Make mock API calls return instantly
Slide 22
Slide 22 text
Success…?
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
Time to try
something else
Slide 25
Slide 25 text
AWS Device Farm
Slide 26
Slide 26 text
• AWS Device Lab failed with cryptic error and no
support
• AWS Device Lab $10/hr
• Firebase Test Lab $1/hr virtual and $5/hr real
devices
Slide 27
Slide 27 text
Easy Decision
Slide 28
Slide 28 text
So much faster!
Slide 29
Slide 29 text
Why faster?
• Do not have to download or install emulator
• Emulator doesn’t slow down the rest of the build
• Tests are now run on faster emulator
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
111 secs = 1m 51 secs
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
Can we go faster?
Slide 34
Slide 34 text
if [ "$COMPONENT" == "checkstyle" ]; then
./gradlew checkstyle
elif [ "$COMPONENT" == "unit" ]; then
./gradlew test
elif [ "$COMPONENT" == "lint" ]; then
./gradlew lint
elif [ "$COMPONENT" == "release" ]; then
./gradlew assembleRelease
elif [ "$COMPONENT" == "instrumentation" ]; then
./gradlew assembleDebug assembleAndroidTest
.travis/run-ui-tests.sh
else
echo "This module doesn't exist"
exit 1
fi
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
Yes, but…
Slide 37
Slide 37 text
Faster individual
builds but slower with
multiple builds.
Slide 38
Slide 38 text
if [ "$COMPONENT" == "build" ]; then
./gradlew checkstyle check assembleRelease
elif [ "$COMPONENT" == "instrumentation" ];
then
./gradlew assembleDebug assembleAndroidTest
.travis/run-ui-tests.sh
else
echo "This module doesn't exist"
exit 1
fi
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
Less is more!
Slide 41
Slide 41 text
Problems solved!
Slide 42
Slide 42 text
…but then we
wrote more UI tests
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
60+ Tests, 11 mins
Slide 45
Slide 45 text
Sharding?
Slide 46
Slide 46 text
Flank!
Slide 47
Slide 47 text
• https://github.com/TestArmada/flank
• Splits up UI tests amongst Firebase Test Lab
devices
• Splits up tests to minimize costs
Slide 48
Slide 48 text
Test time stays constant
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
Prints out cost
Slide 51
Slide 51 text
Still one problem;
manual uploads
Slide 52
Slide 52 text
Automation
Slide 53
Slide 53 text
before_deploy:
# Verify that the apk was signed.
- apksigner verify -v app/build/outputs/apk/release/app-release.apk
# Publish to GitHub releases only on tagged commits.
deploy:
provider: releases
api_key:
secure: blah-blah-blah
file:
- "app/build/outputs/apk/debug/app-debug.apk"
- "app/build/outputs/mapping/release/mapping.txt"
- "app/build/outputs/apk/release/app-release.apk"
skip_cleanup: true
on:
tags: true
condition: $COMPONENT = build
# Publish to Google Play after successful deployment to GitHub releases.
after_deploy:
- ./gradlew publishApkRelease
Slide 54
Slide 54 text
before_deploy:
# Verify that the apk was signed.
- apksigner verify -v app/build/outputs/apk/release/
app-release.apk
Slide 55
Slide 55 text
before_deploy:
# Verify that the apk was signed.
- apksigner verify -v app/build/outputs/apk/release/app-release.apk
# Publish to GitHub releases only on tagged commits.
deploy:
provider: releases
api_key:
secure: blah-blah-blah
file:
- "app/build/outputs/apk/debug/app-debug.apk"
- "app/build/outputs/mapping/release/mapping.txt"
- "app/build/outputs/apk/release/app-release.apk"
skip_cleanup: true
on:
tags: true
condition: $COMPONENT = build
# Publish to Google Play after successful deployment to GitHub releases.
after_deploy:
- ./gradlew publishApkRelease
Slide 56
Slide 56 text
# Publish to GitHub releases only on tagged commits.
deploy:
provider: releases
api_key:
secure: blah-blah-blah
file:
- "app/build/outputs/apk/debug/app-debug.apk"
- "app/build/outputs/mapping/release/mapping.txt"
- "app/build/outputs/apk/release/app-release.apk"
skip_cleanup: true
on:
tags: true
condition: $COMPONENT = build
Slide 57
Slide 57 text
before_deploy:
# Verify that the apk was signed.
- apksigner verify -v app/build/outputs/apk/release/app-release.apk
# Publish to GitHub releases only on tagged commits.
deploy:
provider: releases
api_key:
secure: blah-blah-blah
file:
- "app/build/outputs/apk/debug/app-debug.apk"
- "app/build/outputs/mapping/release/mapping.txt"
- "app/build/outputs/apk/release/app-release.apk"
skip_cleanup: true
on:
tags: true
condition: $COMPONENT = build
# Publish to Google Play after successful deployment to GitHub releases.
after_deploy:
- ./gradlew publishApkRelease
before_deploy:
# Verify that the apk was signed.
- apksigner verify -v app/build/outputs/apk/release/app-release.apk
# Publish to GitHub releases only on tagged commits.
deploy:
provider: releases
api_key:
secure: blah-blah-blah
file:
- "app/build/outputs/apk/debug/app-debug.apk"
- "app/build/outputs/mapping/release/mapping.txt"
- "app/build/outputs/apk/release/app-release.apk"
skip_cleanup: true
on:
tags: true
condition: $COMPONENT = build
# Publish to Google Play after successful deployment to GitHub releases.
after_deploy:
- ./gradlew publishApkRelease
Slide 63
Slide 63 text
# Publish to Google Play after successful
deployment to GitHub releases.
after_deploy:
- ./gradlew publishApkRelease
Slide 64
Slide 64 text
No content
Slide 65
Slide 65 text
TODOs
• Automate bumping version numbers
• Run UI tests on more real devices with
different APIs before release
• Move to AWS or another service with
faster machines
Slide 66
Slide 66 text
TODOs
• Screenshots on failure ( should be built
in)
• Gradle plugin for Firebase tests
• Gradle plugin for Flank (for running
locally)
Slide 67
Slide 67 text
TODOs
• Merge on green button
• Automate release branch cutting
• Speed up local builds (Buck?)