Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Building Android Testing Infrastructure
Search
Mohit S
August 02, 2022
Programming
700
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Building Android Testing Infrastructure
Mohit S
August 02, 2022
Transcript
128 slides
Mohit Sarveiya Building Android Testing Infrastructure @heyitsmohit
Building Android Testing Infrastructure • Automation Tools • Building Infra
pipelines • Snapshot Testing • Gradle Testing
Challenges Teams Monorepo
Arch Fragmentation Codebase Legacy code Feature A Feature B
Testing Codebase 50% Code coverage 20% Code coverage
Test Coverage Gradle Builds Database UI API
Testing Infra Goals • Automate • Scale • Testing Gradle,
Database, UI
UI Test Automation
Test Case Test Suite Test Case Test Case Test Case
Test Case Test Case Test Case Test Case
Test Case Test Case Test Case Test Case Firebase Test
Lab
gCloud Tool to run tests on Firebase Integrates with CI
~ gCloud firebase test android models list
MODEL_ID MAKE MODEL_NAME FORM RESOLUTION OS Nexus4 LG Nexus 4
VIRTUAL 2560 x 1600 19,21,22 sailfish Google Pixel PHYSICAL 1080 x 1920 25, 26 ~ gCloud firebase test android models list
~ gCloud firebase test android run —type robo tests instrumentation
~ gCloud firebase test android run —type instrumentation —app app-debug-unaligned.apk
~ gCloud firebase test android run —type instrumentation —app app-debug-unaligned.apk
—device model=Nexus6, version=21 local=en, orientation=portrait
Upload Google Cloud Storage
Google Cloud Storage Run Tests Report
None
~ gCloud firebase test android run —type robo
None
Get Report Test Lab gCloud Build APK CI Pipeline
UI Tests Time
Flank Run Android and iOS tests in parallel Uses Kotlin
Coroutines
Flank flank: app: ./app-debug.apk test: ./app-debug-androidTest.apk
Flank flank: app: ./app-debug.apk test: ./app-debug-androidTest.apk device: - model: NexusLowRes
version: 28
Test Case Test Case Test Case Test Case Parallel
Flank flank: ## test shards - the amount
of groups to split the test suite into max-test-shards: 2
Flank flank: ## test shards - the amount
of groups to split the test suite into max-test-shards: 2 ## shard time - the amount of time tests within a shard should take shard-time: 2
~ flank android test run
Test Case Test Case Test Case Test Case Parallel
Merged Results Shared 2 Parallel Shard 1
Test Case Test Case Test Case Test Case Problems Flaky
tests
Flaky Test Causes • Concurrency problems • Flaky third party
code • Resource idling.
Flank flank: ## Number of Flaky test attempts
num-flaky-test-attempts: 2
CI Setup Get Report Test Lab Flank Build APK
CI Setup Get Report Test Lab Flank Build APK Docker
Fladle Gradle plugin for using Flank Multi Module Testing support
Recipes Performance Regression
Recipes perfTests { devices.set([ ["model" : "Nexus5", "version" : "28"],
]) testTargets.set([ "class com.test.PerformanceTest" ]) }
Recipes regressionTests { devices.set([ [ "model" : "Nexus4", "version" :
"28"] ]) testTargets.set([ "class com.sample.MyRegressionTest" ]) }
Problems • Tests take too long to run • Flaky
tests • When do you run UI tests?
Nightly CI Job Running UI Tests Get Report Regression
Tests Fladle Build APK
Smoke Tests • Test basic functionality • Run on PRs
Smoke Tests @Retention(AnnotationRetention.RUNTIME) annotation class SmokeTest
Example - Firefox App • Open source • Nightly &
Smoke tests • Shards with Flank
https: / / github.com/mozilla-mobile/fenix
Firefox App Test Rails Smoke Tests Fladle Build APK
Test Rails • Documentation for your test suite • View
results over time • Collaborate with QA
None
None
UI Tests Infra • Run tests in parallel • Use
Flank with Firebase test lab • Flaky tests
Unit Testing Infrastructure
Unit Testing Integration E2E Slower, more expensive Faster,
cheaper
How do we structure unit tests?
Problem class Presenter( val repo: Repo, ... )
Problem interface Repo { fun getData(): Data
... }
Structure Feature (folder) Public (Module) Impl (Module) Fakes (Module)
Structure class PresenterTest { val fakeRepo = FakeRepository()
val presenter = Presenter(fakeRepo) }
Structure Feature (folder) Public (Module) Impl (Module) Fakes (Module)
Dev Tools • Auto Generate module scaffolding • Enforce rules
on how modules depends on each other
Unit Testing Lint Rules • Enforce best practices
Unit Testing Lint Rules • Enforce best practices • Coroutine
lint test rules
Unit Testing Lint Rules • Enforce best practices • Coroutine
lint test rules • Do not mock data classes
Unit Tests Time
Problem bdae142 .. main 50157a .. d89f145 .. f0ddfb ..
None
Run only affected unit tests
Affected Module Dectector Gradle plugin for determine which files changed
Run only affected tests
Affected Module Detector App Module A Module B Module C
Affected Module Detector affectedModuleDetector { baseDir = "${project.rootDir}" compareFrom =
"PreviousCommit" }
Affected Module Detector affectedModuleDetector { baseDir = "${project.rootDir}" compareFrom =
"PreviousCommit" } Fork Commit SpecifiedBranchCommit
Affected Module Detector App Module A Module B Module C
assembleAndroidDebugTest connectedAndroidDebugTest testDebug
Affected Module Detector ~ ./gradlew runAffectedUnitTests // Runs jvm tests
Affected Module Detector ~ ./gradlew runAffectedAndroidTests // Runs UI tests
Run only affected unit tests
Unit Testing Integration E2E Slower, more expensive Faster,
cheaper
Problem • Run only affected unit tests • Snapshot testing
with unit tests
Problem • UI Regressions • Example - Constraint layout changes
Paparazzi Gradle plugin to generate screenshot with unit tests Supports
Compose UI
Paparazzi @get:Rule val paparazzi = Paparazzi( deviceConfig = PIXEL_5,
theme = “android.Theme.Material.Light.NoActionBar” )
Paparazzi @get:Rule val paparazzi = Paparazzi( ... ) @Test fun
testView() { paparazzi.snapshot { UiView(uiState) } }
Paparazzi Git (LFS) Snapshot
Run Paparazzi tests
Paparazzi ~ ./gradlew app:recordPaparazziDebug // Generate Report
Paparazzi ~ ./gradlew app:verifyPaparazziDebug // Run again previously recorded
Problem • Run only affected unit tests • Creating snapshot
with unit testing • Unit testing database migrations
Database Migrations
Database Migrations id user_name user_email 1 User 1
[email protected]
2
User 2
[email protected]
User Table
Database Migrations id user_name user_email 1 User 1
[email protected]
2
User 2
[email protected]
User Table Rename to “email”
Database Migrations @RenameColumn( tableName = "users", fromColumnName =
"user_email", toColumnName = "email" ) class RenameFromUserAddressToAddress : AutoMigrationSpec
Database Migrations @Database( version = 2, autoMigrations = [ AutoMigration(
from = 1, to = 2, spec = UserDatabase.RenameFromUserAddressToAddress :: class ), ], )
How do we test migrations?
Test Migrations @get:Rule val helper: MigrationTestHelper = MigrationTestHelper( UserDatabase ::
class.java, listOf( UserDatabase.RenameFromUserAddressToAddress() ), )
Test Migrations @Test fun migrate1To2() { db = helper.createDatabase(TEST_DB, 1).apply
{ execSQL( """ INSERT INTO users VALUES (1, ‘User 1', ‘
[email protected]
') """.trimIndent() ) close() } }
Test Migrations @Test fun migrate1To2() { db = helper.runMigrationsAndValidate(TEST_DB,
2, true) }
Test Migrations @Test fun migrate1To2() { val resultCursor =
db.query("SELECT * FROM users”) // Perform assertions }
Problem • Test migrations with unit tests
Database Migrations JDBC SQLite Driver
DB Tools Room for Android Test Room with databases Migration
testing
https: // github.com/jeffdcamp/dbtools-room
Test Migrations class MigrationTest: BaseMigrationTest<UserDatabase>()
Test Migrations val db = Room.databaseBuilder( ... ) .openHelperFactory(JdbcSQLiteOpenHelperFactory(…)) .build()
Test Migrations fun testMigration(fromVersion: Int, toVersion: Int) { migrationTestExtension.createDatabase(name, fromVersion)
migrationTestExtension.runMigrationsAndValidate( name, toVersion, *migrations ) }
Problem • Run only affected unit tests • Creating snapshot
with unit testing • Unit testing database migrations
Test Coverage Gradle Builds Database UI API
Gradle Testing
Problem • Detect build regressions
Gradle Profiler Change
Gradle Profiler 1. Write a performance scenario 2. Specify number
of iterations
Gradle Profiler Iteration 1 Iteration 2 Iteration 3
Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Abi Change
Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Result
Performance Scenario incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” }
Performance Scenario incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” apply-android-resource-change-to = “strings.xml”
tasks = ["assemble"] }
Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups=6
Gradle Profiler Mean: 348 ms Min: 319 ms P25: 330
ms Median: 341 ms P75: 368 ms Std dev: 22.71 ms
Gradle Profiler
How do we automate profiling on CI?
CI Pipeline Docker image with Gradle Profiler
CI Pipeline Docker image with Gradle Profiler Performance
Scenarios
CI Pipeline Docker image with Gradle Profiler Performance
Scenarios Run nightly
Use Cases • Introducing Anvil
Benchmarking Build • Benchmark with Anvil change • Benchmark without
Anvil change (Baseline) • Compare results
Results Module A Module B Module C Baseline Anvil
Building Android Testing Infrastructure • Automation Tools • Building Infra
pipelines • Snapshot Testing • Gradle Regressions
Thank You! www.codingwithmohit.com @heyitsmohit
More Decks by Mohit S
See All by Mohit S
Guide to Improving Compose Performance
heyitsmohit
0
370
Building Shared UIs across Platforms with Compose
heyitsmohit
1
760
Building Multiplatform Apps with Compose
heyitsmohit
2
670
Building StateFlows with Jetpack Compose
heyitsmohit
6
2.1k
Migrating to Kotlin State & Shared Flows
heyitsmohit
1
930
Using Square Workflow for Android & iOS
heyitsmohit
1
570
Building Android Infrastructure Teams at Scale
heyitsmohit
3
460
Strategies for Migrating to Jetpack Compose
heyitsmohit
2
710
Challenges of Building Kotlin Multiplatform Libraries
heyitsmohit
1
560
Other Decks in Programming
See All in Programming
AI時代のシニアエンジニアとは
yosuke_furukawa
PRO
15
12k
ローカルLLMの基本と近況 2611 / Local LLM basics and recent situation 2611
kishida
3
590
エラーチャンネルのボールを拾い続けて鍛えるエンジニアの地力 / phpcon-niigata2026
meihei3
1
450
Augmenting AI with the Power of Jakarta EE
ivargrimstad
0
370
そのデザイントークン、Flutterでも同じ見た目になりますか?
nakamizu
0
430
Spec-Driven Development - Why Vaadin Is the Perfect Fit
simas
PRO
0
260
Vapor Mode でアクセシビリティは 壊れないか検証した話
fuqda
1
350
高橋メソッドでCVE解説シリーズ 〜CVE-2026-112816〜
hakkadaikon
0
270
LLM 101
dalinaum
0
510
あなたはMCPの機能を、まだ知らない。
4geru
1
660
プロジェクトコードをキレイキレイしよう!暗黙的な参照を読み解く静的解析
ostk0069
0
220
React Native에 현대적인 빌드 도구 더하기: Rollipop 구현기
leegeunhyeok
0
180
Featured
See All Featured
Building the Perfect Custom Keyboard
takai
2
960
コードの90%をAIが書く世界で何が待っているのか / What awaits us in a world where 90% of the code is written by AI
rkaga
63
48k
Neural Spatial Audio Processing for Sound Field Analysis and Control
skoyamalab
0
620
The Language of Interfaces
destraynor
162
27k
Building Better People: How to give real-time feedback that sticks.
wjessup
371
20k
Game over? The fight for quality and originality in the time of robots
wayneb77
1
330
Deep Space Network (abreviated)
tonyrice
0
420
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
livdayseo
1
250
Getting science done with accelerated Python computing platforms
jacobtomlinson
2
580
Technical Leadership for Architectural Decision Making
baasie
4
650
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
3k
My Coaching Mixtape
mlcsv
0
390