Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Jetpack Benchmark - Droidcon RO 2019
Search
JB Lorenzo
October 04, 2019
140
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Jetpack Benchmark - Droidcon RO 2019
JB Lorenzo
October 04, 2019
Transcript
60 slides
@bennegeek Measure What Matters with Jetpack Benchmark JB Lorenzo |
#DroidconRO19
None
@bennegeek
@bennegeek
@bennegeek
@bennegeek Benchmark 1.0.0-alpha06 September 18, 2019 alpha01 on May 7
@bennegeek From alpha02 to alpha06 * Auto disables test coverage
* AndroidBenchmarkRunner by default * Module structure changed to decouple JUnit4 * Change in package name * Documentation for CI integration
@bennegeek Documentation for CI Integration https://developer.android.com/studio/profile/run-benchmarks-in-ci * Guidelines for benchmark
without Gradle * Self-instrumented APK * Separate app and test APKs * Guidelines for CI * Important flags (e.g. lock clocks, json output)
@bennegeek Setting up Android Studio 3.5 Beta npw.benchmark.template.module=true
@bennegeek Setting up
@bennegeek Setting up your.package.name.
@bennegeek Setting up dependencies { androidTestImplementation “androidx.benchmark:benchmark:1.0.0-alpha06" }
@bennegeek Setting up <application android:debuggable="false" tools:ignore="HardcodedDebugMode" tools:replace="android:debuggable"/>
@bennegeek Let’s measure!
@bennegeek @mootazltaief
@bennegeek @mootazltaief
@bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own
Device Farm
@bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own
Device Farm
@bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own
Device Farm
@bennegeek Amazon EC2 + Emulator AWS: * Does not allow
nested VM * Need to run bare-Metal instance to run emulator Emulator: * Not advised for benchmarks Other cloud providers: * has support for nested VM
@bennegeek Integrating in Gitlab CI // benchmark/build.gradle apply plugin: androidx.benchmark
// if device/emulator is connected to CI > ./gradlew benchmark:connectedCheck
@bennegeek Integrating in Gitlab CI { "context": { "build": {
"device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
@bennegeek Integrating in Gitlab CI { "context": { "build": {
"device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
@bennegeek Integrating in Gitlab CI { "context": { "build": {
"device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
@bennegeek Integrating in Gitlab CI { "context": { "build": {
"device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500, 95250, ...
@bennegeek Integrating in Gitlab CI { "context": { "build": {
"device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Benchmark Warnings
@bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own
Device Farm
@bennegeek Firebase Test Lab Amazon EC2 + Emulator Your Own
Device Farm
@bennegeek Firebase Test Lab Integration with CI Firebase Test Lab
* Problems when running test with mocked network * $5/device/hour after Free tier * Need to configure gcloud in CI * Fetch results in google cloud storage
@bennegeek Firebase Test Lab Integration with CI Firebase Test Lab
How to get the json output from test lab?
@bennegeek Gitlab CI
@bennegeek Gitlab CI
@bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''
import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
@bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''
import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
@bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''
import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
@bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c '''
import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
@bennegeek Gitlab CI PreviousBenchmark = 138375 Threshold = 0.10 if
medians["benchmarkJacksonWithSample1"] > (PreviousBenchmark * Threshold): print("benchmarkJacksonWithSample1 is beyond threshold") exit(1)
@bennegeek Output
@bennegeek Output
@bennegeek Output
@bennegeek Next Steps * Separate app into modules * Mock
more data for benchmarking * Try it out yourself
@bennegeek Next Steps * Separate app into modules * Mock
more data for benchmarking * Try it out yourself * Visit the issue tracker * File bugs
@bennegeek tech.olx.com Example project (No OLX code): http://github.com/mootazltaief/ android-json-benchmark
@bennegeek tech.olx.com https://issuetracker.google.com/ issues?q=componentid: 585351%20status:open
None
@bennegeek tech.olx.com
None
None
Berlin Buenos Aires Lisbon New Delhi Poznan Dubai
@bennegeek Multumesc! vreo întrebare? Get in touch with us olxgroup.com
tech.olx.com olxtechberlin
[email protected]
jblorenzo JB Lorenzo | #DroidconRO19
More Decks by JB Lorenzo
See All by JB Lorenzo
Measure What Matters with Jetpack Benchmark
bennegeek
0
210
Lessons Learned from Using Kotlin Multiplatform internally
bennegeek
0
150
Measure What Matters with Jetpack Benchmark
bennegeek
0
100
Droidkaigi 2019 flutter kotlin native
bennegeek
2
630
Fast Prototypes using Kotlin_Native (and Flutter)
bennegeek
1
97
kotlin-native-workshop.pdf
bennegeek
0
140
Fast Prototypes with Flutter + Kotlin/Native
bennegeek
0
600
Featured
See All Featured
A Tale of Four Properties
chriscoyier
163
24k
世界の人気アプリ100個を分析して見えたペイウォール設計の心得
akihiro_kokubo
PRO
74
43k
Highjacked: Video Game Concept Design
rkendrick25
PRO
2
590
Claude Code のすすめ
schroneko
67
230k
What's in a price? How to price your products and services
michaelherold
247
13k
Breaking role norms: Why Content Design is so much more than writing copy - Taylor Woolridge
uxyall
2
540
Skip the Path - Find Your Career Trail
mkilby
1
300
Mind Mapping
helmedeiros
1
450
Understanding Cognitive Biases in Performance Measurement
bluesmoon
32
3.1k
Building a Scalable Design System with Sketch
lauravandoore
464
34k
Bridging the Design Gap: How Collaborative Modelling removes blockers to flow between stakeholders and teams @FastFlow conf
baasie
0
780
Testing 201, or: Great Expectations
jmmastey
46
8.4k