Measure What Matters with Jetpack Benchmark

Benchmark is one of the new interesting libraries recently released by the Jetpack team at Google. This library allows you to measure the speed of certain parts of your application. This is helpful to validate certain parts of your app that are repeatedly accessed such as going into details of items, or scrolling through an infinite list.
The measurements help you monitor if your app performance is slowing down. Because of this, you can integrate it to your CI. We'll show a real life scenario of benchmarks on the OLX app and show you how you can integrate it in your CI tool.

Transcript

1. @bennegeek Measure What Matters with Jetpack Benchmark JB Lorenzo | #DCBerlin19
3. @bennegeek
4. @bennegeek
5. @bennegeek Setting up Android Studio 3.5 Beta npw.benchmark.template.module=true
6. @bennegeek Setting up
7. @bennegeek Setting up your.package.name.
8. @bennegeek Setting up dependencies { androidTestImplementation "androidx.benchmark:benchmark:1.0.0-alpha02" }
9. @bennegeek Setting up <application android:debuggable="false" tools:ignore="HardcodedDebugMode" tools:replace="android:debuggable"/>
10. @bennegeek Let’s measure!
11. @bennegeek @mootazltaief
12. @bennegeek @mootazltaief
13. @bennegeek Integrating in Gitlab CI // benchmark/build.gradle apply plugin: androidx.benchmark > ./gradlew benchmark:connectedCheck
14. @bennegeek Integrating in Gitlab CI { "context": { "build": { "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
15. @bennegeek Integrating in Gitlab CI { "context": { "build": { "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
16. @bennegeek Integrating in Gitlab CI { "context": { "build": { "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
17. @bennegeek Integrating in Gitlab CI { "context": { "build": { "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500, 95250, ...
18. @bennegeek Integrating in Gitlab CI { "context": { "build": { "device": "generic_x86_64", "fingerprint": "google/sdk_gphone_x86_64/generic_x86_64:9/PSR1.180720.075/5124027:userdebug/dev-keys", "model": "Android SDK built for x86_64", "version": { "sdk": 28 } }, "cpuLocked": false, "sustainedPerformanceModeEnabled": false }, "benchmarks": [ { "name": "EMULATOR_UNLOCKED_benchmarkJacksonWithSample1", "className": "benchmark.JsonBenchmark", "metrics": { "timeNs": { "minimum": 94500, "maximum": 692250, "median": 138375, "runs": [ 96500,
19. @bennegeek Benchmark Warnings
20. @bennegeek Benchmark Warnings
21. @bennegeek Benchmark Warnings
22. @bennegeek Benchmark Warnings
23. @bennegeek Benchmark Warnings
24. @bennegeek Benchmark Warnings
25. @bennegeek Benchmark Warnings
26. @bennegeek Benchmark Warnings
27. @bennegeek Benchmark Warnings
28. @bennegeek Benchmark Warnings
29. @bennegeek Gitlab CI
30. @bennegeek Gitlab CI
31. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c ''' import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
32. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c ''' import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
33. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c ''' import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
34. @bennegeek Gitlab CI #!/bin/sh cat benchmark/build/benchmark_reports/*/*.json | python -c ''' import sys, json; benchmarks = json.load(sys.stdin)["benchmarks"] medians = {v["name"] : v["metrics"]["timeNs"]["median"] for v in benchmarks} … '''
35. @bennegeek Gitlab CI if medians["EMULATOR_UNLOCKED_benchmarkJacksonWithSample1"] > (138375 * 1.10): print("EMULATOR_UNLOCKED_benchmarkJacksonWithSample1 is beyond threshold") exit(1)
36. @bennegeek
37. @bennegeek Failures Learnings
38. @bennegeek Failures Learnings Obey Google docs Move benchmarks to separate module
39. @bennegeek Failures Learnings
40. @bennegeek Failures Learnings Disobey google docs Set debuggable to true
41. @bennegeek Failures Learnings
42. @bennegeek Failures Learnings Obey google docs Add gms services plugin to benchmark/build.gradle
43. @bennegeek Failures Learnings
44. @bennegeek Failures Learnings Disobey google docs Remove gms services plugin And initialize Firebase manually
45. @bennegeek Failures Learnings
46. @bennegeek Output
47. @bennegeek Next Steps * Separate app into modules * Mock more data for benchmarking * Try it out yourself
48. @bennegeek Next Steps * Separate app into modules * Mock more data for benchmarking * Try it out yourself * Visit the issue tracker * File bugs
49. @bennegeek tech.olx.com Example project (No OLX code): http://github.com/mootazltaief/ android-json-benchmark
50. @bennegeek tech.olx.com https://issuetracker.google.com/ issues?q=componentid: 585351%20status:open
52. @bennegeek tech.olx.com
55. Berlin Buenos Aires Lisbon New Delhi Poznan Dubai
56. @bennegeek Vielen Dank! FRAGEN? Get in touch with us olxgroup.com tech.olx.com olxtechberlin jb.lorenzo@olx.com jblorenzo JB Lorenzo | #DCBerlin19