Android Build Performance - DroidconSF 2018

This is a reprisal of my earlier presentation at DroidconNYC, with a few minor updates.

Transcript

1. Improving Android Build Performance Tony Robalik, Gradle Inc.
3. The Cost of Builds (#perfmatters)
4. | X Fast Builds Matter 60s waste * 50 builds per day * 10 developers > 8h wasted per day …and that’s not even considering lost focus https://gradle.com/quantifying-the-costs-of-builds/
5. | X
6. | X
7. | X Fast Builds Matter Don’t even get me started on morale…
8. General Advice
9. | X #science 1. Define scenario to improve 2. Profile scenario 3. Identify biggest bottleneck 4. Fix bottleneck 5. Verify fix by measurement 6. Repeat
10. | X Automate your measurements github.com/gradle/gradle-profiler $ gradle-profiler --benchmark --scenario-file scenarios // Scenario 1 configurationTime { tasks = ["help"] } // Scenario 2 assembleNoCache { tasks = ["clean", “assembleDebug”] gradle-args = ["--no-build-cache"] } // Scenario 3 assembleCache { tasks = ["clean", "assembleDebug"] gradle-args = ["--build-cache"] }
11. | X Automate your measurements github.com/gradle/gradle-profiler $ gradle-profiler --benchmark --scenario-file scenarios // Scenario 1 configurationTime { tasks = ["help"] } // Scenario 2 assembleNoCache { tasks = ["clean", “assembleDebug”] gradle-args = ["--no-build-cache"] } // Scenario 3 assembleCache { tasks = ["clean", "assembleDebug"] gradle-args = ["--build-cache"] }
12. | X Automate your measurements github.com/gradle/gradle-profiler $ gradle-profiler --benchmark --scenario-file scenarios // Scenario 1 configurationTime { tasks = ["help"] } // Scenario 2 assembleNoCache { tasks = ["clean", “assembleDebug”] gradle-args = ["--no-build-cache"] } // Scenario 3 assembleCache { tasks = ["clean", "assembleDebug"] gradle-args = ["--build-cache"] }
13. | X Stay Up-To-Date buildscript { repositories { google() } dependencies { classpath ‘com.android.tools.build:gradle:3.2.1’ } } $ ./gradlew wrapper --gradle-version 4.10.2
14. | X JVM tuning Provide enough heap space Other tweaks often do more harm than good Spend your time on structural improvements
15. | X JVM tuning Provide enough heap space Other tweaks often do more harm than good Spend your time on structural improvements # The only useful argument org.gradle.jvmargs=-Xmx4g
16. | X JVM tuning
17. Where is the Problem?
18. | X Build Scans
19. | X Build Scans $ ./gradlew assembleDebug --scan
20. | X Build Scans $ ./gradlew assembleDebug --scan
21. | X The Build Lifecycle
22. | X Red Flags Startup/Settings/buildSrc > 1s Single-line change ℳ clean build No-op build doing any work at all High GC time
23. | X Red Flags Startup/Settings/buildSrc > 1s Single-line change ℳ clean build No-op build doing any work at all High GC time
24. | X Red Flags Startup/Settings/buildSrc > 1s Single-line change ℳ clean build No-op build doing any work at all High GC time
25. | X Red Flags Startup/Settings/buildSrc > 1s Single-line change ℳ clean build No-op build doing any work at all High GC time
26. | X Red Flags Startup/Settings/buildSrc > 1s Single-line change ℳ clean build No-op build doing any work at all High GC time
27. Your Build
28. Startup, buildSrc, Settings
29. | X Startup, buildSrc, Settings Use the daemon, keep it healthy
30. | X Startup, buildSrc, Settings Don’t do this // settings.gradle rootDir.listFiles({ File dir, String name -> name.endsWith(“.gradle") } as FilenameFilter) .each { include "$it" } ? ¿
31. | X Startup, buildSrc, Settings Don’t do this // settings.gradle rootDir.listFiles({ File dir, String name -> name.endsWith(“.gradle") } as FilenameFilter) .each { include "$it" }
32. | X Startup, buildSrc, Settings ?
33. Configuration Time
34. | X Configuration Time Applying plugins Evaluating build scripts Running afterEvaluate {} blocks
35. | X Configuration Time When running any task Even gradlew help / gradlew tasks Android Studio sync
36. | X Resolution at Configuration Time
37. | X Eager resolution Resolution at Configuration Time task badTask(type: Jar) { from configurations.compile.collect { it.directory ? it : zipTree(it) } classifier “don’t do this” }
38. | X Eager resolution Resolution at Configuration Time task badTask(type: Jar) { from configurations.compile.collect { it.directory ? it : zipTree(it) } classifier “don’t do this” }
39. | X Resolution at Configuration Time task badTask(type: Jar) { from { configurations.compile.collect { it.directory ? it : zipTree(it) } } classifier “don’t do this” } Use lazy evaluation instead
40. | X Configuring Tasks import org.jmailen.gradle.kotlinter.tasks.LintTask tasks.register('lintKotlinAll', LintTask) { group = 'verification' source files(modules.collect { "$it/src" }) reports = [ 'plain': file("$buildDir/reports/ktlint/all-lint.txt"), 'html' : file(“$buildDir/reports/ktlint/all-lint.html”) ] }
41. | X Inefficient Plugins
42. | X Expensive logic for each project Inefficient Plugins // `version.gradle` def out = new ByteArrayOutputStream() exec { commandLine 'git', 'rev-parse', “HEAD” standardOutput = out workingDir = rootDir } version = new String(out.toByteArray()) // root `build.gradle` subprojects { apply from: "$rootDir/version.gradle" }
43. | X Inefficient Plugins // `version.gradle` as before // root `build.gradle` apply from: "$rootDir/version.gradle" subprojects { version = rootProject.version } Reuse expensive calculations
44. | X Variant Explosion https://developer.android.com/studio/build/build-variants#filter-variants variantFilter { variant -> def flavorName = variant.flavors[0].name def freeFlavor = flavorName == "free" if (!freeFlavor && variant.buildType.name == "release") { variant.ignore = true } }
45. | X Variant Explosion https://developer.android.com/studio/build/build-variants#filter-variants variantFilter { variant -> def flavorName = variant.flavors[0].name def freeFlavor = flavorName == "free" if (!freeFlavor && variant.buildType.name == "release") { variant.ignore = true } }
46. | X Configuration Time Avoid dependency resolution Avoid I/O Don’t repeat yourself
47. Optimizing Build Logic
48. | X Extract Script Plugins apply from: "$rootDir/static_analysis.gradle" apply from: "$rootDir/coverage.gradle" apply from: “$rootDir/spock_android_lib.gradle” apply from: "$rootDir/junit5html.gradle" if (isOnCi()) { // This plugin adds ~5s to every build apply plugin: 'com.getkeepsafe.dexcount' } Makes finding issues easier
49. | X Extract Binary Plugins Use buildSrc Use static compilation Keep build scripts declarative https://docs.gradle.org/current/userguide/organizing_gradle_projects.html#sec:build_sources
50. | X Extract Binary Plugins https://docs.gradle.org/current/userguide/organizing_gradle_projects.html#sec:build_sources $rootDir/buildSrc/ src/main/<java|kotlin|groovy>/ com/yourCompany/ all/the/things/
51. Execution Time
52. | X Execution Time Executing selected tasks Incremental Cacheable Parallelizable
53. | X Execution Time Executing selected tasks Incremental Cacheable Parallelizable
54. | X Execution Time Executing selected tasks Incremental Cacheable Parallelizable
55. | X Execution Time Executing selected tasks Incremental Cacheable Parallelizable
56. | X Incremental Builds Nothing changed? Executed tasks should be zero!
57. | X Incremental Builds Nothing changed? Executed tasks should be zero!
58. | X Incremental Builds Find volatile inputs
59. | X Incremental Builds android { defaultConfig { versionName “4.0.0-${new Date()}” } }
60. | X Example: Crashlytics Unique IDs are the bane of local dev performance
61. | X Example: Crashlytics Unique IDs are the bane of local dev performance apply plugin: 'io.fabric' android { buildTypes { debug { ext.alwaysUpdateBuildId = false } } }
62. | X Faster Compilation Modularization => Compile avoidance Decoupled code => Faster incremental compilation Careful with Kotlin annotation processing (for now)
63. | X Faster Compilation Modularization => Compile avoidance Decoupled code => Faster incremental compilation Careful with Kotlin annotation processing (for now) https://youtrack.jetbrains.com/issue/KT-24203
64. | X Faster Compilation Modularization => Compile avoidance Decoupled code => Faster incremental compilation Careful with Kotlin annotation processing (for now)
65. | X Faster Compilation Modularization => Compile avoidance Decoupled code => Faster incremental compilation Careful with Kotlin annotation processing (for now)
66. | X Incremental Annotation Processing Since Gradle 4.7 Early adopters: Lombok, Android-State, Dagger Others: github.com/gradle/gradle/issues/5277 (incap)
67. | X Incremental Annotation Processing Since Gradle 4.7 Early adopters: Lombok, Android-State, Dagger Others: github.com/gradle/gradle/issues/5277 (incap)
68. | X The Build Cache https://guides.gradle.org/using-build-cache/ https://developer.android.com/studio/build/build-cache
69. | X The Build Cache https://guides.gradle.org/using-build-cache/ https://developer.android.com/studio/build/build-cache
70. | X The Build Cache https://guides.gradle.org/using-build-cache/ # gradle.properties org.gradle.caching=true https://developer.android.com/studio/build/build-cache
71. | X The Build Cache https://guides.gradle.org/using-build-cache/ # gradle.properties org.gradle.caching=true $ ./gradlew assembleDebug --build-cache https://developer.android.com/studio/build/build-cache
72. | X The Build Cache https://guides.gradle.org/using-build-cache/ Many tasks are cacheable https://developer.android.com/studio/build/build-cache
73. | X https://developer.android.com/studio/build/build-cache Save time for your whole team with one or more remote caches https://guides.gradle.org/using-build-cache/ The Build Cache
74. | X https://developer.android.com/studio/build/build-cache https://guides.gradle.org/using-build-cache/ The Build Cache // root build.gradle subprojects { pluginManager.withPlugin("kotlin-kapt") { kapt.useBuildCache = true } }
75. | X Parallelism # Or in gradle.properties org.gradle.parallel=true $ ./gradlew assembleDebug --parallel
76. | X Parallelism Serial execution
77. | X Parallelism --parallel and small, decoupled projects
78. Keeping track of performance
79. | X Gradle Enterprise
80. | X See all Builds Find out what colleagues are struggling with
81. | X Detailed Performance Profile
82. | X Watch Performance over Time
83. | X Watch Performance over Time
84. | X
85. Resources
86. | X Gradle performance guide https://guides.gradle.org/performance Android performance guide https://developer.android.com/studio/build/optimize-your-build.html Plugin development guide https://guides.gradle.org/implementing-gradle-plugins Structuring build logic guide https://docs.gradle.org/current/userguide/organizing_build_logic.html Guides
87. | X scans.gradle.com
88. w Thank you DroidconSF 2018 tony@gradle.com @autonomousapps