Improving Android Build Performance

We all know a fast build makes for a happy developer. Unfortunately, as your project grows larger and more complex, its build will become slower if left unchecked. You’ve updated to the latest Android Plugin, but you’re still waiting more than you like, what can you do? Recent Gradle releases give you new capabilities to greatly improve your build speed. This session discusses how you can :

- Identify performance bottlenecks with build scans
- Fix common problems, like slow configuration time, tasks running when they shouldn’t, and builds that don’t parallelize well
- Benefit from incremental annotation processing

Transcript

1. Improving Android Build Performance Cédric Champeau, Gradle Inc.
2. About Gradle
3. | 3 What is Gradle? - Gradle != Gradle plugin for Android != Android Studio - Gradle is a build and automation tool - JVM based - 100% Free Open Source - Apache Standard License 2.0 - Multi-ecosystem - JVM: Java, Kotlin, Groovy, Scala, … - Native: C/C++, Go… - Android - …
4. | 4 Gradle, Inc. The company behind Gradle. • Build Happiness • Employs full time engineers • Providing Gradle Build Scans and Gradle Enterprise • (Gradle consulting, support, development services etc.) • (Training: online, public and in-house)
5. | 5 We’re hiring • Fully distributed development team • Exciting project used by millions • Build tool team and Gradle enterprise positions If anything you hear from now on sounds like a great problem to solve, Talk to us! https:/ /gradle.com/careers/
6. The Cost of Builds
7. | 7 60s waste * 50 builds per day * 50 developers = 42h wasted per day … and that’s not even considering lost focus Fast Builds Matter
8. General Advice
9. | 9 Process 1. Define scenario to improve 2. Profile scenario 3. Identify biggest bottleneck 4. Fix bottleneck 5. Verify fix by measurement 6. Repeat
10. |10 Automate your measurements github.com/gradle/gradle-profiler configurationTime { tasks = ["help"] } sync { tasks = ["generateDebugSources"] } cleanBuild { tasks = ["build"] cleanup-tasks = ["clean"] } cachedCleanBuild { tasks = ["build"] cleanup-tasks = ["clean"] gradle-args = ["--build-cache"] } configurationTime { tasks = ["help"] } sync { tasks = ["generateDebugSources"] } cleanBuild { tasks = ["build"] cleanup-tasks = ["clean"] } cachedCleanBuild { tasks = ["build"] cleanup-tasks = ["clean"] gradle-args = ["--build-cache"] }
11. |11 Stay Up-To-Date ./gradlew wrapper --gradle-version 4.10.2 buildscript { repositories { google() } dependencies { classpath ‘com.android.tools.build:gradle:3.1.2' } } buildscript { repositories { google() } dependencies { classpath ‘com.android.tools.build:gradle:3.1.2' } }
12. |12 JVM tuning Provide enough heap space Other tweaks often do more harm than good Spend your time on structural improvements
13. |13 The Gradle Daemon The Gradle Daemon is enabled by default Provides a number of in-memory caches Useful for repeated builds Do not disable it!
14. Where is the Problem?
15. |15 Build Scans
16. |16 The Build Lifecycle
17. |17 Red Flags Startup/Settings/buildSrc > 1s Configuration time >10ms/project Single-line change ≅ clean build No-op build doing any work at all High GC time
18. Startup, buildSrc, Settings
19. |19 Startup, buildSrc, Settings Use the Daemon, keep it healthy
20. Configuration Time
21. |21 Configuration Time Applying plugins Evaluating build scripts Running afterEvaluate {} blocks
22. |22 Configuration Time When running any task Even gradle help/gradle tasks Android Studio sync
23. |23 Resolution at Configuration Time
24. |24 Eager resolution task uberJar(type:Jar) { from sourceSets.main.output from configurations.runtime.collect { it.directory ? it : zipTree(it) } classifier "uber-jar" } task uberJar(type:Jar) { from sourceSets.main.output from configurations.runtime.collect { it.directory ? it : zipTree(it) } classifier "uber-jar" } Resolution at Configuration Time
25. |25 Use lazy evaluation instead task uberJar(type:Jar) { inputs.files configurations.runtime from sourceSets.main.output from { configurations.runtime.collect { it.directory ? it : zipTree(it) } } classifier "uber-jar" } task uberJar(type:Jar) { inputs.files configurations.runtime from sourceSets.main.output from { configurations.runtime.collect { it.directory ? it : zipTree(it) } } classifier "uber-jar" } Resolution at Configuration Time
26. |26 This build script seems expensive I/O at Configuration Time
27. |27 task projectStats() { def statsFile = new File(buildDir, 'stats.txt') statsFile.parentFile.mkdirs() statsFile.text = "Source files: ${sourceSets.main.java.size()}" } task projectStats() { def statsFile = new File(buildDir, 'stats.txt') statsFile.parentFile.mkdirs() statsFile.text = "Source files: ${sourceSets.main.java.size()}" } I/O at Configuration Time Careful when writing custom tasks!
28. |28 I/O at Configuration Time task projectStats() { def statsFile = new File(buildDir, 'stats.txt') inputs.files sourceSets.main.java outputs.file statsFile doLast { statsFile.parentFile.mkdirs() statsFile.text = "Source files: ${sourceSets.main.java.size()}" } } task projectStats() { def statsFile = new File(buildDir, 'stats.txt') inputs.files sourceSets.main.java outputs.file statsFile doLast { statsFile.parentFile.mkdirs() statsFile.text = "Source files: ${sourceSets.main.java.size()}" } } Don’t forget doLast {}
29. |29 task projectStats(type: ProjectStats) { statsFile = new File(buildDir, 'stats.txt') sources = sourceSets.main.java } class ProjectStats extends DefaultTask { @InputFiles FileCollection sources @OutputFile File statsFile @TaskAction def stats() { statsFile.text = "Source files: ${sources.size()}" } } task projectStats(type: ProjectStats) { statsFile = new File(buildDir, 'stats.txt') sources = sourceSets.main.java } class ProjectStats extends DefaultTask { @InputFiles FileCollection sources @OutputFile File statsFile @TaskAction def stats() { statsFile.text = "Source files: ${sources.size()}" } } I/O at Configuration Time
30. |30 Inefficient Plugins
31. |31 Expensive logic for each project def out = new ByteArrayOutputStream() exec { commandLine 'git', 'rev-parse', "HEAD" standardOutput = out workingDir = rootDir } version = new String(out.toByteArray()) def out = new ByteArrayOutputStream() exec { commandLine 'git', 'rev-parse', "HEAD" standardOutput = out workingDir = rootDir } version = new String(out.toByteArray()) subprojects { apply from: "$rootDir/gradle/version-from-status.gradle" } subprojects { apply from: "$rootDir/gradle/version-from-status.gradle" } Inefficient Plugins
32. |32 Reuse expensive calculations def out = new ByteArrayOutputStream() exec { commandLine 'git', 'rev-parse', "HEAD" standardOutput = out workingDir = rootDir } version = new String(out.toByteArray()) subprojects { version = rootProject.version } def out = new ByteArrayOutputStream() exec { commandLine 'git', 'rev-parse', "HEAD" standardOutput = out workingDir = rootDir } version = new String(out.toByteArray()) subprojects { version = rootProject.version } Inefficient Plugins
33. |33 Variant Explosion Use variantFilter {} to exclude useless flavor/buildType combinations
34. Optimising Build Logic
35. |35 Extract Script Plugins Makes finding issues easier apply from: "$rootDir/gradle/environment.gradle" apply from: "gradle/chuck-norris-fact.gradle" apply from: "$rootDir/gradle/version-from-status.gradle" subprojects { apply plugin: 'java' apply from: "$rootDir/gradle/test-fixtures.gradle" apply from: "$rootDir/gradle/compiler-settings.gradle" } apply from: "$rootDir/gradle/environment.gradle" apply from: "gradle/chuck-norris-fact.gradle" apply from: "$rootDir/gradle/version-from-status.gradle" subprojects { apply plugin: 'java' apply from: "$rootDir/gradle/test-fixtures.gradle" apply from: "$rootDir/gradle/compiler-settings.gradle" }
36. |36 Extract Binary Plugins Use buildSrc Use static compilation Keep build scripts declarative
37. |37 Optimise on the Algorithmic Level gradle-profiler --profile jfr
38. Execution Time
39. |39 Execution Time Executing selected tasks Incremental Cacheable Parallelizable
40. |40 Incremental Builds Nothing changed? Executed tasks should be zero!
41. |41 Incremental Builds Find volatile inputs
42. |42 Example: Crashlytics Unique IDs are the bane of local dev performance apply plugin: 'io.fabric' android { buildT ypes { debug { ext.alwaysUpdateBuildId = false } } } apply plugin: 'io.fabric' android { buildT ypes { debug { ext.alwaysUpdateBuildId = false } } }
43. |43 Faster Compilation Modularization => Compile avoidance Decoupled code => Faster incremental compilation Careful with Kotlin migration (for now)
44. |44 Incremental Annotation Processing Since Gradle 4.7 Early adopters: Lombok, Android-State, Dagger, Parceller Others: github.com/gradle/gradle/issues/5277
45. |45 Caching
46. |46 Parallelism Serial execution
47. |47 Parallelism --parallel and small, decoupled projects
48. Keeping track of performance
49. |50 Gradle Enterprise
50. |51 See all Builds Find out what colleagues are struggling with
51. |52 Get Deep Insights
52. |53 Watch Performance over Time
53. Resources
54. |55 Gradle performance guide https://guides.gradle.org/performance Android performance guide https://developer.android.com/studio/build/optimize-your-build.html Plugin development guide https://guides.gradle.org/implementing-gradle-plugins Structuring build logic guide https://docs.gradle.org/current/userguide/organizing_build_logic.html Guides
55. |56 scans.gradle.com
56. w Thank you Devoxx Belgium 2018