Gradle recipes for reducing your build times (droidcon London 2022)

Lessons learned from how we reduced the build times on a large project from 14 minutes down to 2 minutes.

In this talk, we’ll go through some of the issues that we discovered were causing our builds to take so long and what we learned from them, then we’ll deep-dive into how we went about fixing these issues, and how we tried to ensure that we never end up in this position again.

You don’t need to be a Gradle expert to attend this talk or to make these changes to your code-base to improve your own build times.

Transcript

1. Adam Ahmed — @oheyadam Gradle Recipes for Reducing Your Build Times
2. Background @oheyadam
3. Background • 30+ modules @oheyadam
4. Background • 30+ modules • 50+ screens @oheyadam
5. Background • 30+ modules • 50+ screens • 14 Androids @oheyadam
6. What We’ll Cover @oheyadam
7. What We’ll Cover • Gradle jargon @oheyadam
8. What We’ll Cover • Gradle jargon • The state of our build setup @oheyadam
9. What We’ll Cover • Gradle jargon • The state of our build setup • Easy wins @oheyadam
10. What We’ll Cover • Gradle jargon • The state of our build setup • Easy wins • General advice and best practices @oheyadam
11. What We’ll Cover • Gradle jargon • The state of our build setup • Easy wins • General advice and best practices • Our build setup now @oheyadam
12. What We’ll Cover • Gradle jargon • The state of our build setup • Easy wins • General advice and best practices • Our build setup now • Measuring @oheyadam
13. Gradle Jargon @oheyadam
14. Gradle Jargon • Tasks @oheyadam
15. Gradle Jargon • Tasks • Projects/Modules @oheyadam
16. Gradle Jargon • Tasks • Projects/Modules • Plugins @oheyadam
17. Gradle Jargon • Tasks • Projects/Modules • Plugins • Lifecycle @oheyadam
18. Tasks @oheyadam
19. Tasks • Basic units of work @oheyadam
20. Tasks • Basic units of work • Builds are modeled as Directed Acyclic Graphs of tasks @oheyadam
21. Tasks • Basic units of work • Builds are modeled as Directed Acyclic Graphs of tasks • DAGs are wired and generated based on task dependencies @oheyadam
22. Task Graph @oheyadam
23. Tasks • Basic units of work • Builds are modeled as Directed Acyclic Graphs of tasks • DAGs are wired and generated based on task dependencies • Tasks can have Actions, Inputs, and Outputs @oheyadam
24. Tasks • Basic units of work • Builds are modeled as Directed Acyclic Graphs of tasks • DAGs are wired and generated based on task dependencies • Tasks can have Actions, Inputs, and Outputs • Cacheable @oheyadam
25. Tasks De fi ning Tasks tasks.register("greetings") { doLast { println("Hello Droidcon!”) } } @oheyadam
26. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask : DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
27. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask : DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
28. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask : DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
29. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask : DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
30. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask : DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
31. Projects/Modules @oheyadam
32. Projects/Modules • Gradle builds are made up of one or more projects @oheyadam
33. Projects/Modules • Gradle builds are made up of one or more projects • Work that Gradle can do on a project is defined by one or more tasks @oheyadam
34. Projects/Modules • Gradle builds are made up of one or more projects • Work that Gradle can do on a project is defined by one or more tasks • Tasks are provided by applying a plugin, or you can write them yourself @oheyadam
35. Projects/Modules • Gradle builds are made up of one or more projects • Work that Gradle can do on a project is defined by one or more tasks • Tasks are provided by applying a plugin, or you can write them yourself • Android Modules are Gradle Projects @oheyadam
36. Plugins @oheyadam
37. Plugins • Plugins are an amalgamation of Tasks @oheyadam
38. Plugins • Plugins are an amalgamation of Tasks • Used for configuring Projects @oheyadam
39. Plugins • Plugins are an amalgamation of Tasks • Used for configuring Projects • Encapsulate repetitive logic @oheyadam
40. Plugins • Plugins are an amalgamation of Tasks • Used for configuring Projects • Encapsulate repetitive logic • Android Gradle Plugin @oheyadam
41. Plugins • Plugins are an amalgamation of Tasks • Used for configuring Projects • Encapsulate repetitive logic • Android Gradle Plugin • com.android.application • com.android.library @oheyadam
42. Lifecycle @oheyadam
43. Lifecycle • Initialization @oheyadam
44. Lifecycle • Initialization • Configuration @oheyadam
45. Lifecycle • Initialization • Configuration • Execution @oheyadam
46. The State of Our Build Setup @oheyadam
47. The State of Our Build Setup • An average of 14 minutes for local build times @oheyadam
48. The State of Our Build Setup • An average of 14 minutes for local build times • A lot of code duplication @oheyadam
49. The State of Our Build Setup • An average of 14 minutes for local build times • A lot of code duplication • Outdated tooling dependencies @oheyadam
50. The State of Our Build Setup • An average of 14 minutes for local build times • A lot of code duplication • Outdated tooling dependencies • Plugins applied needlessly everywhere @oheyadam
51. The State of Our Build Setup • An average of 14 minutes for local build times • A lot of code duplication • Outdated tooling dependencies • Plugins applied needlessly everywhere • Tasks are always created eagerly @oheyadam
52. Easy Wins! @oheyadam
53. Easy Wins • Enable file-system watching • org.gradle.vfs.watch=true @oheyadam
54. Easy Wins • Enable file-system watching • org.gradle.vfs.watch=true • Enable configuration on demand • org.gradle.configureondemand=true @oheyadam
55. Easy Wins • Enable file-system watching • org.gradle.vfs.watch=true • Enable configuration on demand • org.gradle.configureondemand=true • Enable parallel execution • org.gradle.parallel=true @oheyadam
56. Easy Wins • Enable parallel execution • org.gradle.parallel=true • Enable build caching • org.gradle.caching=true @oheyadam
57. General Advice and Best Practices @oheyadam
58. General Advice and Best Practices • Be vigilant about adding new plugins to your projects @oheyadam
59. General Advice and Best Practices • Be vigilant about adding new plugins to your projects • Check if you can disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin @oheyadam
60. General Advice and Best Practices • Check if you can disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin android { buildTypes { debug { FirebasePerformance { instrumentationEnabled false } } } }
61. General Advice and Best Practices • Be vigilant about adding new plugins to your projects • Check if you can disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin • Replace KAPT with KSP wherever possible • Room and Moshi are already compatible @oheyadam
62. General Advice and Best Practices • Be vigilant about adding new plugins to your projects • Check if you can disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin • Replace KAPT with KSP wherever possible • Room and Moshi are already compatible • Look into square/anvil or JakeWharton/dagger-reflect https://firebase.google.com/docs/perf-mon/disable-sdk?platform=android#disable-gradle-plugin https://kotlinlang.org/docs/ksp-overview.html#supported-libraries https://github.com/square/anvil https://github.com/JakeWharton/dagger-reflect
63. General Advice and Best Practices • Look into square/anvil or JakeWharton/dagger-reflect • Modularize @oheyadam
64. General Advice and Best Practices • Look into square/anvil or JakeWharton/dagger-reflect • Modularize • Not every module needs to be an Android module @oheyadam
65. General Advice and Best Practices • Look into square/anvil or JakeWharton/dagger-reflect • Modularize • Not every module needs to be an Android module • Favor implementation() configurations over api() @oheyadam
66. General Advice and Best Practices • Look into square/anvil or JakeWharton/dagger-reflect • Modularize • Not every module needs to be an Android module • Favor implementation() configurations over api() • Do as little computation as possible during the configuration phase @oheyadam
67. General Advice and Best Practices • Do as little computation as possible during the configuration phase • Favor custom tasks over scripts @oheyadam
68. General Advice and Best Practices • Do as little computation as possible during the configuration phase • Favor custom tasks over scripts • Make most tasks cacheable using @CacheableTask • copy/jar/zip types are faster to rerun • Non-stable inputs make cache hits difficult @oheyadam
69. General Advice and Best Practices • Do as little computation as possible during the configuration phase • Favor custom tasks over scripts • Make most tasks cacheable using @CacheableTask • copy/jar/zip types are faster to rerun • Non-stable inputs make cache hits difficult • Prefer lazy Gradle APIs over eager ones to take advantage of Configuration Avoidance https://github.com/liutikas/gradle-best-practices
70. Lazily register tasks, don’t eagerly create them tasks.register("greetings") { sleep 2000 doLast { println("Hello Droidcon!”) } } tasks.create(“greetings") { sleep 2000 doLast { println("Hello Droidcon!”) } } @oheyadam
71. General Advice and Best Practices • Prefer lazy Gradle APIs over eager ones to take advantage of Configuration Avoidance • Enable non-transitive R classes @oheyadam
72. General Advice and Best Practices • Prefer lazy Gradle APIs over eager ones to take advantage of Configuration Avoidance • Enable non-transitive R classes • Disable Jetifier @oheyadam
73. General Advice and Best Practices • Prefer lazy Gradle APIs over eager ones to take advantage of Configuration Avoidance • Enable non-transitive R classes • Disable Jetifier • Disable build variants that aren’t relevant https://blog.blundellapps.co.uk/speed-up-your-build-non-transitive-r-files/ https://twitter.com/n8ebel/status/1455347318199259137 https://adambennett.dev/2020/08/disabling-jetifier/
74. General Advice and Best Practices Disable build variants that aren’t relevant androidComponents { beforeVariants { variantBuilder -> if (variantBuilder.productFlavors.containsAll(listOf("api" to "minApi21", "mode" to "demo"))) { variantBuilder.enabled = false } } } https://developer.android.com/studio/build/build-variants#filter-variants
75. General Advice and Best Practices • Disable build variants that aren’t relevant • Don’t use buildSrc. Use Included Builds instead @oheyadam
76. General Advice and Best Practices • Disable build variants that aren’t relevant • Don’t use buildSrc. Use Included Builds instead • Use Version Catalogs and Type-safe Project Accessors for easier dependency management dependencies { implementation(projects.libraryAnalytics) implementation(libs.androidx.fragment.ktx) } https://github.com/android/nowinandroid https://developer.squareup.com/blog/herding-elephants/ https://docs.gradle.org/7.0/userguide/declaring_dependencies.html#sec:type-safe-project-accessors
77. General Advice and Best Practices • Use Version Catalogs and Type-safe Project Accessors for easier dependency management • Enable Configuration Caching • org.gradle.unsafe.configuration-cache=true @oheyadam
78. General Advice and Best Practices • Use Version Catalogs and Type-safe Project Accessors for easier dependency management • Enable Configuration Caching
79. General Advice and Best Practices • Use Version Catalogs and Type-safe Project Accessors for easier dependency management • Enable Configuration Caching
80. General Advice and Best Practices • Enable Configuration Caching • You no longer need to add a Kotlin file to your Java modules to make them cooperate with Kotlin’s IC • kotlin.incremental.useClasspathSnapshot=true @oheyadam
81. General Advice and Best Practices • You no longer need to add a Kotlin file to your Java modules to make them cooperate with Kotlin’s IC* • kotlin.incremental.useClasspathSnapshot=true • Disable BuildConfig generation for modules that don’t need it • android.defaults.buildfeatures.buildconfig=false @oheyadam
82. General Advice and Best Practices • Disable BuildConfig generation for modules that don’t need it • android.defaults.buildfeatures.buildconfig=false • In fact, you can opt out of many android build features that are enabled by default if you don’t need them https://kotlinlang.org/docs/whatsnew17.html#a-new-approach-to-incremental-compilation
83. General Advice and Best Practices • In fact, you can opt out of many android build features that are enabled by default if you don’t need them • android.defaults.buildfeatures.aidl=false • android.defaults.buildfeatures.renderscript=false • android.defaults.buildfeatures.resvalues=false • android.defaults.buildfeatures.shaders=false @oheyadam
84. Our Build Setup Now @oheyadam
85. Our Build Setup Now • ~2 minutes for incremental changes @oheyadam
86. Our Build Setup Now • ~2 minutes for incremental changes • Down to ~1 minute after switching to M1 MacBook Pros @oheyadam
87. Our Build Setup Now • ~2 minutes for incremental changes • Down to ~1 minute after switching to M1 MacBook Pros • Convention plugins to reuse and contain build logic @oheyadam
88. Our Build Setup Now • ~2 minutes for incremental changes • Down to ~1 minute after switching to M1 MacBook Pros • Convention plugins to reuse and contain build logic • Very lean build scripts @oheyadam
89. Our Build Setup Now Very Lean Build Scrips plugins { id("android.feature") } android { namespace = "com.company.module.name" } dependencies { implementation(projects.libraryAnalytics) implementation(libs.androidx.fragment.ktx) testImplementation(projects.coreTesting) androidTestImplementation(projects.coreUiTesting) }
90. Measuring @oheyadam
91. Measuring • Measure, measure, measure! @oheyadam
92. Measuring • Measure, measure, measure! • Use the Gradle-profiler @oheyadam
93. Measuring • Measure, measure, measure! • Use the Gradle-profiler • Use Gradle Enterprise @oheyadam
94. Measuring • Measure, measure, measure! • Use the Gradle-profiler • Use Gradle Enterprise • Write your own BuildLifecycleListener https://github.com/gradle/gradle-pro fi ler https://gradle.com/roi-calculator/ https://gist.github.com/oheyadam/d30a104091753fc79793bc32aea39d2e
95. Write Your Own BuildLifecycleListener class BuildLifecycleListener : BuildAdapter() { override fun buildFinished(result: BuildResult) { // … } } class BuildLifecyclePlugin : Plugin<Project> { override fun apply(target: Project) { target.gradle.addBuildListener(BuildLifecycleListener()) } }
96. Thank you! @oheyadam