$30 off During Our Annual Pro Sale. View Details »

Gradle recipes for reducing your build times (droidcon London 2022)

Adam Ahmed
October 29, 2022

Gradle recipes for reducing your build times (droidcon London 2022)

Lessons learned from how we reduced the build times on a large project from 14 minutes down to 2 minutes.

In this talk, we’ll go through some of the issues that we discovered were causing our builds to take so long and what we learned from them, then we’ll deep-dive into how we went about fixing these issues, and how we tried to ensure that we never end up in this position again.

You don’t need to be a Gradle expert to attend this talk or to make these changes to your code-base to improve your own build times.

Adam Ahmed

October 29, 2022
Tweet

Other Decks in Programming

Transcript

  1. Adam Ahmed — @oheyadam Gradle Recipes for Reducing Your Build

    Times
  2. Background @oheyadam

  3. Background • 30+ modules @oheyadam

  4. Background • 30+ modules • 50+ screens @oheyadam

  5. Background • 30+ modules • 50+ screens • 14 Androids

    @oheyadam
  6. What We’ll Cover @oheyadam

  7. What We’ll Cover • Gradle jargon @oheyadam

  8. What We’ll Cover • Gradle jargon • The state of

    our build setup @oheyadam
  9. What We’ll Cover • Gradle jargon • The state of

    our build setup • Easy wins @oheyadam
  10. What We’ll Cover • Gradle jargon • The state of

    our build setup • Easy wins • General advice and best practices @oheyadam
  11. What We’ll Cover • Gradle jargon • The state of

    our build setup • Easy wins • General advice and best practices • Our build setup now @oheyadam
  12. What We’ll Cover • Gradle jargon • The state of

    our build setup • Easy wins • General advice and best practices • Our build setup now • Measuring @oheyadam
  13. Gradle Jargon @oheyadam

  14. Gradle Jargon • Tasks @oheyadam

  15. Gradle Jargon • Tasks • Projects/Modules @oheyadam

  16. Gradle Jargon • Tasks • Projects/Modules • Plugins @oheyadam

  17. Gradle Jargon • Tasks • Projects/Modules • Plugins • Lifecycle

    @oheyadam
  18. Tasks @oheyadam

  19. Tasks • Basic units of work @oheyadam

  20. Tasks • Basic units of work • Builds are modeled

    as Directed Acyclic Graphs of tasks @oheyadam
  21. Tasks • Basic units of work • Builds are modeled

    as Directed Acyclic Graphs of tasks • DAGs are wired and generated based on task dependencies @oheyadam
  22. Task Graph @oheyadam

  23. Tasks • Basic units of work • Builds are modeled

    as Directed Acyclic Graphs of tasks • DAGs are wired and generated based on task dependencies • Tasks can have Actions, Inputs, and Outputs @oheyadam
  24. Tasks • Basic units of work • Builds are modeled

    as Directed Acyclic Graphs of tasks • DAGs are wired and generated based on task dependencies • Tasks can have Actions, Inputs, and Outputs • Cacheable @oheyadam
  25. Tasks De fi ning Tasks tasks.register("greetings") { doLast { println("Hello

    Droidcon!”) } } @oheyadam
  26. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask :

    DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
  27. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask :

    DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
  28. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask :

    DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
  29. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask :

    DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
  30. Tasks De fi ning Tasks @CacheableTask abstract class GreetingsTask :

    DefaultTask() { @get:Input abstract val place: Property<String> @get:OutputFile abstract val output: RegularFileProperty @TaskAction fun greet() { val greeting = "Hello ${place.get()}" println(greeting) output.asFile.get().writeText(greeting) } }
  31. Projects/Modules @oheyadam

  32. Projects/Modules • Gradle builds are made up of one or

    more projects @oheyadam
  33. Projects/Modules • Gradle builds are made up of one or

    more projects • Work that Gradle can do on a project is defined by one or more tasks @oheyadam
  34. Projects/Modules • Gradle builds are made up of one or

    more projects • Work that Gradle can do on a project is defined by one or more tasks • Tasks are provided by applying a plugin, or you can write them yourself @oheyadam
  35. Projects/Modules • Gradle builds are made up of one or

    more projects • Work that Gradle can do on a project is defined by one or more tasks • Tasks are provided by applying a plugin, or you can write them yourself • Android Modules are Gradle Projects @oheyadam
  36. Plugins @oheyadam

  37. Plugins • Plugins are an amalgamation of Tasks @oheyadam

  38. Plugins • Plugins are an amalgamation of Tasks • Used

    for configuring Projects @oheyadam
  39. Plugins • Plugins are an amalgamation of Tasks • Used

    for configuring Projects • Encapsulate repetitive logic @oheyadam
  40. Plugins • Plugins are an amalgamation of Tasks • Used

    for configuring Projects • Encapsulate repetitive logic • Android Gradle Plugin @oheyadam
  41. Plugins • Plugins are an amalgamation of Tasks • Used

    for configuring Projects • Encapsulate repetitive logic • Android Gradle Plugin • com.android.application • com.android.library @oheyadam
  42. Lifecycle @oheyadam

  43. Lifecycle • Initialization @oheyadam

  44. Lifecycle • Initialization • Configuration @oheyadam

  45. Lifecycle • Initialization • Configuration • Execution @oheyadam

  46. The State of Our Build Setup @oheyadam

  47. The State of Our Build Setup • An average of

    14 minutes for local build times @oheyadam
  48. The State of Our Build Setup • An average of

    14 minutes for local build times • A lot of code duplication @oheyadam
  49. The State of Our Build Setup • An average of

    14 minutes for local build times • A lot of code duplication • Outdated tooling dependencies @oheyadam
  50. The State of Our Build Setup • An average of

    14 minutes for local build times • A lot of code duplication • Outdated tooling dependencies • Plugins applied needlessly everywhere @oheyadam
  51. The State of Our Build Setup • An average of

    14 minutes for local build times • A lot of code duplication • Outdated tooling dependencies • Plugins applied needlessly everywhere • Tasks are always created eagerly @oheyadam
  52. Easy Wins! @oheyadam

  53. Easy Wins • Enable file-system watching • org.gradle.vfs.watch=true @oheyadam

  54. Easy Wins • Enable file-system watching • org.gradle.vfs.watch=true • Enable

    configuration on demand • org.gradle.configureondemand=true @oheyadam
  55. Easy Wins • Enable file-system watching • org.gradle.vfs.watch=true • Enable

    configuration on demand • org.gradle.configureondemand=true • Enable parallel execution • org.gradle.parallel=true @oheyadam
  56. Easy Wins • Enable parallel execution • org.gradle.parallel=true • Enable

    build caching • org.gradle.caching=true @oheyadam
  57. General Advice and Best Practices @oheyadam

  58. General Advice and Best Practices • Be vigilant about adding

    new plugins to your projects @oheyadam
  59. General Advice and Best Practices • Be vigilant about adding

    new plugins to your projects • Check if you can disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin @oheyadam
  60. General Advice and Best Practices • Check if you can

    disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin android { buildTypes { debug { FirebasePerformance { instrumentationEnabled false } } } }
  61. General Advice and Best Practices • Be vigilant about adding

    new plugins to your projects • Check if you can disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin • Replace KAPT with KSP wherever possible • Room and Moshi are already compatible @oheyadam
  62. General Advice and Best Practices • Be vigilant about adding

    new plugins to your projects • Check if you can disable plugins on debug builds • e.g., the Firebase Performance Monitoring plugin • Replace KAPT with KSP wherever possible • Room and Moshi are already compatible • Look into square/anvil or JakeWharton/dagger-reflect https://firebase.google.com/docs/perf-mon/disable-sdk?platform=android#disable-gradle-plugin https://kotlinlang.org/docs/ksp-overview.html#supported-libraries https://github.com/square/anvil https://github.com/JakeWharton/dagger-reflect
  63. General Advice and Best Practices • Look into square/anvil or

    JakeWharton/dagger-reflect • Modularize @oheyadam
  64. General Advice and Best Practices • Look into square/anvil or

    JakeWharton/dagger-reflect • Modularize • Not every module needs to be an Android module @oheyadam
  65. General Advice and Best Practices • Look into square/anvil or

    JakeWharton/dagger-reflect • Modularize • Not every module needs to be an Android module • Favor implementation() configurations over api() @oheyadam
  66. General Advice and Best Practices • Look into square/anvil or

    JakeWharton/dagger-reflect • Modularize • Not every module needs to be an Android module • Favor implementation() configurations over api() • Do as little computation as possible during the configuration phase @oheyadam
  67. General Advice and Best Practices • Do as little computation

    as possible during the configuration phase • Favor custom tasks over scripts @oheyadam
  68. General Advice and Best Practices • Do as little computation

    as possible during the configuration phase • Favor custom tasks over scripts • Make most tasks cacheable using @CacheableTask • copy/jar/zip types are faster to rerun • Non-stable inputs make cache hits difficult @oheyadam
  69. General Advice and Best Practices • Do as little computation

    as possible during the configuration phase • Favor custom tasks over scripts • Make most tasks cacheable using @CacheableTask • copy/jar/zip types are faster to rerun • Non-stable inputs make cache hits difficult • Prefer lazy Gradle APIs over eager ones to take advantage of Configuration Avoidance https://github.com/liutikas/gradle-best-practices
  70. Lazily register tasks, don’t eagerly create them tasks.register("greetings") { sleep

    2000 doLast { println("Hello Droidcon!”) } } tasks.create(“greetings") { sleep 2000 doLast { println("Hello Droidcon!”) } } @oheyadam
  71. General Advice and Best Practices • Prefer lazy Gradle APIs

    over eager ones to take advantage of Configuration Avoidance • Enable non-transitive R classes @oheyadam
  72. General Advice and Best Practices • Prefer lazy Gradle APIs

    over eager ones to take advantage of Configuration Avoidance • Enable non-transitive R classes • Disable Jetifier @oheyadam
  73. General Advice and Best Practices • Prefer lazy Gradle APIs

    over eager ones to take advantage of Configuration Avoidance • Enable non-transitive R classes • Disable Jetifier • Disable build variants that aren’t relevant https://blog.blundellapps.co.uk/speed-up-your-build-non-transitive-r-files/ https://twitter.com/n8ebel/status/1455347318199259137 https://adambennett.dev/2020/08/disabling-jetifier/
  74. General Advice and Best Practices Disable build variants that aren’t

    relevant androidComponents { beforeVariants { variantBuilder -> if (variantBuilder.productFlavors.containsAll(listOf("api" to "minApi21", "mode" to "demo"))) { variantBuilder.enabled = false } } } https://developer.android.com/studio/build/build-variants#filter-variants
  75. General Advice and Best Practices • Disable build variants that

    aren’t relevant • Don’t use buildSrc. Use Included Builds instead @oheyadam
  76. General Advice and Best Practices • Disable build variants that

    aren’t relevant • Don’t use buildSrc. Use Included Builds instead • Use Version Catalogs and Type-safe Project Accessors for easier dependency management dependencies { implementation(projects.libraryAnalytics) implementation(libs.androidx.fragment.ktx) } https://github.com/android/nowinandroid https://developer.squareup.com/blog/herding-elephants/ https://docs.gradle.org/7.0/userguide/declaring_dependencies.html#sec:type-safe-project-accessors
  77. General Advice and Best Practices • Use Version Catalogs and

    Type-safe Project Accessors for easier dependency management • Enable Configuration Caching • org.gradle.unsafe.configuration-cache=true @oheyadam
  78. General Advice and Best Practices • Use Version Catalogs and

    Type-safe Project Accessors for easier dependency management • Enable Configuration Caching
  79. General Advice and Best Practices • Use Version Catalogs and

    Type-safe Project Accessors for easier dependency management • Enable Configuration Caching
  80. General Advice and Best Practices • Enable Configuration Caching •

    You no longer need to add a Kotlin file to your Java modules to make them cooperate with Kotlin’s IC • kotlin.incremental.useClasspathSnapshot=true @oheyadam
  81. General Advice and Best Practices • You no longer need

    to add a Kotlin file to your Java modules to make them cooperate with Kotlin’s IC* • kotlin.incremental.useClasspathSnapshot=true • Disable BuildConfig generation for modules that don’t need it • android.defaults.buildfeatures.buildconfig=false @oheyadam
  82. General Advice and Best Practices • Disable BuildConfig generation for

    modules that don’t need it • android.defaults.buildfeatures.buildconfig=false • In fact, you can opt out of many android build features that are enabled by default if you don’t need them https://kotlinlang.org/docs/whatsnew17.html#a-new-approach-to-incremental-compilation
  83. General Advice and Best Practices • In fact, you can

    opt out of many android build features that are enabled by default if you don’t need them • android.defaults.buildfeatures.aidl=false • android.defaults.buildfeatures.renderscript=false • android.defaults.buildfeatures.resvalues=false • android.defaults.buildfeatures.shaders=false @oheyadam
  84. Our Build Setup Now @oheyadam

  85. Our Build Setup Now • ~2 minutes for incremental changes

    @oheyadam
  86. Our Build Setup Now • ~2 minutes for incremental changes

    • Down to ~1 minute after switching to M1 MacBook Pros @oheyadam
  87. Our Build Setup Now • ~2 minutes for incremental changes

    • Down to ~1 minute after switching to M1 MacBook Pros • Convention plugins to reuse and contain build logic @oheyadam
  88. Our Build Setup Now • ~2 minutes for incremental changes

    • Down to ~1 minute after switching to M1 MacBook Pros • Convention plugins to reuse and contain build logic • Very lean build scripts @oheyadam
  89. Our Build Setup Now Very Lean Build Scrips plugins {

    id("android.feature") } android { namespace = "com.company.module.name" } dependencies { implementation(projects.libraryAnalytics) implementation(libs.androidx.fragment.ktx) testImplementation(projects.coreTesting) androidTestImplementation(projects.coreUiTesting) }
  90. Measuring @oheyadam

  91. Measuring • Measure, measure, measure! @oheyadam

  92. Measuring • Measure, measure, measure! • Use the Gradle-profiler @oheyadam

  93. Measuring • Measure, measure, measure! • Use the Gradle-profiler •

    Use Gradle Enterprise @oheyadam
  94. Measuring • Measure, measure, measure! • Use the Gradle-profiler •

    Use Gradle Enterprise • Write your own BuildLifecycleListener https://github.com/gradle/gradle-pro fi ler https://gradle.com/roi-calculator/ https://gist.github.com/oheyadam/d30a104091753fc79793bc32aea39d2e
  95. Write Your Own BuildLifecycleListener class BuildLifecycleListener : BuildAdapter() { override

    fun buildFinished(result: BuildResult) { // … } } class BuildLifecyclePlugin : Plugin<Project> { override fun apply(target: Project) { target.gradle.addBuildListener(BuildLifecycleListener()) } }
  96. Thank you! @oheyadam