Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Improving Gradle Builds

B3f560d34c14a9113e5024bc34ac26a0?s=47 Mohit S
January 25, 2022

Improving Gradle Builds

As the number of modules in your code base increases, speed and consistency of your Gradle builds becomes very important. However, improving your Gradle builds can be a tricky endeavor. In this talk, I’ll address these common questions in improving Gradle builds.

- How do we benchmark & profile our builds?
- How do you read build scans to identify problems?
- How do we maintain a consistent modular structure?
- What tools can you use to speed up builds?

I’ll take a deep dive into these questions with examples and insights from my experiences.

B3f560d34c14a9113e5024bc34ac26a0?s=128

Mohit S

January 25, 2022
Tweet

More Decks by Mohit S

Other Decks in Programming

Transcript

  1. Mohit Sarveiya Improving Gradle Builds @heyitsmohit

  2. Improving Gradle Builds • Measure & Benchmark • Modularization •

    Optimizations • Tools
  3. Measure Gradle Enterprise, Build Analyzer

  4. https: // github.com/uber-common/android-build-eval Github

  5. Uber Build Eval • Similar to Uber Production mobile apps

    • 1292 Modules • 21,235 Build Graph Edges
  6. How do we analyze our builds?

  7. How do we analyze our builds? • Build Analyzer in

    Android Studio
  8. Team 1 Team 2 Team 3 Team 4 Build Data

    Collect
  9. Team 1 Team 2 Team 3 Team 4 Build Data

    Analyze
  10. How do we analyze our builds? • Build Analyzer in

    Android Studio • Gradle Enterprise
  11. https: // gradle.com/ Gradle

  12. Build Scan $ ./gradlew rootModule:assembleDebug —scan

  13. Build Scan $ ./gradlew rootModule:assembleDebug —scan Publishing build scan ...

    https: // gradle.com/s/46dna3y4xauze
  14. https: // gradle.com/ Gradle

  15. https: // gradle.com/ Gradle

  16. Build Times • Configuration time • Task execution time •

    Cache Hits/Misses
  17. https: // gradle.com/ Gradle

  18. https: // gradle.com/ Gradle

  19. https: // gradle.com/ Gradle

  20. Exploring Config Time • Plugin config • Script config

  21. https: // gradle.com/ Gradle

  22. https: // gradle.com/ Gradle

  23. https: // gradle.com/ Gradle

  24. https: // gradle.com/ Gradle

  25. https: // gradle.com/ Gradle

  26. https: // gradle.com/ Gradle

  27. https: // gradle.com/ Gradle

  28. https: // gradle.com/ Gradle

  29. https: // gradle.com/ Gradle

  30. Build Tags • Local • CI

  31. https: // gradle.com/ Gradle

  32. https: // gradle.com/ Gradle

  33. How do we integrate for Local and CI?

  34. Build Tags plugins { id "com.gradle.enterprise" version "3.8.1" }

  35. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag

    "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
  36. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag

    "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
  37. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag

    "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
  38. Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag

    "CI" } else { tag "Local" } tag System.getProperty("os.name") } }
  39. https: // gradle.com/ Gradle

  40. Summary • Gradle Enterprise • How do we view build

    times across teams
  41. Optimizations

  42. Optimizations • Config time • Dependencies • Build cache

  43. Optimizations • Turn off unused Android Gradle Plugin features •

    Only apply AGP plugin where needed
  44. https: // gradle.com/ Gradle

  45. Optimizations android.defaults.buildfeatures.renderscript = false

  46. Optimizations android.defaults.buildfeatures.shaders = false

  47. AGP Features • Build Config • Data binding • View

    binding
  48. Optimizations • Config time • Dependencies • Build cache

  49. How do we remove unused plugins?

  50. Dependency Analysis Plugin • Removed unused plugins & dependencies •

    Provides suggestions on using api vs impl
  51. Dependency Analysis Plugin plugins { id 'com.autonomousapps.dependency-analysis' }

  52. Dependency Analysis Plugin ./gradlew projectHealth

  53. Dependency Analysis Plugin > Task :LibraryB:projectHealth Unused dependencies which should

    be removed: implementation("androidx.appcompat:appcompat:1.4.1") implementation("androidx.core:core-ktx:1.7.0") implementation("com.google.android.material:material:1.5.0")
  54. Dependency Analysis Plugin Plugin advice: kotlin-kapt: no used annotation processors

  55. Optimizations • Config time • Dependencies • Build cache

  56. Build Cache Task Output Cache (id) Result

  57. Build Cache Task Output Cache (id) Get

  58. Gradle Doctor Plugin • Slower tasks from cache • Build

    scan tags
  59. Gradle Doctor Plugin plugins { id "com.osacky.doctor" version "0.7.3" }

  60. Gradle Doctor Plugin plugins { id "com.osacky.doctor" version "0.7.3" }

  61. https: // gradle.com/ Gradle

  62. Gradle Doctor Plugin • doctor-negative-savings • doctor-slow-build-cache-connection

  63. How do you disable caching for a task?

  64. Disable Local Caching tasks.named("<taskname>").configure { outputs.cacheIf { false } }

  65. Disable Remote Caching tasks.named("<taskname>").configure { outputs.cacheIf { !isCI } }

  66. Gradle Doctor Plugin • Set threshold on Dagger processing

  67. Disable Remote Caching doctor { /** * Print a warning

    to the console if we spend more than this * amount of time with Dagger annotation processors. */ daggerThreshold = 5000 }
  68. Android Build Cache Plugin • Plugin by Gradle • Gradle

    plugin to fix Android build problems • Workarounds for Room
  69. Summary • Dependency Analysis Plugin • Gradle Doctor Plugin •

    Android Build Cache Fix Plugin
  70. Modularization Graph Assertions, Version Catalogs

  71. Modularization Module A Module B Module C

  72. How do we enforce best practices?

  73. Module Graph Assert Plugin • Check graph depth • Enforce

    conventions
  74. Module Graph Assert Plugin id "com.jraska.module.graph.assertion"

  75. Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 }

  76. Module Graph Assert Plugin Max height the number of edges

    on the longest path from the node to a leaf.
  77. Modularization Max height - 4

  78. Module Graph Assert Plugin ./gradlew assertModuleGraph

  79. Module Graph Assert Plugin > Task :app:assertMaxHeight FAILED 
 Module

    :app is allowed to have maximum height of 3, but has 4.
  80. Modularization Max height - 3

  81. Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 allowed

    = [‘:app’ -> ‘:feature-[a-z]’, ‘:feature.* -> :lib’] }
  82. Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 allowed

    = [‘:app’ -> ‘:feature-[a-z]’, ‘:feature.* -> :lib’] restricted = [‘:feature-[a-z]*’ :forbidden-module,] }
  83. How do we visualize all Gradle modules?

  84. Module Graph Assert Plugin • Generate full graph of all

    modules • Visualize a subgraph
  85. GraphViz • Tool to create graphs • Color nodes based

    on config
  86. Module Graph Assert Plugin ./gradlew generateModulesGraphvizText

  87. Module Graph Assert Plugin ./gradlew generateModulesGraphvizText 
 -Pmodules.graph.of.module=all_modules

  88. Module Graph Assert Plugin Generate Graph GraphViz File Visualize with

    GraphViz
  89. Module Graph Assert Plugin digraph G { ":leafModuleAvg" -> ":module785"

    ":leafModuleAvg" -> ":module574" ... }
  90. Full Graph

  91. Full Graph

  92. Module Graph Assert Plugin ./gradlew generateModulesGraphvizText 
 -Pmodules.graph.of.module=:feature_a

  93. How do we gets stats on all modules?

  94. Module Graph Assert Plugin ./gradlew generateModulesGraphStatistics 


  95. Module Graph Stats • Number of modules • Number of

    edges • Longest path
  96. How do we setup dependencies?

  97. Version Catalogs • Define all deps in TOML file •

    Gradle feature to share deps between modules
  98. Version Catalogs [versions] androidx-core = “1.6.0-beta01”

  99. Version Catalogs [versions] androidx-core = “1.6.0-beta01” [libraries] androidx-core-ktx = {

    module = "androidx.core:core-ktx", version.ref = “androidx-core" }
  100. Version Catalogs dependencies { implementation(libs.androidx.core.ktx) }

  101. Version Catalogs [bundles] compose = [ “androidx-composeUi", “androidx-composeMaterial”, “androidx-composeUiTooling" ]

  102. Version Catalogs dependencies { implementation(libs.bundles.compose) }

  103. Version Catalogs enableFeaturePreview("VERSION_CATALOGS")

  104. Summary • Module Assert Graph Plugin • Version Catalogs

  105. Benchmark Gradle Profiler

  106. Gradle Profiler • Automates benchmarking builds • Detect regressions

  107. Gradle Profiler

  108. Gradle Profiler Change

  109. How do we find regressions?

  110. Gradle Profiler 1. Write a performance scenario 2. Specify number

    of iterations
  111. Gradle Profiler brew install gradle-profiler

  112. Performance Scenario assemble { # Show a slightly more human-readable

    title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } assemble { }
  113. Performance Scenario assemble { # Show a slightly more human-readable

    title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } assemble { tasks = ["assemble"] }
  114. Gradle Profiler gradle-profiler —benchmark

  115. Gradle Profiler gradle-profiler —benchmark —iterations=

  116. Gradle Profiler gradle-profiler —benchmark —iterations= Profiling - 1 Benchmarking -

    10
  117. Gradle Profiler gradle-profiler —benchmark —iterations=10

  118. Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups= Profiling - 2 Benchmarking

    - 6
  119. Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups=6

  120. Gradle Profiler

  121. Gradle Profiler Mean: 348 ms Min: 319 ms P25: 330

    ms Median: 341 ms P75: 368 ms Std dev: 22.71 ms
  122. How do we test incremental builds?

  123. Gradle Profiler Change

  124. Gradle Profiler Iteration 1 Iteration 2 Iteration 3

  125. Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Abi Change

  126. Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Result

  127. Performance Scenario assemble { # Show a slightly more human-readable

    title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” }
  128. Performance Scenario assemble { # Show a slightly more human-readable

    title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” apply-android-resource-change-to = “strings.xml” tasks = ["assemble"] }
  129. Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups=6

  130. Gradle Profiler

  131. How do we test Gradle sync?

  132. Performance Scenario assemble { # Show a slightly more human-readable

    title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } androidStudioSync { android-studio-sync { } }
  133. Performance Scenario assemble { # Show a slightly more human-readable

    title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } androidStudioSync { android-studio-sync { studio-jvm-args = ["-Xms256m", "-Xmx4096m"] } }
  134. Gradle Profiler 1. Write a performance scenario 2. Specify number

    of iterations
  135. How do we automate profiling on CI?

  136. CI Pipeline 1. Create pipeline runs nightly 2. Collect metrics

    overtime
  137. CI Pipeline Docker image with 
 Gradle Profiler

  138. CI Pipeline Docker image with 
 Gradle Profiler Performance 


    Scenarios
  139. CI Pipeline Docker image with 
 Gradle Profiler Performance 


    Scenarios Run 
 nightly
  140. Use Cases 1. Anvil 2. KSP

  141. Anvil • Kotlin Compiler Plugin • Generate factories • Improve

    build times
  142. Anvil Dagger Module A Module B Module C

  143. How do we use Anvil to generate factories?

  144. Anvil anvil { generateDaggerFactories = true }

  145. Benchmarking Build • Benchmark with Anvil change • Benchmark without

    Anvil change (Baseline) • Compare results
  146. Performance Scenario assemble { # Show a slightly more human-readable

    title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” tasks = ["assemble"] }
  147. Results Module A Module B Module C Baseline Anvil

  148. Summary • How to setup & run Gradle Profiler •

    Performance scenarios • Anvil
  149. Thank You! www.codingwithmohit.com @heyitsmohit