Slide 1

Slide 1 text

Mohit Sarveiya Improving Gradle Builds @heyitsmohit

Slide 2

Slide 2 text

Improving Gradle Builds ● Measure & Benchmark ● Modularization ● Optimizations ● Tools

Slide 3

Slide 3 text

Measure Gradle Enterprise, Build Analyzer

Slide 4

Slide 4 text

https: // github.com/uber-common/android-build-eval Github

Slide 5

Slide 5 text

Uber Build Eval ● Similar to Uber Production mobile apps ● 1292 Modules ● 21,235 Build Graph Edges

Slide 6

Slide 6 text

How do we analyze our builds?

Slide 7

Slide 7 text

How do we analyze our builds? ● Build Analyzer in Android Studio

Slide 8

Slide 8 text

Team 1 Team 2 Team 3 Team 4 Build Data Collect

Slide 9

Slide 9 text

Team 1 Team 2 Team 3 Team 4 Build Data Analyze

Slide 10

Slide 10 text

How do we analyze our builds? ● Build Analyzer in Android Studio ● Gradle Enterprise

Slide 11

Slide 11 text

https: // gradle.com/ Gradle

Slide 12

Slide 12 text

Build Scan $ ./gradlew rootModule:assembleDebug —scan

Slide 13

Slide 13 text

Build Scan $ ./gradlew rootModule:assembleDebug —scan Publishing build scan ... https: // gradle.com/s/46dna3y4xauze

Slide 14

Slide 14 text

https: // gradle.com/ Gradle

Slide 15

Slide 15 text

https: // gradle.com/ Gradle

Slide 16

Slide 16 text

Build Times ● Configuration time ● Task execution time ● Cache Hits/Misses

Slide 17

Slide 17 text

https: // gradle.com/ Gradle

Slide 18

Slide 18 text

https: // gradle.com/ Gradle

Slide 19

Slide 19 text

https: // gradle.com/ Gradle

Slide 20

Slide 20 text

Exploring Config Time ● Plugin config ● Script config

Slide 21

Slide 21 text

https: // gradle.com/ Gradle

Slide 22

Slide 22 text

https: // gradle.com/ Gradle

Slide 23

Slide 23 text

https: // gradle.com/ Gradle

Slide 24

Slide 24 text

https: // gradle.com/ Gradle

Slide 25

Slide 25 text

https: // gradle.com/ Gradle

Slide 26

Slide 26 text

https: // gradle.com/ Gradle

Slide 27

Slide 27 text

https: // gradle.com/ Gradle

Slide 28

Slide 28 text

https: // gradle.com/ Gradle

Slide 29

Slide 29 text

https: // gradle.com/ Gradle

Slide 30

Slide 30 text

Build Tags ● Local ● CI

Slide 31

Slide 31 text

https: // gradle.com/ Gradle

Slide 32

Slide 32 text

https: // gradle.com/ Gradle

Slide 33

Slide 33 text

How do we integrate for Local and CI?

Slide 34

Slide 34 text

Build Tags plugins { id "com.gradle.enterprise" version "3.8.1" }

Slide 35

Slide 35 text

Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }

Slide 36

Slide 36 text

Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }

Slide 37

Slide 37 text

Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }

Slide 38

Slide 38 text

Build Tags gradleEnterprise { buildScan { if (System.getenv("CI")) { tag "CI" } else { tag "Local" } tag System.getProperty("os.name") } }

Slide 39

Slide 39 text

https: // gradle.com/ Gradle

Slide 40

Slide 40 text

Summary ● Gradle Enterprise ● How do we view build times across teams

Slide 41

Slide 41 text

Optimizations

Slide 42

Slide 42 text

Optimizations ● Config time ● Dependencies ● Build cache

Slide 43

Slide 43 text

Optimizations • Turn off unused Android Gradle Plugin features • Only apply AGP plugin where needed

Slide 44

Slide 44 text

https: // gradle.com/ Gradle

Slide 45

Slide 45 text

Optimizations android.defaults.buildfeatures.renderscript = false

Slide 46

Slide 46 text

Optimizations android.defaults.buildfeatures.shaders = false

Slide 47

Slide 47 text

AGP Features • Build Config • Data binding • View binding

Slide 48

Slide 48 text

Optimizations ● Config time ● Dependencies ● Build cache

Slide 49

Slide 49 text

How do we remove unused plugins?

Slide 50

Slide 50 text

Dependency Analysis Plugin • Removed unused plugins & dependencies • Provides suggestions on using api vs impl

Slide 51

Slide 51 text

Dependency Analysis Plugin plugins { id 'com.autonomousapps.dependency-analysis' }

Slide 52

Slide 52 text

Dependency Analysis Plugin ./gradlew projectHealth

Slide 53

Slide 53 text

Dependency Analysis Plugin > Task :LibraryB:projectHealth Unused dependencies which should be removed: implementation("androidx.appcompat:appcompat:1.4.1") implementation("androidx.core:core-ktx:1.7.0") implementation("com.google.android.material:material:1.5.0")

Slide 54

Slide 54 text

Dependency Analysis Plugin Plugin advice: kotlin-kapt: no used annotation processors

Slide 55

Slide 55 text

Optimizations ● Config time ● Dependencies ● Build cache

Slide 56

Slide 56 text

Build Cache Task Output Cache (id) Result

Slide 57

Slide 57 text

Build Cache Task Output Cache (id) Get

Slide 58

Slide 58 text

Gradle Doctor Plugin • Slower tasks from cache • Build scan tags

Slide 59

Slide 59 text

Gradle Doctor Plugin plugins { id "com.osacky.doctor" version "0.7.3" }

Slide 60

Slide 60 text

Gradle Doctor Plugin plugins { id "com.osacky.doctor" version "0.7.3" }

Slide 61

Slide 61 text

https: // gradle.com/ Gradle

Slide 62

Slide 62 text

Gradle Doctor Plugin • doctor-negative-savings • doctor-slow-build-cache-connection

Slide 63

Slide 63 text

How do you disable caching for a task?

Slide 64

Slide 64 text

Disable Local Caching tasks.named("").configure { outputs.cacheIf { false } }

Slide 65

Slide 65 text

Disable Remote Caching tasks.named("").configure { outputs.cacheIf { !isCI } }

Slide 66

Slide 66 text

Gradle Doctor Plugin • Set threshold on Dagger processing

Slide 67

Slide 67 text

Disable Remote Caching doctor { /** * Print a warning to the console if we spend more than this * amount of time with Dagger annotation processors. */ daggerThreshold = 5000 }

Slide 68

Slide 68 text

Android Build Cache Plugin • Plugin by Gradle • Gradle plugin to fix Android build problems • Workarounds for Room

Slide 69

Slide 69 text

Summary ● Dependency Analysis Plugin ● Gradle Doctor Plugin ● Android Build Cache Fix Plugin

Slide 70

Slide 70 text

Modularization Graph Assertions, Version Catalogs

Slide 71

Slide 71 text

Modularization Module A Module B Module C

Slide 72

Slide 72 text

How do we enforce best practices?

Slide 73

Slide 73 text

Module Graph Assert Plugin • Check graph depth • Enforce conventions

Slide 74

Slide 74 text

Module Graph Assert Plugin id "com.jraska.module.graph.assertion"

Slide 75

Slide 75 text

Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 }

Slide 76

Slide 76 text

Module Graph Assert Plugin Max height the number of edges on the longest path from the node to a leaf.

Slide 77

Slide 77 text

Modularization Max height - 4

Slide 78

Slide 78 text

Module Graph Assert Plugin ./gradlew assertModuleGraph

Slide 79

Slide 79 text

Module Graph Assert Plugin > Task :app:assertMaxHeight FAILED 
 Module :app is allowed to have maximum height of 3, but has 4.

Slide 80

Slide 80 text

Modularization Max height - 3

Slide 81

Slide 81 text

Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 allowed = [‘:app’ -> ‘:feature-[a-z]’, ‘:feature.* -> :lib’] }

Slide 82

Slide 82 text

Module Graph Assert Plugin moduleGraphAssert { maxHeight = 3 allowed = [‘:app’ -> ‘:feature-[a-z]’, ‘:feature.* -> :lib’] restricted = [‘:feature-[a-z]*’ :forbidden-module,] }

Slide 83

Slide 83 text

How do we visualize all Gradle modules?

Slide 84

Slide 84 text

Module Graph Assert Plugin • Generate full graph of all modules • Visualize a subgraph

Slide 85

Slide 85 text

GraphViz • Tool to create graphs • Color nodes based on config

Slide 86

Slide 86 text

Module Graph Assert Plugin ./gradlew generateModulesGraphvizText

Slide 87

Slide 87 text

Module Graph Assert Plugin ./gradlew generateModulesGraphvizText 
 -Pmodules.graph.of.module=all_modules

Slide 88

Slide 88 text

Module Graph Assert Plugin Generate Graph GraphViz File Visualize with GraphViz

Slide 89

Slide 89 text

Module Graph Assert Plugin digraph G { ":leafModuleAvg" -> ":module785" ":leafModuleAvg" -> ":module574" ... }

Slide 90

Slide 90 text

Full Graph

Slide 91

Slide 91 text

Full Graph

Slide 92

Slide 92 text

Module Graph Assert Plugin ./gradlew generateModulesGraphvizText 
 -Pmodules.graph.of.module=:feature_a

Slide 93

Slide 93 text

How do we gets stats on all modules?

Slide 94

Slide 94 text

Module Graph Assert Plugin ./gradlew generateModulesGraphStatistics 


Slide 95

Slide 95 text

Module Graph Stats • Number of modules • Number of edges • Longest path

Slide 96

Slide 96 text

How do we setup dependencies?

Slide 97

Slide 97 text

Version Catalogs • Define all deps in TOML file • Gradle feature to share deps between modules

Slide 98

Slide 98 text

Version Catalogs [versions] androidx-core = “1.6.0-beta01”

Slide 99

Slide 99 text

Version Catalogs [versions] androidx-core = “1.6.0-beta01” [libraries] androidx-core-ktx = { module = "androidx.core:core-ktx", version.ref = “androidx-core" }

Slide 100

Slide 100 text

Version Catalogs dependencies { implementation(libs.androidx.core.ktx) }

Slide 101

Slide 101 text

Version Catalogs [bundles] compose = [ “androidx-composeUi", “androidx-composeMaterial”, “androidx-composeUiTooling" ]

Slide 102

Slide 102 text

Version Catalogs dependencies { implementation(libs.bundles.compose) }

Slide 103

Slide 103 text

Version Catalogs enableFeaturePreview("VERSION_CATALOGS")

Slide 104

Slide 104 text

Summary ● Module Assert Graph Plugin ● Version Catalogs

Slide 105

Slide 105 text

Benchmark Gradle Profiler

Slide 106

Slide 106 text

Gradle Profiler ● Automates benchmarking builds ● Detect regressions

Slide 107

Slide 107 text

Gradle Profiler

Slide 108

Slide 108 text

Gradle Profiler Change

Slide 109

Slide 109 text

How do we find regressions?

Slide 110

Slide 110 text

Gradle Profiler 1. Write a performance scenario 2. Specify number of iterations

Slide 111

Slide 111 text

Gradle Profiler brew install gradle-profiler

Slide 112

Slide 112 text

Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } assemble { }

Slide 113

Slide 113 text

Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } assemble { tasks = ["assemble"] }

Slide 114

Slide 114 text

Gradle Profiler gradle-profiler —benchmark

Slide 115

Slide 115 text

Gradle Profiler gradle-profiler —benchmark —iterations=

Slide 116

Slide 116 text

Gradle Profiler gradle-profiler —benchmark —iterations= Profiling - 1 Benchmarking - 10

Slide 117

Slide 117 text

Gradle Profiler gradle-profiler —benchmark —iterations=10

Slide 118

Slide 118 text

Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups= Profiling - 2 Benchmarking - 6

Slide 119

Slide 119 text

Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups=6

Slide 120

Slide 120 text

Gradle Profiler

Slide 121

Slide 121 text

Gradle Profiler Mean: 348 ms Min: 319 ms P25: 330 ms Median: 341 ms P75: 368 ms Std dev: 22.71 ms

Slide 122

Slide 122 text

How do we test incremental builds?

Slide 123

Slide 123 text

Gradle Profiler Change

Slide 124

Slide 124 text

Gradle Profiler Iteration 1 Iteration 2 Iteration 3

Slide 125

Slide 125 text

Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Abi Change

Slide 126

Slide 126 text

Gradle Profiler Iteration 1 Iteration 2 Iteration 3 Result

Slide 127

Slide 127 text

Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” }

Slide 128

Slide 128 text

Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” apply-android-resource-change-to = “strings.xml” tasks = ["assemble"] }

Slide 129

Slide 129 text

Gradle Profiler gradle-profiler —benchmark —iterations=10 —warmups=6

Slide 130

Slide 130 text

Gradle Profiler

Slide 131

Slide 131 text

How do we test Gradle sync?

Slide 132

Slide 132 text

Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } androidStudioSync { android-studio-sync { } }

Slide 133

Slide 133 text

Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } androidStudioSync { android-studio-sync { studio-jvm-args = ["-Xms256m", "-Xmx4096m"] } }

Slide 134

Slide 134 text

Gradle Profiler 1. Write a performance scenario 2. Specify number of iterations

Slide 135

Slide 135 text

How do we automate profiling on CI?

Slide 136

Slide 136 text

CI Pipeline 1. Create pipeline runs nightly 2. Collect metrics overtime

Slide 137

Slide 137 text

CI Pipeline Docker image with 
 Gradle Profiler

Slide 138

Slide 138 text

CI Pipeline Docker image with 
 Gradle Profiler Performance 
 Scenarios

Slide 139

Slide 139 text

CI Pipeline Docker image with 
 Gradle Profiler Performance 
 Scenarios Run 
 nightly

Slide 140

Slide 140 text

Use Cases 1. Anvil 2. KSP

Slide 141

Slide 141 text

Anvil • Kotlin Compiler Plugin • Generate factories • Improve build times

Slide 142

Slide 142 text

Anvil Dagger Module A Module B Module C

Slide 143

Slide 143 text

How do we use Anvil to generate factories?

Slide 144

Slide 144 text

Anvil anvil { generateDaggerFactories = true }

Slide 145

Slide 145 text

Benchmarking Build • Benchmark with Anvil change • Benchmark without Anvil change (Baseline) • Compare results

Slide 146

Slide 146 text

Performance Scenario assemble { # Show a slightly more human-readable title in reports title = "Assemble" # Run the 'assemble' task tasks = ["assemble"] } incremental_build { apply-abi-change-to = “src/main/java/StringUtils.kt” tasks = ["assemble"] }

Slide 147

Slide 147 text

Results Module A Module B Module C Baseline Anvil

Slide 148

Slide 148 text

Summary • How to setup & run Gradle Profiler • Performance scenarios • Anvil

Slide 149

Slide 149 text

Thank You! www.codingwithmohit.com @heyitsmohit