Share
Embed
#AndroidBuilds at Twitter
https://www.youtube.com/watch?v=rG6nzrRPMRI
As codebases become larger and more complex, build performance is impacted. At Twitter, we’ve seen the existing tools struggle in a project with more than 100 modules and 1 million lines of code. This has forced us to explore a number of solutions to maintain good build speeds, and minimize resource usage.
We’ll share strategies that we’ve employed that were successful, and things that weren’t - to avoid hurting developer productivity.
These tips include:
- Strategies to effectively break down larger modules
- How to best utilize build tooling
- Profiling techniques to help find bottlenecks
Michael Evans
April 21, 2022
Transcript
231 slides
-
Don't think of them as Gradle build times. Think of
them as job security for your Twitter account 13 72 2:41 PM - 17 Jun 2017 Lisa Wray @lisawrayz -
54 Modules • Mostly inside large app module, and one
large library module • AGP 2.2.2 • Gradle 3.1 (In October 2016) Our build Seconds 0 75 150 225 300 Clean Incremental -
"I always see you just staring at your screen. You
Android devs must have it easy." I AM WAITING FOR GRADLE OKAY 66 144 3:03 AM - 28 Apr 2017 Zarah Dominguez 🦉 @zarahjutz -
Very excited about fi nally switching entirely to Buck soon
and never going back to Gradle. Subsecond builds are coming 😂 1 6 2:31 PM - 4 Nov 2016 Felipe Lima @felipecsl -
Buck • Super fast • OkBuck plugin makes it super
easy to get started • Exopackage is like Instant Run (but works) -
Seconds 0 75 150 225 300 Gradle 2.3.3 Buck Clean
Incremental -
Buck • Less fl exible • Not o ff i
cially supported • High maintenance cost • No Kotlin support -
Avoid Legacy Multidex Disable Multi-APK 1 2 Some Tips from
James Lau Google IO 2017 -
Avoid Legacy Multidex Disable Multi-APK Disable PNG Crunching 1 2
3 Some Tips from James Lau Google IO 2017 -
Avoid Legacy Multidex Disable Multi-APK Disable PNG Crunching 1 2
3 Avoid inadvertent changes 4 Some Tips from James Lau Google IO 2017 -
Avoid Legacy Multidex Disable Multi-APK Disable PNG Crunching 1 2
3 Avoid inadvertent changes Watch the memory 4 5 Some Tips from James Lau Google IO 2017 -
Avoid Legacy Multidex Disable Multi-APK Disable PNG Crunching 1 2
3 Avoid inadvertent changes Watch the memory Enable Gradle caching 4 5 6 Some Tips from James Lau Google IO 2017 -
More simple improvements Other Tips • Enable Parallel Builds •
--parallel • Con fi gure on demand • --configure-on-demand • org.gradle.configureondemand=true -
More simple improvements Other Tips • Enable Parallel Builds •
--parallel • Con fi gure on demand • --configure-on-demand • org.gradle.configureondemand=true • Partial builds (-a) • Ignore all dependent projects and only execute tasks in the target project -
Leveraging Gradle features 3.0 Daemon 3.1 Faster Dependency Resolution 3.4
Compile Avoidance Incremental Compilation 4.0 Build Cache -
3.0 Daemon 3.1 Faster Dependency Resolution 3.4 Compile Avoidance Incremental
Compilation 4.0 Build Cache Leveraging Gradle features -
3.0 Daemon 3.1 Faster Dependency Resolution 3.4 Compile Avoidance Incremental
Compilation 4.0 Build Cache Leveraging Gradle features -
Compiler avoidance In Java projects… • Replace java with java-library
• Use the api and implementation configurations • Declare annotation processors in the processor classpath -
Compiler avoidance In Java projects… • Replace java with java-library
• Use the api and implementation configurations • Declare annotation processors in the processor classpath -
Compiler avoidance In Java projects… • Replace java with java-library
• Use the api and implementation configurations • Declare annotation processors in the processor classpath -
Compiler avoidance In Java projects… • Replace java with java-library
• Use the api and implementation configurations • Declare annotation processors in the processor classpath -
Compiler avoidance In Java projects… configurations { // A configuration
that collects annotation processing dependencies // to add to the annotation processsor path. annotationProcessor } compileJava { options.annotationProcessorPath = configurations.annotationProcessor } dependencies { annotationProcessor 'com.google.dagger:dagger-compiler:2.11' } -
Compiler avoidance In Java projects… configurations { // A configuration
that collects annotation processing dependencies // to add to the annotation processsor path. annotationProcessor } compileJava { options.annotationProcessorPath = configurations.annotationProcessor } dependencies { annotationProcessor 'com.google.dagger:dagger-compiler:2.11' } -
Compiler avoidance In Java projects… configurations { // A configuration
that collects annotation processing dependencies // to add to the annotation processsor path. annotationProcessor } compileJava { options.annotationProcessorPath = configurations.annotationProcessor } dependencies { annotationProcessor 'com.google.dagger:dagger-compiler:2.11' } -
Compiler avoidance In Java projects… configurations { // A configuration
that collects annotation processing dependencies // to add to the annotation processsor path. annotationProcessor } compileJava { options.annotationProcessorPath = configurations.annotationProcessor } dependencies { annotationProcessor 'com.google.dagger:dagger-compiler:2.11' } -
3.0 Daemon 3.1 Faster Dependency Resolution 3.4 Compile Avoidance Incremental
Compilation 4.0 Build Cache Leveraging Gradle features -
3.0 Daemon 3.1 Faster Dependency Resolution 3.4 Compile Avoidance Incremental
Compilation 4.0 Build Cache Leveraging Gradle features -
Enabling the build cache Enable cache in gradle.properties org.gradle.caching=true Con
fi gure cache in settings.gradle buildCache { local(DirectoryBuildCache) { directory = System.env.GRADLE_USER_HOME != null ? new File(System.env.GRADLE_USER_HOME, '.buildcache') : new File(rootDir, '.buildcache') enabled = true } remote(HttpBuildCache) { url = 'https://ge.twitter.biz/cache/' push = true enabled = true } } -
Enable cache in gradle.properties org.gradle.caching=true Con fi gure cache in
settings.gradle buildCache { local(DirectoryBuildCache) { directory = System.env.GRADLE_USER_HOME != null ? new File(System.env.GRADLE_USER_HOME, '.buildcache') : new File(rootDir, '.buildcache') enabled = true } remote(HttpBuildCache) { url = 'https://ge.twitter.biz/cache/' push = true enabled = true } } Enabling the build cache -
Enabling the build cache Enable cache in gradle.properties org.gradle.caching=true Con
fi gure cache in settings.gradle buildCache { local(DirectoryBuildCache) { directory = System.env.GRADLE_USER_HOME != null ? new File(System.env.GRADLE_USER_HOME, '.buildcache') : new File(rootDir, '.buildcache') enabled = true } remote(HttpBuildCache) { url = 'https://ge.twitter.biz/cache/' push = true enabled = true } } -
Give me six hours to chop down a tree and
I will spend the fi rst four sharpening the axe 16 68 2:48 PM – 15 Jul 1860 Abraham Lincoln @POTUS16 -
--profile is your friend • Generates a report in build/reports/profile
• Lists timing for con fi guration and task execution • Indicates if tasks were skipped/did no work -
The Gradle profiler ./gradle-profiler \ --profile chrome-trace \ --project-dir ~/src/twitter-android/
\ --scenario-file tfa.scenarios \ nonAbiChange -
The Gradle profiler ./gradle-profiler \ --profile chrome-trace \ --project-dir ~/src/twitter-android/
\ --scenario-file tfa.scenarios \ nonAbiChange -
The Gradle profiler ./gradle-profiler \ --profile chrome-trace \ --project-dir ~/src/twitter-android/
\ --scenario-file tfa.scenarios \ nonAbiChange -
The Gradle profiler nonAbiChange { tasks = [":app:twitter:assembleDefaultDebug"] apply-non-abi-change-to =
"lib/twitter/Example.java" gradle-args = ["-PminSdkVersion=23"] warm-ups = 3 } tfa.scenarios -
The Gradle profiler nonAbiChange { tasks = [":app:twitter:assembleDefaultDebug"] apply-non-abi-change-to =
"lib/twitter/Example.java" gradle-args = ["-PminSdkVersion=23"] warm-ups = 3 } tfa.scenarios -
The Gradle profiler nonAbiChange { tasks = [":app:twitter:assembleDefaultDebug"] apply-non-abi-change-to =
"lib/twitter/Example.java" gradle-args = ["-PminSdkVersion=23"] warm-ups = 3 } tfa.scenarios -
The Gradle profiler nonAbiChange { tasks = [":app:twitter:assembleDefaultDebug"] apply-non-abi-change-to =
"lib/twitter/Example.java" gradle-args = ["-PminSdkVersion=23"] warm-ups = 3 } tfa.scenarios -
The Gradle profiler nonAbiChange { tasks = [":app:twitter:assembleDefaultDebug"] apply-non-abi-change-to =
"lib/twitter/Example.java" gradle-args = ["-PminSdkVersion=23"] warm-ups = 3 } tfa.scenarios -
YourKit changed my life. 4:12 PM - 18 Apr 2017
Gwen (Chen) Shapira @gwenshap -
YourKit https://www.yourkit.com/docs/java/help/agent.jsp https://www.yourkit.com/docs/java/help/startup_options.jsp org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 -Djava.awt.headless=true \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false
org.gradle.parallel=true org.gradle.configureondemand=true gradle.properties -
YourKit https://www.yourkit.com/docs/java/help/agent.jsp https://www.yourkit.com/docs/java/help/startup_options.jsp org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 -Djava.awt.headless=true \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false
org.gradle.parallel=true org.gradle.configureondemand=true gradle.properties -
YourKit https://www.yourkit.com/docs/java/help/agent.jsp https://www.yourkit.com/docs/java/help/startup_options.jsp org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 -Djava.awt.headless=true \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false
org.gradle.parallel=true org.gradle.configureondemand=true gradle.properties -
YourKit Or use the Gradle profiler! ./gradle-profiler \ --profile yourkit
--yourkit-sampling \ --project-dir ~/src/twitter-android/ \ --scenario-file tfa.scenarios \ nonAbiChange -
YourKit Or use the Gradle profiler! ./gradle-profiler \ --profile yourkit
--yourkit-sampling \ --project-dir ~/src/twitter-android/ \ --scenario-file tfa.scenarios \ nonAbiChange -
Use the logs! Why is your compilation not doing the
right thing? ➡ check out the command-line parameters and the classpath ./gradlew assemble --info -
Use the logs! Why is your compilation not doing the
right thing? ➡ check out the command-line parameters and the classpath Compiler arguments: -source 1.7 -target 1.7 -d /Users/ mevans/src/twitter-android/lib/core/database/lru/build/ intermediates/classes/release -encoding UTF-8 -bootclasspath /opt/twitter/Caskroom/android-sdk/25.2.3/ platforms/android-25/android.jar -g -sourcepath -processorpath /Users/mevans/.gradle/caches/modules-2/ files-2.1/javax.inject/javax.inject/ 1/6975da39a7040257bd51d21a231b76c915872d38/ javax.inject-1.jar… -classpath /Users/mevans/src/twitter-android/lib/core/util/ android/common/build/intermediates/bundles/default/ classes.jar… -
Use the logs! Why is your compilation not doing the
right thing? ➡ check out the command-line parameters and the classpath Compiler arguments: -source 1.7 -target 1.7 -d /Users/ mevans/src/twitter-android/lib/core/database/lru/build/ intermediates/classes/release -encoding UTF-8 -bootclasspath /opt/twitter/Caskroom/android-sdk/25.2.3/ platforms/android-25/android.jar -g -sourcepath -processorpath /Users/mevans/.gradle/caches/modules-2/ files-2.1/javax.inject/javax.inject/ 1/6975da39a7040257bd51d21a231b76c915872d38/ javax.inject-1.jar… -classpath /Users/mevans/src/twitter-android/lib/core/util/ android/common/build/intermediates/bundles/default/ classes.jar… -
Android Gradle Plugin optimizations 2.2 Pre-dexing 2.3 Gradle Build Cache
3.0 Compile Avoidance Variant-aware dependency Management … -
Android Gradle Plugin optimizations 2.2 Pre-dexing 2.3 Gradle Build Cache
3.0 Compile Avoidance Variant-aware dependency Management … -
AGP 3.0 Trying the plugin early • Enable with a
flag • Define flavor dimensions • Disable aapt2 and new resource processing • Define configurations for 2.3 -
AGP 3.0 Trying the plugin early • Enable with a
flag • Define flavor dimensions • Disable aapt2 and new resource processing • Define configurations for 2.3 -
AGP 3.0 Trying the plugin early ext.isNewBuild = System.properties['build.new'] !=
null buildscript { repositories { google() } dependencies { classpath isNewBuild ? 'com.android.tools.build:gradle:3.0.0-alpha4' : 'com.android.tools.build:gradle:2.3.0' } } -
AGP 3.0 Trying the plugin early ext.isNewBuild = System.properties['build.new'] !=
null buildscript { repositories { google() } dependencies { classpath isNewBuild ? 'com.android.tools.build:gradle:3.0.0-alpha4' : 'com.android.tools.build:gradle:2.3.0' } } -
AGP 3.0 Trying the plugin early ext.isNewBuild = System.properties['build.new'] !=
null buildscript { repositories { google() } dependencies { classpath isNewBuild ? 'com.android.tools.build:gradle:3.0.0-alpha4' : 'com.android.tools.build:gradle:2.3.0' } } -
AGP 3.0 Trying the plugin early • Enable with a
flag • Define flavor dimensions • Disable aapt2 and new resource processing • Define configurations for 2.3 -
AGP 3.0 Trying the plugin early android { if (isNewBuild)
{ flavorDimensions 'type' } productFlavors { 'default' { if (isNewBuild) { dimension 'type' } } } } -
AGP 3.0 Trying the plugin early • Enable with a
flag • Define flavor dimensions • Disable aapt2 and new resource processing • Define configurations for 2.3 -
AGP 3.0 Trying the plugin early Disable aapt2 and resource
processing in gradle.properties android.enableNewResourceProcessing=false android.enableAapt2=false -
AGP 3.0 Trying the plugin early • Enable with a
flag • Define flavor dimensions • Disable aapt2 and new resource processing • Define configurations for 2.3 -
AGP 3.0 if (!isNewBuild) { configurations { // Aliases for
the new configurations in 3.0.0. api implementation compile.extendsFrom(api, implementation) debugImplementation debugCompile.extendsFrom(debugImplementation) dogfoodImplementation dogfoodCompile.extendsFrom(dogfoodImplementation) testImplementation testCompile.extendsFrom(testImplementation) androidTestImplementation androidTestCompile.extendsFrom(androidTestImplementation) compileOnly provided.extendsFrom(compileOnly) } } Trying the plugin early -
AGP 3.0 if (!isNewBuild) { configurations { // Aliases for
the new configurations in 3.0.0. api implementation compile.extendsFrom(api, implementation) debugImplementation debugCompile.extendsFrom(debugImplementation) dogfoodImplementation dogfoodCompile.extendsFrom(dogfoodImplementation) testImplementation testCompile.extendsFrom(testImplementation) androidTestImplementation androidTestCompile.extendsFrom(androidTestImplementation) compileOnly provided.extendsFrom(compileOnly) } } Trying the plugin early -
AGP 3.0 if (!isNewBuild) { configurations { // Aliases for
the new configurations in 3.0.0. api implementation compile.extendsFrom(api, implementation) debugImplementation debugCompile.extendsFrom(debugImplementation) dogfoodImplementation dogfoodCompile.extendsFrom(dogfoodImplementation) testImplementation testCompile.extendsFrom(testImplementation) androidTestImplementation androidTestCompile.extendsFrom(androidTestImplementation) compileOnly provided.extendsFrom(compileOnly) } } Trying the plugin early -
Up-to-date check issues SkipUpToDateTaskExecuter in the logs Determining if task
':lib:core:util:android:common:extractReleaseAnnotations' is up-to-date Executing task ':lib:core:util:android:common:extractReleaseAnnotations' (up-to-date check took 0.012 secs) due to: Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v14/preference/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/appcompat/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/preference/R$id.class has changed. Determining if task ':lib:core:util:android:common:mergeReleaseAssets' is up-to-date Skipping task ':lib:core:util:android:common:mergeReleaseAssets' as it is up-to-date (took 0.003 secs). Determining if task ':lib:core:util:android:common:packageReleaseResources' is up-to-date Executing task ':lib:core:util:android:common:packageReleaseResources' (up-to-date check took 0.035 secs) due to: Value of input property 'minSdk' has changed for task ':lib:core:util:android:common:packageReleaseResources' -
Up-to-date check issues SkipUpToDateTaskExecuter in the logs Determining if task
':lib:core:util:android:common:extractReleaseAnnotations' is up-to-date Executing task ':lib:core:util:android:common:extractReleaseAnnotations' (up-to-date check took 0.012 secs) due to: Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v14/preference/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/appcompat/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/preference/R$id.class has changed. Determining if task ':lib:core:util:android:common:mergeReleaseAssets' is up-to-date Skipping task ':lib:core:util:android:common:mergeReleaseAssets' as it is up-to-date (took 0.003 secs). Determining if task ':lib:core:util:android:common:packageReleaseResources' is up-to-date Executing task ':lib:core:util:android:common:packageReleaseResources' (up-to-date check took 0.035 secs) due to: Value of input property 'minSdk' has changed for task ':lib:core:util:android:common:packageReleaseResources' -
Up-to-date check issues SkipUpToDateTaskExecuter in the logs Determining if task
':lib:core:util:android:common:extractReleaseAnnotations' is up-to-date Executing task ':lib:core:util:android:common:extractReleaseAnnotations' (up-to-date check took 0.012 secs) due to: Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v14/preference/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/appcompat/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/preference/R$id.class has changed. Determining if task ':lib:core:util:android:common:mergeReleaseAssets' is up-to-date Skipping task ':lib:core:util:android:common:mergeReleaseAssets' as it is up-to-date (took 0.003 secs). Determining if task ':lib:core:util:android:common:packageReleaseResources' is up-to-date Executing task ':lib:core:util:android:common:packageReleaseResources' (up-to-date check took 0.035 secs) due to: Value of input property 'minSdk' has changed for task ':lib:core:util:android:common:packageReleaseResources' -
Up-to-date check issues SkipUpToDateTaskExecuter in the logs Determining if task
':lib:core:util:android:common:extractReleaseAnnotations' is up-to-date Executing task ':lib:core:util:android:common:extractReleaseAnnotations' (up-to-date check took 0.012 secs) due to: Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v14/preference/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/appcompat/R$id.class has changed. Input property 'classDir' file /Users/cpuerta/workspace/twitter-android/lib/core/util/android/common/ build/intermediates/classes/release/android/support/v7/preference/R$id.class has changed. Determining if task ':lib:core:util:android:common:mergeReleaseAssets' is up-to-date Skipping task ':lib:core:util:android:common:mergeReleaseAssets' as it is up-to-date (took 0.003 secs). Determining if task ':lib:core:util:android:common:packageReleaseResources' is up-to-date Executing task ':lib:core:util:android:common:packageReleaseResources' (up-to-date check took 0.035 secs) due to: Value of input property 'minSdk' has changed for task ':lib:core:util:android:common:packageReleaseResources' -
Up-to-date check issues • Android Studio invalidating caches • Annotation
processors that don’t generate consistent code • Missing dependencies among tasks • MinSDK Some examples -
Up-to-date check issues • Android Studio invalidating caches • Annotation
processors that don’t generate consistent code • Missing dependencies among tasks • MinSDK Some examples -
Up-to-date check issues buildscript { // Remove Android Studio's local
Maven repository, since it pollutes // the classpath and causes rebuilds. allprojects { buildscript { repositories.clear() } } } Some examples -
Up-to-date check issues • Android Studio invalidating caches • Annotation
processors that don’t generate consistent code • Missing dependencies among tasks • MinSDK Some examples -
Up-to-date check issues • Android Studio invalidating caches • Annotation
processors that don’t generate consistent code • Missing dependencies among tasks • MinSDK Some examples -
Up-to-date check issues afterEvaluate { // Work around missing task
dependency getProjectVariants(project).each { variant -> def safeguardTask = tasks.getByName( "incremental${variant.name.capitalize()}JavaCompilationSafeguard") variant.variantData.outputs.each { output -> safeguardTask.dependsOn(output.scope.processResourcesTask.name) } } } Some examples -
Up-to-date check issues • Android Studio invalidating caches • Annotation
processors that don’t generate consistent code • Missing dependencies among tasks • MinSDK Some examples -
Debugging AGP Downloading the sources… wget https://maven.google.com/com/android/tools/build/ gradle-core/3.0.0-alpha4/gradle-core-3.0.0-alpha4-sources.jar wget https://maven.google.com/com/android/tools/build/
builder/3.0.0-alpha4/builder-3.0.0-alpha4-sources.jar -
Debugging AGP Downloading the sources… wget https://maven.google.com/com/android/tools/build/ gradle-core/3.0.0-alpha4/gradle-core-3.0.0-alpha4-sources.jar wget https://maven.google.com/com/android/tools/build/
builder/3.0.0-alpha4/builder-3.0.0-alpha4-sources.jar -
Addressing memory issues Adjust daemon heap size in gradle.properties org.gradle.jvmargs=-Xms512m
-Xmx4g -Dfile.encoding=utf-8 Fork compilation to a separate process tasks.withType(JavaCompile) { options.fork = true } Leaks? org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false -
Addressing memory issues Adjust daemon heap size in gradle.properties org.gradle.jvmargs=-Xms512m
-Xmx4g -Dfile.encoding=utf-8 Fork compilation to a separate process tasks.withType(JavaCompile) { options.fork = true } Leaks? org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false -
Addressing memory issues Adjust daemon heap size in gradle.properties org.gradle.jvmargs=-Xms512m
-Xmx4g -Dfile.encoding=utf-8 Fork compilation to a separate process tasks.withType(JavaCompile) { options.fork = true } Leaks? org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false -
Addressing memory issues Adjust daemon heap size in gradle.properties org.gradle.jvmargs=-Xms512m
-Xmx4g -Dfile.encoding=utf-8 Fork compilation to a separate process tasks.withType(JavaCompile) { options.fork = true } Leaks? org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false -
Addressing memory issues Adjust daemon heap size in gradle.properties org.gradle.jvmargs=-Xms512m
-Xmx4g -Dfile.encoding=utf-8 Fork compilation to a separate process tasks.withType(JavaCompile) { options.fork = true } Leaks? org.gradle.jvmargs=-Xms512m -Xmx4g -Dfile.encoding=utf-8 \ -agentpath:/Applications/YourKit.app/Contents/Resources/bin/mac/libyjpagent.jnilib=sampling,onexit=memory org.gradle.daemon=false -
And “ fi xing” the leak Addressing memory issues tasks.withType(MergeResources).each
{ task -> task.doLast { Utils.setPrivateFieldValue(task, MergeResources, 'processedInputs', null) task.sourceFolderInputs = { [] } } } And report an issue! -
Why? • Improve compiler avoidance • Optimize caching of build
artifacts • More e ffi cient parallelization • Better code architecture • Finer job granularity in CI -
Why? • Improve compiler avoidance • Optimize caching of build
artifacts • More e ffi cient parallelization • Better code architecture • Finer job granularity in CI -
Why? • Improve compiler avoidance • Optimize caching of build
artifacts • More e ffi cient parallelization • Better code architecture • Finer job granularity in CI -
Why? • Improve compiler avoidance • Optimize caching of build
artifacts • More e ffi cient parallelization • Better code architecture • Finer job granularity in CI -
Why? • Improve compiler avoidance • Optimize caching of build
artifacts • More e ffi cient parallelization • Better code architecture • Finer job granularity in CI -
Why? • Improve compiler avoidance • Optimize caching of build
artifacts • More e ffi cient parallelization • Better code architecture • Finer job granularity in CI -
Configuring Gradle • Shared gradle con fi guration fi les
• Making sense of build errors • Select projects to run your tasks on -
Configuring Gradle • Shared gradle con fi guration fi les
• Making sense of build errors • Select projects to run your tasks on -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle apply plugin: 'checkstyle' apply from: "${project.rootDir}/config/gradle-common/common-properties.gradle" checkstyle {
configProperties = [ configDir:"${rootDir}/config/checkstyle" ] configFile = file("${rootDir}/config/checkstyle/checkstyle.xml") toolVersion = checkstyleVersion showViolations = false } dependencies { checkstyle project(':tools:checkstyle-rules') } task checkstyle(type: Checkstyle) { source 'src' include '**/*.java' exclude '*/resources/**' classpath = files() group = 'Verification' description = 'Run all checkstyle tasks.' } checkstyle.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle apply plugin: 'java-library' apply from: "${project.rootDir}/config/gradle-common/common-properties.gradle" apply from:
"${project.rootDir}/config/gradle-common/checkstyle.gradle" apply from: "${project.rootDir}/config/gradle-common/robolectric-test-config.gradle" sourceCompatibility = JavaVersion.VERSION_1_7 targetCompatibility = JavaVersion.VERSION_1_7 dependencies { api project.jetbrainsAnnotations } configurations { // A configuration that collects annotation processing dependencies to add to the // annotation processsor path. annotationProcessor } compileJava { options.annotationProcessorPath = configurations.annotationProcessor } java-library-config.gradle -
Configuring Gradle apply plugin: 'java-library' apply from: "${project.rootDir}/config/gradle-common/common-properties.gradle" apply from:
"${project.rootDir}/config/gradle-common/checkstyle.gradle" apply from: "${project.rootDir}/config/gradle-common/robolectric-test-config.gradle" sourceCompatibility = JavaVersion.VERSION_1_7 targetCompatibility = JavaVersion.VERSION_1_7 dependencies { api project.jetbrainsAnnotations } configurations { // A configuration that collects annotation processing dependencies to add to the // annotation processsor path. annotationProcessor } compileJava { options.annotationProcessorPath = configurations.annotationProcessor } java-library-config.gradle -
Configuring Gradle apply plugin: 'java-library' apply from: "${project.rootDir}/config/gradle-common/common-properties.gradle" apply from:
"${project.rootDir}/config/gradle-common/checkstyle.gradle" apply from: "${project.rootDir}/config/gradle-common/robolectric-test-config.gradle" sourceCompatibility = JavaVersion.VERSION_1_7 targetCompatibility = JavaVersion.VERSION_1_7 dependencies { api project.jetbrainsAnnotations } configurations { // A configuration that collects annotation processing dependencies to add to the // annotation processsor path. annotationProcessor } compileJava { options.annotationProcessorPath = configurations.annotationProcessor } java-library-config.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle apply plugin: 'com.android.library' apply from: "${project.rootDir}/config/gradle-common/common-properties.gradle" apply from:
"${project.rootDir}/config/gradle-common/android-common-config.gradle" apply from: "${project.rootDir}/config/gradle-common/checkstyle.gradle" apply from: "${project.rootDir}/config/gradle-common/lint.gradle" android-library-config.gradle -
Configuring Gradle • common-properties.gradle • checkstyle.gradle • lint.gradle • java-library-config.gradle
• java-annotation-processor-config.gradle • android-common-config.gradle • android-library-config.gradle • android-application-config.gradle -
Configuring Gradle apply from: "${project.rootDir}/config/gradle-common/android-library-config.gradle" dependencies { api 'com.twitter.android:media:1.22.1' api
"com.android.support:exifinterface:${supportLibraryVersion}" implementation project(':lib:core:async:common') implementation project(':lib:core:util:android:io') } /lib/core/media/common/build.gradle -
Configuring Gradle apply from: "${project.rootDir}/config/gradle-common/android-library-config.gradle" dependencies { api 'com.twitter.android:media:1.22.1' api
"com.android.support:exifinterface:${supportLibraryVersion}" implementation project(':lib:core:async:common') implementation project(':lib:core:util:android:io') } /lib/core/media/common/build.gradle -
Configuring Gradle apply from: "${project.rootDir}/config/gradle-common/android-library-config.gradle" dependencies { api 'com.twitter.android:media:1.22.1' api
"com.android.support:exifinterface:${supportLibraryVersion}" implementation project(':lib:core:async:common') implementation project(':lib:core:util:android:io') } /lib/core/media/common/build.gradle -
Configuring Gradle • Shared gradle con fi guration fi les
• Making sense of build errors • Select projects to run your tasks on -
Configuring gradle • Shared gradle con fi guration fi les
• Making sense of build errors • Select projects to run your tasks on -
Some advice • Use api dependencies sparingly • Isolate annotation
processing • Remove bottlenecks in your graph • Use Java modules when possible • Use the new refactors in Android Studio to move resources -
• Use api dependencies sparingly • Isolate annotation processing •
Remove bottlenecks in your graph • Use Java modules when possible • Use the new refactors in Android Studio to move resources Some advice -
• Use api dependencies sparingly • Isolate annotation processing •
Remove bottlenecks in your graph • Use Java modules when possible • Use the new refactors in Android Studio to move resources Some advice -
• Use api dependencies sparingly • Isolate annotation processing •
Remove bottlenecks in your graph • Use Java modules when possible • Use the new refactors in Android Studio to move resources Some advice -
• Use api dependencies sparingly • Isolate annotation processing •
Remove bottlenecks in your graph • Use Java modules when possible • Use the new refactors in Android Studio to move resources Some advice -
• Use api dependencies sparingly • Isolate annotation processing •
Remove bottlenecks in your graph • Use Java modules when possible • Use the new refactors in Android Studio to move resources Some advice -
54 Modules • Mostly inside large app module, and one
large library module • AGP 2.2.2 • Gradle 3.1 (In October 2016) Our build -
93 Modules • Still have 2 larger modules, but considerably
smaller than months ago • AGP 2.3.3 - using 3.0.0 also • Gradle 4.0 (In June 2017) Our build -
Seconds 0 75 150 225 300 Gradle 2.3.3 Buck Gradle
3.0.0 Clean Incremental -
Seconds 0 75 150 225 300 Gradle 2.3.3 Buck Gradle
3.0.0 Clean Incremental