Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Dive into the Android Gradle Plugin (Droidcon UK 2018)

Deep Dive into the Android Gradle Plugin (Droidcon UK 2018)

Android developers use it every day, but who knows what actually goes on under the hood of the Android Gradle Plugin?

In this talk, you'll learn about AGP internals and answer questions like:

* What's D8 and R8?
* What is mergeResources and why does it take so long?
* Why are there so many tasks?
* How can I measure and speed up my builds?

You'll come away with a better understanding of the toolchain which will make you a better equipped Android developer!

Video: http://uk.droidcon.com/skillscasts/12718-deep-dive-into-the-android-gradle-plugin

6c8b509fe5422470d148c2c4cf2eb4b0?s=128

John Rodriguez

October 26, 2018
Tweet

Transcript

  1. Deep Dive into the Android Gradle Plugin John Rodriguez

  2. None
  3. None
  4. Build phases

  5. • Initialization • Configuration • Execution Build phases rootProject.name =

    'cash' include ':analytics' include ':app' include ':db' include ':keypad' include ':presenters' include ':protos' include ':screens' include ':viewmodels'
  6. • Initialization • Configuration • Execution Build phases Settings rootProject.name

    = 'cash' include ':analytics' include ':app' include ':db' include ':keypad' include ':presenters' include ':protos' include ':screens' include ':viewmodels' project project project project project project project project project
  7. • Initialization • Configuration • Execution Build phases project('cash') project(':analytics')

    project(':db') project(':app') project(':keypad') project(':presenters') project(':protos') project(':screens') project(':viewmodels')
  8. • Initialization • Configuration • Execution Build phases project('cash') project(':analytics')

    project(':db') project(':app') project(':keypad') project(':presenters') project(':protos') project(':screens') project(':viewmodels') > Configure project : > Configure project :analytics > Configure project :app > Configure project :db > Configure project :keypad > Configure project :presenters > Configure project :protos > Configure project :screens > Configure project :viewmodels
  9. • Initialization • Configuration • Execution Build phases project('cash') project(':analytics')

    project(':db') project(':app') project(':keypad') project(':presenters') project(':protos') project(':screens') project(':viewmodels') > Configure project : > Configure project :db > Configure project :protos ./gradlew :db:assembleDebug —configure-on-demand
  10. Configuration buildscript { repositories { google() jcenter() }d dependencies {

    classpath 'com.android.tools.build:gradle:3.2.1' }f }e
  11. Configuration buildscript { repositories { google() jcenter() }d dependencies {

    classpath 'com.android.tools.build:gradle:3.2.1' classpath 'org.jetbrains.kotlin:kotlin-gradle-plugin:1.2.71' classpath 'net.ltgt.gradle:gradle-errorprone-plugin:0.0.16' classpath 'com.squareup.sqldelight:android-gradle-plugin:1.0.0' }f }e
  12. Configuration apply plugin: 'com.android.application' apply plugin: 'com.android.library'

  13. class GreetingPlugin implements Plugin<Project> { @Override void apply(Project project) {

    }f }e
  14. class GreetingPlugin implements Plugin<Project> { @Override void apply(Project project) {

    project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e
  15. class GreetingPlugin implements Plugin<Project> { @Override void apply(Project project) {

    project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e project.apply plugin: GreetingPlugin
  16. class GreetingPlugin implements Plugin<Project> { @Override void apply(Project project) {

    project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e apply plugin: GreetingPlugin
  17. class GreetingPluginExtension { String message String greeter }a class GreetingPlugin

    implements Plugin<Project> { @Override void apply(Project project) { project.tasks.register('Greeting') { it.doLast { println 'Hello from the Greeting Plugin' }b }a }f }e apply plugin: GreetingPlugin
  18. class GreetingPluginExtension {1 String message String greeter }a class GreetingPlugin

    implements Plugin<Project> {2 @Override void apply(Project project) {3 def extension = project.extensions.create('greeting', GreetingPluginExtension) project.tasks.register('Greeting') {4 it.doLast {0 println 'Hello from the Greeting Plugin' }b }a }f }e apply plugin: GreetingPlugin
  19. class GreetingPluginExtension {1 String message String greeter }a class GreetingPlugin

    implements Plugin<Project> {2 @Override void apply(Project project) {3 def extension = project.extensions.create('greeting', GreetingPluginExtension) project.tasks.register('Greeting') {4 it.doLast {0 println "${extension.message} from ${extension.greeter}" }b }a }f }e apply plugin: GreetingPlugin
  20. apply plugin: GreetingPlugin greeting { message 'Hi' greeter 'Gradle' }

    $ ./gradlew Greeting > Task :Greeting Hi from Gradle
  21. interface Project extends ExtensionAware, PluginAware { } interface ExtensionAware {

    ExtensionContainer getExtensions(); } interface PluginAware { void apply(…); PluginManager getPluginManager(); }
  22. None
  23. None
  24. None
  25. class SomeTask extends DefaultTask { @OutputDirectory File getOutputDir() {…} @InputFiles

    List<File> getInputFiles() {…} @TaskAction void doStuff() {…}
  26. None
  27. None
  28. None
  29. • Initialization • Configuration • Execution Build phases

  30. • Initialization • Configuration • Execution Build phases ./gradlew :db:assembleDebug

  31. Notifications project.tasks.whenTaskAdded {} project.beforeEvaluate {} project.afterEvaluate {} gradle.projectsEvaluated {} gradle.taskGraph.whenReady

    {} gradle.taskGraph.beforeTask {} gradle.taskGraph.afterTask {}
  32. Debugging Builds

  33. $ ./gradlew task

  34. $ ./gradlew -Dorg.gradle.debug=true task

  35. $ ./gradlew --no-daemon --no-build-cache —rerun-tasks / -Dorg.gradle.debug=true task

  36. $ ./gradlew --no-daemon --no-build-cache —rerun-tasks / -Dorg.gradle.debug=true task To honour

    the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/4.10.2/ userguide/gradle_daemon.html. > Starting Daemon
  37. None
  38. dependencies { compileOnly ‘com.android.tools.build:gradle:3.2.1' }

  39. ?

  40. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') gradle.taskGraph.allTasks.each {

    task -> task.taskDependencies.getDependencies(task).each { dep -> // add task -> dep to graph }a }b 'dot -Tpng -O project.dot'.execute(…) }c
  41. None
  42. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') gradle.taskGraph.allTasks.each {

    task -> task.taskDependencies.getDependencies(task).each { dep -> // add task -> dep to graph }a }b 'dot -Tpng -O project.dot'.execute(…) }c
  43. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') gradle.taskGraph.allTasks.each {

    task -> task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  44. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') def startTasks

    = gradle.startParameter.getTaskNames().join(" ") println "command: ./gradlew " + startTasks gradle.taskGraph.allTasks.each { task -> println "task: " + task.name + ", class: " + task.class.name task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  45. command: ./gradlew app:assembleDebug compileJava -> JavaCompile processResources -> ProcessResources classes

    -> DefaultTask jar -> Jar checkDebugClasspath -> AppClasspathCheckTask preBuild -> DefaultTask preDebugBuild -> AppPreBuildTask compileDebugAidl -> AidlCompile compileDebugRenderscript -> RenderscriptCompile checkDebugManifest -> CheckManifest generateDebugBuildConfig -> GenerateBuildConfig prepareLintJar -> PrepareLintJar mainApkListPersistenceDebug -> MainApkListPersistence generateDebugResValues -> GenerateResValues generateDebugResources -> DefaultTask mergeDebugResources -> MergeResources …e
  46. command: ./gradlew app:assembleDebug compileJava -> JavaCompile processResources -> ProcessResources classes

    -> DefaultTask jar -> Jar checkDebugClasspath -> AppClasspathCheckTask preBuild -> DefaultTask preDebugBuild -> AppPreBuildTask compileDebugAidl -> AidlCompile compileDebugRenderscript -> RenderscriptCompile checkDebugManifest -> CheckManifest generateDebugBuildConfig -> GenerateBuildConfig prepareLintJar -> PrepareLintJar mainApkListPersistenceDebug -> MainApkListPersistence generateDebugResValues -> GenerateResValues generateDebugResources -> DefaultTask mergeDebugResources -> MergeResources …e
  47. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') def startTasks

    = gradle.startParameter.getTaskNames().join(" ") println "command: ./gradlew " + startTasks gradle.taskGraph.allTasks.each { task -> println "task: " + task.name + ", class: " + task.class.name task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  48. gradle.taskGraph.whenReady { def dot = new File(rootProject.buildDir, 'project.dot') def startTasks

    = gradle.startParameter.getTaskNames().join(" ") println "command: ./gradlew " + startTasks gradle.taskGraph.allTasks.each { task -> println "task: " + task.name + ", class: " + task.class.name println " inputs: " task.inputs.each { it.files.each { println " " + it } } println " outputs: " task.outputs.each { it.files.each { println " " + it } } task.taskDependencies.getDependencies(task).each {…}a }b 'dot -Tpng -O project.dot'.execute(…) }c
  49. gradle.taskGraph

  50. gradle.taskGraph.afterTask { task -> println " inputs: " task.inputs.files.each {

    println " " + it } println " outputs: " task.outputs.files.each { println " " + it } }
  51. > Task :app:checkDebugClasspath inputs: …/gradle-3.2.1.jar …/builder-3.2.1.jar …/tracker-26.2.1.jar …/shared-26.2.1.jar …/crash-26.2.1.jar …

    outputs: …/app/build/intermediates/checkDebugClasspath/debug
  52. None
  53. None
  54. None
  55. buildTypes { debug { applicationIdSuffix '.debug' minifyEnabled false proguardFile file('proguard-cash.pro')

    proguardFile file('proguard-cash-debug.pro') } release { minifyEnabled true shrinkResources true proguardFile file('proguard-cash.pro') } }
  56. productFlavors { flavorDimensions 'environment' internal { dimension 'environment' applicationId 'com.squareup.cash.beta'

    } production { dimension 'environment' applicationId 'com.squareup.cash' } } debug release
  57. production internal release debug

  58. production internal release debug internal Release

  59. production internal release debug internal Release Release

  60. production internal release debug internalDebug internalRelease productionDebug productionRelease release

  61. Configurations

  62. dependencies { implementation project(‘:api') api project(‘:db’) }a

  63. dependencies { implementation project(‘:api') api project(‘:db’) kapt project(‘:dagger’) testImplementation deps.mockito

    }a
  64. Images from https://jeroenmols.com/blog/2017/06/14/androidstudio3/

  65. Images from https://jeroenmols.com/blog/2017/06/14/androidstudio3/

  66. Images from https://jeroenmols.com/blog/2017/06/14/androidstudio3/

  67. :app dependencies { implementation project(':api') }a

  68. :app dependencies { implementation project(':api') }a :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 }
  69. :app dependencies { implementation project(':api') } :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 }
  70. :app dependencies { implementation project(':api') }a :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 }a
  71. :app dependencies { implementation project(':api') } :api dependencies { implementation

    project(':protos') implementation deps.retrofit implementation deps.rx2 } :protos dependencies { api deps.wire.runtime }
  72. :app dependencies { implementation project(':api') } :api dependencies { api

    project(':protos') implementation deps.retrofit implementation deps.rx2 } :protos dependencies { api deps.wire.runtime }
  73. :app dependencies { implementation project(':api') } :api dependencies { api

    project(':protos') implementation deps.retrofit implementation deps.rx2 } :protos dependencies { api deps.wire.runtime }
  74. :app dependencies { implementation project(':api') }a :api dependencies { api

    project(':protos') implementation deps.retrofit api deps.rx2 }b :protos dependencies { api deps.wire.runtime }c
  75. :app dependencies { implementation project(':api') } :api dependencies { api

    project(':protos') api deps.retrofit api deps.rx2 } :protos dependencies { api deps.wire.runtime }
  76. None
  77. None
  78. None
  79. None
  80. class AppClasspathCheckTask extends ClasspathComparisionTask { @TaskAction void run() { compareClasspaths();

    } }e
  81. class AppClasspathCheckTask extends ClasspathComparisionTask { @Override void onDifferentVersionsFound(…)a{ String message

    = String.format( "Conflict with dependency '%1$s:%2$s' in project '%3$s'. Resolved versions for runtime classpath (%4$s) and compile classpath (%5$s) differ. This can lead to runtime crashes. To resolve this issue …\n”, …); reporter.reportWarning(EvalIssueReporter.Type.GENERIC, message); } @TaskAction void run() { compareClasspaths(); } }e
  82. class AppClasspathCheckTask extends ClasspathComparisionTask { a{…} }e

  83. class AppClasspathCheckTask extends ClasspathComparisionTask { a{…} static class ConfigAction TaskConfigAction<AppClasspathCheckTask>

    { String getName() { return variantScope.getTaskName("check", "Classpath"); } @Override void execute(@NonNull AppClasspathCheckTask task) { task.setVariantName(variantScope.getFullVariantName()); task.runtimeClasspath = variantScope.getArtifactCollection(RUNTIME_CLASSPATH, …); task.compileClasspath = variantScope.getArtifactCollection(COMPILE_CLASSPATH, …); } } }e
  84. None
  85. None
  86. :moduleA dependencies { api libs.retrofit implementation libs.okhttp } :moduleB dependencies

    { implementation libs.retrofit } :app dependencies { implementation project(':module-a') implementation project(':module-b') }
  87. $ ./gradlew app:dependencies debugCompileClasspath: debug +--- project :module-a | \---

    com.squareup.retrofit2:retrofit:2.3.0 | \--- com.squareup.okhttp3:okhttp:3.8.0 | \--- com.squareup.okio:okio:1.13.0 \--- project :module-b … debugRuntimeClasspath: debug +--- project :module-a | +--- com.squareup.retrofit2:retrofit:2.3.0 | | \--- com.squareup.okhttp3:okhttp:3.8.0 -> 3.9.0 | | \--- com.squareup.okio:okio:1.13.0 | \--- com.squareup.okhttp3:okhttp:3.9.0 (*) \--- project :module-b \--- com.squareup.retrofit2:retrofit:2.3.0 (*)
  88. None
  89. None
  90. None
  91. None
  92. class JavaPreCompileTask extends AndroidBuilderTask { @OutputFile File getProcessorListFile() { return

    processorListFile; } @TaskAction void preCompile() { Set<String> classNames = Sets.newHashSet(); classNames.addAll( convertArtifactsToNames( collectAnnotationProcessors(annotationProcessorConfiguration) )); Gson gson = new GsonBuilder().create(); try (FileWriter writer = new FileWriter(processorListFile)) { gson.toJson(classNames, writer); } } /** * Returns a List of packages in the configuration believed to
  93. } } /** * Returns a List of packages in

    the configuration believed to * contain an annotation processor. * * We assume a package has an annotation processor if it contains the * META-INF/services/javax.annotation.processing.Processor file. */ List<…> collectAnnotationProcessors(ArtifactCollection config) {}
  94. None
  95. None
  96. > Task :app:mergeDebugResources inputs: ~/.gradle/caches/…/aapt2-3.2.1-4818971-osx.jar/… …/app/build/generated/res/resValues/debug …/app/build/generated/res/rs/debug …/app/src/main/res …/app/src/debug/res outputs:

    …/app/build/intermediates/blame/res/debug …/app/build/generated/res/pngs/debug …/app/build/intermediates/incremental/mergeDebugResources …/app/build/intermediates/res/merged/debug
  97. None
  98. None
  99. > Task :app:processDebugResources inputs: ~/.gradle/caches/…/aapt2-3.2.1-4818971-osx.jar/… …/app/build/intermediates/apk_list/…/apk-list.gson …/app/build/intermediates/res/merged/debug …/app/build/intermediates/split_list/…/split-list.gson …/app/build/intermediates/merged_manifests/…/merged outputs:

    …/app/build/intermediates/incremental/processDebugResources …/app/build/intermediates/processed_res/…/out …/app/build/generated/not_namespaced_r_class_sources/…/r …/app/build/intermediates/res/symbol-table…/…/package-aware-r.txt …/app/build/intermediates/symbols/debug/R.txt
  100. Resource processing • Before in aapt: take all resources and

    outputs one binary • Now in aapt2: compile + link • converts individual resources into binary flat files • merge at the end • uses Gradle incremental check feature for compile avoidance • Improving mergeReleaseResources for libraries • avoids full merge of resources for all the transitive dependencies
  101. ArtifactTransform API Request • aar -> android-manifest Internally • aar

    -> exploded-aar • exploded-aar -> android-manifest • Runs once, cached globally • Downloaded, transformed on demand
  102. android.libraryVariants.all { variant -> def runtimeConfiguration = variant.getRuntimeConfiguration() runtimeConfiguration.getIncoming() .artifactView(new

    Action<ArtifactView.ViewConfiguration>() { void execute(ArtifactView.ViewConfiguration config2) { config2.attributes(new Action<AttributeContainer>() { void execute(AttributeContainer c) { c.attribute(Attribute.of("artifactType", String), “android-assets“) } }) } }) .artifacts.artifacts.each { result -> // This file is an asset def file = result.file }
  103. android.libraryVariants.all { variant -> def runtimeConfiguration = variant.getRuntimeConfiguration() runtimeConfiguration.getIncoming() .artifactView(new

    Action<ArtifactView.ViewConfiguration>() { void execute(ArtifactView.ViewConfiguration config2) { config2.attributes(new Action<AttributeContainer>() { void execute(AttributeContainer c) { c.attribute(Attribute.of("artifactType", String), “android-assets“) } }) } }) .artifacts.artifacts.each { result -> // This file is an asset def file = result.file }
  104. ArtifactTransform API “give me all the assets” Before: view =

    configuration.getFiles() // check if AAR (could have JARS), then find assets… After: view = configuration.getIncoming().artifactView(config -> {...}) Downstream tasks can set up their inputs as a lazy FileCollection • allowing resolution on demand during task execution, as opposed to configuration. FileCollection will only contain the assets and tasks generating them • helpful for variant-awareness by not generating extra tasks
  105. • the process of transforming .class bytecode into .dex bytecode

    for Android
 • D8 - new dex compiler, replaces DX
 • DX takes a class,zip,jar,apk and converts to a single dex
 • When comparing with the current DX compiler, D8 compiles faster and outputs smaller .dex files, while having the same or better app runtime performance.
 • To test in 3.0, android.enableD8=true
 • Set to default in 3.1.
 Dexing
  106. DX $ $ANDROID_HOME/build-tools/LATEST/dx --dex —output=out input.jar D8 Debug mode build:

    $ java -jar build/libs/d8.jar --output out input.jar Release mode build: $ java -jar build/libs/d8.jar --release --output out input.jar Dexing
  107. • the process of transforming .class bytecode into .dex bytecode

    for Android
 • D8 - new dex compiler, replaces DX
 • DX takes a class,zip,jar,apk and converts to a single dex
 • When comparing with the current DX compiler, D8 compiles faster and outputs smaller .dex files, while having the same or better app runtime performance.
 • To test in 3.0, android.enableD8=true
 • Set to default in 3.1.
 Desugaring
  108. • Migration from PSI to UAST as of AGP 3.0

    • Jetbrains + Google • Kotlin UAST support as of AGP 3.1 • Quick fixes • Lots more detail here: • Lint to the Finish Line: https://www.youtube.com/watch?v=wFe0WZm_xm8 • Kotlin Static Analysis with Android Lint: https://www.youtube.com/watch?v=p8yX5-lPS6o Lint
  109. Improving your Builds

  110. enum class BooleanOption( override val propertyName: String, override val defaultValue:

    Boolean = false, override val status: Option.Status = Option.Status.EXPERIMENTAL, override val additionalInfo: String = "" ) : Option<Boolean> { ENABLE_AAPT2("android.enableAapt2", true, DeprecationReporter.DeprecationTarget.AAPT), ENABLE_BUILD_CACHE("android.enableBuildCache", true), ENABLE_PROFILE_JSON("android.enableProfileJson", false), // Used by Studio as workaround for b/71054106, b/75955471 ENABLE_SDK_DOWNLOAD("android.builder.sdkDownload", true, status = Option.Status.STABLE), ENABLE_TEST_SHARDING("android.androidTest.shardBetweenDevices"), ENABLE_DEX_ARCHIVE( "android.useDexArchive", true, aDeprecationReporter.DeprecationTarget.LEGACY_DEXER ),a
  111. VERSION_CHECK_OVERRIDE_PROPERTY("android.overrideVersionCheck"), OVERRIDE_PATH_CHECK_PROPERTY("android.overridePathCheck"), ENABLE_DESUGAR( "android.enableDesugar", true, DeprecationReporter.DeprecationTarget.DESUGAR_TOOL), ENABLE_INCREMENTAL_DESUGARING( “android.enableIncrementalDesugaring", true, DeprecationReporter.DeprecationTarget.INCREMENTAL_DESUGARING),

    ENABLE_GRADLE_WORKERS("android.enableGradleWorkers", false), ENABLE_AAPT2_WORKER_ACTIONS( “android.enableAapt2WorkerActions", true), ENABLE_CORE_LAMBDA_STUBS( "android.enableCoreLambdaStubs", true, DeprecationReporter.DeprecationTarget.CORE_LAMBDA_STUBS), ENABLE_D8("android.enableD8", true, DeprecationReporter.DeprecationTarget.LEGACY_DEXER),
  112. AndroidProject.PROPERTY_REFRESH_EXTERNAL_NATIVE_MODEL, status = Option.Status.STABLE), IDE_GENERATE_SOURCES_ONLY( AndroidProject.PROPERTY_GENERATE_SOURCES_ONLY, status = Option.Status.STABLE), ENABLE_SEPARATE_APK_RESOURCES("android.enableSeparateApkRes",

    true), ENABLE_EXPERIMENTAL_FEATURE_DATABINDING( "android.enableExperimentalFeatureDatabinding", false), ENABLE_SEPARATE_R_CLASS_COMPILATION( "android.enableSeparateRClassCompilation"), ENABLE_JETIFIER("android.enableJetifier", false, status = Option.Status.STABLE), USE_ANDROID_X("android.useAndroidX", false, status = Option.Status.STABLE), ENABLE_UNIT_TEST_BINARY_RESOURCES( "android.enableUnitTestBinaryResources", false), DISABLE_EARLY_MANIFEST_PARSING(
  113. ~/.gradle/gradle.properties org.gradle.caching=true

  114. ~/.gradle/gradle.properties org.gradle.caching=true …/build.gradle kapt { useBuildCache true }

  115. ~/.gradle/gradle.properties org.gradle.caching=true …/build.gradle kapt { useBuildCache !isCi }

  116. variantFilter { }a

  117. variantFilter { variant -> if (variant.flavors[0].name == 'production' && variant.buildType.name

    == 'debug') { variant.ignore = true } }a
  118. $ ./gradlew app:assembleDebug --profile

  119. None
  120. None
  121. None
  122. None
  123. None
  124. None
  125. None
  126. None
  127. None
  128. None
  129. None
  130. http://bit.ly/sdk-search-graph

  131. Deep Dive into the Android Gradle Plugin @jrodbx