Improving Android Build Performance

We all know a fast build makes for a happy developer. Unfortunately, as your project grows larger and more complex, its build will become slower if left unchecked. You’ve updated to the latest Android Plugin, but you’re still waiting more than you like, what can you do? Recent Gradle releases give you new capabilities to greatly improve your build speed. This session discusses how you can :

- Identify performance bottlenecks with build scans
- Fix common problems, like slow configuration time, tasks running when they shouldn’t, and builds that don’t parallelize well
- Benefit from incremental annotation processing

Transcript

1. Improving Android Build
   Performance
   Cédric Champeau, Gradle Inc.
   

   View Slide

2. About Gradle
   

   View Slide

3. | 3
   What is Gradle?
   - Gradle != Gradle plugin for Android != Android Studio
   - Gradle is a build and automation tool
   - JVM based
   - 100% Free Open Source - Apache Standard License 2.0
   - Multi-ecosystem
   - JVM: Java, Kotlin, Groovy, Scala, …
   - Native: C/C++, Go…
   - Android
   - …
   

   View Slide

4. | 4
   Gradle, Inc.
   The company behind Gradle.
   • Build Happiness
   • Employs full time engineers
   • Providing Gradle Build Scans and Gradle Enterprise
   • (Gradle consulting, support, development services etc.)
   • (Training: online, public and in-house)
   

   View Slide

5. | 5
   We’re hiring
   • Fully distributed development team
   • Exciting project used by millions
   • Build tool team and Gradle enterprise positions
   If anything you hear from now on sounds like a great problem to solve,
   Talk to us!
   https:/
   /gradle.com/careers/
   

   View Slide

6. The Cost of Builds
   

   View Slide

7. | 7
   60s waste * 50 builds per day * 50 developers
   = 42h wasted per day
   … and that’s not even considering lost focus
   Fast Builds Matter
   

   View Slide

8. General Advice
   

   View Slide

9. | 9
   Process
   1. Define scenario to improve
   2. Profile scenario
   3. Identify biggest bottleneck
   4. Fix bottleneck
   5. Verify fix by measurement
   6. Repeat
   

   View Slide

10. |10
    Automate your measurements
    github.com/gradle/gradle-profiler
    configurationTime {
    tasks = ["help"]
    }
    sync {
    tasks = ["generateDebugSources"]
    }
    cleanBuild {
    tasks = ["build"]
    cleanup-tasks = ["clean"]
    }
    cachedCleanBuild {
    tasks = ["build"]
    cleanup-tasks = ["clean"]
    gradle-args = ["--build-cache"]
    }
    configurationTime {
    tasks = ["help"]
    }
    sync {
    tasks = ["generateDebugSources"]
    }
    cleanBuild {
    tasks = ["build"]
    cleanup-tasks = ["clean"]
    }
    cachedCleanBuild {
    tasks = ["build"]
    cleanup-tasks = ["clean"]
    gradle-args = ["--build-cache"]
    }
    

    View Slide

11. |11
    Stay Up-To-Date
    ./gradlew wrapper --gradle-version 4.10.2
    buildscript {
    repositories {
    google()
    }
    dependencies {
    classpath ‘com.android.tools.build:gradle:3.1.2'
    }
    }
    buildscript {
    repositories {
    google()
    }
    dependencies {
    classpath ‘com.android.tools.build:gradle:3.1.2'
    }
    }
    

    View Slide

12. |12
    JVM tuning
    Provide enough heap space
    Other tweaks often do more harm than good
    Spend your time on structural improvements
    

    View Slide

13. |13
    The Gradle Daemon
    The Gradle Daemon is enabled by default
    Provides a number of in-memory caches
    Useful for repeated builds
    Do not disable it!
    

    View Slide

14. Where is the Problem?
    

    View Slide

15. |15
    Build Scans
    

    View Slide

16. |16
    The Build Lifecycle
    

    View Slide

17. |17
    Red Flags
    Startup/Settings/buildSrc > 1s
    Configuration time >10ms/project
    Single-line change ≅ clean build
    No-op build doing any work at all
    High GC time
    

    View Slide

18. Startup, buildSrc, Settings
    

    View Slide

19. |19
    Startup, buildSrc, Settings
    Use the Daemon, keep it healthy
    

    View Slide

20. Configuration Time
    

    View Slide

21. |21
    Configuration Time
    Applying plugins
    Evaluating build scripts
    Running afterEvaluate {} blocks
    

    View Slide

22. |22
    Configuration Time
    When running any task
    Even gradle help/gradle tasks
    Android Studio sync
    

    View Slide

23. |23
    Resolution at Configuration
    Time
    

    View Slide

24. |24
    Eager resolution
    task uberJar(type:Jar) {
    from sourceSets.main.output
    from configurations.runtime.collect { it.directory ? it : zipTree(it) }
    classifier "uber-jar"
    }
    task uberJar(type:Jar) {
    from sourceSets.main.output
    from configurations.runtime.collect { it.directory ? it : zipTree(it) }
    classifier "uber-jar"
    }
    Resolution at Configuration Time
    

    View Slide

25. |25
    Use lazy evaluation instead
    task uberJar(type:Jar) {
    inputs.files configurations.runtime
    from sourceSets.main.output
    from { configurations.runtime.collect { it.directory ? it : zipTree(it) } }
    classifier "uber-jar"
    }
    task uberJar(type:Jar) {
    inputs.files configurations.runtime
    from sourceSets.main.output
    from { configurations.runtime.collect { it.directory ? it : zipTree(it) } }
    classifier "uber-jar"
    }
    Resolution at Configuration Time
    

    View Slide

26. |26
    This build script seems expensive
    I/O at Configuration Time
    

    View Slide

27. |27
    task projectStats() {
    def statsFile = new File(buildDir, 'stats.txt')
    statsFile.parentFile.mkdirs()
    statsFile.text = "Source files: ${sourceSets.main.java.size()}"
    }
    task projectStats() {
    def statsFile = new File(buildDir, 'stats.txt')
    statsFile.parentFile.mkdirs()
    statsFile.text = "Source files: ${sourceSets.main.java.size()}"
    }
    I/O at Configuration Time
    Careful when writing custom tasks!
    

    View Slide

28. |28
    I/O at Configuration Time
    task projectStats() {
    def statsFile = new File(buildDir, 'stats.txt')
    inputs.files sourceSets.main.java
    outputs.file statsFile
    doLast {
    statsFile.parentFile.mkdirs()
    statsFile.text = "Source files: ${sourceSets.main.java.size()}"
    }
    }
    task projectStats() {
    def statsFile = new File(buildDir, 'stats.txt')
    inputs.files sourceSets.main.java
    outputs.file statsFile
    doLast {
    statsFile.parentFile.mkdirs()
    statsFile.text = "Source files: ${sourceSets.main.java.size()}"
    }
    }
    Don’t forget doLast {}
    

    View Slide

29. |29
    task projectStats(type: ProjectStats) {
    statsFile = new File(buildDir, 'stats.txt')
    sources = sourceSets.main.java
    }
    class ProjectStats extends DefaultTask {
    @InputFiles
    FileCollection sources
    @OutputFile
    File statsFile
    @TaskAction
    def stats() {
    statsFile.text = "Source files: ${sources.size()}"
    }
    }
    task projectStats(type: ProjectStats) {
    statsFile = new File(buildDir, 'stats.txt')
    sources = sourceSets.main.java
    }
    class ProjectStats extends DefaultTask {
    @InputFiles
    FileCollection sources
    @OutputFile
    File statsFile
    @TaskAction
    def stats() {
    statsFile.text = "Source files: ${sources.size()}"
    }
    }
    I/O at Configuration Time
    

    View Slide

30. |30
    Inefficient Plugins
    

    View Slide

31. |31
    Expensive logic for each project
    def out = new ByteArrayOutputStream()
    exec {
    commandLine 'git', 'rev-parse', "HEAD"
    standardOutput = out
    workingDir = rootDir
    }
    version = new String(out.toByteArray())
    def out = new ByteArrayOutputStream()
    exec {
    commandLine 'git', 'rev-parse', "HEAD"
    standardOutput = out
    workingDir = rootDir
    }
    version = new String(out.toByteArray())
    subprojects {
    apply from: "$rootDir/gradle/version-from-status.gradle"
    }
    subprojects {
    apply from: "$rootDir/gradle/version-from-status.gradle"
    }
    Inefficient Plugins
    

    View Slide

32. |32
    Reuse expensive calculations
    def out = new ByteArrayOutputStream()
    exec {
    commandLine 'git', 'rev-parse', "HEAD"
    standardOutput = out
    workingDir = rootDir
    }
    version = new String(out.toByteArray())
    subprojects {
    version = rootProject.version
    }
    def out = new ByteArrayOutputStream()
    exec {
    commandLine 'git', 'rev-parse', "HEAD"
    standardOutput = out
    workingDir = rootDir
    }
    version = new String(out.toByteArray())
    subprojects {
    version = rootProject.version
    }
    Inefficient Plugins
    

    View Slide

33. |33
    Variant Explosion
    Use variantFilter {} to exclude useless
    flavor/buildType combinations
    

    View Slide

34. Optimising Build Logic
    

    View Slide

35. |35
    Extract Script Plugins
    Makes finding issues easier
    apply from: "$rootDir/gradle/environment.gradle"
    apply from: "gradle/chuck-norris-fact.gradle"
    apply from: "$rootDir/gradle/version-from-status.gradle"
    subprojects {
    apply plugin: 'java'
    apply from: "$rootDir/gradle/test-fixtures.gradle"
    apply from: "$rootDir/gradle/compiler-settings.gradle"
    }
    apply from: "$rootDir/gradle/environment.gradle"
    apply from: "gradle/chuck-norris-fact.gradle"
    apply from: "$rootDir/gradle/version-from-status.gradle"
    subprojects {
    apply plugin: 'java'
    apply from: "$rootDir/gradle/test-fixtures.gradle"
    apply from: "$rootDir/gradle/compiler-settings.gradle"
    }
    

    View Slide

36. |36
    Extract Binary Plugins
    Use buildSrc
    Use static compilation
    Keep build scripts declarative
    

    View Slide

37. |37
    Optimise on the Algorithmic
    Level
    gradle-profiler --profile jfr
    

    View Slide

38. Execution Time
    

    View Slide

39. |39
    Execution Time
    Executing selected tasks
    Incremental
    Cacheable
    Parallelizable
    

    View Slide

40. |40
    Incremental Builds
    Nothing changed? Executed tasks should be zero!
    

    View Slide

41. |41
    Incremental Builds
    Find volatile inputs
    

    View Slide

42. |42
    Example: Crashlytics
    Unique IDs are the bane of local dev performance
    apply plugin: 'io.fabric'
    android {
    buildT
    ypes {
    debug {
    ext.alwaysUpdateBuildId = false
    }
    }
    }
    apply plugin: 'io.fabric'
    android {
    buildT
    ypes {
    debug {
    ext.alwaysUpdateBuildId = false
    }
    }
    }
    

    View Slide

43. |43
    Faster Compilation
    Modularization => Compile avoidance
    Decoupled code => Faster incremental compilation
    Careful with Kotlin migration (for now)
    

    View Slide

44. |44
    Incremental Annotation
    Processing
    Since Gradle 4.7
    Early adopters: Lombok, Android-State, Dagger,
    Parceller
    Others: github.com/gradle/gradle/issues/5277
    

    View Slide

45. |45
    Caching
    

    View Slide

46. |46
    Parallelism
    Serial execution
    

    View Slide

47. |47
    Parallelism
    --parallel and small, decoupled projects
    

    View Slide

48. Keeping track of
    performance
    

    View Slide

49. |50
    Gradle Enterprise
    

    View Slide

50. |51
    See all Builds
    Find out what colleagues are struggling with
    

    View Slide

51. |52
    Get Deep Insights
    

    View Slide

52. |53
    Watch Performance over Time
    

    View Slide

53. Resources
    

    View Slide

54. |55
    Gradle performance guide
    https://guides.gradle.org/performance
    Android performance guide
    https://developer.android.com/studio/build/optimize-your-build.html
    Plugin development guide
    https://guides.gradle.org/implementing-gradle-plugins
    Structuring build logic guide
    https://docs.gradle.org/current/userguide/organizing_build_logic.html
    Guides
    

    View Slide

55. |56
    scans.gradle.com
    

    View Slide

56. w
    Thank you
    Devoxx Belgium 2018
    

    View Slide