Demystify and Optimize Your Android Gradle Builds - Mobiconf 2021

Gradle and the Android Gradle Plugin are the backbone of most Android projects today. They are also the source of much frustration for dev teams wishing their builds were faster or easier to work with.

This talk aims to demystify your Android build by helping you understand how Gradle works, what happens when you build your project, and how to start optimizing your build for increased developer productivity.

During this talk, we’ll focus on a few key areas including
- The difference between Gradle and Android Gradle Plugin
- What happens when you run a Gradle task
- How to investigate your project’s build performance
- Performance optimizations

By the end of the talk, you should better understand what happens when you build your Android app, and be equipped to start optimizing your builds for faster builds and happier developers.

Transcript

1. DEMYSTIFY & OPTIMIZE
   YOUR ANDROID GRADLE
   BUILDS
   

   View Slide

2. Mobiconf 2021
   Demystify & Optimize Your
   Android Gradle Builds
   Android Developer & Instructor
   
   Kotlin GDE
   @n8ebel goobar.dev
   Nate Ebel
   

   View Slide

3. What is Gradle?
   

   View Slide

4. What is Gradle?
   A. The source of 94.3% of Android dev Twitter tra
   ffi
   c
   
   B. A viable means of heating a small room
   
   C. A general purpose build tool used for Android and other projects
   
   D. All of the above
   

   View Slide

5. What is Gradle?
   A. The source of 94.3% of Android dev Twitter tra
   ffi
   c
   
   B. A viable means of heating a small room
   
   C. A general purpose build tool used for Android and other projects
   
   D. All of the above
   

   View Slide

6. Android Builds
   Android
   
   Gradle Plugin
   Gradle
   

   View Slide

7. Android Builds
   Android
   
   Gradle Plugin
   Gradle
   

   View Slide

8. Android Builds
   Gradle & the Android Gradle Plugin are the
   backbone of most Android projects today
   

   View Slide

9. Android Builds
   Backbone of our Android projects
   

   View Slide

10. Android Builds
    Backbone of our
    Android projects
    

    View Slide

11. Android Builds
    Backbone of our
    Android projects
    Source of frustration
    and lost productivity
    

    View Slide

12. Builders want to Build
    

    View Slide

13. Developer Productivity Engineering
    

    View Slide

14. Developer Productivity Engineering
    “The use of data analytics and build
    acceleration technologies to speed up critical
    software development processes - and to make
    them more e
    ff
    i
    cient.”
    https://gradle.com/blog/top-three-reasons-to-launch-a-dedicated-developer-
    productivity-engineering-team/
    

    View Slide

15. Where are we going?
    Understanding Profiling
    Optimizing Maintaining
    

    View Slide

16. My hope for you
    1. Better understand what happens when you build your project
    
    2. Identify at least 1 potential tool or optimization to try out today
    

    View Slide

17. Understanding
    Your Build
    

    View Slide

18. Run ‘app’ ?
    

    View Slide

19. Executing tasks
    [:app:assembleDebug]
    

    View Slide

20. Gradle Task
    “… a single atomic piece of work for a build,
    such as compiling classes or generating
    javadoc.”
    https://docs.gradle.org/current/dsl/org.gradle.api.Task.html
    

    View Slide

21. Where do Tasks come from?
    

    View Slide

22. Where to Tasks come from?
    • From Gradle
    
    • From our own custom Tasks
    
    • From Plugins
    

    View Slide

23. Tasks coming from Gradle
    • help
    • dependencies
    • tasks
    • projects
    • properties
    

    View Slide

24. Custom Gradle Tasks
    De
    fi
    ne your own tasks to perform whatever task you need
    Custom Tasks
    task clean(type: Delete) {
    
    delete rootProject.buildDir
    
    }
    
    task helloWorld {
    
    println("Hello World")
    
    }
    
    

    View Slide

25. Tasks coming from Plugins
    • assembleDebug
    • artifactoryPublish
    • appDistributionUploadDebug
    

    View Slide

26. Gradle Plugins
    “[… plugins add new tasks, domain objects,
    conventions, and extend core objects…]”
    https://docs.gradle.org/current/userguide/plugins.html
    

    View Slide

27. Custom Gradle Plugins
    De
    fi
    ne your own Gradle plugins for common con
    fi
    guration or custom tasks
    Custom Tasks
    class VersionCodePlugin : Plugin {
    
    override fun apply(project: Project) {
    
    project.version = calculateVersion()
    
    …
    
    }
    
    }
    
    apply plugin: VersionCodePlugin
    
    

    View Slide

28. What does the most basic
    Gradle project look like?
    

    View Slide

29. The interesting
    functionality comes from
    Gradle Plugins
    

    View Slide

30. What tasks are available
    in a brand new Android
    project?
    

    View Slide

31. Android Gradle Plugin
    

    View Slide

32. What is the relationship?
    Android
    
    Gradle Plugin
    Gradle
    

    View Slide

33. Slow Builds?
    

    View Slide

34. Slow builds come from
    • Underpowered hardware
    
    • Computationally intensive tasks
    
    • Building more than is needed
    

    View Slide

35. Faster Machines
    ==
    Faster Builds
    

    View Slide

36. Avoid computationally intensive
    tasks when possible
    

    View Slide

37. Avoid doing more work than is
    necessary
    

    View Slide

38. Phases of Gradle Builds
    Initialization Configuration Execution
    

    View Slide

39. Initialization
    

    View Slide

40. Configuration
    

    View Slide

41. Initialization & Configuration
    

    View Slide

42. Execution
    

    View Slide

43. Execution
    Task
    Input Output
    

    View Slide

44. Task Execution Status
    • EXECUTED
    • UP-TO-DATE
    • FROM-CACHE
    • SKIPPED
    • NO-SOURCE
    

    View Slide

45. Common Build Scenarios
    First Time Build Up-To-Date Build
    Clean Build Incremental Build
    

    View Slide

46. First Time Build
    • Fresh checkout of a project
    
    • No previously built outputs/artifacts
    
    • May pull from a remote cache
    

    View Slide

47. Up-To-Date Build
    • Run a task
    
    • Run the same task again with no changes
    
    • Ideally, nothing should run
    
    • Everything should be UP-TO-DATE
    

    View Slide

48. Clean Build
    • Delete local build artifacts
    
    • Run your task again
    
    • May need to rebuild full project
    
    • Should ideally pull many artifacts from build cache
    

    View Slide

49. Incremental Build
    • Make a change to your project code/resources
    
    • Run your task again
    
    • Ideally, only a portion of your project should need to rebuild
    

    View Slide

50. Execute as few tasks as possible
    for any given build
    

    View Slide

51. The Story So Far
    • Click ‘run app’
    
    • Android Studio executes an Android Speci
    fi
    c Gradle task
    
    • That task comes from Android Gradle Plugin
    
    • That task will run via Gradle’s infrastructure
    
    • Gradle is designed to avoid doing unnecessary work
    
    • Our goal, then, is to con
    fi
    gure our builds to do as little work as possible
    

    View Slide

52. Profiling
    
    Your Build
    

    View Slide

53. What gets measured, gets improved
    

    View Slide

54. Profiling/Monitoring Tools
    Gradle Build Output
    Android Studio
    
    Build Analyzer
    Gradle Buildscans Gradle Enterprise
    

    View Slide

55. Gradle Build Output
    • Build time
    
    • Number of tasks executed
    
    • Number of tasks from cache
    
    • Number of tasks up-to-date
    
    • Task status
    

    View Slide

56. Android Studio Build Analyzer
    • Build overview
    
    • Plugin analysis
    
    • Task execution time
    
    • Warnings & recommendations
    

    View Slide

57. Profiling Across Machines
    

    View Slide

58. Gradle Buildscans
    • Detailed report of a speci
    fi
    c build
    
    • Easily shareable between team members and across machines
    
    • Can extend to
    fi
    t the needs of your team
    

    View Slide

59. Gradle Buildscans
    • Con
    fi
    guration, Initialization, and Execution time
    
    • Queryable build timeline
    
    • Console outputs
    
    • Project structure
    
    • Environment details
    
    • Cache hit/miss report
    
    • Test outputs
    

    View Slide

60. How to Generate a Buildscan?
    • add --scan to the end of your Gradle Wrapper invocation
    
    • accept the TOS
    
    • click the generated link and enter an email
    
    • click the link in the email to view your scan
    
    • subsequent scans won’t require the email
    

    View Slide

61. How can we use a Buildscan?
    • Analyze con
    fi
    guration and initialization times to ensure a properly con
    fi
    gured build
    
    • Analyze task execution to identify slow running tasks
    
    • Use the timeline to determine if any cacheable tasks were executed
    
    • Diagnose build failures for distributed teammates
    
    • Diagnose CI build failures
    
    • Identify hardware factors contributing to slow builds
    

    View Slide

62. Analyzing Trends
    

    View Slide

63. Gradle Enterprise
    

    View Slide

64. Gradle Enterprise
    “Gradle Enterprise leverages acceleration
    technologies to speed up the software build
    and test process and data analytics to make
    troubleshooting more e
    ff i
    cient”
    https://gradle.com/
    

    View Slide

65. What does Gradle Enterprise do?
    • Store buildscans on remote server for historical access and analysis
    
    • Automatically store buildscans in a single location
    
    • Provides a remote build cache
    
    • Analyze build trends over time
    
    • Enables support for custom tagging and
    fi
    ltering
    
    • Supports test distribution and analysis
    

    View Slide

66. Gradle Enterprise at Premise
    • Identify build cache issues across local and CI builds
    
    • Identify regionally speci
    fi
    c build cache issues
    
    • Identify build speed regressions due to AGP and NDK
    
    • Quickly diagnose build failures for distributed team
    
    • Compare buildscans to diagnose build issues
    

    View Slide

67. Gradle Enterprise at Premise
    • Is a paid product
    
    • Is an incredibly valuable tool for our team
    

    View Slide

68. Optimizing
    
    Your Build
    

    View Slide

69. Give your devs faster machines
    

    View Slide

70. Slow build times * cost/dev
    =
    =
    money for faster hardware
    

    View Slide

71. Enable build acceleration features
    

    View Slide

72. • Parallel execution
    
    • Local build cache
    
    • Remote build cache
    
    • Con
    fi
    guration cache
    
    • File-system watching
    Build acceleration features
    

    View Slide

73. Parallel Execution
    • Build projects in parallel
    
    • Very bene
    fi
    cial in multi-modular projects with reasonable dependency graph
    
    • Add org.gradle.parallel=true to gradle.properties
    

    View Slide

74. Gradle Build Cache
    • Reuse the outputs from previous builds even after a clean
    
    • Local Cache
    
    • Add org.gradle.caching=true to your gradle.properties
    fi
    le
    
    • Remote Cache
    
    • Setup with, or without, Gradle Enterprise
    

    View Slide

75. Configuration Caching
    • Cache con
    fi
    guration data
    
    • Add org.gradle.unsafe.con
    fi
    guration-cache=true to gradle.properties
    
    • Very experimental / Plugins must support it
    https://github.com/gradle/gradle/issues/13490
    

    View Slide

76. File System Watching
    • Reduce I/O impact of builds using in-memory
    fi
    le system info
    
    • Add org.gradle.vfs.watch=true to gradle.properties
    
    • On by default in Gradle 7+
    

    View Slide

77. Optimize memory settings
    

    View Slide

78. Optimize memory settings
    • Allocate more memory to Gradle daemon
    
    • Add org.gradle.jvmargs=-Xmx2048M to gradle.properties
    
    • Allocate more memory to Kotlin compiler
    
    • Add -Dkotlin.daemon.jvm.options=-Xmx2048M to gradle.properties
    
    • Play with this setting based on your build and hardware
    

    View Slide

79. Optimize memory settings
    • Remember that builds run in environments with limited resources
    
    • (Gradle daemon + Kotlinc) * numberOfSeparateBuildProcesses = total memory
    
    • Your machine, your teammate’s machine, and your CI machine might di
    ff
    er
    

    View Slide

80. Optimize how you build
    

    View Slide

81. Optimize how you build
    • Use a single JDK
    
    • Keep libraries and tooling up to date
    
    • Build only what is needed
    
    • Modularize your project
    
    • Reduce the use of kapt
    

    View Slide

82. Use the same JDK
    • Use the same JDK for Android Studio and command line builds
    
    • Set JAVA_HOME environment variable
    
    • Con
    fi
    gure Android Studio to use that JDK rather than embedded
    
    • This will speed up builds when switching from cmd line & Android Studio
    

    View Slide

83. Keep libraries up-to-date
    • Stay up to date with libraries, plugins, languages
    
    • Helps pick up performance improvements, cache miss
    fi
    xes, etc
    

    View Slide

84. Build only what is needed
    • Don’t minify your app if you don’t need to
    
    • Don’t build multiple variants if you don’t need them all
    
    • Exclude locales/resources for dev builds when not needed
    
    • Disable plugins such as Crashlytics if not needed for dev builds
    
    • Build/test/analyze in parallel when possible
    

    View Slide

85. Modularize your project
    • Modules/projects can build in parallel
    
    • Splitting your project into smaller modules enables greater parallelization
    
    • Limited bene
    fi
    t when most of the code is all in a single module
    
    • Limited bene
    fi
    t if all modules are highly coupled
    

    View Slide

86. Reduce the use of kapt
    • Kapt is very slow for Kotlin builds
    
    • Avoid annotation processors with kapt if possible
    
    • Limit to speci
    fi
    c modules if possible
    
    • Consider using KSP if applicable for your project
    

    View Slide

87. Maintaining
    
    Your Build
    

    View Slide

88. Monitor and analyze your build
    

    View Slide

89. Gradle Enterprise
    

    View Slide

90. Gradle Doctor
    

    View Slide

91. Gradle Doctor
    • Plugin to diagnose & prevent common build issues
    
    • Con
    fi
    gurable warnings for things such as
    
    • Build speed issues
    
    • Annotation processing time
    
    • Mismatched JDK versions
    
    • Building all variants
    
    • Remote cache speed issues
    https://runningcode.github.io/gradle-doctor/
    

    View Slide

92. android-gradle-cache-fix Plugin
    

    View Slide

93. android-gradle-cache-fix Plugin
    • Plugin to
    fi
    x known build cache issues in Android Gradle Plugin
    
    • Add to your project and start seeing more cache hits
    https://github.com/gradle/android-cache-
    fi
    x-gradle-plugin
    

    View Slide

94. Benchmark your builds
    

    View Slide

95. Gradle Profiler
    

    View Slide

96. Gradle Profiler
    • Run multiple Gradle builds and aggregate the results
    
    • Highlights statistically signi
    fi
    cant changes to build performance
    
    • Use to pro
    fi
    le the impact of build changes such as enabling caching
    
    • Can be run manually or automated for common scenarios
    https://github.com/gradle/gradle-pro
    fi
    ler
    

    View Slide

97. Gradle Profiler
    

    View Slide

98. Keep your project up-to-date
    • Stay up to date with libraries, plugins, languages
    
    • Helps pick up performance improvements, cache miss
    fi
    xes, etc
    

    View Slide

99. Make your build easier to
    work with
    

    View Slide

100. Wrapping Up
     

     View Slide

101. Android relies on Android Gradle
     Plugin and Gradle to build our
     projects
     

     View Slide

102. Gradle provides infrastructure
     and plugins provide functionality
     

     View Slide

103. There’s very little magic
     

     View Slide

104. To speed up our builds, we can
     use faster hardware, do less
     intensive work, or do less work.
     

     View Slide

105. Tools such as buildscans and
     Gradle Enterprise help diagnose
     build issues
     

     View Slide

106. We can optimize our builds
     through caching, modularization,
     and other build acceleration
     strategies
     

     View Slide

107. By monitoring and profiling
     performance, we may maintain
     efficiency over time
     

     View Slide

108. Develop a culture of continued
     investment into developer
     productivity for happier, more
     productive, teams.
     

     View Slide

109. Thank You
     @n8ebel
     
     goobar.dev
     

     View Slide