Perspectives from a Solutions Engineer

Presented at Droidcon America Online Nov 16, 2020

This talk covers some of the most common build issues causing slow Gradle builds from seeing many different builds as a Gradle Solutions engineer. This also covers future performance improvements to the Gradle Build tool regarding ease of use and performance.

Interested in a Gradle Enterprise Trial?
https://gradle.com/enterprise/trial/

Gradle is hiring Solutions Engineers
https://go.gradle.com/solutions-engineer

Video is available here: https://www.droidcon.com/media-detail?video=481160907

Transcript

1. Perspectives from a Solutions Engineer Nelson Osacky
2. Toro y Moi - Freelance
3. Perspectives from a Solutions Engineer •Why is productivity important? •What does Nelson do as a Solutions Engineer? •How can I make my build faster? •What is coming up in Gradle?
4. Me • Previously Android Engineer • Large projects • SoundCloud • Square • Small startups • Solutions Engineer at Gradle
5. Me • Gradle Plugin Maintainer • Fladle - Easily Scale Instrumentation Tests on Firebase https://github.com/runningcode/fladle • Delect - Replace Dagger with Dagger Reflect https://github.com/soundcloud/delect/ • Gradle Doctor - Actionable Insights for your build https://github.com/runningcode/gradle-doctor
6. Solutions Engineer https://gradle.com/enterprise/trial/
8. Gradle Build Tool Gradle Enterprise
9. Android Gradle Plugin Android Studio
10. Kotlin Intellij
11. Why is productivity important?
12. Android Development Feature Development Tech Debt (There are other things too)
13. Tech Debt Refactoring Build Speed
14. Build Speed is Tech Debt And it always pays off
15. Cost of Builds 60s waste * 50 builds / day * 50 devs = 42 hours lost / day
16. Cost of Builds 60s waste * 50 builds / day * 50 devs = 42 hours lost / day not including lost focus https://gradle.com/roi-calculator
17. Fast Builds Matter
18. Cost of Builds 60s waste * 50 builds / day * 50 devs = 42 hours lost / day hire 5 new people without paying them! no recruiting https://gradle.com/roi-calculator
19. Build Speed is Tech Debt And it always pays off And is easy to justify working on it
20. Motivated?
21. Solutions Engineer https://gradle.com/enterprise/trial/
22. Solutions Engineer What do I do?
23. Work with trial prospects
24. Help customers get the most out of Gradle Enterprise
25. Help Gradle address customers needs
26. I see a lot of builds from 10 person teams to 1000+ Android, Gradle and Maven
27. Optimize and understand builds Reduce failures Preach Developer Productivity Help customers maximize trials
28. Help prioritize issues with Gradle Build tools and Gradle Enterprise Present business cases for purchasing Gradle Enterprise
29. Meet with Google and Jetbrains to help prioritize issues impacting customers
30. Fix issues in open source plugin affecting customers
31. Solutions Team at Gradle • Gary Hale • Daz Deboer • Etienne Studer • Nelson Osacky (me)
32. Solutions Team at Gradle Supported by Operations and Build Tool teams help me sound like I know what I'm talking about
33. Gradle Enterprise Trials
34. Work with a champion
35. Meet once a week
36. Experiments Examine build scans Help with other Gradle questions
37. What do I see?
38. Understanding of Gradle can vary wildly
39. Optimization level of build can vary wildly
40. Always room for improvements! but watch out for diminishing returns
41. How can I go faster?
42. Depends on your build
43. Build Lifecycle •Initialization •Configuration •Execution
44. Build Lifecycle Initialization Sets up the environment for the build and determines which projects will take part in it.
45. Build Lifecycle Configuration All build scripts of all projects are executed. Constructs and configures the task graph.
46. Build Lifecycle Execution Runs the tasks graph in order.
47. Where to start?
48. Check for Red Flags
49. What is a red flag?
50. If nothing changes, no tasks should execute.
51. ./gradlew assembleDebug ./gradlew assembleDebug BUILD SUCCESSFUL in 2s 58 actionable tasks: 58 up-to-date
52. ./gradlew assembleDebug ./gradlew assembleDebug BUILD SUCCESSFUL in 5s 359 actionable tasks: 3 executed, 356 from cache
53. Check for Red Flags Regularly
54. BUILD SUCCESSFUL in 5s 359 actionable tasks: 3 executed, 356 up-to-date Publishing build scan... https://gradle.com/s/yj5jtd4kh6ucc Build Analyzer results available
55. BUILD SUCCESSFUL in 5s 359 actionable tasks: 3 executed, 356 up-to-date Publishing build scan... https://gradle.com/s/yj5jtd4kh6ucc Build Analyzer results available Gradle Build Scans Android Studio Build Analyzer
56. Android Studio Build Analyzer
57. Run a build scan ./gradlew assembleDebug --scan https://scans.gradle.com/
59. Filter
62. 9 tasks ran Bugsnag
64. Enable parallel, enable caching, enable daemon
65. 16627 tasks is a red flag
66. Use Task Configuration Avoidance https://docs.gradle.org/current/userguide/task_configuration_avoidance.html
67. Configuration time should be roughly 5s per 100 projects
68. Just released
69. File system watching https:///blog.gradle.org/introducing-file-system-watching
70. org.gradle.vfs.watch=true Easiest build speed improvement! gradle.properties
71. Faster input snapshotting
72. Jetifier performance improvements Released in AGP 4.1.0
73. Jetifier - please kill it
75. Stuck on the Jetifier?
76. plugins { id "com.github.plnice.canidropjetifier" version "0.5" } Can I drop Jetifier? ./gradlew -Pandroid.enableJetifier=false canIDropJetifier
77. Jetify At Home ./jetifier-standalone -i <source-library> -o <output-library> https://developer.android.com/studio/command-line/jetifier
78. Jetify At Home Upload to nexus
79. Update Dependencies
81. Update All The Things ⬢ Gradle 6.7 ⬢ Android Gradle Plugin 4.1.1 ⬢ Gradle Enterprise Plugin 3.5 ⬢ Kotlin 1.4.10 ⬢ Third party plugins ⬢ Third party libraries Stable File Watching Caching Improvements Task Configuration Avoidance Compressed scan upload Incremental Annotation Processors Fixes for caching bugs
82. Update All The Things plugins { id "com.github.ben-manes.versions" version "0.33.0" } ./gradlew dependencyUpdates -Drevision=release https://github.com/ben-manes/gradle-versions-plugin
84. Let's talk caching
85. Local Cache and Remote Cache
86. https://docs.gradle.org/current/userguide/ build_cache.html https://gradle.com/training/build-cache-deep-dive
87. Scaling Android Builds in Pandemic Times Inaki Villar 3:00pm PST - Nov 17th https://www.online.droidcon.com/americas-speaker/inaki-villar
88. If a task has been run before, no need to re-run
89. Compilation, generation, packaging, tests
90. Fix cache misses
91. Gradle Enterprise
92. Public Gradle Enterprise instances • ge.gradle.org • ge.spring.io • ge.junit.org • ge.jetbrains.com
93. Task Input Comparison • https://ge.gradle.org/s/vmpk3ahkdvf34 • https://ge.gradle.org/c/xjsxszonlcuj6/vmpk3ahkdvf34/task-inputs? cacheability=cacheable&expanded=WyJsbTVwNjU1bWFqbGx5LWNsYXNzcGF0aCIsIjdlN mtzNWVidnRoeTYtY2xhc3NwYXRoIiwidDd4dDZjZ3d2d3ZqcS0kMSJd
94. Failure Analysis • https://ge.gradle.org/s/ryzmyn67iahnw
95. Flaky Test Analysis • https://ge.gradle.org/scans/tests? search.relativeStartTime=P7D&search.timeZoneId=Europe/ Zurich&tests.container=org.gradle.performance.regression.java.JavaUpToDatePerformanc eTest&tests.sortField=FAILED&tests.test=up-to- date%20assemble%20with%20local%20build%20cache%20enabled%20(parallel%20false) &tests.unstableOnly=true
96. Upcoming performance enhancements
97. Configuration Cache https://docs.gradle.org/current/userguide/configuration_cache.html
98. Kotlin 1.4.20 AGP 4.2.0 Full configuration caching support
99. Bugfixes to incremental Kotlin Compiler
100. Performance improvements when restoring from cache for incremental compile https://youtrack.jetbrains.com/issue/KT-34862
101. Gradle 6.8 Improvements https://docs.gradle.org/6.8-milestone-3/release-notes.html#performance-improvements
102. Faster kts build script compilation
103. Compile avoidance for kts build script compilation
104. Configuration cache for included builds
105. Future improvements in AGP and Gradle
106. Unbounded thread executor used for transformations https://github.com/gradle/gradle/pull/15025
107. Android Studio Sync improvements in 4.2.0
108. Empty Source Directory Cache misses
109. Empty Source Directory Cache miss
110. Empty Source Directory Cache misses Gary Hale
111. Ease of Use
112. Idiomatic Gradle
113. Upcoming usability improvements
114. Idiomatic Gradle Jendrik Johannes https:///github.com/jjohannes/idiomatic-gradle
115. Idiomatic Gradle ./gradlew my-awesome-library:compileTestJava ./gradlew mAL:cT Kebab case https:///docs.gradle.org/6.7/release-notes.html#other-new-features-and-usability-improvements
116. Idiomatic Gradle allProjects { repositories { mavenCentral() } } Central Repository Declaration https:///docs.gradle.org/6.8-milestone-3/release-notes.html#dependency-resolution-improvements
117. Idiomatic Gradle dependencyResolutionManagement { components { withModule('com.google.guava:guava', GuavaRule) } } Central Component Metadata Rules https:///docs.gradle.org/6.8-milestone-3/release-notes.html#dependency-resolution-improvements
118. Idiomatic Gradle ./gradlew :my-other-project:sub:foo Executing Tasks From Included Builds https:///docs.gradle.org/6.7/release-notes.html#other-new-features-and-usability-improvements
119. More Resources
120. Gradle Training https://gradle.com/training/ https://gradle.com/training/introduction-to-gradle https://gradle.com/training/build-cache-deep-dive
121. Gradle Community Slack https://gradle.com/slack-invite
122. Benchmarking Builds with the Gradle Profiler Tony Robalik https://dev.to/autonomousapps/benchmarking-builds-with-gradle- profiler-oa8
123. Understanding, Profiling, and Optimizing Gradle in Android Builds Nate Ebel https://goobar.io/understanding-profiling-and-optimizing-gradle-in- android-builds/
124. Improving Android Build Performance Devoxx Belgium 2018 Cédric Champeau https://www.youtube.com/watch?v=StahHZ-L83k https://speakerdeck.com/melix/improving-android-build-performance
125. Build Bigger, Better: Gradle for Large Projects Google I/O 2019 Aurimas Liutikas and Xavier Ducrohet https://www.youtube.com/watch?v=sQC9-Rj2yLI
126. Beyond Modularization - Scaling Android Builds Nelson Osacky https://vimeo.com/430648415 https://speakerdeck.com/runningcode/beyond-modularization-scaling- your-android-build-with-gradle
127. The Secrets of the Build Scan Plugin and the Internals of Gradle Virtual Android Makers Paris 2020 Me https://www.youtube.com/watch?v=lgaqS0pmUzk
128. Measuring remote build cache performance https://medium.com/@runningcode https://github.com/runningcode/gradle-doctor
129. Scaling Android Builds in Pandemic Times Inaki Villar 3:00pm PST - Nov 17th https://www.online.droidcon.com/americas-speaker/inaki-villar
130. Android Builds At Scale Zac Sweers, Chiu-Ki Chan, Israel Ferrero Camacho, John Rodriguez, Ivan Gavrilogic 1:10pm PST - Nov 17th https://www.online.droidcon.com/americas-panels-workshops
131. Thank you! Gradle Enterprise Trial osacky.com https://gradle.com/enterprise/trial/ https://speakerdeck.com/runningcode/perspectives-from-a-solutions-engineer https://go.gradle.com/solutions-engineer Gradle Solutions is hiring