Perspectives from a Solutions Engineer

Perspectives from a Solutions Engineer

Presented at Droidcon America Online Nov 16, 2020

This talk covers some of the most common build issues causing slow Gradle builds from seeing many different builds as a Gradle Solutions engineer. This also covers future performance improvements to the Gradle Build tool regarding ease of use and performance.

Interested in a Gradle Enterprise Trial?
https://gradle.com/enterprise/trial/

Gradle is hiring Solutions Engineers
https://go.gradle.com/solutions-engineer

Video is available here: https://www.droidcon.com/media-detail?video=481160907

5f69045a2ca496221cfc624405917cdf?s=128

Nelson Osacky

November 16, 2020
Tweet

Transcript

  1. Perspectives from a Solutions Engineer Nelson Osacky

  2. Toro y Moi - Freelance

  3. Perspectives from a Solutions Engineer •Why is productivity important? •What

    does Nelson do as a Solutions Engineer? •How can I make my build faster? •What is coming up in Gradle?
  4. Me • Previously Android Engineer • Large projects • SoundCloud

    • Square • Small startups • Solutions Engineer at Gradle
  5. Me • Gradle Plugin Maintainer • Fladle - Easily Scale

    Instrumentation Tests on Firebase https://github.com/runningcode/fladle • Delect - Replace Dagger with Dagger Reflect https://github.com/soundcloud/delect/ • Gradle Doctor - Actionable Insights for your build https://github.com/runningcode/gradle-doctor
  6. Solutions Engineer https://gradle.com/enterprise/trial/

  7. None
  8. Gradle Build Tool Gradle Enterprise

  9. Android Gradle Plugin Android Studio

  10. Kotlin Intellij

  11. Why is productivity important?

  12. Android Development Feature Development Tech Debt (There are other things

    too)
  13. Tech Debt Refactoring Build Speed

  14. Build Speed is Tech Debt And it always pays off

  15. Cost of Builds 60s waste * 50 builds / day

    * 50 devs = 42 hours lost / day
  16. Cost of Builds 60s waste * 50 builds / day

    * 50 devs = 42 hours lost / day not including lost focus https://gradle.com/roi-calculator
  17. Fast Builds Matter

  18. Cost of Builds 60s waste * 50 builds / day

    * 50 devs = 42 hours lost / day hire 5 new people without paying them! no recruiting https://gradle.com/roi-calculator
  19. Build Speed is Tech Debt And it always pays off

    And is easy to justify working on it
  20. Motivated?

  21. Solutions Engineer https://gradle.com/enterprise/trial/

  22. Solutions Engineer What do I do?

  23. Work with trial prospects

  24. Help customers get the most out of Gradle Enterprise

  25. Help Gradle address customers needs

  26. I see a lot of builds from 10 person teams

    to 1000+ Android, Gradle and Maven
  27. Optimize and understand builds Reduce failures Preach Developer Productivity Help

    customers maximize trials
  28. Help prioritize issues with Gradle Build tools and Gradle Enterprise

    Present business cases for purchasing Gradle Enterprise
  29. Meet with Google and Jetbrains to help prioritize issues impacting

    customers
  30. Fix issues in open source plugin affecting customers

  31. Solutions Team at Gradle • Gary Hale • Daz Deboer

    • Etienne Studer • Nelson Osacky (me)
  32. Solutions Team at Gradle Supported by Operations and Build Tool

    teams help me sound like I know what I'm talking about
  33. Gradle Enterprise Trials

  34. Work with a champion

  35. Meet once a week

  36. Experiments Examine build scans Help with other Gradle questions

  37. What do I see?

  38. Understanding of Gradle can vary wildly

  39. Optimization level of build can vary wildly

  40. Always room for improvements! but watch out for diminishing returns

  41. How can I go faster?

  42. Depends on your build

  43. Build Lifecycle •Initialization •Configuration •Execution

  44. Build Lifecycle Initialization Sets up the environment for the build

    and determines which projects will take part in it.
  45. Build Lifecycle Configuration All build scripts of all projects are

    executed. Constructs and configures the task graph.
  46. Build Lifecycle Execution Runs the tasks graph in order.

  47. Where to start?

  48. Check for Red Flags

  49. What is a red flag?

  50. If nothing changes, no tasks should execute.

  51. ./gradlew assembleDebug ./gradlew assembleDebug BUILD SUCCESSFUL in 2s 58 actionable

    tasks: 58 up-to-date
  52. ./gradlew assembleDebug ./gradlew assembleDebug BUILD SUCCESSFUL in 5s 359 actionable

    tasks: 3 executed, 356 from cache
  53. Check for Red Flags Regularly

  54. BUILD SUCCESSFUL in 5s 359 actionable tasks: 3 executed, 356

    up-to-date Publishing build scan... https://gradle.com/s/yj5jtd4kh6ucc Build Analyzer results available
  55. BUILD SUCCESSFUL in 5s 359 actionable tasks: 3 executed, 356

    up-to-date Publishing build scan... https://gradle.com/s/yj5jtd4kh6ucc Build Analyzer results available Gradle Build Scans Android Studio Build Analyzer
  56. Android Studio Build Analyzer

  57. Run a build scan ./gradlew assembleDebug --scan https://scans.gradle.com/

  58. None
  59. Filter

  60. None
  61. None
  62. 9 tasks ran Bugsnag

  63. None
  64. Enable parallel, enable caching, enable daemon

  65. 16627 tasks is a red flag

  66. Use Task Configuration Avoidance https://docs.gradle.org/current/userguide/task_configuration_avoidance.html

  67. Configuration time should be roughly 5s per 100 projects

  68. Just released

  69. File system watching https:///blog.gradle.org/introducing-file-system-watching

  70. org.gradle.vfs.watch=true Easiest build speed improvement! gradle.properties

  71. Faster input snapshotting

  72. Jetifier performance improvements Released in AGP 4.1.0

  73. Jetifier - please kill it

  74. None
  75. Stuck on the Jetifier?

  76. plugins { id "com.github.plnice.canidropjetifier" version "0.5" } Can I drop

    Jetifier? ./gradlew -Pandroid.enableJetifier=false canIDropJetifier
  77. Jetify At Home ./jetifier-standalone -i <source-library> -o <output-library> https://developer.android.com/studio/command-line/jetifier

  78. Jetify At Home Upload to nexus

  79. Update Dependencies

  80. None
  81. Update All The Things ⬢ Gradle 6.7 ⬢ Android Gradle

    Plugin 4.1.1 ⬢ Gradle Enterprise Plugin 3.5 ⬢ Kotlin 1.4.10 ⬢ Third party plugins ⬢ Third party libraries Stable File Watching Caching Improvements Task Configuration Avoidance Compressed scan upload Incremental Annotation Processors Fixes for caching bugs
  82. Update All The Things plugins { id "com.github.ben-manes.versions" version "0.33.0"

    } ./gradlew dependencyUpdates -Drevision=release https://github.com/ben-manes/gradle-versions-plugin
  83. None
  84. Let's talk caching

  85. Local Cache and Remote Cache

  86. https://docs.gradle.org/current/userguide/ build_cache.html https://gradle.com/training/build-cache-deep-dive

  87. Scaling Android Builds in Pandemic Times Inaki Villar 3:00pm PST

    - Nov 17th https://www.online.droidcon.com/americas-speaker/inaki-villar
  88. If a task has been run before, no need to

    re-run
  89. Compilation, generation, packaging, tests

  90. Fix cache misses

  91. Gradle Enterprise

  92. Public Gradle Enterprise instances • ge.gradle.org • ge.spring.io • ge.junit.org

    • ge.jetbrains.com
  93. Task Input Comparison • https://ge.gradle.org/s/vmpk3ahkdvf34 • https://ge.gradle.org/c/xjsxszonlcuj6/vmpk3ahkdvf34/task-inputs? cacheability=cacheable&expanded=WyJsbTVwNjU1bWFqbGx5LWNsYXNzcGF0aCIsIjdlN mtzNWVidnRoeTYtY2xhc3NwYXRoIiwidDd4dDZjZ3d2d3ZqcS0kMSJd

  94. Failure Analysis • https://ge.gradle.org/s/ryzmyn67iahnw

  95. Flaky Test Analysis • https://ge.gradle.org/scans/tests? search.relativeStartTime=P7D&search.timeZoneId=Europe/ Zurich&tests.container=org.gradle.performance.regression.java.JavaUpToDatePerformanc eTest&tests.sortField=FAILED&tests.test=up-to- date%20assemble%20with%20local%20build%20cache%20enabled%20(parallel%20false) &tests.unstableOnly=true

  96. Upcoming performance enhancements

  97. Configuration Cache https://docs.gradle.org/current/userguide/configuration_cache.html

  98. Kotlin 1.4.20 AGP 4.2.0 Full configuration caching support

  99. Bugfixes to incremental Kotlin Compiler

  100. Performance improvements when restoring from cache for incremental compile https://youtrack.jetbrains.com/issue/KT-34862

  101. Gradle 6.8 Improvements https://docs.gradle.org/6.8-milestone-3/release-notes.html#performance-improvements

  102. Faster kts build script compilation

  103. Compile avoidance for kts build script compilation

  104. Configuration cache for included builds

  105. Future improvements in AGP and Gradle

  106. Unbounded thread executor used for transformations https://github.com/gradle/gradle/pull/15025

  107. Android Studio Sync improvements in 4.2.0

  108. Empty Source Directory Cache misses

  109. Empty Source Directory Cache miss

  110. Empty Source Directory Cache misses Gary Hale

  111. Ease of Use

  112. Idiomatic Gradle

  113. Upcoming usability improvements

  114. Idiomatic Gradle Jendrik Johannes https:///github.com/jjohannes/idiomatic-gradle

  115. Idiomatic Gradle ./gradlew my-awesome-library:compileTestJava ./gradlew mAL:cT Kebab case https:///docs.gradle.org/6.7/release-notes.html#other-new-features-and-usability-improvements

  116. Idiomatic Gradle allProjects { repositories { mavenCentral() } } Central

    Repository Declaration https:///docs.gradle.org/6.8-milestone-3/release-notes.html#dependency-resolution-improvements
  117. Idiomatic Gradle dependencyResolutionManagement { components { withModule('com.google.guava:guava', GuavaRule) } }

    Central Component Metadata Rules https:///docs.gradle.org/6.8-milestone-3/release-notes.html#dependency-resolution-improvements
  118. Idiomatic Gradle ./gradlew :my-other-project:sub:foo Executing Tasks From Included Builds https:///docs.gradle.org/6.7/release-notes.html#other-new-features-and-usability-improvements

  119. More Resources

  120. Gradle Training https://gradle.com/training/ https://gradle.com/training/introduction-to-gradle https://gradle.com/training/build-cache-deep-dive

  121. Gradle Community Slack https://gradle.com/slack-invite

  122. Benchmarking Builds with the Gradle Profiler Tony Robalik https://dev.to/autonomousapps/benchmarking-builds-with-gradle- profiler-oa8

  123. Understanding, Profiling, and Optimizing Gradle in Android Builds Nate Ebel

    https://goobar.io/understanding-profiling-and-optimizing-gradle-in- android-builds/
  124. Improving Android Build Performance Devoxx Belgium 2018 Cédric Champeau https://www.youtube.com/watch?v=StahHZ-L83k

    https://speakerdeck.com/melix/improving-android-build-performance
  125. Build Bigger, Better: Gradle for Large Projects Google I/O 2019

    Aurimas Liutikas and Xavier Ducrohet https://www.youtube.com/watch?v=sQC9-Rj2yLI
  126. Beyond Modularization - Scaling Android Builds Nelson Osacky https://vimeo.com/430648415 https://speakerdeck.com/runningcode/beyond-modularization-scaling-

    your-android-build-with-gradle
  127. The Secrets of the Build Scan Plugin and the Internals

    of Gradle Virtual Android Makers Paris 2020 Me https://www.youtube.com/watch?v=lgaqS0pmUzk
  128. Measuring remote build cache performance https://medium.com/@runningcode https://github.com/runningcode/gradle-doctor

  129. Scaling Android Builds in Pandemic Times Inaki Villar 3:00pm PST

    - Nov 17th https://www.online.droidcon.com/americas-speaker/inaki-villar
  130. Android Builds At Scale Zac Sweers, Chiu-Ki Chan, Israel Ferrero

    Camacho, John Rodriguez, Ivan Gavrilogic 1:10pm PST - Nov 17th https://www.online.droidcon.com/americas-panels-workshops
  131. Thank you! Gradle Enterprise Trial osacky.com https://gradle.com/enterprise/trial/ https://speakerdeck.com/runningcode/perspectives-from-a-solutions-engineer https://go.gradle.com/solutions-engineer Gradle

    Solutions is hiring