Save 37% off PRO during our Black Friday Sale! »

Cash App & Gradle: A Journey in Android Developer Productivity

Cash App & Gradle: A Journey in Android Developer Productivity

A case study on how Cash App and Gradle have been working together to optimize Cash App builds, find bottlenecks, and improve developer productivity. John Rodriguez from Square and Rooz Mohazzabi from Gradle will share wins and findings from their journey on...

* How Cash App does builds
* Biggest wins and findings debugging Cash App local and CI builds
* Best performance practices for builds at scale
* Optimizing for your expensive build tasks, i.e., assemble, lint, etc
* Tracking down bottlenecks in 3rd-party plugins
* Speeding up local builds with the remote build cache during pandemic

Video: coming soon!

6c8b509fe5422470d148c2c4cf2eb4b0?s=128

John Rodriguez

October 28, 2021
Tweet

Transcript

  1. A Journey in Android Developer Productivity Cash App & Gradle

  2. How Cash App Does Builds

  3. None
  4. None
  5. None
  6. None
  7. The Tools We Used

  8. None
  9. ./gradlew app:assembleDebug

  10. ./gradlew app:assembleDebug --scan

  11. None
  12. None
  13. None
  14. None
  15. None
  16. None
  17. Biggest wins and findings

  18. What we discovered - Discovered negative savings from the remote

    cache 
 
 - After optimizations and applying the cache fix plugin, most instances of negative savings were eliminated, even in East Coast regions. 
 
 - Wire plugin cacheability issues 
 
 - The configuration for enabling the remote cache was not working properly 
 
 - Unnecessary VPN issues causing long build times
  19. After identifying and fixing cache misses avoidance savings from the

    remote cache started to increase
  20. AssembleDebug Build Time Down by ~50%

  21. Remote Cache Saving CI Builds an average ~8 min per

    build CI Compute Resource Savings from remote cache: 55 Days & 16 hours per week
  22. Optimizing for your expensive build tasks

  23. None
  24. None
  25. None
  26. None
  27. apply plugin: 'com.android.library ' apply plugin: 'org.jetbrains.kotlin.android ' apply plugin:

    'org.jetbrains.kotlin.kapt ' apply plugin: 'org.jetbrains.kotlin.plugin.parcelize ' apply plugin: 'app.cash.paparazzi ' apply plugin: 'app.cash.treehouse'
  28. Section Title apply plugin: 'java-library'

  29. apply plugin: 'com.android.library '

  30. JavaCompile task

  31. JavaCompile task class files

  32. JavaCompile task class files source files

  33. JavaCompile task class files source files target sdk

  34. JavaCompile task class files source files target sdk max heap

    size
  35. task task outputs task inputs

  36. task task outputs task inputs UP-TO-DATE (aka incremental build)

  37. JavaCompile task class files source files target sdk max heap

    size
  38. JavaCompile task A.class A.java JDK 11 4 GB

  39. JavaCompile task A.class A.java JDK 11 4 GB EXECUTED

  40. JavaCompile task A.class, B.class A.java, B.java JDK 11 4 GB

  41. JavaCompile task A.class, B.class A.java, B.java JDK 11 4 GB

    EXECUTED
  42. JavaCompile task A.class, B.class A.java, B.java JDK 11 4 GB

    UP-TO-DATE x 2
  43. JavaCompile task A.class A.java JDK 11 4 GB

  44. JavaCompile task A.class A.java JDK 11 4 GB CACHED echo

    “org.gradle.caching=true” >> gradle.properties
  45. Speeding Up Local Builds with the Remote Build Cache

  46. None
  47. None
  48. None
  49. None
  50. Average AssembleDebug savings from the remote cache were low: 20

    sec wall clock time
  51. private void addOsXMetadata() { buildScan.background { def json = new

    JsonSlurper( ) def ip = exec('curl', '-s', 'https://ipinfo.io/ip' ) def ge o if (ip) { geo = json.parseText ( exec('curl', '-s', “https://json.geoiplookup.io/$ip” ) ) } if (ip && geo) { value 'OS: Location', "${geo.city}, ${geo.region}, ${geo.country} " value 'OS: Coordinates', "${geo.latitude}, ${geo.longitude} " } else { value 'OS: Location', 'Unknown ' } } }
  52. None
  53. NYC Developers Saving 1 min 12 sec of build execution

    time per build from the remote build cache 
 Wall Clock Time: 38 sec of eng time savings
  54. SF Developers Saving 3 min 41 sec of build execution

    time per build from the remote build cache Wall Clock Time: 3 min 10 sec eng time savings
  55. SF vs NYC difference in remote cache avoidance savings: 2

    min 12 sec East Coast Local AssembleDebug builds: ~709 per week Remote Cache Node in the East Coast would result in... 709 * 2 min 12 sec = 1559.8 min of saved eng time per week * 48 work weeks per year = 1247.84 hours of saved eng time per year
  56. $ grep "include '" settings.gradle | wc -l 590

  57. $ 590 grep "include '" settings.gradle | wc -l

  58. $ grep "include '" settings.gradle | wc -l 590

  59. None
  60. None
  61. None
  62. Tracking down bottlenecks in 3rd-party plugins

  63. None
  64. None
  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. None
  72. None
  73. None
  74. None
  75. None
  76. None
  77. Time to run task Time loading from cache Avoidance Savings!

  78. None
  79. None
  80. Best practices for build performance

  81. None
  82. None
  83. None
  84. @Overrid e <T extends Task> T create ( String name,

    Class<T> type, Action<? super T> configuratio n ) throws InvalidUserDataException;
  85. @Overrid e <T extends Task> T create ( String name,

    Class<T> type, Action<? super T> configuratio n ) throws InvalidUserDataException;
  86. @Overrid e <T extends Task> T create ( String name,

    Class<T> type, Action<? super T> configuratio n ) throws InvalidUserDataException; @Overrid e TaskProvider<Task> register ( String name, Action<? super Task> configurationActio n ) throws InvalidUserDataException ;
  87. @Overrid e <T extends Task> T create ( String name,

    Class<T> type, Action<? super T> configuratio n ) throws InvalidUserDataException; @Overrid e TaskProvider<Task> register ( String name, Action<? super Task> configurationActio n ) throws InvalidUserDataException ;
  88. None
  89. None
  90. Upgrade!

  91. None
  92. None
  93. None
  94. $ grep "include '" settings.gradle | wc -l 590

  95. None
  96. None
  97. buildCache { remote(HttpBuildCache) { if (!isCi) { def pingCache =

    'ping -c1 -W1 -q gradle-enterprise.aws.square.com'.execute( ) def isCacheUnreachable = !pingCache.waitFor(2, TimeUnit.SECONDS) | | pingCache.exitValue() == 6 8 if (isCacheUnreachable) { throw new GradleException(... ) } url = 'https://gradle-build-cache.aws.square.com/cache/ ' }
  98. boolean isVpnCheckEnabled = hasProperty('com.squareup.cash.vpn.check.enabled') ? getProperty('com.squareup.cash.vpn.check.enabled').toBoolean() : true buildCache {

    remote(HttpBuildCache) { if (!isCi) { def pingCache = 'ping -c1 -W1 -q gradle-enterprise.aws.square.com'.execute( ) def isCacheUnreachable = !pingCache.waitFor(2, TimeUnit.SECONDS) | | pingCache.exitValue() == 6 8 if (isCacheUnreachable && isVpnCheckEnabled) { throw new GradleException(... ) } url = 'https://gradle-build-cache.aws.square.com/cache/ ' }
  99. void addGradleBuildMetadata() { def filteredProperties = project.propertie s .findAll {

    def key = it.ke y key.startsWith('android.') | | key.startsWith('org.gradle.') | | key.startsWith('com.squareup.') | | key.startsWith('app.cash.') | | key.startsWith('kotlin.') | | key.startsWith('kapt.' ) } filteredProperties.sort()*.ke y .forEach { key - > println("$key = ${filteredProperties[key]}" ) buildScan.value("Property: $key", "${filteredProperties[key]}" ) } }
  100. None
  101. None
  102. None
  103. None
  104. http://bit.ly/34ZN5UQ

  105. https://github.com/JetBrains/kotlin/blob/master/libraries/tools/kotlin-gradle-plugin/src/main/kotlin/org/ jetbrains/kotlin/gradle/tasks/Tasks.kt#L223

  106. Recap of Build Performance Essentials Build Scans = your best

    friend Daemon Parallel Max Workers Configuration On Demand
  107. None
  108. None
  109. None
  110. None
  111. A Journey in Android Developer Productivity Cash App & Gradle

    @jrodbx @DJRooz