Lightning fast Android builds with Gradle + Buck

Lightning fast Android builds with Gradle + Buck

Gradle is the typical build system of choice for android development, but it's performance tends to degrade as code bases become more complex. There exist alternate build systems like Buck (built by Facebook) that can build significantly faster than Gradle for Android, but they can be hard to migrate to and maintain. What if it is possible to have both the simplicity of expressing build logic in Gradle and also the fast build speeds of Buck? In this talk, Gautam will go over the open source OkBuck Gradle plugin and how it can help improve your android build speeds without having to learn and maintain a new build system.

3f9ebfb254c56d281d91bbee960219a8?s=128

Gautam Korlam

January 31, 2017
Tweet

Transcript

  1. 1.

    Lightning fast Android builds with Gradle + Buck Gautam Korlam

    Android Developer Experience, Uber Slides: https://goo.gl/JXrjk3
  2. 3.

    Gradle for Android Development • Most widely used build system

    for Android • Officially recommended by Google • Tools ecosystem - Android Gradle Plugin, Android Studio, Third party plugins • Easy to understand, maintain and extend
  3. 5.

    Single Gradle Module Pros • Simple to start a project

    • Fast gradle configuration time Cons • Builds start becoming slower as code size grows • Instant run/incremental compilation - do not work with all annotation processors
  4. 6.

    Multiple Remote Dependencies Pros • Reusable and focused dependencies •

    No need to build and test precompiled artifacts Cons • Making changes in them is no longer atomic • Dependency Hell
  5. 7.

    Multiple Gradle Modules Pros • No dependency hell • Build

    and test as needed Cons • Configuration and Gradle sync times start to grow • Memory usage goes up a lot
  6. 9.

    100+ modules • We found that gradle does not scale

    very well with large android projects • A single line change can take several minutes to build and test • Daemon, parallel builds and configuration on demand help, but not by much • Gradle build caching/better incremental builds are still not quite there yet • As code is split into smaller modules, Android Studio becomes slow
  7. 11.

    Alternative Build Systems • Buck (Facebook), Bazel (Google) and Pants

    (Twitter) • Focused around what to produce rather than what to run • Efficient Parallelization of projects with many small modules • Cache outputs and avoid rebuilding if public api does not change • Advanced capabilities like remote build cache
  8. 12.

    Why Buck? • Most mature among all and actively developed

    • Heaviest focus on mobile and scaling complex builds • Closest to the Android Gradle Plugin in terms of supported features • Provides fast and correct incremental builds and tests for Android Development • Used at Uber for building iOS as well https://buckbuild.com
  9. 13.

    How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  10. 14.

    How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  11. 15.

    How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  12. 16.

    How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  13. 17.

    Hidden Cost of Build Time 3 mins per iteration 25

    Hours spent per day 30 secs per iteration 4 Hours spent per day 20 Developers Build and Deploy 25 times a day VS 2 Extra Developers!
  14. 18.

    Android Builds at Uber • 300+ gradle modules in project

    • Clean buck builds are 180x faster than gradle (network cache!) • Incremental buck builds are ~8x faster than gradle • Most day to day tasks benefit greatly from the network cache • Clean builds are rare. CI builds are fully incremental with buck https://github.com/uber/buck-http-cache
  15. 22.

    Other Considerations • Migrating completely can be painful and time

    consuming • Need to rewrite a lot of existing tools • Engineers need to learn a new build system • Isolation from Android tools, knowledge and community focused around gradle • Cannot take coffee breaks during builds
  16. 24.

    OkBuck • Open source gradle plugin that lets you build

    gradle projects with buck! • Generates buck build files for complex android gradle projects seamlessly • Provides many configuration options for customizing generated BUCK files • Bridges the gap between simplicity of gradle and complexity of buck • Used at Uber for building android projects with buck https://github.com/uber/okbuck
  17. 25.

    Under the Hood • Creates a model of the gradle

    dependency graph in memory • Downloads external dependencies to a local cache • Translates the project’s gradle task model into buck’s rule based model
  18. 26.
  19. 27.
  20. 28.

    Seamless Transition • Extracts android gradle plugin options to use

    during buck builds ◦ Sources, resources and manifests from flavors/build types ◦ Annotation processors ◦ Consumer proguard rules ◦ Custom lint jars • Provides a wrapper around buck similar to the gradle wrapper • Wrapper downloads/installs/updates buck • Watches project changes and automatically regenerates BUCK files if needed
  21. 29.

    • Build Flavors/Variants • Multidex • Exopackage • Annotation Processors

    • Proguard • Manifest Placeholders • Retrolambda • Java Unit Tests • Lint Supported Features What is Supported? • Robolectric Unit Tests • Espresso UI Tests • Aidl • Code Coverage • Intellij Project Generation • Custom Gradle Plugin Hooks • Vector Drawables • Custom BuildConfig Fields • Transform API • Flavor Dimensions • Kotlin • Databinding • NDK Integration • Jack Not Supported Yet (Planned)
  22. 39.

    Configure Buck to Build Fast • Move all dependencies to

    a top level configuration file • Split up your project into a number of smaller modules • Disable unnecessary gradle plugins when generating BUCK files • Many configuration options can be tweaked in the.buckconfig file • Invest in adding exopackage support to your apps https://buckbuild.com/concept/buckconfig.html
  23. 40.

    Exopackage • An exopackage is a small shell of an

    Android app that contains the minimal code and resources needed to run • Loading the application code at runtime avoids a full reinstall when testing a Java change • < 15s average incremental deploy https://buckbuild.com/article/exopackage.html
  24. 41.

    Making code modular • Breakup the gigantic app module into

    smaller focused modules • Less frequently touched code is a good candidate for separation • Keeping top level app/apk module small allows for fastest exopackage builds • Smaller modules build/test faster and cache better • buck suggest gives possible ways to untangle class level dependencies https://buckbuild.com/command/suggest.html
  25. 42.

    Setup your IDE • Install the Buck Intellij Plugin •

    Lets you perform frequent tasks like deploying apks and debugging • Configure the Buck Plugin to invoke all buck commands via the wrapper • Intellij Community Edition plays nicer with buck projects than Android Studio • This means no more Gradle Sync! https://plugins.jetbrains.com/plugin/7826?pr=idea
  26. 43.

    What’s Next? • Full compatibility with android gradle plugin •

    Better Kotlin Support • More docs, samples and best practices • Fully open source most of our android tooling stack