Lightning fast Android builds with Gradle + Buck

Lightning fast Android builds with Gradle + Buck

Gradle is the typical build system of choice for android development, but it's performance tends to degrade as code bases become more complex. There exist alternate build systems like Buck (built by Facebook) that can build significantly faster than Gradle for Android, but they can be hard to migrate to and maintain. What if it is possible to have both the simplicity of expressing build logic in Gradle and also the fast build speeds of Buck? In this talk, Gautam will go over the open source OkBuck Gradle plugin and how it can help improve your android build speeds without having to learn and maintain a new build system.

3f9ebfb254c56d281d91bbee960219a8?s=128

Gautam Korlam

January 31, 2017
Tweet

Transcript

  1. Lightning fast Android builds with Gradle + Buck Gautam Korlam

    Android Developer Experience, Uber Slides: https://goo.gl/JXrjk3
  2. What do you use to build Android Apps?

  3. Gradle for Android Development • Most widely used build system

    for Android • Officially recommended by Google • Tools ecosystem - Android Gradle Plugin, Android Studio, Third party plugins • Easy to understand, maintain and extend
  4. Gradle Project Structure

  5. Single Gradle Module Pros • Simple to start a project

    • Fast gradle configuration time Cons • Builds start becoming slower as code size grows • Instant run/incremental compilation - do not work with all annotation processors
  6. Multiple Remote Dependencies Pros • Reusable and focused dependencies •

    No need to build and test precompiled artifacts Cons • Making changes in them is no longer atomic • Dependency Hell
  7. Multiple Gradle Modules Pros • No dependency hell • Build

    and test as needed Cons • Configuration and Gradle sync times start to grow • Memory usage goes up a lot
  8. How does Gradle Scale?

  9. 100+ modules • We found that gradle does not scale

    very well with large android projects • A single line change can take several minutes to build and test • Daemon, parallel builds and configuration on demand help, but not by much • Gradle build caching/better incremental builds are still not quite there yet • As code is split into smaller modules, Android Studio becomes slow
  10. Waiting for Gradle builds Good, we’ll wait for our gradle

    builds here!
  11. Alternative Build Systems • Buck (Facebook), Bazel (Google) and Pants

    (Twitter) • Focused around what to produce rather than what to run • Efficient Parallelization of projects with many small modules • Cache outputs and avoid rebuilding if public api does not change • Advanced capabilities like remote build cache
  12. Why Buck? • Most mature among all and actively developed

    • Heaviest focus on mobile and scaling complex builds • Closest to the Android Gradle Plugin in terms of supported features • Provides fast and correct incremental builds and tests for Android Development • Used at Uber for building iOS as well https://buckbuild.com
  13. How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  14. How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  15. How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  16. How Fast is Buck? Build all Apps Gradle Buck Speed

    Up Clean 8 mins 40 secs 3 mins 20 secs 2.6x Incremental - ABI Change 3 mins 47 secs 43 secs 5.3x Incremental - Same ABI 2 mins 47 secs 30 secs 5.6x No-op 3 mins 38 secs 0.2 secs 1090x • Test project on github with many gradle modules • Can be built with both gradle and buck https://github.com/kageiit/android-studio-gradle-test Gradle - 3.4-rc-1 Android Gradle Plugin - 2.3.0-beta3
  17. Hidden Cost of Build Time 3 mins per iteration 25

    Hours spent per day 30 secs per iteration 4 Hours spent per day 20 Developers Build and Deploy 25 times a day VS 2 Extra Developers!
  18. Android Builds at Uber • 300+ gradle modules in project

    • Clean buck builds are 180x faster than gradle (network cache!) • Incremental buck builds are ~8x faster than gradle • Most day to day tasks benefit greatly from the network cache • Clean builds are rare. CI builds are fully incremental with buck https://github.com/uber/buck-http-cache
  19. Sounds awesome but why is buck not used widely?

  20. Verbose Build Configuration

  21. No Dependency Resolution

  22. Other Considerations • Migrating completely can be painful and time

    consuming • Need to rewrite a lot of existing tools • Engineers need to learn a new build system • Isolation from Android tools, knowledge and community focused around gradle • Cannot take coffee breaks during builds
  23. Simplicity of Gradle + Fast Build Speeds of Buck

  24. OkBuck • Open source gradle plugin that lets you build

    gradle projects with buck! • Generates buck build files for complex android gradle projects seamlessly • Provides many configuration options for customizing generated BUCK files • Bridges the gap between simplicity of gradle and complexity of buck • Used at Uber for building android projects with buck https://github.com/uber/okbuck
  25. Under the Hood • Creates a model of the gradle

    dependency graph in memory • Downloads external dependencies to a local cache • Translates the project’s gradle task model into buck’s rule based model
  26. None
  27. None
  28. Seamless Transition • Extracts android gradle plugin options to use

    during buck builds ◦ Sources, resources and manifests from flavors/build types ◦ Annotation processors ◦ Consumer proguard rules ◦ Custom lint jars • Provides a wrapper around buck similar to the gradle wrapper • Wrapper downloads/installs/updates buck • Watches project changes and automatically regenerates BUCK files if needed
  29. • Build Flavors/Variants • Multidex • Exopackage • Annotation Processors

    • Proguard • Manifest Placeholders • Retrolambda • Java Unit Tests • Lint Supported Features What is Supported? • Robolectric Unit Tests • Espresso UI Tests • Aidl • Code Coverage • Intellij Project Generation • Custom Gradle Plugin Hooks • Vector Drawables • Custom BuildConfig Fields • Transform API • Flavor Dimensions • Kotlin • Databinding • NDK Integration • Jack Not Supported Yet (Planned)
  30. Migration Guide

  31. Apply the Plugin

  32. Quick Start

  33. Quick Start

  34. Quick Start

  35. Quick Start

  36. Quick Start

  37. Quick Start

  38. Quick Start https://github.com/uber/okbuck/blob/master/Usage.md

  39. Configure Buck to Build Fast • Move all dependencies to

    a top level configuration file • Split up your project into a number of smaller modules • Disable unnecessary gradle plugins when generating BUCK files • Many configuration options can be tweaked in the.buckconfig file • Invest in adding exopackage support to your apps https://buckbuild.com/concept/buckconfig.html
  40. Exopackage • An exopackage is a small shell of an

    Android app that contains the minimal code and resources needed to run • Loading the application code at runtime avoids a full reinstall when testing a Java change • < 15s average incremental deploy https://buckbuild.com/article/exopackage.html
  41. Making code modular • Breakup the gigantic app module into

    smaller focused modules • Less frequently touched code is a good candidate for separation • Keeping top level app/apk module small allows for fastest exopackage builds • Smaller modules build/test faster and cache better • buck suggest gives possible ways to untangle class level dependencies https://buckbuild.com/command/suggest.html
  42. Setup your IDE • Install the Buck Intellij Plugin •

    Lets you perform frequent tasks like deploying apks and debugging • Configure the Buck Plugin to invoke all buck commands via the wrapper • Intellij Community Edition plays nicer with buck projects than Android Studio • This means no more Gradle Sync! https://plugins.jetbrains.com/plugin/7826?pr=idea
  43. What’s Next? • Full compatibility with android gradle plugin •

    Better Kotlin Support • More docs, samples and best practices • Fully open source most of our android tooling stack
  44. Who is using Okbuck + Buck?

  45. Questions? Thank You @kageiit