Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cache once and use everywhere

Cache once and use everywhere

We, developers, write lots of code and compile a lot. We get others' contributions and compile them as well. Most of our time is wasted while staring at compile output without any productive outcome.

Do you ever feel that you compile the same thing over and over? Do you hate Gradle without knowing why? You are right, most of the compilations are unnecessary. And the majority of the problems are simple ones.

By fixing simple build configurations we can utilize local and remote Gradle cache and enjoy incremental builds with less time and focus on developing more.

Sinan Kozak

October 07, 2022
Tweet

More Decks by Sinan Kozak

Other Decks in Programming

Transcript

  1. Cache Once and
    Use Everywhere
    Sinan Kozak
    Delivery Hero
    @snnkzk
    Leveraging Gradle build cache

    View full-size slide

  2. How to speed up Android build times?
    Avoid building as much as possible

    View full-size slide

  3. Lifecycle of build
    Not every compile is equal
    1. Pull the source code
    2. Start build
    3. Download dependencies
    4. Compile without any cache
    5. Change something
    6. Build everything again

    View full-size slide

  4. Gradle task execution output
    Execution of task can be avoided
    ● No source
    ● From source
    ● Up to date
    ● Incremental from build output
    ● Local cache
    ● Remote cache

    View full-size slide

  5. Cache is hardware or software component that stores data
    so that future requests for that data can be served faster;
    the data stored in a cache might be the result of an earlier
    computation or a copy of data stored elsewhere.
    https://en.wikipedia.org/wiki/Cache_(computing)
    What is cache?

    View full-size slide

  6. Which cache is useful
    for development?
    There are different caches

    Android Studio

    Downloaded dependencies

    Android build pipeline

    Incremental build

    Gradle configuration

    Gradle build

    View full-size slide

  7. Which cache is broken?
    What is not related to build
    Android Studio cache
    Build output
    ./gradlew clean
    Incremental build

    View full-size slide

  8. The Gradle build cache is a cache mechanism that aims to
    save time by reusing outputs produced by other builds.
    The build cache works by storing (locally or remotely) build
    outputs and allowing builds to fetch these outputs from the
    cache when it is determined that inputs have not changed,
    avoiding the expensive work of regenerating them.
    https://docs.gradle.org/current/userguide/build_cache.html

    View full-size slide

  9. Benefits of cache
    Remote and Local cache usage

    Less compile

    Less time to develop

    Less context switch because of waiting

    Less energy usage

    Less money spending
    More productivity ☀🌈

    View full-size slide

  10. “The world ain't all sunshine and rainbows”
    Rocky Balboa

    View full-size slide

  11. Why does build miss cache?
    Some examples
    Not deterministic generated files

    Time

    Commit hash

    Machine host name
    Non deterministic configuration

    View full-size slide

  12. Cache miss
    About Gradle tasks
    No input or output
    Non deterministic task input
    Same input and output target
    Not properly declared input or output
    Room library - https://issuetracker.google.com/issues/132245929

    View full-size slide

  13. How to fix cache issues?
    Is it easy to find cache miss reason?

    View full-size slide

  14. Cache on CI
    Local vs CI

    Single source for cache

    Consistent cache without local
    changes

    Less upload and bandwidth usage
    for local devices

    Only affected Unit Tests will run

    View full-size slide

  15. Gradle remote cache server
    How can I use?
    A.
    Gradle Enterprise
    ○ Build in cache server node
    B.
    You can manage own cache server with persistent storage and
    ○ Jar
    ○ Docker
    ○ Kubernetes
    https://docs.gradle.com/build-cache-node/

    View full-size slide

  16. Don’t want to setup a server;
    Use Google Cloud Storage buckets
    https://github.com/androidx/gcp-gradle-build-cache

    View full-size slide

  17. AndroidX repo uses Google Cloud Storage
    solution.
    https://github.com/androidx/androidx

    View full-size slide

  18. Is remote cache always better?
    Local build can be faster
    Some tasks could execute faster than using remote cache
    Copy, package, merge …

    View full-size slide

  19. Cache or not to Cache
    Is cache a Golden Hammer? 🔨

    DataBindingMergeDependencyArtifacts

    BundleLibraryClasses

    BundleLibraryClassesDir
    Those were cacheable until AGP 7.2.0-alpha06
    Executing them locally is unlikely slower than cache usage
    Android Gradle Cache fix plugin helps lots of issues
    https://github.com/gradle/android-cache-fix-gradle-plugin

    View full-size slide

  20. Lifecycle of remote cache
    Download and zip could be slow
    Remote cache
    ● Download
    ● Unzip
    ● Use
    Big cache entries will also affect network usage and bandwidth usage
    Gradle Doctor helps to detect negative gains
    https://github.com/runningcode/gradle-doctor

    View full-size slide

  21. Gradle build cache
    How much we can reuse?
    Hitting 80%-90% range of cache usage is great
    When we started our remote cache usage was 30%
    Our current cache usage is above 85%

    View full-size slide

  22. How to go forward?
    Disabling cache should be a temporary solution
    It will cost more time and money in long run
    If you have reproducible case, please report issue
    Millions of developers will benefit from it
    Consider remote cache and calculate the difference between
    Start with suitable solution

    View full-size slide

  23. Grazie mille
    Q/A?
    Sinan Kozak
    Delivery Hero
    @snnkzk

    View full-size slide