Android Builds: Simple Recipes for faster feedback loop

Slide 1

Slide 1 text

Android Builds: Simple recipes for faster feedback loop Aida Issayeva @aida_say

Slide 2

Slide 2 text

About me ● 9 years as an Android engineer ● Staff Software Engineer at Toast, Inc ● Google Developer Expert for Android ● Women Techmakers Ambassador

Slide 3

Slide 3 text

Table of Contents 1 Why care? 2 Bottlenecks and fixes a locally in a project b remotely on CI 3 Recap

Slide 4

Slide 4 text

Context: POS App 1 2 a b 3 294 modules 110 contributors >2M lines of code 150 avg remote builds/day

Slide 5

Slide 5 text

Context: Tools 1 2 a b 3

Slide 6

Slide 6 text

Why care? 1 2 a b 3 2022 H1 2023 227% Contributors Number of remote builds 305% Avg build time in mins 51.5 73.88 44%

Slide 7

Slide 7 text

1 2 a b 3 Main priority is a build acceleration. for us

Slide 8

Slide 8 text

2. Bottlenecks 1 2 a b 3 LOCAL REMOTE

Slide 9

Slide 9 text

2a. Local bottlenecks 1 2 a b 3 gradle.properties unused dependencies outdated dependencies

Slide 10

Slide 10 text

2a. Callout: gradle.properties 1 2 a b 3 No space between key and value

Slide 11

Slide 11 text

2a. Solution: Parallel tasks 1 2 a b 3 org.gradle.parallel=true

Slide 12

Slide 12 text

2a. Solution: Gradle cache 1 2 a b 3 org.gradle.caching=true

Slide 13

Slide 13 text

2a. Solution: Memory configuration org.gradle.jvmargs=-Xmx10g 1 2 a b 3 heap size

Slide 14

Slide 14 text

2a. Solution: Memory configuration org.gradle.jvmargs=-Xmx10g -Xms10g 1 2 a b 3 max heap size initial heap size

Slide 15

Slide 15 text

2a. Solution: Initial heap size calculation 1 2 a b 3 100% 80% Xms Xmx

Slide 16

Slide 16 text

2a. Solution: Max heap size calculation 1 2 a b 3 100% 70% JVM Memory Xmx Metaspace and other native memory Hungriest task

Slide 17

Slide 17 text

2a. Local bottlenecks: Gradle Workers 1 2 a b 3 CPU Core 1 CPU Core 2 CPU Core 3 Gradle Worker 1 Gradle Worker 2 Gradle Worker 3 … OutOfMemoryError

Slide 18

Slide 18 text

2a. Solution: Gradle Workers Count Calculation 1 2 a b 3 org.gradle.workers.max=8 Total CPU cores/1.5~2

Slide 19

Slide 19 text

2a. Local bottlenecks: Unused Dependencies 1 2 a b 3 Dependency resolution Downloading artifacts Configuration time uncached Classpath size

Slide 20

Slide 20 text

2a. Local bottlenecks: Outdated Dependencies 1 2 a b 3 Inefficient build processes Increased resolution time Compatibility issues Deprecated features

Slide 21

Slide 21 text

2a. Solution: Unused & Outdated Dependencies 1 2 a b 3 Automated dependency management: Use plugins (Dependabot, etc)

Slide 22

Slide 22 text

2b. Remote Bottlenecks 1 2 a b 3 outdated hardware container configuration + plugins' settings inefficient scripts applicable locally too

Slide 23

Slide 23 text

2b. Remote bottlenecks: Hardware age 1 2 a b 3 RAM CPU

Slide 24

Slide 24 text

2b. Solution: Hardware age 1 2 a b 3 CPU * More RAM * More > 10 > 32 = Optimal Performance* *Hardware

Slide 25

Slide 25 text

2b. Solution: Jenkins Agent Required Memory 1 2 a b 3 (XmxSize + XmxSize*30%) + ((max.workers * XmxSize)* 30%) + ((1 + 2) * 30%) = optimal memory JVM and OS overhead Daemon 1 2 % of all workers at max heap not modularized project

Slide 26

Slide 26 text

2b. Remote bottlenecks: Container Settings & Config 1 2 a b 3 Container startup time Resource Allocation Old Instance Wait times

Slide 27

Slide 27 text

2b. Solution: Container Settings & Config 1 2 a b 3 1.Allocate high number of agents 2.Keep agents warm 3.Migrate to a newer instance

Slide 28

Slide 28 text

2b. Remote bottlenecks: Jenkins Plugin Settings 1 2 a b 3 Build Queue Congestion Build trigger on PR metadata change

Slide 29

Slide 29 text

2b. Solution: Jenkins Plugin Settings 1 2 a b 3 1. Don't trigger a build on PR metadata change 2. Cancel ongoing builds after a new commit is pushed

Slide 30

Slide 30 text

2b. Remote bottlenecks: inefficient CI scripts 1 2 a b 3 Serial execution of most stages Timeouts on different plugin responses Staggered gradle tasks

Slide 31

Slide 31 text

2b. Remote bottlenecks: inefficient CI scripts 1 2 a b 3 1. Parallelize stages 2. Investigate and reduce timeouts 3. Combine logically related gradle tasks

Slide 32

Slide 32 text

Local: - enable parallel - enable cache - optimize memory config - manage gradle workers 1 2 a b 3 3. Recap Remote: - upgrade hardware - optimize container settings - fine-tune plugin events - improve CI scripts

Slide 33

Slide 33 text

1 2 a b 3 3. Recap Before H2 2023 After H2 2023 After Q1 2024 43% 31% 61%

Slide 34

Slide 34 text

1. Dependency Analysis Gradle Plugin - unused & transitive dependencies 2. Dependabot - automated dependency updates in Github 3. RenovateBot 4. Gradle Profiler - profiling & benchmarking gradle builds 5. Gradle Process Plugin - information about gradle processes 1 2 a b 3 3. Resources: Plugins & Tools

Slide 35

Slide 35 text

1. Java Memory model specification 2. Introduction to Java's Memory model 3. Essential Tips to minimize Gradle Build Time 4. From laptops to advanced CI 5. Managing nodes in Jenkins 1 2 a b 3 3. Resources: References

Slide 36

Slide 36 text

THANK YOU @aida_isay Q&A Let's connect