Embracing Productivity Engineering in Android

Productivity Engineering is fast becoming an integral part of the development & release process in the Software Engineering world today. The benefits of implementing Productivity Engineering significantly outweigh the cost and frustration of inactivity in this space.

In this session, we will discuss the importance of productivity engineering and why Android developers & teams should embrace productivity engineering as part of their development lifecycle.

We will cover topics on how to make a compelling case for productivity engineering to your team and manager using data, we will also cover tips to improve your build, work with build scans to quickly find build problems & collaborate with team members to resolve them, and build caching to significantly speed-up builds.

I will take you through a real-life example of using Gradle Enterprise to diagnose builds, improve build experience, and make a business case for a focus on Productivity Engineering.

You will leave this talk with a better understanding and appreciation for Productivity Engineering. You will also learn the current tips & tools on how to improve your android builds and the value proposition of Gradle Enterprise to make data-driven decisions to continuously improve your build process and developer productivity.

I gave this talk at Droidcon APAC 2020

Transcript

1. 1 14/12/20 - DroidCon APAC
3. 3 • Definition of Productivity Engineering. • How to get buy-in and make a compelling case for Productivity Engineering to your manager & team. • Gradle Enterprise - to diagnose builds and improve the build experience. • Tips and Tools to improve your Android Builds
4. 4 • Are you currently experiencing long build times? • Are you currently working on a project and no one looks into the build? • Have inherited an old codebase? • Are you working in a team where the developers’ productivity is hindered by the build process? • Are you currently looking to improve your knowledge on Productivity Engineering?
5. 5 • The use of engineering practices to solve problems around developer productivity. • The discipline of using data to optimize software development practices, thereby improving the happiness and enhancing the productivity of software developers.
6. 6 Definition
7. 7 • The job of developer productivity engineering is to improve productivity through the entire lifecycle from the IDE to production. • For a team to be successful, it needs a culture where the whole organization commits to improving developer productivity.
8. 8 • There is a significant gap between the actual productivity of software development teams and their full potential. • Bridges this gap. • The gap is growing over the lifetime of a project for most organizations but it can be significantly reduced with the practice of developer productivity engineering.
9. 9 • The happiness of your developers depends on their productivity. • Organizations that cannot provide such an environment are seeing a drain of talent leading to even lower productivity.
10. 10 • The economic benefits of not applying this practice are dramatic. • The amount of features you can ship is heavily affected by it. • The productivity gap that comes from not applying the practice of developer productivity engineering is a significant competitive disadvantage. • That means the productivity of your software development teams is crucial for your business to survive.
11. 11 DPE Landscape
12. 12 IDENTIFY • Identify productivity bottlenecks. MEASURE • Measure Productivity. PRIORITIZE • Prioritize according to cost.
13. 13 13
14. 14 ▪ Average Build Times ▪ PR Build - 30-40min ▪ Release Build - 50-60min ▪ Gradle Version - 4.10.2 ▪ Release Date - Sep 19, 2018 ▪ AGP Version - 3.3.1 ▪ Release Date - February 2019 ▪ Broken local build for tests. ▪ Team Size - 5 engineers ▪ Size of the project - 8 Modules and 187606 LOC
15. 15 • Painfully long build times. • Long feedback cycles for test feedback. • Long waiting times for merging PRs, QA builds to be distributed and RC builds to be cut. • Significantly negative impact on engineers productivity. • Flaky builds - Broken builds and Build timeouts.
16. 16 16
17. 17 • Installing the Gradle Enterprise Server (GCP Instance) • Collecting Baseline Data using Build Scans • Proper Configuration • Local and Remote Cache Work • Cache Node Replication
18. 18 • Gradle Task • ./gradlew assembleDebug --scan https://scans.gradle.com/
19. 19 1. Incremental build - in between consecutive runs. 2. Local Cache. 3. Remote Cache.
20. 20 • Helps improve build cache speeds • Reduces round trip time to cache node server
21. 21 During Premise Data’s trial of Gradle Enterprise… • Average build time on CI went from 23 min down to 8.27 min per build • Average local build time went from 3.16 min down to 1.33 min per build • Build scans have been intensively used to diagnose, collaborate on, and fix build issues • Discovered incorrect heap size configuration causing high GC • Discovered mismatched JDK spawning multiple daemons
22. 22 During Premise Data’s trial of Gradle Enterprise… • it was found that a large portion of unit tests were disabled for the project • it was found that < 10% of unit tests account for 60% of test execution time • we uncovered inefficiencies in our build task configurations • we uncovered inefficiencies and duplication of our QA/Release build distribution process • we were able to quickly discover and diagnose build regressions due to NDK introduction
23. 23 • Accelerating failure analysis and resolution on local and CI builds ◦ Collaborating on build issues via build scans ◦ Investigating build failures via build scans ◦ Investigating slow builds via build scans • Accelerating feedback cycle ◦ Improving CI pipeline configuration ◦ Diagnosing and improving build cache performance ◦ Insights for switching CI providers (Travis queue time is most of the wait) • Enabling changes ◦ Better observability leads to more confidence in making changes to build
25. 25 Ensure that your project is using the latest version of the following: • Gradle • Android Gradle Plugin • Gradle Enterprise Plugin * • Kotlin • 3rd Party Plugins • 3rd Party libraries
26. 26 • This plugin provides a task to determine which dependencies have updates. • It also checks for updates to Gradle itself. • Gradle Task • ./gradlew dependencyUpdates -Drevision=release Github URL: https://github.com/ben-manes/gradle-versions-plugin
27. 27 You can use this plugin to identify/address certain issues in your project: • Configurable warnings for build speed problems • Ensure JAVA_HOME is set and matches IDE's JAVA_HOME Github URL: https://github.com/runningcode/gradle-doctor
28. 28 Get understanding and buy-in early on from major stakeholders by communicating the value of Productivity Engineering • Do small experiments that clearly communicate the value • Share findings with major stakeholders (Manager, Team) to get buy in. NOTE: Productiving Eng is Tech Debt.
29. 29 • Engineering Needs Sync (Bi-Weekly) • Epic on JIRA (Android Build Performance)
30. 30 • Local Build Cache • Remote Build Cache • A combination of Local and Remote build cache can significantly improve your build times. Before Caching After Caching
31. 31 • Some Android plugin versions have issues with Gradle's build cache feature. • This results in build cache misses for some tasks affecting build speeds. • When this plugin is added to an Android project, it applies workarounds for these issues based on the Android plugin and Gradle versions. Github URL: https://github.com/gradle/android-cache-fix-gradle-plugin
32. 32 • Successful migration from Travis CI to Github Actions. • Results: • Improvements in the CI build pipeline. • Running full test suite for every PR.
33. 33 • Making benchmarking accessible to all • Make Gradle Enterprise to all • Documentation • Lunch and Learn (Knowledge Share team syncs)
34. 34 • Configuration Caching • File System Watching • Update amount of memory allocated for CI builds • IDE Optimizations
35. 35 ▪ Average Build Times ▪ PR Build - 9 min (vs 30min) ▪ Release Build - 15min (vs 50min) ▪ Gradle Version - 6.7.1 ▪ AGP Version - 4.1.0 ▪ Team Size - 9 engineers ▪ Size of the project - 10 Modules and 247,655 LOC
36. 36 ▪ Measure what you want to Improve ▪ Productivity Engineering is Tech Debt. Plan for it. ▪ Use Experiments and Data to get Buy-in ▪ Use the available build speed acceleration technologies. ▪ Productivity Engineering is a continuous process and not a one off event.
37. 37 37