The Road To Single Dex (GDG San Francisco Meetup)

Lean Android applications with small APK sizes and low method counts are hard to come by nowadays as some of the most used libraries such as AppCompat and Play Services continue to grow in size. In an increasingly mobile first world, it important to try and maintain a lean application in order to avoid loading extra DEX files, slowing down local development builds and increasing the size of the updates you ship to your customers.

This talk is centered around how I was able to lower Yammer for Android’s method count down to a single DEX and will give several tips on how to help you lower the number of methods and overall APK size of your application.

Meetup: https://www.meetup.com/google-developer-group-san-francisco/events/244566100

Transcript

1. Jared Burrows The Road to Single Dex Jared Burrows
2. Motivation • Yammer is a social networking service used for communication within an organization • 3 APKs: Min SDK 15, 16 and 19, Target SDK 26 • Loading extra DEX files may lead to possible ANRs • Multidex makes very large memory allocation requests may crash during run time
3. Focus • Reducing DEX method count • Reducing APK size
4. Results 1 DEX file, Smaller APK, All features kept! Before After Reduction Method Count 85k+ 60.8k 28.47% APK Size 10.7MB+ 7.1MB 33.64%
5. What makes an APK so big? 
 Multiple DEX files + Many Resources
6. What is inside an APK? • AndroidManifest.xml • assets/ • classes.dex • META-INF/
 - CERT.SF
 - MANIFEST.MF
 - CERT.RSA • lib/ • res/
 - drawable/
 - layout/ • resources.arsc
7. Big APKs - DEX size • Remove all/any dead code • Remove any old/unused libraries • Refactor out large/non-mobile libraries • Verify Gradle configurations
8. Big APKs - Resources • Remove extra and unused files in your “/res” folder • Remove extra and used files in your “/assets” folder
9. Before Optimizations - Debug build $ gradlew countDebugDexMethods sizeDebugApk
 
 > Task :countDebugDexMethods
 Total methods in debug.apk: 113007 (172.44% used) 
 Total fields in debug.apk: 50547 (77.13% used) 
 Methods remaining in debug.apk: 0 
 Fields remaining in debug.apk: 14988
 
 > Task :sizeDebugApk
 Total APK Size in debug.apk in bytes: 12386152 (12.3MB)
10. Before Optimizations - Release build $ gradlew countReleaseDexMethods sizeReleaseApk (with Proguard)
 
 > Task :countReleaseDexMethods
 Total methods in release.apk: 85259 (130.10% used) 
 Total fields in release.apk: 39887 (60.86% used) 
 Methods remaining in release.apk: 0 
 Fields remaining in release.apk: 25648
 
 > Task :sizeReleaseApk
 Total APK Size in release.apk in bytes: 10764242 (10.7MB)
11. Baselined Results 85k+ > 64k DEX limit = 2 DEX files Debug Release Method Count 113k+ 85k+ APK Size 12.3MB+ 10.7MB+
12. Optimizing the DEX size
13. Trimming the DEX size - Dead Code • Remove any and all unused code
 - PMD - Comes with Gradle
 - Proguard - Comes with Android Gradle Plugin
 - Intellij/Android Studio Inspections - Click “Analyze” > Click “Inspect Code…” • Remove all unused libraries
 - We chose HockeyApp over an open source analytics library called ApplicationInsights
 - HockeyApp’s functionality encompasses all of the functionality that we need from ApplicationInsights
14. Trimming the DEX size - Old Libraries • Remove old libraries
 - Using HockeyApp for releasing versioning/nightlies - Allowed us to remove AndroidQuery - version checking • Remove unnecessary libraries
 - Try not add libraries that duplicate Android’s existing functionality
 - Able to remove Apache Commons validator - validate emails addresses • Removing differential versions of same library
 - Jackson 1 and Jackson 2 are the same library but Jackson 2 has a different package name
 - Duplicating functionality across across prevent tools such as Proguard from optimizing
15. Trimming the DEX size - Transitive Libraries • Watch out for Library Duplication
 - Jackson 1 vs Jackson 2 vs Gson
 - Consider using a Groovy map in your build.gradle • Bringing in extra libraries based on a libraries’ dependencies
 - Use “gradlew dependencies” to see what you are actually using
16. Trimming the DEX size - Transitive Libraries ext {
 supportLibraryVersion = "27.0.0"
 }
 
 ext.deps = [
 "design" : "com.android.support:design:$supportLibraryVersion",
 "recyclerviewV7" : "com.android.support:recyclerview-v7:$supportLibraryVersion",
 "appcompatV7" : "com.android.support:appcompat-v7:$supportLibraryVersion",
 "cardviewV7" : "com.android.support:cardview-v7:$supportLibraryVersion"
 ] dependencies {
 compile deps.design
 } Keep track of dependencies, Enforce versions
17. Trimming the DEX size - Transitive Libraries gradlew dependencies --configuration compile* compile - Classpath for compiling the main sources. +--- project :yammer-common | \--- com.google.code.gson:gson:2.8.2 +--- project :yammer-data | \--- com.squareup.retrofit:retrofit:1.9.0 | \--- com.google.code.gson:gson:2.3.1 -> 2.8.2 +--- com.microsoft.aad:adal:1.12.0 | \--- com.google.code.gson:gson:2.2.4 -> 2.8.2
18. Trimming the DEX size - Large Libraries • Try not to use Large/Non-mobile libraries
 - Guava - 15k+ methods
 - Jackson2 2.9.0.pr3 - 12k+ methods
 - Check size of libraries with methodscount.com • Necessary Evils
 - AppCompat - 25.4.0 has 16.5k+ methods, 26.0.0-beta2 has 18.3k+ (continues to increase)
 - Google Play Services GCM 11.0.0 - 16.7k+ methods
 - Simply using both App Compat + Play Services together brings your APK to 30k+ methods, halfway to the 64k limit per DEX file!
19. Trimming the DEX size - MethodsCount methodscount.com/?lib=com.android.support:appcompat-v7:24.2.1
20. Trimming the DEX size - Gradle Configurations dependencies { compile project(":common")
 compile project(":common-test")
 } Pre Android Gradle Plugin 3.0
21. Trimming the DEX size - Gradle Configurations dependencies { compile project(":common")
 compile project(":common-test")
 } Pre Android Gradle Plugin 3.0
22. Trimming the DEX size - Gradle Configurations Pre Android Gradle Plugin 3.0 dependencies { compile project(":common")
 testCompile project(":common-test")
 }
23. Trimming the DEX size - Gradle Configurations dependencies { implementation project(":common")
 testImplementation project(":common-test")
 } With Android Gradle Plugin 3.0
24. Trimming the DEX size - Proguard • Use Proguard on Release builds
 - Set “minifyEnabled” to true!
 - Easiest way to help lower your DEX size • Consider using “proguard-android-optimize.txt” over “proguard-android.txt”
 - Proguard will make 5 optimization passes on your code
25. Trimming the DEX size - Proguard android {
 buildTypes {
 release {
 minifyEnabled true
 proguardFile getDefaultProguardFile("proguard-android.txt")
 proguardFile "custom-proguard-rules.txt"
 signingConfig signingConfigs.release
 }
 }
 }
 Make sure you turn on “minifyEnabled”
26. Trimming the DEX size - Proguard android {
 buildTypes {
 release {
 minifyEnabled true
 proguardFile getDefaultProguardFile("proguard-android.txt")
 proguardFile "custom-proguard-rules.txt"
 signingConfig signingConfigs.release
 }
 }
 }
 Make sure you turn on “minifyEnabled”
27. Trimming the DEX size - Proguard Make sure you provide Proguard rules android {
 buildTypes {
 release {
 minifyEnabled true
 proguardFile getDefaultProguardFile("proguard-android.txt")
 proguardFile "custom-proguard-rules.txt"
 signingConfig signingConfigs.release
 }
 }
 }

28. Trimming the DEX size - Proguard Try using more aggressive Proguard rules android {
 buildTypes {
 release {
 minifyEnabled true
 proguardFile getDefaultProguardFile("proguard-android-optimize.txt")
 proguardFile "custom-proguard-rules.txt"
 signingConfig signingConfigs.release
 }
 }
 }

29. Trimming the DEX size - Proguard • Remove extra Proguard rules • Keep them up to date with the current libraries in your app • Make sure to use “-keep” sparingly
30. Trimming the DEX size - Proguard Over using “-keep” can be costly -keep class okhttp3.* { ; }
 -keep interface okhttp3.* { ; }
 -dontwarn okhttp3.**
 -dontnote okhttp3.**
31. Trimming the DEX size - Proguard Over using “-keep” can be costly -keep class okhttp3.* { ; }
 -keep interface okhttp3.* { ; }
 -dontwarn okhttp3.**
 -dontnote okhttp3.**
32. Trimming the DEX size - Proguard Try and let Proguard decide -dontwarn okhttp3.**
 -dontnote okhttp3.**
33. Optimizing the Resources
34. Trimming the Resources - Lint • Make sure to use lint to check for “UnusedResources” • Do not suppress this lint rule unless you are writing an Android library • Consider failing the builds based on lint warnings/errors
35. Trimming the Resources - Lint Strict lint options android {
 lintOptions {
 textReport true
 textOutput "stdout"
 checkAllWarnings true
 warningsAsErrors true
 }
 }
36. Trimming the Resources - Lint android {
 lintOptions {
 error "UnusedResources"
 }
 } Less strict, check for “UnusedResources”
37. Trimming the Resources - ShrinkResources • What Lint does not find, you have to remove manually • Remove all unused resources in “assets/” folder
 - Raw media such as Font files • Use “shrinkResources” to remove unused resources
 - Great for removing library dependencies resources in your final APK
38. Trimming the Resources - ShrinkResources android {
 buildTypes {
 release {
 minifyEnabled true
 proguardFile getDefaultProguardFile("proguard-android-optimize.txt")
 proguardFile "custom-proguard-rules.txt"
 signingConfig signingConfigs.release
 }
 }
 }
 Missing “shrinkResources”
39. Trimming the Resources - ShrinkResources Added “shrinkResources” android {
 buildTypes {
 release {
 minifyEnabled true shrinkResources true
 proguardFile getDefaultProguardFile("proguard-android-optimize.txt")
 proguardFile "custom-proguard-rules.txt"
 signingConfig signingConfigs.release
 }
 }
 }

40. Trimming the Resources - ShrinkResources Using “shrinkResources” android {
 buildTypes {
 release {
 minifyEnabled true shrinkResources true
 proguardFile getDefaultProguardFile("proguard-android-optimize.txt")
 proguardFile "custom-proguard-rules.txt"
 signingConfig signingConfigs.release
 }
 }
 }

41. Trimming the Resources - ResConfigs • Use “resConfigs” to filter out localizations that you do not want/support in your app
42. Trimming the Resources - ResConfig Using “resConfigs” to keep multiple resources or languages android { defaultConfig { resConfigs "en" } }
43. Trimming the Resources - ResConfig Using “resConfigs” to keep multiple languages android { defaultConfig {
 resConfigs "en","de","fr"
 }
 }
44. Trimming the Resources - PackagingOptions • Remove extra files that are baked into your APK • Use the APK Analyzer to find these files
 - In Android Studio, Click “Build” > Click “Analyze APK…” • Use “packagingOptions” to filter out these files
45. Trimming the Resources - PackagingOptions android {
 packagingOptions {
 exclude "META-INF/LICENSE.txt"
 exclude "META-INF/NOTICE.txt"
 }
 } Common “packagingOptions” excludes
46. Trimming the Resources - PackagingOptions After using common “packagingOptions” excludes
47. Trimming the Resources - PackagingOptions Strict “packagingOptions” excludes android {
 packagingOptions {
 exclude "**/*.kotlin_module" exclude "**/*.version"
 exclude "**/kotlin/**"
 exclude "**/*.xml"
 exclude “**/*.txt" exclude "**/*.properties"
 }
 }
48. Trimming the Resources - PackagingOptions After using stricter “packagingOptions” excludes
49. Trimming the Resources - Vector Drawables • Consider converting simple PNGs to “Vector” Drawables • Single XML file is smaller than multiple PNGs • Use tools such as “svg2android” to convert SVGs to Android Vector Drawables
50. Trimming the Resources - Vector Drawables android {
 defaultConfig {
 resConfigs "en", "de", "fr"
 vectorDrawables {
 useSupportLibrary = true
 }
 }
 } Using “vectorDrawables.useSupportLibrary = true” to prevent PNG generation
51. Trimming the Resources - Vector Drawables android {
 defaultConfig {
 resConfigs "en", "de", "fr"
 vectorDrawables {
 useSupportLibrary = true
 }
 }
 } Using “vectorDrawables.useSupportLibrary = true” to prevent PNG generation
52. Trimming the Resources - Vector Drawables inloop.github.io/svg2android/
53. Trimming the Resources - Vector Drawables After dropping your SVG file, Copy it to your project in “res/drawable”
54. Trimming the Resources - Vector Drawables Make sure to use “app:srcCompat” your in XML <?xml version="1.0" encoding="utf-8"?>
 <LinearLayout
 xmlns:android="http://schemas.android.com/apk/res/android"
 xmlns:app="http://schemas.android.com/apk/res-auto"
 android:orientation="vertical"
 android:layout_width="match_parent"
 android:layout_height="match_parent">
 
 <ImageView
 android:layout_width="30dp"
 android:layout_height="30dp"
 android:src="@drawable/circle"/>
 
 <ImageView
 android:layout_width="25dp"
 android:layout_height="25dp"
 android:src="@drawable/hello"/>
 </LinearLayout>
55. Trimming the Resources - Vector Drawables Make sure to use “app:srcCompat” your in XML <?xml version="1.0" encoding="utf-8"?>
 <LinearLayout
 xmlns:android="http://schemas.android.com/apk/res/android"
 xmlns:app="http://schemas.android.com/apk/res-auto"
 android:orientation="vertical"
 android:layout_width="match_parent"
 android:layout_height="match_parent">
 
 <ImageView
 android:layout_width="30dp"
 android:layout_height="30dp"
 android:src="@drawable/circle"/>
 
 <ImageView
 android:layout_width="25dp"
 android:layout_height="25dp"
 android:src="@drawable/hello"/>
 </LinearLayout>
56. Trimming the Resources - Vector Drawables Using “app:srcCompat” your in XML <?xml version="1.0" encoding="utf-8"?>
 <LinearLayout
 xmlns:android="http://schemas.android.com/apk/res/android"
 xmlns:app="http://schemas.android.com/apk/res-auto"
 android:orientation="vertical"
 android:layout_width="match_parent"
 android:layout_height="match_parent">
 
 <ImageView
 android:layout_width="30dp"
 android:layout_height="30dp"
 app:srcCompat="@drawable/circle"/>
 
 <ImageView
 android:layout_width="25dp"
 android:layout_height="25dp"
 app:srcCompat="@drawable/hello"/>
 </LinearLayout>
57. Trimming the Resources - WebP • Reduces image sizes by converting to WebP • Does not need to compress at build time(PNG crunching) • By converting beforehand, this will help speed up your build
58. Trimming the Resources - WebP Right-click on “drawable” folders, click on “Convert to WebP”
59. After Optimizations - Debug build $ gradlew countDebugDexMethods sizeDebugApk
 
 > Task :countDebugDexMethods
 Total methods in debug.apk: 100595 (153.50% used) 
 Total fields in debug.apk: 51278 (78.25% used) 
 Methods remaining in debug.apk: 0 
 Fields remaining in debug.apk: 14257
 
 > Task :sizeDebugApk
 Total APK Size in debug.apk in bytes: 10053036 (10.1MB)
60. After Optimizations - Release build $ gradlew countReleaseDexMethods sizeReleaseApk (with Proguard)
 
 > Task :countReleaseDexMethods
 Total methods in release.apk: 60810 (92.79% used) 
 Total fields in release.apk: 29445 (44.93% used) 
 Methods remaining in release.apk: 4725
 Fields remaining in release.apk: 36090
 
 > Task :sizeReleaseApk
 Total APK Size in release.apk in bytes: 7049930 (7.1MB)
61. Optimized Results 60.8k+ < 64k DEX limit = 1 DEX file! Debug Release Method Count 100k+ 60.8k APK Size 10.1MB+ 7.1MB
62. Before vs After Release Builds 1 DEX file, Smaller APK, All features kept! Before After Reduction Method Count 85k+ 60.8k 28.47% APK Size 10.7MB+ 7.1MB 33.64%
63. Automating Dex Count Plugin gradle.taskGraph.whenReady { taskGraph ->
 if (taskGraph.hasTask(":assembleRelease")) {
 dexcount {
 maxMethodCount = 61000 // 61k methods
 }
 }
 } github.com/KeepSafe/dexcount-gradle-plugin
64. Automating APK Size Plugin gradle.taskGraph.whenReady { taskGraph ->
 if (taskGraph.hasTask(":assembleRelease")) {
 apkSize {
 maxApkSize = 8000 // in Kb
 }
 }
 } github.com/vanniktech/gradle-android-apk-size-plugin
65. Thank you! Questions? twitter.com/jaredsburrows github.com/jaredsburrows jaredsburrows@gmail.com jaredsburrows.com