Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's in the Android Toolbox

Bryan Herbst
February 10, 2018

What's in the Android Toolbox

A look into the Android build tools. How are Android apps built? What's new in the Android build tools?

Bryan Herbst

February 10, 2018
Tweet

More Decks by Bryan Herbst

Other Decks in Programming

Transcript

  1. What’s in the
    Android Toolbox?
    Bryan Herbst
    (@bryancherbst)

    View Slide

  2. What tools?

    View Slide

  3. $ sdkmanager --list

    View Slide

  4. $ sdkmanager --list
    Tools

    View Slide

  5. $ sdkmanager --list
    Tools
    Platform-Tools

    View Slide

  6. $ sdkmanager --list
    Tools
    Platform-Tools
    Build-Tools

    View Slide

  7. SDK tools

    View Slide

  8. $ ls $ANDROID_HOME/tools

    View Slide

  9. $ ls $ANDROID_HOME/tools
    android
    emulator
    monitor

    View Slide

  10. $ ls $ANDROID_HOME/platform-tools

    View Slide

  11. $ ls $ANDROID_HOME/platform-tools
    adb
    fastboot
    systrace

    View Slide

  12. $ ls $ANDROID_HOME/build-tools/27.0.3

    View Slide

  13. $ ls $ANDROID_HOME/build-tools/27.0.3
    aapt
    apksigner
    dx

    View Slide

  14. What’s in an APK?

    View Slide

  15. APK ≈ JAR ≈ Zip

    View Slide

  16. APK
    META_INF/ APK metadata

    View Slide

  17. APK
    META_INF/
    lib/ Native libs

    View Slide

  18. APK
    META_INF/
    lib/
    res/ Static resources
    (PNGs, XML layouts, etc.)

    View Slide

  19. APK
    META_INF/
    lib/
    res/
    assets/ Static assets
    (fonts, HTML, etc.)

    View Slide

  20. APK
    META_INF/
    lib/
    res/
    assets/
    AndroidManifest.xml Hey, this looks familiar!

    View Slide

  21. APK
    META_INF/
    lib/
    res/
    assets/
    AndroidManifest.xml
    classes.dex Your code

    View Slide

  22. APK
    META_INF/
    lib/
    res/
    assets/
    AndroidManifest.xml
    classes.dex
    resources.arsc Compiled resources
    (strings, dimens, etc.)

    View Slide

  23. Dex?

    View Slide

  24. Code
    (.c, etc)
    Machine
    code

    View Slide

  25. Code
    (.c, etc)
    Machine code
    (arm)
    Machine code
    (x86)
    Machine code
    (mips)

    View Slide

  26. Java

    View Slide

  27. Java
    (.java)
    Bytecode
    (.class)
    Machine
    code

    View Slide

  28. Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Ahead-of-time

    View Slide

  29. Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Just in time

    View Slide

  30. Android

    View Slide

  31. Ahead-of-time
    Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Dex Bytecode
    (.dex)

    View Slide

  32. Just-in-time
    Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Dex Bytecode
    (.dex)
    Dalvik

    View Slide

  33. Ahead-of-time
    Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Dex Bytecode
    (.dex)
    ART

    View Slide

  34. Ahead-of-time
    (on-device)
    Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Dex Bytecode
    (.dex)
    ART

    View Slide

  35. Ahead-of-time
    (on-device)
    (with JIT)
    Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Dex Bytecode
    (.dex)
    ART

    View Slide

  36. Compile
    Transform
    Package

    View Slide

  37. Compile
    Transform
    Package
    .java à .class

    View Slide

  38. Compile
    Transform
    Package
    .classà .dex

    View Slide

  39. Compile
    Transform
    Package everything à .apk

    View Slide

  40. Shift from
    generic tools to
    Android-specific
    tools

    View Slide

  41. incrementality
    parallelism

    View Slide

  42. Gradle
    The orchestrator

    View Slide

  43. By default
    Gradle
    does nothing

    View Slide

  44. Plugins
    add
    functionality

    View Slide

  45. apply plugin: 'com.android.application' Android functionality

    View Slide

  46. apply plugin: 'com.android.application'
    apply plugin: 'java' Java functionality

    View Slide

  47. apply plugin: 'com.android.application'
    apply plugin: 'java'
    apply plugin: 'kotlin' Kotlin functionality

    View Slide

  48. Plugins can
    extend the DSL

    View Slide

  49. Plugins can
    add dependency
    configurations

    View Slide

  50. Plugins can
    add tasks

    View Slide

  51. Android Plugin 3.x
    &
    Gradle 4.x

    View Slide

  52. compile
    Configurations

    View Slide

  53. compile
    testCompile
    Configurations

    View Slide

  54. compile
    testCompile
    debugCompile
    Configurations

    View Slide

  55. compile
    testCompile
    debugCompile
    apk
    Configurations

    View Slide

  56. compile
    testCompile
    debugCompile
    apk
    provided
    Configurations

    View Slide

  57. compile
    testCompile
    debugCompile
    apk
    provided
    Configurations

    View Slide

  58. api Available at compile time
    and runtime
    Configurations

    View Slide

  59. api
    implementation
    Available at compile time,
    Only available to
    consumers at runtime
    Configurations

    View Slide

  60. api
    implementation
    compileOnly
    Only available at
    compile time
    Configurations

    View Slide

  61. api
    implementation
    compileOnly
    runtimeOnly
    Only available at
    runtime
    Configurations

    View Slide

  62. Lib B Lib A App
    compile ':libB'

    View Slide

  63. Lib B Lib A App
    compile ':libB'

    View Slide

  64. Lib B Lib A App
    compile ':libB'

    View Slide

  65. Lib B Lib A App
    compile ':libB'

    View Slide

  66. Lib B Lib A App
    implemenation ':libB'

    View Slide

  67. Lib B Lib A App
    implemenation ':libB'

    View Slide

  68. Lib B Lib A App
    implemenation ':libB'

    View Slide

  69. And so much
    more

    View Slide

  70. Compile
    .java à .class

    View Slide

  71. Before we compile…

    View Slide

  72. Code Generation
    Dagger
    Butterknife
    Databinding

    View Slide

  73. buildConfigField "String", "FOO", '"bar"'

    View Slide

  74. resValue "string", "FOO", "bar"

    View Slide

  75. Compile
    javac / kotlinc

    View Slide

  76. Transforms
    .class à .dex

    View Slide

  77. Java 8
    bytecode
    Java 6/7
    bytecode

    View Slide

  78. Java 8
    bytecode
    Java 6/7
    bytecode
    Bytecode
    Optimized
    bytecode

    View Slide

  79. Java 8
    bytecode
    Java 6/7
    bytecode
    Bytecode
    Optimized
    bytecode
    Java
    bytecode
    Dex bytecode

    View Slide

  80. import com.android.build.api.transform.Transform

    View Slide

  81. import com.android.build.api.transform.Transform
    class RetrolambdaTransform : Transform {
    }

    View Slide

  82. app/build
    ├── generated
    ├── intermediates
    │ ├── classes
    │ └── transforms
    │ ├── dex[...]
    │ ├── mergeJavaRes
    │ ├── profilers-transform
    │ └── proguard
    └── outputs
    └── apk

    View Slide

  83. DX and D8
    The dexing duo

    View Slide

  84. DX is the
    current dexing tool

    View Slide

  85. $ javac Main.java
    $ dx --dex --output=. Main.class

    View Slide

  86. dexdump lets you
    disassemble dex files

    View Slide

  87. $ dexdump –d classes.dex

    View Slide

  88. [000148] Main.main(String)
    0000: sget-object v0, System;.out:PrintStream;
    0002: const-string v1, "Hello World"
    0004: invoke-virtual {v0, v1}, println:(String)V
    0007: return-void

    View Slide

  89. [000148] Main.main(String)
    0000: sget-object v0, System;.out:PrintStream;
    0002: const-string v1, "Hello World"
    0004: invoke-virtual {v0, v1}, println:(String)V
    0007: return-void

    View Slide

  90. D8 is the
    next-gen dexing tool

    View Slide

  91. Faster compilation!

    View Slide

  92. Faster compilation!
    Smaller dex files!

    View Slide

  93. Faster compilation!
    Smaller dex files!
    R8 integration!

    View Slide

  94. gradle.properties
    android.enableD8=true

    View Slide

  95. Default in Studio
    + Gradle Plugin 3.1

    View Slide

  96. Desugar
    Artificial Java sweetener

    View Slide

  97. Retrolambda

    View Slide

  98. Retrolambda

    View Slide

  99. Retrolambda
    Jack

    View Slide

  100. Retrolambda
    Jack

    View Slide

  101. Retrolambda
    Jack
    Desugar

    View Slide

  102. Retrolambda
    Jack
    Desugar
    D8

    View Slide

  103. Jack and Jill
    Dead tools

    View Slide

  104. Shrink + Obfsucate
    ProGuard and R8

    View Slide

  105. Activity
    Class A Class B

    View Slide

  106. Activity
    Class A Class B

    View Slide

  107. Activity
    Class A

    View Slide

  108. A
    B

    View Slide

  109. ProGuard
    The old guard

    View Slide

  110. ProGuard is an open-source
    Java shrinker, obfuscation,
    and optimization tool

    View Slide

  111. minifyEnabled true
    proguardFiles 'proguard-rules.pro'

    View Slide

  112. -keep class com.model.ImportantData { *; }

    View Slide

  113. -keep class * extends android.view.View {
    (...);
    }

    View Slide

  114. AAPT
    automatically generates
    ProGuard rules for
    Activities

    View Slide

  115. AAPT
    automatically generates
    ProGuard rules for
    Views

    View Slide

  116. AAPT
    automatically generates
    ProGuard rules for
    Services

    View Slide

  117. AAPT
    automatically generates
    ProGuard rules for
    ContentProviders

    View Slide

  118. AAPT
    automatically generates
    ProGuard rules for
    BroadcastReceivers

    View Slide

  119. ProGuard is an open-source
    Java shrinker, obfuscation,
    and optimization tool

    View Slide

  120. ProGuard is an open-source
    Java shrinker, obfuscation,
    and optimization tool

    View Slide

  121. Activity
    Drawable Class A

    View Slide

  122. Drawable Class A

    View Slide

  123. R8
    The new (pro)guard

    View Slide

  124. R8
    The new (pro)guard
    And D8

    View Slide

  125. R8 does
    full program
    Shrinking, obfuscation,
    and optimization

    View Slide

  126. Better incremental builds!

    View Slide

  127. Better incremental builds!
    New DSL!

    View Slide

  128. Not available yet :(

    View Slide

  129. Package
    Making the APK

    View Slide

  130. A
    A
    P
    T

    View Slide

  131. Android
    Asset
    Packaging
    Tool

    View Slide

  132. AAPT packages,
    reads, and updates
    APKs

    View Slide

  133. AAPT also…

    View Slide

  134. AAPT also…
    Processes resources

    View Slide

  135. AAPT also…
    Processes resources
    Generates R class

    View Slide

  136. AAPT also…
    Processes resources
    Generates R class
    Compiles resources

    View Slide

  137. AAPT2
    Android Asset Packaging Tool 2

    View Slide

  138. AAPT2 is enabled by
    default as of
    plugin 3.0.0

    View Slide

  139. AAPT2 split
    linking and compilation

    View Slide

  140. AAPT2 is
    more restrictive

    View Slide









  141. View Slide









  142. View Slide

  143. APK Signing

    View Slide

  144. V1 signatures use
    jarsigner

    View Slide

  145. V2 signatures use
    apksigner

    View Slide

  146. Whole-file signature
    https://source.android.com/security/apksigning/v2

    View Slide

  147. Gradle Plugin 2.2+
    defaults to v2 signatures

    View Slide

  148. Zipalign

    View Slide

  149. Zipalign aligns APKs
    on 4-byte boundaries

    View Slide

  150. Memory CPU

    View Slide

  151. Memory CPU
    Data

    View Slide

  152. Memory CPU
    Data

    View Slide

  153. Memory
    CPU

    View Slide

  154. Memory
    CPU
    Data

    View Slide

  155. Memory
    CPU
    Data

    View Slide

  156. Memory
    CPU
    Data

    View Slide

  157. Memory
    CPU
    Data

    View Slide

  158. Memory
    CPU
    Data

    View Slide

  159. V1 signature: After signing
    V2 signature: Before signing

    View Slide

  160. apkzlib

    View Slide

  161. Packing tools
    aapt apksigner
    zipalign

    View Slide

  162. Packing tools
    aapt apksigner
    zipalign
    Gradle Android Plugin
    apkzlib

    View Slide

  163. Post-build

    View Slide

  164. ART
    Android Runtime

    View Slide

  165. Ahead-of-time
    (on-device)
    (with JIT)
    Java
    (.java)
    Bytecode
    (.class)
    Machine
    code
    Dex Bytecode
    (.dex)

    View Slide

  166. Compiled APK
    (.dex)

    View Slide

  167. Compiled APK
    (.dex)
    optimize
    (dex2oat) Optimized
    APK
    (.dex, .oat)

    View Slide

  168. Compiled APK
    (.dex)
    optimize
    (dex2oat) Optimized
    APK
    (.dex, .oat)
    JIT Profile

    View Slide

  169. Compiled APK
    (.dex)
    optimize
    (dex2oat) Optimized
    APK
    (.dex, .oat)
    JIT Profile
    optimize
    (dex2oat)
    Optimized
    .oat

    View Slide

  170. Compiled APK
    (.dex)
    optimize
    (dex2oat) Optimized
    APK
    (.dex, .oat)
    JIT Profile
    optimize
    (dex2oat)
    Optimized
    .oat

    View Slide

  171. Compiled APK
    (.dex)
    optimize
    (dex2oat) Optimized
    APK
    (.dex, .oat)
    JIT Profile
    optimize
    (dex2oat)
    Optimized
    .oat
    Android 5.0

    View Slide

  172. Compiled APK
    (.dex)
    optimize
    (dex2oat) Optimized
    APK
    (.dex, .oat)
    JIT Profile
    optimize
    (dex2oat)
    Optimized
    .oat
    Android 7.0

    View Slide

  173. State of the ART

    View Slide

  174. State of the ART
    Interpreted (.dex)

    View Slide

  175. State of the ART
    Interpreted (.dex)
    JIT Compiled (in memory)

    View Slide

  176. State of the ART
    Interpreted (.dex)
    JIT Compiled (in memory)
    AOT compiled (.oat)

    View Slide

  177. Tools: android.googlesource.com/platform/tools/base
    ART, APK Signing: source.android.com/
    R8 + D8: r8.googlesource.com

    View Slide

  178. https://goo.gl/cMsvmf
    @bryancherbst

    View Slide