Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Journey of APK from Compile to Start

Journey of APK from Compile to Start

This was the keynote presentation at Droidkaigi, 2019 in Tokyo, Japan.

Amanjeet Singh

February 08, 2019
Tweet

More Decks by Amanjeet Singh

Other Decks in Technology

Transcript

  1. Journey of APK from
    Compile to Start
    @droid_singh @_romichandra_

    View Slide

  2. “Success is a journey not a destination”
    - Internet

    View Slide

  3. Components of APK

    View Slide

  4. Components of APK
    Basic

    View Slide

  5. Components of APK
    Complex

    View Slide

  6. Where do these come from?

    View Slide

  7. Lets magnify this

    View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. AAPT
    •Android Asset Packaging tool.
    •Creates R.java file.
    •Tool for creating, updating and viewing
    components the zip archive.

    View Slide

  12. AAPT
    •Android Asset Packaging tool.
    •Creates R.java file.
    •Tool for creating, updating and viewing
    components the zip archive.

    View Slide

  13. •IDs are generated by AAPT to refer resources.
    •Resource bit:
    How R is prepared?
    P P T T N N N N
    0x7f

    View Slide

  14. Category
    •IDs are generated by AAPT to refer resources.
    •Resource bit:
    How R is prepared?
    P P T T N N N N
    0x7f Next hex available

    View Slide

  15. public static final class id {
    public static final int ALT=0x7f070000;
    public static final int CTRL=0x7f070001;
    public static final int FUNCTION=0x7f070002;
    public static final int META=0x7f070003;
    public static final int SHIFT=0x7f070004;
    public static final int SYM=0x7f070005;
    public static final int action_bar=0x7f070006;
    public static final int action_bar_activity_content=0x7f070007;
    public static final int action_bar_container=0x7f070008;
    }

    View Slide


  16. xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height=“match_parent">

    View Slide


  17. xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/action_bar_constraintLayout"
    android:layout_width="match_parent"
    android:layout_height=“match_parent">

    View Slide

  18. public static final class id {
    public static final int ALT=0x7f070000;
    public static final int CTRL=0x7f070001;
    public static final int FUNCTION=0x7f070002;
    public static final int META=0x7f070003;
    public static final int SHIFT=0x7f070004;
    public static final int SYM=0x7f070005;
    public static final int action_bar=0x7f070006;
    public static final int action_bar_activity_content=0x7f070007;
    public static final int action_bar_constraintLayout=0x7f070008;
    public static final int action_bar_container=0x7f070009;
    }
    PP
    TT
    NNNN

    View Slide

  19. R
    Java/Kt files Resources
    R.id.action_constraintLayout

    View Slide

  20. View Slide

  21. Java
    Java
    class
    Java
    .
    .
    .
    .
    javac
    Java
    javac
    javac
    javac
    class
    class
    class
    dx
    or
    d8
    dex
    Proguard

    View Slide

  22. Java
    Java
    class
    Java
    .
    .
    .
    .
    javac
    Java
    javac
    javac
    javac
    class
    class
    class
    dx
    or
    d8
    dex
    Proguard

    View Slide

  23. Java
    Java
    class
    Java
    .
    .
    .
    .
    javac
    Java
    javac
    javac
    javac
    class
    class
    class
    dx
    or
    d8
    dex
    Proguard

    View Slide

  24. Java
    Java
    class
    Java
    .
    .
    .
    .
    javac
    Java
    javac
    javac
    javac
    class
    class
    class
    dx
    or
    d8
    dex
    Proguard

    View Slide

  25. Java
    Java
    class
    Java
    .
    .
    .
    .
    javac
    Java
    javac
    javac
    javac
    class
    class
    class
    dex
    Proguard
    Optimization 

    layers
    dx
    or
    d8

    View Slide

  26. public static long sumArray(int[] arr){
    long sum = 0;
    for(int i: arr){
    sum += i;
    }
    return sum;
    }

    View Slide

  27. public static long sumArray(int[] arr){
    long sum = 0;
    for(int i: arr){
    sum += i;
    }
    return sum;
    }
    public static long sumArray(int[]);
    Code:
    0: lconst_0
    1: lstore_1
    2: aload_0
    3: astore_3
    4: aload_3
    5: arraylength
    6: istore 4
    8: iconst_0
    9: istore 5
    11: iload 5
    13: iload 4
    15: if_icmpge 36
    18: aload_3
    19: iload 5
    21: iaload
    22: istore 6
    24: lload_1
    25: iload 6
    27: i2l
    28: ladd
    29: lstore_1
    30: iinc 5, 1
    33: goto 11
    36: lload_1
    37: lreturn
    .class

    View Slide

  28. public static long sumArray(int[] arr){
    long sum = 0;
    for(int i: arr){
    sum += i;
    }
    return sum;
    }
    public static long sumArray(int[]);
    Code:
    0: lconst_0
    1: lstore_1
    2: aload_0
    3: astore_3
    4: aload_3
    5: arraylength
    6: istore 4
    8: iconst_0
    9: istore 5
    11: iload 5
    13: iload 4
    15: if_icmpge 36
    18: aload_3
    19: iload 5
    21: iaload
    22: istore 6
    24: lload_1
    25: iload 6
    27: i2l
    28: ladd
    29: lstore_1
    30: iinc 5, 1
    33: goto 11
    36: lload_1
    37: lreturn
    .class
    11: iload 5
    13: iload 4
    15: if_icmpge 36
    18: aload_3
    19: iload 5
    21: iaload
    22: istore 6
    24: lload_1
    25: iload 6
    27: i2l
    28: ladd
    29: lstore_1
    30: iinc 5, 1
    33: goto 11

    Dispatches 14
    Read 20
    Write 16

    View Slide

  29. public static long sumArray(int[] arr){
    long sum = 0;
    for(int i: arr){
    sum += i;
    }
    return sum;
    }

    View Slide

  30. public static long sumArray(int[] arr){
    long sum = 0;
    for(int i: arr){
    sum += i;
    }
    return sum;
    }
    0000: const-wide/16 v2, #int 0 // #0
    0002: array-length v1, v6
    0003: const/4 v0, #int 0 // #0
    0004: if-ge v0, v1, 000d // +0009
    0006: aget v4, v6, v0
    0008: int-to-long v4, v4
    0009: add-long/2addr v2, v4
    000a: add-int/lit8 v0, v0, #int 1 // #01
    000c: goto 0004 // -0008
    000d: return-wide v2
    .dex:

    View Slide

  31. public static long sumArray(int[] arr){
    long sum = 0;
    for(int i: arr){
    sum += i;
    }
    return sum;
    }
    0000: const-wide/16 v2, #int 0 // #0
    0002: array-length v1, v6
    0003: const/4 v0, #int 0 // #0
    0004: if-ge v0, v1, 000d // +0009
    0006: aget v4, v6, v0
    0008: int-to-long v4, v4
    0009: add-long/2addr v2, v4
    000a: add-int/lit8 v0, v0, #int 1 // #01
    000c: goto 0004 // -0008
    000d: return-wide v2
    .dex:
    0004: if-ge v0, v1, 000d // +0009
    0006: aget v4, v6, v0
    0008: int-to-long v4, v4
    0009: add-long/2addr v2, v4
    000a: add-int/lit8 v0, v0, #int 1 // #01
    000c: goto 0004 // -0008

    Dispatches 6
    Reads 10
    Writes 6

    View Slide

  32. View Slide

  33. View Slide

  34. ZipAlign

    View Slide

  35. ZipAlign
    •Android has a 4 byte boundary memory mapping

    View Slide

  36. ZipAlign
    •Misaligned data slows data access performance.

    View Slide

  37. ZipAlign
    •Misaligned data slows data access performance.
    •ZipAlign step results in aligning all the data in 4 byte
    boundary.

    View Slide

  38. Compilation
    Installation

    View Slide

  39. View Slide

  40. Dalvik
    Upto KitKat

    View Slide

  41. Dalvik
    Upto KitKat
    •Byte Code Verification

    View Slide

  42. Dalvik
    Upto KitKat
    •Byte Code Verification
    To check the integrity of Dex

    View Slide

  43. Dalvik
    Upto KitKat
    •Byte Code Verification
    To check the integrity of Dex
    Done at install time rather than run
    time.

    View Slide

  44. Dalvik
    Upto KitKat
    •Byte Code Verification
    To check the integrity of Dex
    Done at install time rather than run
    time.
    Validity of indices and structure of
    file.

    View Slide

  45. Dalvik
    Upto KitKat
    •Byte Code Verification
    To check the integrity of Dex
    Done at install time rather than run
    time.
    Validity of indices and structure of
    file.
    Ordering of the instructions.

    View Slide

  46. Dalvik
    Upto KitKat
    •Optimizations

    View Slide

  47. Dalvik
    Upto KitKat
    •Optimizations
    VTable Method Index for method
    invocation.

    View Slide

  48. Dalvik
    Upto KitKat
    •Optimizations
    VTable Method Index for method
    invocation.
    Replacing high volume of methods
    with their inline versions.

    View Slide

  49. Dalvik
    Upto KitKat
    •Optimizations
    VTable Method Index for method
    invocation.
    Replacing high volume of methods
    with their inline versions.
    Prune empty methods.

    View Slide

  50. Dalvik
    Upto KitKat
    •Optimizations
    VTable Method Index for method
    invocation.
    Replacing high volume of methods
    with their inline versions.
    Prune empty methods.
    Pre computations.

    View Slide

  51. Dalvik
    Upto KitKat
    •Optimizations
    VTable Method Index for method
    invocation.
    Replacing high volume of methods
    with their inline versions.
    Prune empty methods.
    Pre computations.
    dexopt
    ODEX

    View Slide

  52. ART
    In Lollipop

    View Slide

  53. ART
    In Lollipop
    •Preloading
    boot.art
    boot.oat

    View Slide

  54. ART
    In Lollipop
    •Preloading
    boot.art
    boot.oat
    Compiler

    Driver
    ELF file
    •Load Compiler

    View Slide

  55. ART
    In Lollipop
    •Preloading
    boot.art
    boot.oat
    Compiler

    Driver
    ELF file
    dex2Oat
    •Load Compiler (AOT)

    View Slide

  56. ART
    •Preloading
    boot.art
    boot.oat
    Compiler

    Driver
    ELF file
    •Load Compiler
    Dex
    Resolve
    classes
    Verify
    classes
    Init
    classes
    Compile
    classes
    ELF
    File

    View Slide

  57. Problem:

    View Slide

  58. Problem:

    View Slide

  59. Hybrid: JIT+AOT

    View Slide

  60. Hybrid: JIT+AOT
    From Android Nougat

    View Slide

  61. Profile guided Compilation
    Profile Files
    App
    App Execution

    (Interpret/JIT)
    Profile guided 

    compile
    Compiled App

    View Slide

  62. dex file,
    oat file

    View Slide

  63. dex file,
    oat file
    ART

    View Slide

  64. dex file,
    oat file
    ART
    Execute

    method

    View Slide

  65. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?

    View Slide

  66. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Y

    View Slide

  67. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y

    View Slide

  68. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N

    View Slide

  69. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N
    Run code

    from jit cache
    Y

    View Slide

  70. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N
    N
    Interpreter
    Run code

    from jit cache
    Y

    View Slide

  71. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N
    N
    Interpreter
    Run code

    from jit cache
    Y
    Record
    Profiles
    info

    View Slide

  72. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N
    N
    Interpreter
    Is

    it hot?
    Run code

    from jit cache
    Y
    Record
    Profiles
    info

    View Slide

  73. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N
    N
    Interpreter
    Is

    it hot?
    JIT

    Cache
    Run code

    from jit cache
    Y
    Record
    Profiles
    info
    Y

    View Slide

  74. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N
    N
    Interpreter
    Is

    it hot?
    JIT

    Cache
    Run code

    from jit cache
    Y
    Record
    Profiles
    info
    N
    Y

    View Slide

  75. dex file,
    oat file
    ART
    Execute

    method
    Is

    it compiled

    ?
    Is it

    JIT
    compiled
    Y
    Run code

    from .oat
    file
    N
    N
    Interpreter
    Is

    it hot?
    JIT

    Cache
    Run code

    from jit cache
    Y
    Record
    Profiles
    info
    N
    Y
    When Profile guided
    compilation 

    occurs?

    View Slide

  76. When Profile guided
    compilation 

    occurs?

    View Slide

  77. Compilation
    Installation
    Optimizations

    View Slide

  78. “Abstractions increase the productivity of developer but they
    can be overhead for runtime”

    View Slide

  79. Optimizations
    1. Constant Folding
    numberOfDays = 2 + 2 + previous;
    add-int/lit8 v0, v2, #int 4

    View Slide

  80. 2. Inlining
    fun len (string: String): Int{
    return string.length
    }

    View Slide

  81. 2. Inlining
    fun len (string: String): Int{
    return string.length
    }
    const-string v0, "string"
    invoke-static {v1, v0}, Lkotlin/jvm/internal/Intrinsics;.checkParameterIsNotNull
    invoke-virtual {v1}, Ljava/lang/String;.length
    move-result v0
    return v0

    View Slide

  82. 2. Inlining
    fun len (string: String): Int{
    return string.length
    }
    const-string v0, "string"
    invoke-static {v1, v0}, Lkotlin/jvm/internal/Intrinsics;.checkParameterIsNotNull
    invoke-virtual {v1}, Ljava/lang/String;.length
    move-result v0
    return v0

    View Slide

  83. const-string v0, "string"
    if-nez v1, :continue
    invoke-static {v1}, throwParameterIsNullException
    :continue
    invoke-virtual {v1}, Ljava/lang/String;.length
    move-result v0
    return v0

    View Slide

  84. 3. Bounds Check Elimination
    int sum = 0;
    for (int i = 0; i < size; i++) {
    sum += array[i];
    }

    View Slide

  85. 3. Bounds Check Elimination
    int sum = 0;
    for (int i = 0; i < size; i++) {
    sum += array[i];
    }

    View Slide

  86. 4. Strength Reduction
    5. Dead Code Elimination
    6. DexLayout
    7. Class Hierarchy Analysis
    And much more…..

    View Slide

  87. Compilation
    ??

    View Slide

  88. View Slide

  89. View Slide

  90. View Slide

  91. View Slide

  92. View Slide

  93. View Slide

  94. View Slide

  95. MULTIDEX: The 64k LIMIT

    View Slide

  96. MULTIDEX: THE AFTERMATH
    •Adverse effect on App Startup
    •Build time increases.
    •Strange errors during runtime
    •Strangers in Crashyltics..

    View Slide

  97. MULTIDEX: THE AFTERMATH
    •Adverse effect on App Startup
    •Build time increases.
    •Strange errors during runtime
    •Strangers in Crashyltics..

    View Slide

  98. MULTIDEX: THE AFTERMATH
    •Adverse effect on App Startup
    •Build time increases.
    •Strange errors during runtime
    •Strangers in Crashyltics..

    View Slide

  99. MULTIDEX: HOW?

    View Slide

  100. DirectClass

    FileConsumer
    .class
    Validate & Lock
    ClassTranslatorTask
    ClassDefItemConsumer
    Update global indices
    Release the lock
    Write
    Copyright: AOSP

    View Slide

  101. DirectClass

    FileConsumer
    .class
    ClassTranslatorTask
    ClassDefItemConsumer
    Update global indices
    Release the lock
    Write
    Copyright: AOSP
    Validate & Lock

    View Slide

  102. DirectClass

    FileConsumer
    .class
    ClassTranslatorTask
    ClassDefItemConsumer
    Update global indices
    Release the lock
    Write
    Copyright: AOSP
    Validate & Lock

    View Slide

  103. DirectClass

    FileConsumer
    .class
    ClassTranslatorTask
    ClassDefItemConsumer
    Update global indices
    Release the lock
    Write
    Copyright: AOSP
    Validate & Lock

    View Slide

  104. DirectClass

    FileConsumer
    .class
    ClassTranslatorTask
    ClassDefItemConsumer
    Update global indices
    Release the lock
    Write
    Copyright: AOSP
    Validate & Lock

    View Slide

  105. DirectClass

    FileConsumer
    .class
    ClassTranslatorTask
    ClassDefItemConsumer
    Update global indices
    Release the lock
    Write
    Copyright: AOSP
    Validate & Lock

    View Slide

  106. Compilation
    Installation
    Optimizations
    App

    Startup

    View Slide

  107. App Startup: 

    Evolving Better with ART

    View Slide

  108. View Slide

  109. View Slide

  110. What’s wrong?

    View Slide

  111. View Slide

  112. 10s

    View Slide

  113. View Slide

  114. Time for tips
    • Use lazy initializations.
    • Don’t use reflections at time of APP startup
    • Initialize all the SDKs in background
    • Load images asynchronously.
    • Avoid nesting layout

    View Slide

  115. Let us all be an ARTist

    View Slide

  116. Thank you'
    @droid_singh
    amanjeetsingh150
    @_romichandra_
    romichandra

    View Slide