Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Journey of APK from Compile to Start

Journey of APK from Compile to Start

This was the keynote presentation at Droidkaigi, 2019 in Tokyo, Japan.

Amanjeet Singh

February 08, 2019
Tweet

More Decks by Amanjeet Singh

Other Decks in Technology

Transcript

  1. Journey of APK from Compile to Start @droid_singh @_romichandra_

  2. “Success is a journey not a destination” - Internet

  3. Components of APK

  4. Components of APK Basic

  5. Components of APK Complex

  6. Where do these come from?

  7. Lets magnify this

  8. None
  9. None
  10. None
  11. AAPT •Android Asset Packaging tool. •Creates R.java file. •Tool for

    creating, updating and viewing components the zip archive.
  12. AAPT •Android Asset Packaging tool. •Creates R.java file. •Tool for

    creating, updating and viewing components the zip archive.
  13. •IDs are generated by AAPT to refer resources. •Resource bit:

    How R is prepared? P P T T N N N N 0x7f
  14. Category •IDs are generated by AAPT to refer resources. •Resource

    bit: How R is prepared? P P T T N N N N 0x7f Next hex available
  15. public static final class id { public static final int

    ALT=0x7f070000; public static final int CTRL=0x7f070001; public static final int FUNCTION=0x7f070002; public static final int META=0x7f070003; public static final int SHIFT=0x7f070004; public static final int SYM=0x7f070005; public static final int action_bar=0x7f070006; public static final int action_bar_activity_content=0x7f070007; public static final int action_bar_container=0x7f070008; }
  16. <?xml version="1.0" encoding="utf-8"?> <android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height=“match_parent"> </android.support.constraint.ConstraintLayout>

  17. <?xml version="1.0" encoding="utf-8"?> <android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" android:id="@+id/action_bar_constraintLayout" android:layout_width="match_parent" android:layout_height=“match_parent"> </android.support.constraint.ConstraintLayout>

  18. public static final class id { public static final int

    ALT=0x7f070000; public static final int CTRL=0x7f070001; public static final int FUNCTION=0x7f070002; public static final int META=0x7f070003; public static final int SHIFT=0x7f070004; public static final int SYM=0x7f070005; public static final int action_bar=0x7f070006; public static final int action_bar_activity_content=0x7f070007; public static final int action_bar_constraintLayout=0x7f070008; public static final int action_bar_container=0x7f070009; } PP TT NNNN
  19. R Java/Kt files Resources R.id.action_constraintLayout

  20. None
  21. Java Java class Java . . . . javac Java

    javac javac javac class class class dx or d8 dex Proguard
  22. Java Java class Java . . . . javac Java

    javac javac javac class class class dx or d8 dex Proguard
  23. Java Java class Java . . . . javac Java

    javac javac javac class class class dx or d8 dex Proguard
  24. Java Java class Java . . . . javac Java

    javac javac javac class class class dx or d8 dex Proguard
  25. Java Java class Java . . . . javac Java

    javac javac javac class class class dex Proguard Optimization 
 layers dx or d8
  26. public static long sumArray(int[] arr){ long sum = 0; for(int

    i: arr){ sum += i; } return sum; }
  27. public static long sumArray(int[] arr){ long sum = 0; for(int

    i: arr){ sum += i; } return sum; } public static long sumArray(int[]); Code: 0: lconst_0 1: lstore_1 2: aload_0 3: astore_3 4: aload_3 5: arraylength 6: istore 4 8: iconst_0 9: istore 5 11: iload 5 13: iload 4 15: if_icmpge 36 18: aload_3 19: iload 5 21: iaload 22: istore 6 24: lload_1 25: iload 6 27: i2l 28: ladd 29: lstore_1 30: iinc 5, 1 33: goto 11 36: lload_1 37: lreturn .class
  28. public static long sumArray(int[] arr){ long sum = 0; for(int

    i: arr){ sum += i; } return sum; } public static long sumArray(int[]); Code: 0: lconst_0 1: lstore_1 2: aload_0 3: astore_3 4: aload_3 5: arraylength 6: istore 4 8: iconst_0 9: istore 5 11: iload 5 13: iload 4 15: if_icmpge 36 18: aload_3 19: iload 5 21: iaload 22: istore 6 24: lload_1 25: iload 6 27: i2l 28: ladd 29: lstore_1 30: iinc 5, 1 33: goto 11 36: lload_1 37: lreturn .class 11: iload 5 13: iload 4 15: if_icmpge 36 18: aload_3 19: iload 5 21: iaload 22: istore 6 24: lload_1 25: iload 6 27: i2l 28: ladd 29: lstore_1 30: iinc 5, 1 33: goto 11 Dispatches 14 Read 20 Write 16
  29. public static long sumArray(int[] arr){ long sum = 0; for(int

    i: arr){ sum += i; } return sum; }
  30. public static long sumArray(int[] arr){ long sum = 0; for(int

    i: arr){ sum += i; } return sum; } 0000: const-wide/16 v2, #int 0 // #0 0002: array-length v1, v6 0003: const/4 v0, #int 0 // #0 0004: if-ge v0, v1, 000d // +0009 0006: aget v4, v6, v0 0008: int-to-long v4, v4 0009: add-long/2addr v2, v4 000a: add-int/lit8 v0, v0, #int 1 // #01 000c: goto 0004 // -0008 000d: return-wide v2 .dex:
  31. public static long sumArray(int[] arr){ long sum = 0; for(int

    i: arr){ sum += i; } return sum; } 0000: const-wide/16 v2, #int 0 // #0 0002: array-length v1, v6 0003: const/4 v0, #int 0 // #0 0004: if-ge v0, v1, 000d // +0009 0006: aget v4, v6, v0 0008: int-to-long v4, v4 0009: add-long/2addr v2, v4 000a: add-int/lit8 v0, v0, #int 1 // #01 000c: goto 0004 // -0008 000d: return-wide v2 .dex: 0004: if-ge v0, v1, 000d // +0009 0006: aget v4, v6, v0 0008: int-to-long v4, v4 0009: add-long/2addr v2, v4 000a: add-int/lit8 v0, v0, #int 1 // #01 000c: goto 0004 // -0008 Dispatches 6 Reads 10 Writes 6
  32. None
  33. None
  34. ZipAlign

  35. ZipAlign •Android has a 4 byte boundary memory mapping

  36. ZipAlign •Misaligned data slows data access performance.

  37. ZipAlign •Misaligned data slows data access performance. •ZipAlign step results

    in aligning all the data in 4 byte boundary.
  38. Compilation Installation

  39. None
  40. Dalvik Upto KitKat

  41. Dalvik Upto KitKat •Byte Code Verification

  42. Dalvik Upto KitKat •Byte Code Verification To check the integrity

    of Dex
  43. Dalvik Upto KitKat •Byte Code Verification To check the integrity

    of Dex Done at install time rather than run time.
  44. Dalvik Upto KitKat •Byte Code Verification To check the integrity

    of Dex Done at install time rather than run time. Validity of indices and structure of file.
  45. Dalvik Upto KitKat •Byte Code Verification To check the integrity

    of Dex Done at install time rather than run time. Validity of indices and structure of file. Ordering of the instructions.
  46. Dalvik Upto KitKat •Optimizations

  47. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation.

  48. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation.

    Replacing high volume of methods with their inline versions.
  49. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation.

    Replacing high volume of methods with their inline versions. Prune empty methods.
  50. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation.

    Replacing high volume of methods with their inline versions. Prune empty methods. Pre computations.
  51. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation.

    Replacing high volume of methods with their inline versions. Prune empty methods. Pre computations. dexopt ODEX
  52. ART In Lollipop

  53. ART In Lollipop •Preloading boot.art boot.oat

  54. ART In Lollipop •Preloading boot.art boot.oat Compiler
 Driver ELF file

    •Load Compiler
  55. ART In Lollipop •Preloading boot.art boot.oat Compiler
 Driver ELF file

    dex2Oat •Load Compiler (AOT)
  56. ART •Preloading boot.art boot.oat Compiler
 Driver ELF file •Load Compiler

    Dex Resolve classes Verify classes Init classes Compile classes ELF File
  57. Problem:

  58. Problem:

  59. Hybrid: JIT+AOT

  60. Hybrid: JIT+AOT From Android Nougat

  61. Profile guided Compilation Profile Files App App Execution
 (Interpret/JIT) Profile

    guided 
 compile Compiled App
  62. dex file, oat file

  63. dex file, oat file ART

  64. dex file, oat file ART Execute
 method

  65. dex file, oat file ART Execute
 method Is
 it compiled


    ?
  66. dex file, oat file ART Execute
 method Is
 it compiled


    ? Y
  67. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y
  68. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N
  69. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N Run code
 from jit cache Y
  70. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Run code
 from jit cache Y
  71. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Run code
 from jit cache Y Record Profiles info
  72. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? Run code
 from jit cache Y Record Profiles info
  73. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? JIT
 Cache Run code
 from jit cache Y Record Profiles info Y
  74. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? JIT
 Cache Run code
 from jit cache Y Record Profiles info N Y
  75. dex file, oat file ART Execute
 method Is
 it compiled


    ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? JIT
 Cache Run code
 from jit cache Y Record Profiles info N Y When Profile guided compilation 
 occurs?
  76. When Profile guided compilation 
 occurs?

  77. Compilation Installation Optimizations

  78. “Abstractions increase the productivity of developer but they can be

    overhead for runtime”
  79. Optimizations 1. Constant Folding numberOfDays = 2 + 2 +

    previous; add-int/lit8 v0, v2, #int 4
  80. 2. Inlining fun len (string: String): Int{ return string.length }

  81. 2. Inlining fun len (string: String): Int{ return string.length }

    const-string v0, "string" invoke-static {v1, v0}, Lkotlin/jvm/internal/Intrinsics;.checkParameterIsNotNull invoke-virtual {v1}, Ljava/lang/String;.length move-result v0 return v0
  82. 2. Inlining fun len (string: String): Int{ return string.length }

    const-string v0, "string" invoke-static {v1, v0}, Lkotlin/jvm/internal/Intrinsics;.checkParameterIsNotNull invoke-virtual {v1}, Ljava/lang/String;.length move-result v0 return v0
  83. const-string v0, "string" if-nez v1, :continue invoke-static {v1}, throwParameterIsNullException :continue

    invoke-virtual {v1}, Ljava/lang/String;.length move-result v0 return v0
  84. 3. Bounds Check Elimination int sum = 0; for (int

    i = 0; i < size; i++) { sum += array[i]; }
  85. 3. Bounds Check Elimination int sum = 0; for (int

    i = 0; i < size; i++) { sum += array[i]; }
  86. 4. Strength Reduction 5. Dead Code Elimination 6. DexLayout 7.

    Class Hierarchy Analysis And much more…..
  87. Compilation ??

  88. None
  89. None
  90. None
  91. None
  92. None
  93. None
  94. None
  95. MULTIDEX: The 64k LIMIT

  96. MULTIDEX: THE AFTERMATH •Adverse effect on App Startup •Build time

    increases. •Strange errors during runtime •Strangers in Crashyltics..
  97. MULTIDEX: THE AFTERMATH •Adverse effect on App Startup •Build time

    increases. •Strange errors during runtime •Strangers in Crashyltics..
  98. MULTIDEX: THE AFTERMATH •Adverse effect on App Startup •Build time

    increases. •Strange errors during runtime •Strangers in Crashyltics..
  99. MULTIDEX: HOW?

  100. DirectClass
 FileConsumer .class Validate & Lock ClassTranslatorTask ClassDefItemConsumer Update global

    indices Release the lock Write Copyright: AOSP
  101. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the

    lock Write Copyright: AOSP Validate & Lock
  102. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the

    lock Write Copyright: AOSP Validate & Lock
  103. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the

    lock Write Copyright: AOSP Validate & Lock
  104. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the

    lock Write Copyright: AOSP Validate & Lock
  105. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the

    lock Write Copyright: AOSP Validate & Lock
  106. Compilation Installation Optimizations App
 Startup

  107. App Startup: 
 Evolving Better with ART

  108. None
  109. None
  110. What’s wrong?

  111. None
  112. 10s

  113. None
  114. Time for tips • Use lazy initializations. • Don’t use

    reflections at time of APP startup • Initialize all the SDKs in background • Load images asynchronously. • Avoid nesting layout
  115. Let us all be an ARTist

  116. Thank you' @droid_singh amanjeetsingh150 @_romichandra_ romichandra