Journey of APK from Compile to Start

This was the keynote presentation at Droidkaigi, 2019 in Tokyo, Japan.

Transcript

1. Journey of APK from Compile to Start @droid_singh @_romichandra_
2. “Success is a journey not a destination” - Internet
3. Components of APK
4. Components of APK Basic
5. Components of APK Complex
6. Where do these come from?
7. Lets magnify this
11. AAPT •Android Asset Packaging tool. •Creates R.java file. •Tool for creating, updating and viewing components the zip archive.
12. AAPT •Android Asset Packaging tool. •Creates R.java file. •Tool for creating, updating and viewing components the zip archive.
13. •IDs are generated by AAPT to refer resources. •Resource bit: How R is prepared? P P T T N N N N 0x7f
14. Category •IDs are generated by AAPT to refer resources. •Resource bit: How R is prepared? P P T T N N N N 0x7f Next hex available
15. public static final class id { public static final int ALT=0x7f070000; public static final int CTRL=0x7f070001; public static final int FUNCTION=0x7f070002; public static final int META=0x7f070003; public static final int SHIFT=0x7f070004; public static final int SYM=0x7f070005; public static final int action_bar=0x7f070006; public static final int action_bar_activity_content=0x7f070007; public static final int action_bar_container=0x7f070008; }
16. <?xml version="1.0" encoding="utf-8"?> <android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height=“match_parent"> </android.support.constraint.ConstraintLayout>
17. <?xml version="1.0" encoding="utf-8"?> <android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android" android:id="@+id/action_bar_constraintLayout" android:layout_width="match_parent" android:layout_height=“match_parent"> </android.support.constraint.ConstraintLayout>
18. public static final class id { public static final int ALT=0x7f070000; public static final int CTRL=0x7f070001; public static final int FUNCTION=0x7f070002; public static final int META=0x7f070003; public static final int SHIFT=0x7f070004; public static final int SYM=0x7f070005; public static final int action_bar=0x7f070006; public static final int action_bar_activity_content=0x7f070007; public static final int action_bar_constraintLayout=0x7f070008; public static final int action_bar_container=0x7f070009; } PP TT NNNN
19. R Java/Kt files Resources R.id.action_constraintLayout
21. Java Java class Java . . . . javac Java javac javac javac class class class dx or d8 dex Proguard
22. Java Java class Java . . . . javac Java javac javac javac class class class dx or d8 dex Proguard
23. Java Java class Java . . . . javac Java javac javac javac class class class dx or d8 dex Proguard
24. Java Java class Java . . . . javac Java javac javac javac class class class dx or d8 dex Proguard
25. Java Java class Java . . . . javac Java javac javac javac class class class dex Proguard Optimization 
 layers dx or d8
26. public static long sumArray(int[] arr){ long sum = 0; for(int i: arr){ sum += i; } return sum; }
27. public static long sumArray(int[] arr){ long sum = 0; for(int i: arr){ sum += i; } return sum; } public static long sumArray(int[]); Code: 0: lconst_0 1: lstore_1 2: aload_0 3: astore_3 4: aload_3 5: arraylength 6: istore 4 8: iconst_0 9: istore 5 11: iload 5 13: iload 4 15: if_icmpge 36 18: aload_3 19: iload 5 21: iaload 22: istore 6 24: lload_1 25: iload 6 27: i2l 28: ladd 29: lstore_1 30: iinc 5, 1 33: goto 11 36: lload_1 37: lreturn .class
28. public static long sumArray(int[] arr){ long sum = 0; for(int i: arr){ sum += i; } return sum; } public static long sumArray(int[]); Code: 0: lconst_0 1: lstore_1 2: aload_0 3: astore_3 4: aload_3 5: arraylength 6: istore 4 8: iconst_0 9: istore 5 11: iload 5 13: iload 4 15: if_icmpge 36 18: aload_3 19: iload 5 21: iaload 22: istore 6 24: lload_1 25: iload 6 27: i2l 28: ladd 29: lstore_1 30: iinc 5, 1 33: goto 11 36: lload_1 37: lreturn .class 11: iload 5 13: iload 4 15: if_icmpge 36 18: aload_3 19: iload 5 21: iaload 22: istore 6 24: lload_1 25: iload 6 27: i2l 28: ladd 29: lstore_1 30: iinc 5, 1 33: goto 11 Dispatches 14 Read 20 Write 16
29. public static long sumArray(int[] arr){ long sum = 0; for(int i: arr){ sum += i; } return sum; }
30. public static long sumArray(int[] arr){ long sum = 0; for(int i: arr){ sum += i; } return sum; } 0000: const-wide/16 v2, #int 0 // #0 0002: array-length v1, v6 0003: const/4 v0, #int 0 // #0 0004: if-ge v0, v1, 000d // +0009 0006: aget v4, v6, v0 0008: int-to-long v4, v4 0009: add-long/2addr v2, v4 000a: add-int/lit8 v0, v0, #int 1 // #01 000c: goto 0004 // -0008 000d: return-wide v2 .dex:
31. public static long sumArray(int[] arr){ long sum = 0; for(int i: arr){ sum += i; } return sum; } 0000: const-wide/16 v2, #int 0 // #0 0002: array-length v1, v6 0003: const/4 v0, #int 0 // #0 0004: if-ge v0, v1, 000d // +0009 0006: aget v4, v6, v0 0008: int-to-long v4, v4 0009: add-long/2addr v2, v4 000a: add-int/lit8 v0, v0, #int 1 // #01 000c: goto 0004 // -0008 000d: return-wide v2 .dex: 0004: if-ge v0, v1, 000d // +0009 0006: aget v4, v6, v0 0008: int-to-long v4, v4 0009: add-long/2addr v2, v4 000a: add-int/lit8 v0, v0, #int 1 // #01 000c: goto 0004 // -0008 Dispatches 6 Reads 10 Writes 6
34. ZipAlign
35. ZipAlign •Android has a 4 byte boundary memory mapping
36. ZipAlign •Misaligned data slows data access performance.
37. ZipAlign •Misaligned data slows data access performance. •ZipAlign step results in aligning all the data in 4 byte boundary.
38. Compilation Installation
40. Dalvik Upto KitKat
41. Dalvik Upto KitKat •Byte Code Verification
42. Dalvik Upto KitKat •Byte Code Verification To check the integrity of Dex
43. Dalvik Upto KitKat •Byte Code Verification To check the integrity of Dex Done at install time rather than run time.
44. Dalvik Upto KitKat •Byte Code Verification To check the integrity of Dex Done at install time rather than run time. Validity of indices and structure of file.
45. Dalvik Upto KitKat •Byte Code Verification To check the integrity of Dex Done at install time rather than run time. Validity of indices and structure of file. Ordering of the instructions.
46. Dalvik Upto KitKat •Optimizations
47. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation.
48. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation. Replacing high volume of methods with their inline versions.
49. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation. Replacing high volume of methods with their inline versions. Prune empty methods.
50. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation. Replacing high volume of methods with their inline versions. Prune empty methods. Pre computations.
51. Dalvik Upto KitKat •Optimizations VTable Method Index for method invocation. Replacing high volume of methods with their inline versions. Prune empty methods. Pre computations. dexopt ODEX
52. ART In Lollipop
53. ART In Lollipop •Preloading boot.art boot.oat
54. ART In Lollipop •Preloading boot.art boot.oat Compiler
 Driver ELF file •Load Compiler
55. ART In Lollipop •Preloading boot.art boot.oat Compiler
 Driver ELF file dex2Oat •Load Compiler (AOT)
56. ART •Preloading boot.art boot.oat Compiler
 Driver ELF file •Load Compiler Dex Resolve classes Verify classes Init classes Compile classes ELF File
57. Problem:
58. Problem:
59. Hybrid: JIT+AOT
60. Hybrid: JIT+AOT From Android Nougat
61. Profile guided Compilation Profile Files App App Execution
 (Interpret/JIT) Profile guided 
 compile Compiled App
62. dex file, oat file
63. dex file, oat file ART
64. dex file, oat file ART Execute
 method
65. dex file, oat file ART Execute
 method Is
 it compiled
 ?
66. dex file, oat file ART Execute
 method Is
 it compiled
 ? Y
67. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y
68. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N
69. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N Run code
 from jit cache Y
70. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Run code
 from jit cache Y
71. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Run code
 from jit cache Y Record Profiles info
72. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? Run code
 from jit cache Y Record Profiles info
73. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? JIT
 Cache Run code
 from jit cache Y Record Profiles info Y
74. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? JIT
 Cache Run code
 from jit cache Y Record Profiles info N Y
75. dex file, oat file ART Execute
 method Is
 it compiled
 ? Is it
 JIT compiled Y Run code
 from .oat file N N Interpreter Is
 it hot? JIT
 Cache Run code
 from jit cache Y Record Profiles info N Y When Profile guided compilation 
 occurs?
76. When Profile guided compilation 
 occurs?
77. Compilation Installation Optimizations
78. “Abstractions increase the productivity of developer but they can be overhead for runtime”
79. Optimizations 1. Constant Folding numberOfDays = 2 + 2 + previous; add-int/lit8 v0, v2, #int 4
80. 2. Inlining fun len (string: String): Int{ return string.length }
81. 2. Inlining fun len (string: String): Int{ return string.length } const-string v0, "string" invoke-static {v1, v0}, Lkotlin/jvm/internal/Intrinsics;.checkParameterIsNotNull invoke-virtual {v1}, Ljava/lang/String;.length move-result v0 return v0
82. 2. Inlining fun len (string: String): Int{ return string.length } const-string v0, "string" invoke-static {v1, v0}, Lkotlin/jvm/internal/Intrinsics;.checkParameterIsNotNull invoke-virtual {v1}, Ljava/lang/String;.length move-result v0 return v0
83. const-string v0, "string" if-nez v1, :continue invoke-static {v1}, throwParameterIsNullException :continue invoke-virtual {v1}, Ljava/lang/String;.length move-result v0 return v0
84. 3. Bounds Check Elimination int sum = 0; for (int i = 0; i < size; i++) { sum += array[i]; }
85. 3. Bounds Check Elimination int sum = 0; for (int i = 0; i < size; i++) { sum += array[i]; }
86. 4. Strength Reduction 5. Dead Code Elimination 6. DexLayout 7. Class Hierarchy Analysis And much more…..
87. Compilation ??
95. MULTIDEX: The 64k LIMIT
96. MULTIDEX: THE AFTERMATH •Adverse effect on App Startup •Build time increases. •Strange errors during runtime •Strangers in Crashyltics..
97. MULTIDEX: THE AFTERMATH •Adverse effect on App Startup •Build time increases. •Strange errors during runtime •Strangers in Crashyltics..
98. MULTIDEX: THE AFTERMATH •Adverse effect on App Startup •Build time increases. •Strange errors during runtime •Strangers in Crashyltics..
99. MULTIDEX: HOW?
100. DirectClass
 FileConsumer .class Validate & Lock ClassTranslatorTask ClassDefItemConsumer Update global indices Release the lock Write Copyright: AOSP
101. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the lock Write Copyright: AOSP Validate & Lock
102. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the lock Write Copyright: AOSP Validate & Lock
103. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the lock Write Copyright: AOSP Validate & Lock
104. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the lock Write Copyright: AOSP Validate & Lock
105. DirectClass
 FileConsumer .class ClassTranslatorTask ClassDefItemConsumer Update global indices Release the lock Write Copyright: AOSP Validate & Lock
106. Compilation Installation Optimizations App
 Startup
107. App Startup: 
 Evolving Better with ART
110. What’s wrong?
112. 10s
114. Time for tips • Use lazy initializations. • Don’t use reflections at time of APP startup • Initialize all the SDKs in background • Load images asynchronously. • Avoid nesting layout
115. Let us all be an ARTist
116. Thank you' @droid_singh amanjeetsingh150 @_romichandra_ romichandra