Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Decoding Android Runtime

Ce1ca64f3265f01a8718a622427f0a1d?s=47 Jitin
September 19, 2019

Decoding Android Runtime

While writing code, we mostly care about that it should be efficient and should not break. How that code executes at runtime is a completely different story in case of Android. The .dex format has been here for years and everyone is accustomed to it. But there are a lot of complexities going on once the .dex reaches your device. This talk will focussed on analyzing on what happens after your code in Kotlin/Java is compiled, packaged and run on an actual device. We will take a deep dive into .dex format as well as see how runtime has evolved since Android 1.0 and taken a great leap with Android 5.0 and above. In the process, we will also learn how to write runtime efficient code.

Ce1ca64f3265f01a8718a622427f0a1d?s=128

Jitin

September 19, 2019
Tweet

More Decks by Jitin

Other Decks in Technology

Transcript

  1. JITIN S HARMA D E C O D I N

    G A N D R O I D R U N T I M E @_jitinsharma jitinsharma.in
  2. @_jitinsharma / jitinsharma.in What is Android -Some Hardware -Some Linux

    -Some C++ -Some Java
  3. @_jitinsharma / jitinsharma.in

  4. @_jitinsharma / jitinsharma.in Boot-Up

  5. @_jitinsharma / jitinsharma.in Boot-Up Hardware Start-Up 02 Software Start-Up ZYGOTE

    03 01
  6. @_jitinsharma / jitinsharma.in Boot-Up init.rc AndroidRuntime.cpp Zygote main() Transition to

    Java based execution Linux Kernel Start
  7. @_jitinsharma / jitinsharma.in Zygote - Zygote Process acts a base

    process from which all processes are forked for subsequent applications. - This allows Android to run in a Multi-Process environment where all processes run independent of each other. - All resources preloaded by Zygote are accessed by subsequent applications through shared memory.
  8. @_jitinsharma / jitinsharma.in Zygote System Server Load core libraries and

    resources Starts Activity Manager Notification Manager, Bluetooth Service, Package Manager, Sensor Service…
  9. @_jitinsharma / jitinsharma.in Zygote

  10. @_jitinsharma / jitinsharma.in Phone is ON

  11. @_jitinsharma / jitinsharma.in Building APK java/kt files .class files .dex

    javac kotlinc dx/D8 APK
  12. @_jitinsharma / jitinsharma.in Installing Application

  13. @_jitinsharma / jitinsharma.in Manifest Parsing - All data from AndroidManifest.xml

    is read and is written to data/system/packages.xml - This file acts as a global source for all packages and is used by PackageManager API - This happens only once either when device is booted or an application is being installed.
  14. @_jitinsharma / jitinsharma.in Manifest Parsing - Identifies various elements within

    your application - At the same time manifest performs checks which can cause installation errors if elements are not defined properly.
  15. @_jitinsharma / jitinsharma.in Code Compilation and Other Stuff

  16. @_jitinsharma / jitinsharma.in Code Compilation and Other Stuff Dalvik ART

  17. @_jitinsharma / jitinsharma.in dex and Dalvik Era Android 1.0 -

    4.4 (2008-2014)
  18. @_jitinsharma / jitinsharma.in .dex Dalvik VM A register based VM

    optimised for memory constrained device .dex -> Dalvik Executable
  19. @_jitinsharma / jitinsharma.in Stack Machines value1 value2 value3 value4 value1

    value2 operation pop pop value5 value3 value4 push
  20. @_jitinsharma / jitinsharma.in Stack Machines Register Machines value1 value2 value3

    value4 value1 value2 operation pop pop value5 value3 value4 push addr1 value1 operation addr2 value2 addr3 value3
  21. @_jitinsharma / jitinsharma.in Stack Machines Register Machines value1 value2 value3

    value4 value1 value2 operation pop pop value5 value3 value4 push addr1 value1 operation addr2 value2 addr3 value3 ARM Optimized
  22. @_jitinsharma / jitinsharma.in dex - Uses 16 bit instruction format.

    - Bytecode tends to be larger due to register based architecture for storing addresses
  23. @_jitinsharma / jitinsharma.in dex - Uses 16 bit instruction format.

    - Bytecode tends to be larger due to register based architecture for storing addresses ‣header ‣string_ids ‣type_ids ‣proto_ids ‣field_ids ‣method_ids ‣class_defs ‣call_site_ids ‣method_handles ‣data ‣link_data
  24. @_jitinsharma / jitinsharma.in dex - Uses 16 bit instruction format.

    - Bytecode tends to be larger due to register based architecture for storing addresses ‣header ‣string_ids ‣type_ids ‣proto_ids ‣field_ids ‣method_ids ‣class_defs ‣call_site_ids ‣method_handles ‣data ‣link_data Contains verification signatures
  25. @_jitinsharma / jitinsharma.in dex

  26. @_jitinsharma / jitinsharma.in dex

  27. @_jitinsharma / jitinsharma.in public class Data { int id; String

    name = ""; public int getId() { return id; } public void setId(int id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } }
  28. @_jitinsharma / jitinsharma.in public int getId() { return id; }

    .METHOD getId : int .MODIFIERS public .REGISTERS 2 .CODE iget v0, v1 field@8046 return v0 .METHOD setName : void .PARAM java.lang.String .MODIFIERS public .REGISTERS 2 .CODE iput-object v1, v0 field@8047 return-void public void setName(String name) { this.name = name; }
  29. @_jitinsharma / jitinsharma.in private fun forEachTest(): Int { var sum

    = 0 (0..50).forEach { sum += it } return sum } private fun forTest(): Int { var sum = 0 for (i in 0 until 50) { sum += i } return sum } VS
  30. @_jitinsharma / jitinsharma.in private fun forTest(): Int { var sum

    = 0 for (i in 0 until 50) { sum += i } return sum } .METHOD forTest : int .MODIFIERS private final .REGISTERS 4 .CODE const/4 v0, #0 const/4 v1, #0 const/16 v2, #50 if-ge v1, v2, 6 add-int/2addr v1, v0 add-int/lit8 v1, v1, #1 goto -7 return v0
  31. @_jitinsharma / jitinsharma.in private fun forEachTest(): Int { var sum

    = 0 (0..50).forEach { sum += it } return sum } .METHOD forEachTest : int .MODIFIERS private final .REGISTERS 8 .CODE const/4 v0, #0 new-instance v1, type@2439 const/4 v2, #0 const/16 v3, #50 invoke-direct {v1, v2, v3}, meth@20280 check-cast v1, type@1851 move v3, v2 invoke-interface {v1}, meth@15176 move-result-object v4 invoke-interface {v4}, meth@15640 move-result v5 if-eqz v5, 13 move-object v5, v4 check-cast v5, type@2142 invoke-virtual {v5}, meth@18564 move-result v5 move v6, v5 add-int/2addr v6, v0 goto -16 return v0
  32. @_jitinsharma / jitinsharma.in A.class B.class Z.class classes.dex

  33. @_jitinsharma / jitinsharma.in A.class B.class Z.class classes.dex - Reduction in

    bytecode size during conversion. - Duplicate constants from class files are converted to a single one. - Each method needs have a reference which is limited by dex instruction format of 16 bits, hence total method references which a single dex file can hold is 65,536 (2^16)
  34. @_jitinsharma / jitinsharma.in dex++

  35. @_jitinsharma / jitinsharma.in Multidexing classes.dex java/kt classes2.dex classesN.dex …

  36. @_jitinsharma / jitinsharma.in Installation data/dalvik-cache data/data/com.example.myapplication

  37. @_jitinsharma / jitinsharma.in dexopt -Verification and Optimization of contents of

    .dex file -Result differ from device to device. -Starts a small version of VM to optimize dex files. -Contents of system/framework/ can be used to optimize dex files outside Android OS. -Code is optimized for runtime not memory.
  38. @_jitinsharma / jitinsharma.in classes.dex My Application Zygote Process

  39. @_jitinsharma / jitinsharma.in Multidex.install(this) classes.dex classes2.dex classes3.dex …. System Application

    classesN.dex Inject values into ClassLoader
  40. @_jitinsharma / jitinsharma.in Multidex.install(this) classes.dex classes2.dex classes3.dex …. System Application

    Must have Application class classesN.dex Inject values into ClassLoader
  41. @_jitinsharma / jitinsharma.in Multidex.install(this) classes.dex classes2.dex classes3.dex …. System Application

    Must have Application class Library dependencies classesN.dex Inject values into ClassLoader
  42. @_jitinsharma / jitinsharma.in Multidex.install(this) classes.dex classes2.dex classes3.dex …. System Application

    Must have Application class Library dependencies (multiDexKeepFile) classesN.dex Inject values into ClassLoader
  43. @_jitinsharma / jitinsharma.in DexClassLoader DexClassLoader dexClassLoader = new DexClassLoader(dexFile.getAbsolutePath(), getCodeCacheDir().getAbsolutePath(),

    null, context.getClassLoader());
  44. @_jitinsharma / jitinsharma.in DexClassLoader DexClassLoader dexClassLoader = new DexClassLoader(dexFile.getAbsolutePath(), getCodeCacheDir().getAbsolutePath(),

    null, context.getClassLoader()); Network
  45. @_jitinsharma / jitinsharma.in DexClassLoader DexClassLoader dexClassLoader = new DexClassLoader(dexFile.getAbsolutePath(), getCodeCacheDir().getAbsolutePath(),

    null, context.getClassLoader()); Class<?> dynamicClass = dexClassLoader.loadClass("com.example.myapplication.runtime"); Method dynamicMethod = dynamicClass.getDeclaredMethod("executeDataLoad", Integer.class, Integer.class); Integer value = (Integer) dynamicMethod.invoke(dynamicClass.newInstance(), 1, 2); Network
  46. @_jitinsharma / jitinsharma.in DexClassLoader DexClassLoader dexClassLoader = new DexClassLoader(dexFile.getAbsolutePath(), getCodeCacheDir().getAbsolutePath(),

    null, context.getClassLoader()); Class<?> dynamicClass = dexClassLoader.loadClass("com.example.myapplication.runtime"); Method dynamicMethod = dynamicClass.getDeclaredMethod("executeDataLoad", Integer.class, Integer.class); Integer value = (Integer) dynamicMethod.invoke(dynamicClass.newInstance(), 1, 2); Network public Integer executeDataLoad(Integer value1, Integer value2) { // return result; }
  47. @_jitinsharma / jitinsharma.in DexClassLoader DexClassLoader dexClassLoader = new DexClassLoader(dexFile.getAbsolutePath(), getCodeCacheDir().getAbsolutePath(),

    null, context.getClassLoader()); Class<?> dynamicClass = dexClassLoader.loadClass("com.example.myapplication.runtime"); Method dynamicMethod = dynamicClass.getDeclaredMethod("executeDataLoad", Integer.class, Integer.class); Integer value = (Integer) dynamicMethod.invoke(dynamicClass.newInstance(), 1, 2); Network public Integer executeDataLoad(Integer value1, Integer value2) { // return result; }
  48. @_jitinsharma / jitinsharma.in JIT

  49. @_jitinsharma / jitinsharma.in JIT private void handleIntent() { ... public

    RequestQueue getRequestQueue() { if (requestQueue == null) { requestQueue = Volley.newRequestQueue(this); } return requestQueue; } for (String s : networkRequestMap.keySet()) { if (BuildConfig.DEBUG) { //
  50. @_jitinsharma / jitinsharma.in Tracing and Profiling Low Memory Optimized (100K/200K)

    Faster results Google I/O 2010
  51. @_jitinsharma / jitinsharma.in Shortcomings of Dalvik Time

  52. @_jitinsharma / jitinsharma.in ART (Android Runtime) Android 5.0 - present

    (2014-present)
  53. @_jitinsharma / jitinsharma.in Embracing the GBs…

  54. @_jitinsharma / jitinsharma.in

  55. @_jitinsharma / jitinsharma.in

  56. @_jitinsharma / jitinsharma.in Embracing the GBs…

  57. @_jitinsharma / jitinsharma.in Revamped GC -ART keeps track of application

    going in non-UI state. -Seperate types of GC for low memory and non-low memory devices. -GC pauses reduced from 2 to 1. -Larger objects like bitmap have seperate heap for faster allocation. -Young Objects are moved to stack for faster reclamation of memory.
  58. @_jitinsharma / jitinsharma.in Dalvik GC GC GC 1MB 3MB 5MB

  59. @_jitinsharma / jitinsharma.in ART GC GC GC 1MB 3MB 5MB

  60. @_jitinsharma / jitinsharma.in ART GC GC GC 1MB 3MB 5MB

    9MB Heap Compaction
  61. @_jitinsharma / jitinsharma.in AOT Compilation classes.dex classes2.dex classesN.dex .oat dex2oat

    - AOT works on principle of compile once and then run. - Compilation takes significant time and memory since whole bytecode is converted to machine code at installation. - AOT compilers do not benefit from profile guided compilation which JIT compilers have instead use a restrictive approach to do compilation. Android Runtime VM
  62. @_jitinsharma / jitinsharma.in .oat header instruction set dex file count

    dex2oat cmd line code data/app/
  63. @_jitinsharma / jitinsharma.in DEX CODE: 0x0000: 5210 e10f | iget

    v0, v1, I com.example.myapplication.Data.id // field@4065 0x0002: 0f00 | return v0 public int getId() { return id; } Java
  64. @_jitinsharma / jitinsharma.in CODE: (code_offset=0x00d6a3fc size_offset=0x00d6a3f8 size=68)... 0x00d6a3fc: 4885842400E0FFFF testq

    rax, [rsp + -8192] suspend point dex PC: 0x0000 GC map objects: v1 ([sp + #40]) 0x00d6a404: 4883EC18 subq rsp, 24 0x00d6a408: 48893C24 movq [rsp], rdi 0x00d6a40c: 89742428 mov [rsp + 40], esi 0x00d6a410: 6566833C250000000000 cmpw gs:[0], 0 ; state_and_flags 0x00d6a41a: 0F8516000000 jnz/ne +22 (0x00d6a436) 0x00d6a420: 8B442428 mov eax, [rsp + 40] 0x00d6a424: 8500 test eax, [rax] suspend point dex PC: 0x0000 GC map objects: v1 ([sp + #40]) 0x00d6a426: 8B480C mov ecx, [rax + 12] suspend point dex PC: 0x0000 GC map objects: v1 ([sp + #40]) 0x00d6a429: 894C2410 mov [rsp + 16], ecx 0x00d6a42d: 8B442410 mov eax, [rsp + 16] 0x00d6a431: 4883C418 addq rsp, 24 0x00d6a435: C3 ret 0x00d6a436: 65FF142538040000 call gs:[1080] ; pTestSuspend ……..
  65. @_jitinsharma / jitinsharma.in Problems with AOT compilation

  66. @_jitinsharma / jitinsharma.in

  67. @_jitinsharma / jitinsharma.in JIT/AOT Hybrid ‣System boot after OS update

    will no longer optimise the app, a job solely performed by JIT. ‣dex2oat is still used for .oat creation but it uses profile data generated by JIT to do so. ‣Code may be executed by directed .dex interpretation or JIT cache or AOT compiled. ‣At any time JIT compiled data will be more efficient because of availability of better environment data than AOT compiled code, and the same is preferred if both are present.
  68. @_jitinsharma / jitinsharma.in DEX CODE: 0x0000: 5210 e10f | iget

    v0, v1, I com.example.myapplication.Data.id // field@4065 0x0002: 0f00 | return v0 public int getId() { return id; } Java CODE: (code_offset=0x00000000 size_offset=0x00aea110 size=0) NO CODE!
  69. @_jitinsharma / jitinsharma.in Cloud Profiles

  70. @_jitinsharma / jitinsharma.in Third party frameworks

  71. @_jitinsharma / jitinsharma.in Future

  72. @_jitinsharma / jitinsharma.in Future A.I

  73. @_jitinsharma / jitinsharma.in Thanks