Obfuscation: The Fundamentals

Understand the Android Build Pipeline and how obfuscation comes into play.

Transcript

1. $ export ANDROID_HOME=/path/to/my/sdk (on mac this is typically ~/Library/Android/sdk/)
2. $ export ANDROID_HOME=/path/to/my/sdk (on mac this is typically ~/Library/Android/sdk/) $ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
3. $ export ANDROID_HOME=/path/to/my/sdk (on mac this is typically ~/Library/Android/sdk/) $ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git Cloning into 'depot_tools'... remote: Sending approximately 25.76 MiB ... remote: Counting objects: 3, done remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done.
4. $ export ANDROID_HOME=/path/to/my/sdk (on mac this is typically ~/Library/Android/sdk/) $ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git Cloning into 'depot_tools'... remote: Sending approximately 25.76 MiB ... remote: Counting objects: 3, done remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done. $ export PATH=$PATH:`pwd`/depot_tools
5. $ export ANDROID_HOME=/path/to/my/sdk (on mac this is typically ~/Library/Android/sdk/) $ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git Cloning into 'depot_tools'... remote: Sending approximately 25.76 MiB ... remote: Counting objects: 3, done remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done. $ export PATH=$PATH:`pwd`/depot_tools $ git clone https://r8.googlesource.com/r8
6. $ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git Cloning into 'depot_tools'... remote: Sending approximately 25.76 MiB ... remote: Counting objects: 3, done remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done. $ export PATH=$PATH:`pwd`/depot_tools $ git clone https://r8.googlesource.com/r8 Cloning into 'r8'... remote: Sending approximately 38.71 MiB ... remote: Counting objects: 386, done remote: Finding sources: 100% (218/218) remote: Total 107455 (delta 48242), reused 107418 (delta 48242) Receiving objects: 100% (107455/107455), 38.60 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (48242/48242), done.
7. $ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git Cloning into 'depot_tools'... remote: Sending approximately 25.76 MiB ... remote: Counting objects: 3, done remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done. $ export PATH=$PATH:`pwd`/depot_tools $ git clone https://r8.googlesource.com/r8 Cloning into 'r8'... remote: Sending approximately 38.71 MiB ... remote: Counting objects: 386, done remote: Finding sources: 100% (218/218) remote: Total 107455 (delta 48242), reused 107418 (delta 48242) Receiving objects: 100% (107455/107455), 38.60 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (48242/48242), done. $ cd r8
8. $ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git Cloning into 'depot_tools'... remote: Sending approximately 25.76 MiB ... remote: Counting objects: 3, done remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done. $ export PATH=$PATH:`pwd`/depot_tools $ git clone https://r8.googlesource.com/r8 Cloning into 'r8'... remote: Sending approximately 38.71 MiB ... remote: Counting objects: 386, done remote: Finding sources: 100% (218/218) remote: Total 107455 (delta 48242), reused 107418 (delta 48242) Receiving objects: 100% (107455/107455), 38.60 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (48242/48242), done. $ cd r8 $ export R8_HOME=`pwd`
9. remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done. $ export PATH=$PATH:`pwd`/depot_tools $ git clone https://r8.googlesource.com/r8 Cloning into 'r8'... remote: Sending approximately 38.71 MiB ... remote: Counting objects: 386, done remote: Finding sources: 100% (218/218) remote: Total 107455 (delta 48242), reused 107418 (delta 48242) Receiving objects: 100% (107455/107455), 38.60 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (48242/48242), done. $ cd r8 $ export R8_HOME=`pwd` $ tools/gradle.py d8 r8
10. remote: Finding sources: 100% (3/3) remote: Total 33588 (delta 23410), reused 33587 (delta 23410) Receiving objects: 100% (33588/33588), 25.75 MiB | 9.92 MiB/s, done. Resolving deltas: 100% (23410/23410), done. $ export PATH=$PATH:`pwd`/depot_tools $ git clone https://r8.googlesource.com/r8 Cloning into 'r8'... remote: Sending approximately 38.71 MiB ... remote: Counting objects: 386, done remote: Finding sources: 100% (218/218) remote: Total 107455 (delta 48242), reused 107418 (delta 48242) Receiving objects: 100% (107455/107455), 38.60 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (48242/48242), done. $ cd r8 $ export R8_HOME=`pwd` $ tools/gradle.py d8 r8 (... gradle output removed…) (... this takes time...)
11. Obfuscation: The Fundamentals Presented By: Dustin Summers
12. Overview • Introduction • Compilation in Action ◦ Test File 1 • Compiling with DX ◦ Test File 1 • Multiple Classes > One Dex ◦ Test File 2 • Introducing… Lambdas! ◦ Test File 3 • Compiling with D8 ◦ Test File 3 • Dedupe with D8 ◦ Test File 4 • Compiling with R8 / Looking at APKs ◦ Test File 5
13. $ git clone https://r8.googlesource.com/r8 Cloning into 'r8'... remote: Sending approximately 38.71 MiB ... remote: Counting objects: 386, done remote: Finding sources: 100% (218/218) remote: Total 107455 (delta 48242), reused 107418 (delta 48242) Receiving objects: 100% (107455/107455), 38.60 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (48242/48242), done. $ cd r8 $ tools/gradle.py d8 r8 (... gradle output removed…) (... this takes time...) $ BUILD SUCCESSFUL in 1m 57s
14. $ git clone https://r8.googlesource.com/r8 Cloning into 'r8'... remote: Sending approximately 38.71 MiB ... remote: Counting objects: 386, done remote: Finding sources: 100% (218/218) remote: Total 107455 (delta 48242), reused 107418 (delta 48242) Receiving objects: 100% (107455/107455), 38.60 MiB | 4.72 MiB/s, done. Resolving deltas: 100% (48242/48242), done. $ cd r8 $ tools/gradle.py d8 r8 (... gradle output removed…) (... this takes time...) $ BUILD SUCCESSFUL in 1m 57s #ONLY DOWNLOAD IF INTERESTED!!! $ tools/test.py --no-internal (... gradle output removed…) (... this takes even longer...)
15. Goals - Comfortable reading Dalvik/Smali - Understand Android Compilation, and how it plays into Obfuscation - The stages that transform source code into bytecode
16. What this course will not teach... - Various keep rules and examples (follow-on lesson) - How to hack...
17. Whats in an APK?
18. Why .dex and not just .class? - Typical Java runs multiple .class files in JVM (jar) - .dex optimized to run on Android in Android VM’s (Dalvik/ART) - Combine everything from all .class files into one .dex file - or multiple if you hit the 64k method limit and use the -multidex option) - Duplicate strings/other constants used in multiple class files can be included only once in the .dex output to conserve space.
19. Compilation Pipeline Source TestFile1.java (Java/Kotlin) Source Compilers Javac/Kotlinc ByteCode TestFile1.class
20. Compilation in Action Test File 1
21. $ cd .. #(Note: You can make a separate directory for test files if you wish) $ git clone https://github.com/dustinsummers/obfuscation.git $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java
22. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java
23. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ java TestFile1
24. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ java TestFile1 Hello, I'm compiled now
25. Compiling with DX Source TestFile1.java (Java/Kotlin) Source Compilers Javac/Kotlinc ByteCode TestFile1.class Dex Compilers DX Dalvik Bytecode classes.dex
26. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ dx --dex --verbose TestFile1.class
27. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ dx --dex --verbose --output=TestFile1.dex TestFile1.class processing TestFile1.class... │
28. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ dx --dex --verbose --output=TestFile1.dex TestFile1.class processing TestFile1.class... │ $ ls
29. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ dx --dex --verbose --output=TestFile1.dex TestFile1.class processing TestFile1.class... │ $ ls TestFile1.class TestFile1.dex TestFile1.java
30. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ dx --dex --verbose --output=TestFile1.dex TestFile1.class processing TestFile1.class... │ $ ls TestFile1.class TestFile1.dex TestFile1.java #Or just type in dexdump below if you have it mapped it like I do : ) $ $ANDROID_HOME/build-tools/29.0.1/dexdump -d TestFile1.dex
31. $ cd obfuscation/TestFile1 $ ls TestFile1.java $ javac -source 1.7 -target 1.7 TestFile1.java $ ls TestFile1.class TestFile1.java $ dx --dex --verbose --output=TestFile1.dex TestFile1.class processing TestFile1.class... │ $ ls TestFile1.class TestFile1.dex TestFile1.java #Or just type in dexdump below if you have it mapped it like I do : ) $ $ANDROID_HOME/build-tools/29.0.1/dexdump -d TestFile1.dex $ export PATH=$PATH:$ANDROID_HOME/build-tools/BUILD_TOOLS_#/
32. Multiple Classes > One Dex Test File 2
33. $ cd ../TestFile2
34. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java
35. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java $ ls Message.class TestFile2$Messenger.class TestFile2$PostMan.class TestFile2.class TestFile2.java
36. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java $ ls Message.class TestFile2$Messenger.class TestFile2$PostMan.class TestFile2.class TestFile2.java $ java TestFile2 Sending a message: I’m implementing interfaces!!
37. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java $ ls Message.class TestFile2$Messenger.class TestFile2$PostMan.class TestFile2.class TestFile2.java $ java TestFile2 Sending a message: I’m implementing interfaces!! $ dx --dex --verbose *.class
38. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java $ ls Message.class TestFile2$Messenger.class TestFile2$PostMan.class TestFile2.class TestFile2.java $ java TestFile2 Sending a message: I’m implementing interfaces!! $ dx --dex --verbose *.class processing Message.class... processing TestFile2$Messenger.class... processing TestFile2$PostMan.class... processing TestFile2.class...
39. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java $ ls Message.class TestFile2$Messenger.class TestFile2$PostMan.class TestFile2.class TestFile2.java $ java TestFile2 Sending a message: I’m implementing interfaces!! $ dx --dex --verbose *.class processing Message.class... processing TestFile2$Messenger.class... processing TestFile2$PostMan.class... processing TestFile2.class... $ ls
40. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java $ ls Message.class TestFile2$Messenger.class TestFile2$PostMan.class TestFile2.class TestFile2.java $ java TestFile2 Sending a message: I’m implementing interfaces!! $ dx --dex --verbose *.class processing Message.class... processing TestFile2$Messenger.class... processing TestFile2$PostMan.class... processing TestFile2.class... $ ls Message.class TestFile2$PostMan.class TestFile2.java classes.dex TestFile2$Messenger.class TestFile2.class
41. $ cd ../TestFile2 $ javac -source 1.7 -target 1.7 TestFile2.java $ ls Message.class TestFile2$Messenger.class TestFile2$PostMan.class TestFile2.class TestFile2.java $ java TestFile2 Sending a message: I’m implementing interfaces!! $ dx --dex --verbose *.class processing Message.class... processing TestFile2$Messenger.class... processing TestFile2$PostMan.class... processing TestFile2.class... $ ls Message.class TestFile2$PostMan.class TestFile2.java classes.dex TestFile2$Messenger.class TestFile2.class $ dexdump -d classes0.dex
42. Processing 'TestFile2.dex'... Opened 'TestFile2.dex', DEX version '035' Class #0 - Class descriptor : 'LTestFile2$PostMan;' Access flags : 0x0600 (INTERFACE ABSTRACT) Superclass : 'Ljava/lang/Object;' Interfaces - Static fields - Instance fields - Direct methods - Virtual methods - #0 : (in LTestFile2$PostMan;) name : 'letterTime' type : '(Ljava/lang/String;)V' access : 0x0401 (PUBLIC ABSTRACT) code : (none) source_file_idx : 18 (TestFile2.java) Class #1 - Class descriptor : 'LTestFile2$Messenger;' Access flags : 0x0000 () Superclass : 'Ljava/lang/Object;' Interfaces - #0 : 'LTestFile2$PostMan;' Static fields - Instance fields - Direct methods - #0 : (in LTestFile2$Messenger;) name : '<init>' type : '()V' access : 0x10000 (CONSTRUCTOR) code - registers : 1 ins : 1 outs : 1 insns size : 4 16-bit code units 00024c: |[00024c] TestFile2$Messenger.<init>:()V 00025c: 7010 0700 0000 |0000: invoke-direct {v0}, Ljava/lang/Object;.<init>:()V // method@0007 000262: 0e00 |0003: return-void catches : (none) positions : 0x0000 line=7 locals : 0x0000 - 0x0004 reg=0 this LTestFile2$Messenger; Virtual methods - #0 : (in LTestFile2$Messenger;) name : 'letterTime' type : '(Ljava/lang/String;)V' access : 0x0001 (PUBLIC) code - registers : 5 ins : 2 outs : 2 insns size : 25 16-bit code units 000264: |[000264] TestFile2$Messenger.letterTime:(Ljava/lang/String;)V 000274: 6200 0000 |0000: sget-object v0, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0000 000278: 2201 0900 |0002: new-instance v1, Ljava/lang/StringBuilder; // type@0009 00027c: 7010 0800 0100 |0004: invoke-direct {v1}, Ljava/lang/StringBuilder;.<init>:()V // method@0008 000282: 1a02 1100 |0007: const-string v2, "Sending a message: " // string@0011 000286: 6e20 0900 2100 |0009: invoke-virtual {v1, v2}, Ljava/lang/StringBuilder;.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; // method@0009 00028c: 0c01 |000c: move-result-object v1 00028e: 6e20 0900 4100 |000d: invoke-virtual {v1, v4}, Ljava/lang/StringBuilder;.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; // method@0009 000294: 0c01 |0010: move-result-object v1 000296: 6e10 0a00 0100 |0011: invoke-virtual {v1}, Ljava/lang/StringBuilder;.toString:()Ljava/lang/String; // method@000a 00029c: 0c01 |0014: move-result-object v1 00029e: 6e20 0600 1000 |0015: invoke-virtual {v0, v1}, Ljava/io/PrintStream;.println:(Ljava/lang/String;)V // method@0006 0002a4: 0e00 |0018: return-void catches : (none) positions : 0x0000 line=11 0x0018 line=12 locals : 0x0000 - 0x0019 reg=3 this LTestFile2$Messenger; source_file_idx : 18 (TestFile2.java) Class #2 - Class descriptor : 'LTestFile2;' Access flags : 0x0000 () Superclass : 'Ljava/lang/Object;' Interfaces - Static fields - Instance fields - Direct methods - #0 : (in LTestFile2;) name : '<init>' type : '()V' access : 0x10000 (CONSTRUCTOR) code - registers : 1 ins : 1 outs : 1 insns size : 4 16-bit code units 0002a8: |[0002a8] TestFile2.<init>:()V 0002b8: 7010 0700 0000 |0000: invoke-direct {v0}, Ljava/lang/Object;.<init>:()V // method@0007 0002be: 0e00 |0003: return-void catches : (none) positions : 0x0000 line=1 locals : 0x0000 - 0x0004 reg=0 this LTestFile2; #1 : (in LTestFile2;) name : 'main' type : '([Ljava/lang/String;)V' access : 0x0009 (PUBLIC STATIC) code - registers : 2 ins : 1 outs : 1 insns size : 9 16-bit code units 0002c0: |[0002c0] TestFile2.main:([Ljava/lang/String;)V 0002d0: 2200 0000 |0000: new-instance v0, LTestFile2$Messenger; // type@0000 0002d4: 7010 0000 0000 |0002: invoke-direct {v0}, LTestFile2$Messenger;.<init>:()V // method@0000 0002da: 7110 0500 0000 |0005: invoke-static {v0}, LTestFile2;.sendMessage:(LTestFile2$Messenger;)V // method@0005 0002e0: 0e00 |0008: return-void catches : (none) positions : 0x0000 line=17 0x0008 line=18 locals : #2 : (in LTestFile2;) name : 'sendMessage' type : '(LTestFile2$Messenger;)V' access : 0x000a (PRIVATE STATIC) code - registers : 2
43. Introducing… Lambdas!!!! TestFile3
44. $ cd ../TestFile3
45. $ cd ../TestFile3 $ javac TestFile3.java
46. $ cd ../TestFile3 $ javac TestFile3.java $ ls TestFile3.class TestFile3.java 'TestFile3$PostMan.class'
47. $ cd ../TestFile3 $ javac TestFile3.java $ ls TestFile3.class TestFile3.java 'TestFile3$PostMan.class' $ java TestFile3 Sending a message: I'm Using Lamdas
48. $ cd ../TestFile3 $ javac TestFile3.java $ ls TestFile3.class TestFile3.java 'TestFile3$PostMan.class' $ java TestFile3 Sending a message: I'm Using Lamdas $ dx --dex --verbose *.class
49. $ cd ../TestFile3 $ javac TestFile3.java $ ls TestFile3.class TestFile3.java 'TestFile3$PostMan.class' $ java TestFile3 Sending a message: I'm Using Lamdas $ dx --dex --verbose --output=TestFile3.dex *.class processing TestFile3$PostMan.class... Uncaught translation error: com.android.dx.cf.code.SimException: ERROR in TestFile3.main:([Ljava/lang/String;)V: invalid opcode ba - invokedynamic requires --min-sdk-version >= 26 (currently 13) 1 error; aborting
50. What Happened? Source TestFile3.java (Java/Kotlin) Source Compilers Javac/Kotlinc ByteCode TestFile3.class Dex Compilers DX Dalvik Bytecode TestFile3.dex
51. Compilation Pipeline Transformers DeSugar Source TestFile3.java (Java/Kotlin) Source Compilers Javac/Kotlinc
52. $ dx --dex --verbose --output=TestFile2.dex *.class processing TestFile3$PostMan.class... PARSE ERROR: unsupported class file version 55.0 ...while parsing TestFile3$PostMan.class processing TestFile3.class... PARSE ERROR: unsupported class file version 55.0 ...while parsing TestFile3.class 2 errors; aborting $ javac -source 1.7 -target 1.7 TestFile3.java warning: [options] bootstrap class path not set in conjunction with -source 7 TestFile3.java:7: error: lambda expressions are not supported in -source 7 PostMan postMan = message -> System.out.println(message); ^ (use -source 8 or higher to enable lambda expressions) 1 error 1 warning
53. Compilation Pipeline ByteCode TestFile3.class Transformers DeSugar Source TestFile3.java (Java/Kotlin) Source Compilers Javac/Kotlinc Dex Compilers DX
54. Compilation Pipeline ByteCode TestFile3.class Transformers DeSugar Source TestFile3.java (Java/Kotlin) Source Compilers Javac/Kotlinc Dex Compilers DX, D8(DeSugar) Dalvik Bytecode TestFile3.dex
55. DeSugaring in Action TestFile3… continued
56. $ dx --dex --verbose --output=TestFile2.dex *.class processing TestFile3$PostMan.class... PARSE ERROR: unsupported class file version 55.0 ...while parsing TestFile3$PostMan.class processing TestFile3.class... PARSE ERROR: unsupported class file version 55.0 ...while parsing TestFile3.class 2 errors; aborting $ javac -source 1.7 -target 1.7 TestFile3.java warning: [options] bootstrap class path not set in conjunction with -source 7 TestFile3.java:7: error: lambda expressions are not supported in -source 7 PostMan postMan = message -> System.out.println(message); ^ (use -source 8 or higher to enable lambda expressions) 1 error 1 warning
57. PARSE ERROR: unsupported class file version 55.0 ...while parsing TestFile3.class 2 errors; aborting $ javac -source 1.7 -target 1.7 TestFile3.java warning: [options] bootstrap class path not set in conjunction with -source 7 TestFile3.java:7: error: lambda expressions are not supported in -source 7 PostMan postMan = message -> System.out.println(message); ^ (use -source 8 or higher to enable lambda expressions) 1 error 1 warning $ java -jar $R8_HOME/build/libs/d8.jar \ --lib $ANDROID_HOME/platforms/android-29/android.jar \ --release \ --output . \ *.class
58. PARSE ERROR: unsupported class file version 55.0 ...while parsing TestFile3.class 2 errors; aborting $ javac -source 1.7 -target 1.7 TestFile3.java warning: [options] bootstrap class path not set in conjunction with -source 7 TestFile3.java:7: error: lambda expressions are not supported in -source 7 PostMan postMan = message -> System.out.println(message); ^ (use -source 8 or higher to enable lambda expressions) 1 error 1 warning $ java -jar $R8_HOME/build/libs/d8.jar \ --lib $ANDROID_HOME/platforms/android-29/android.jar \ --release \ --output . \ *.class $ ls
59. PARSE ERROR: unsupported class file version 55.0 ...while parsing TestFile3.class 2 errors; aborting $ javac -source 1.7 -target 1.7 TestFile3.java warning: [options] bootstrap class path not set in conjunction with -source 7 TestFile3.java:7: error: lambda expressions are not supported in -source 7 PostMan postMan = message -> System.out.println(message); ^ (use -source 8 or higher to enable lambda expressions) 1 error 1 warning $ java -jar $R8_HOME/build/libs/d8.jar \ --lib $ANDROID_HOME/platforms/android-29/android.jar \ --release \ --output . \ *.class $ ls classes.dex TestFile3.class TestFile3.java 'TestFile3$PostMan.class'
60. $ javac -source 1.7 -target 1.7 TestFile3.java warning: [options] bootstrap class path not set in conjunction with -source 7 TestFile3.java:7: error: lambda expressions are not supported in -source 7 PostMan postMan = message -> System.out.println(message); ^ (use -source 8 or higher to enable lambda expressions) 1 error 1 warning $ java -jar $R8_HOME/build/libs/d8.jar \ --lib $ANDROID_HOME/platforms/android-29/android.jar \ --release \ --output . \ *.class $ ls classes.dex TestFile3.class TestFile3.java 'TestFile3$PostMan.class' $ dexdump -d classes.dex > TestFile3_dump.txt
61. [000280] TestFile3.main:([Ljava/lang/String;)V 0000: sget-object v1, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 0002: const-string v0, "Sending a message: I'm Using Lamdas" // string@0011 0004: invoke-interface {v1, v0}, LTestFile3$PostMan;.letterTime:(Ljava/lang/Object;)V // method@0003 0007: return-void [000264] TestFile3.lambda$main$0:(Ljava/lang/Object;)V 0000: sget-object v0, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0001 0002: invoke-virtual {v0, v1}, Ljava/io/PrintStream;.println:(Ljava/lang/Object;)V // method@0007 0005: return-void |[0001fc] -$$Lambda$TestFile3$1.<clinit>:()V |0000: new-instance v0, L-$$Lambda$TestFile3$1; // type@0000 |0002: invoke-direct {v0}, L-$$Lambda$TestFile3$1;.<init>:()V // method@0001 |0005: sput-object v0, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 |0007: return-void |[000234] -$$Lambda$TestFile3$1.letterTime:(Ljava/lang/Object;)V |0000: invoke-static {v1}, LTestFile3;.lambda$main$0:(Ljava/lang/Object;)V // method@0005 |0003: return-void
62. [000280] TestFile3.main:([Ljava/lang/String;)V 0000: sget-object v1, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 0002: const-string v0, "Sending a message: I'm Using Lamdas" // string@0011 0004: invoke-interface {v1, v0}, LTestFile3$PostMan;.letterTime:(Ljava/lang/Object;)V // method@0003 0007: return-void [000264] TestFile3.lambda$main$0:(Ljava/lang/Object;)V 0000: sget-object v0, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0001 0002: invoke-virtual {v0, v1}, Ljava/io/PrintStream;.println:(Ljava/lang/Object;)V // method@0007 0005: return-void |[0001fc] -$$Lambda$TestFile3$1.<clinit>:()V |0000: new-instance v0, L-$$Lambda$TestFile3$1; // type@0000 |0002: invoke-direct {v0}, L-$$Lambda$TestFile3$1;.<init>:()V // method@0001 |0005: sput-object v0, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 |0007: return-void |[000234] -$$Lambda$TestFile3$1.letterTime:(Ljava/lang/Object;)V |0000: invoke-static {v1}, LTestFile3;.lambda$main$0:(Ljava/lang/Object;)V // method@0005 |0003: return-void
63. [000280] TestFile3.main:([Ljava/lang/String;)V 0000: sget-object v1, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 0002: const-string v0, "Sending a message: I'm Using Lamdas" // string@0011 0004: invoke-interface {v1, v0}, LTestFile3$PostMan;.letterTime:(Ljava/lang/Object;)V // method@0003 0007: return-void [000264] TestFile3.lambda$main$0:(Ljava/lang/Object;)V 0000: sget-object v0, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0001 0002: invoke-virtual {v0, v1}, Ljava/io/PrintStream;.println:(Ljava/lang/Object;)V // method@0007 0005: return-void |[0001fc] -$$Lambda$TestFile3$1.<clinit>:()V |0000: new-instance v0, L-$$Lambda$TestFile3$1; // type@0000 |0002: invoke-direct {v0}, L-$$Lambda$TestFile3$1;.<init>:()V // method@0001 |0005: sput-object v0, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 |0007: return-void |[000234] -$$Lambda$TestFile3$1.letterTime:(Ljava/lang/Object;)V |0000: invoke-static {v1}, LTestFile3;.lambda$main$0:(Ljava/lang/Object;)V // method@0005 |0003: return-void
64. [000280] TestFile3.main:([Ljava/lang/String;)V 0000: sget-object v1, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 0002: const-string v0, "Sending a message: I'm Using Lamdas" // string@0011 0004: invoke-interface {v1, v0}, LTestFile3$PostMan;.letterTime:(Ljava/lang/Object;)V // method@0003 0007: return-void [000264] TestFile3.lambda$main$0:(Ljava/lang/Object;)V 0000: sget-object v0, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0001 0002: invoke-virtual {v0, v1}, Ljava/io/PrintStream;.println:(Ljava/lang/Object;)V // method@0007 0005: return-void |[0001fc] -$$Lambda$TestFile3$1.<clinit>:()V |0000: new-instance v0, L-$$Lambda$TestFile3$1; // type@0000 |0002: invoke-direct {v0}, L-$$Lambda$TestFile3$1;.<init>:()V // method@0001 |0005: sput-object v0, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 |0007: return-void |[000234] -$$Lambda$TestFile3$1.letterTime:(Ljava/lang/Object;)V |0000: invoke-static {v1}, LTestFile3;.lambda$main$0:(Ljava/lang/Object;)V // method@0005 |0003: return-void
65. [000280] TestFile3.main:([Ljava/lang/String;)V 0000: sget-object v1, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 0002: const-string v0, "Sending a message: I'm Using Lamdas" // string@0011 0004: invoke-interface {v1, v0}, LTestFile3$PostMan;.letterTime:(Ljava/lang/Object;)V // method@0003 0007: return-void [000264] TestFile3.lambda$main$0:(Ljava/lang/Object;)V 0000: sget-object v0, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0001 0002: invoke-virtual {v0, v1}, Ljava/io/PrintStream;.println:(Ljava/lang/Object;)V // method@0007 0005: return-void |[0001fc] -$$Lambda$TestFile3$1.<clinit>:()V |0000: new-instance v0, L-$$Lambda$TestFile3$1; // type@0000 |0002: invoke-direct {v0}, L-$$Lambda$TestFile3$1;.<init>:()V // method@0001 |0005: sput-object v0, L-$$Lambda$TestFile3$1;.INSTANCE:L-$$Lambda$TestFile3$1; // field@0000 |0007: return-void |[000234] -$$Lambda$TestFile3$1.letterTime:(Ljava/lang/Object;)V |0000: invoke-static {v1}, LTestFile3;.lambda$main$0:(Ljava/lang/Object;)V // method@0005 |0003: return-void
66. Quick side notes... - “Instance” fields contain Hash - D8 can deduplicate method references - Saves on overall method count - TestFile4
67. $ cd ../TestFile4
68. $ cd ../TestFile4 $ javac TestFile4.java
69. $ cd ../TestFile4 $ javac TestFile4.java $ java TestFile4
70. $ cd ../TestFile4 $ javac TestFile4.java $ java TestFile4 My new fancy message My second fancy message Sending a default message Sending a default message
71. $ cd ../TestFile4 $ javac TestFile4.java $ java TestFile4 My new fancy message My second fancy message Sending a default message Sending a default message $ java -jar $R8_HOME/build/libs/d8.jar \ --lib $ANDROID_HOME/platforms/android-29/android.jar \ --release \ --output . \ *.class
72. $ cd ../TestFile4 $ javac TestFile4.java $ java TestFile4 My new fancy message My second fancy message Sending a default message Sending a default message $ java -jar $R8_HOME/build/libs/d8.jar \ --lib $ANDROID_HOME/platforms/android-29/android.jar \ --release \ --output . \ *.class $ dexdump -d classes.dex > TestFile4_dump.txt
73. |[000440] TestFile4.main:([Ljava/lang/String;)V |0000: sget-object v1, L-$$Lambda$TestFile4$1;.INSTANCE:L-$$Lambda$TestFile4$1; // field@0001 |0002: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0005: sget-object v1, L-$$Lambda$TestFile4$2;.INSTANCE:L-$$Lambda$TestFile4$2; // field@0000 |0007: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |000a: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |000c: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |000f: new-instance v0, L-$$Lambda$3; // type@0002 |0011: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0014: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0017: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |0019: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |001c: new-instance v0, L-$$Lambda$3; // type@0002 |001e: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0021: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0024: return-void
74. |[000440] TestFile4.main:([Ljava/lang/String;)V |0000: sget-object v1, L-$$Lambda$TestFile4$1;.INSTANCE:L-$$Lambda$TestFile4$1; // field@0001 |0002: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0005: sget-object v1, L-$$Lambda$TestFile4$2;.INSTANCE:L-$$Lambda$TestFile4$2; // field@0000 |0007: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |000a: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |000c: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |000f: new-instance v0, L-$$Lambda$3; // type@0002 |0011: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0014: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0017: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |0019: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |001c: new-instance v0, L-$$Lambda$3; // type@0002 |001e: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0021: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0024: return-void
75. |[000440] TestFile4.main:([Ljava/lang/String;)V |0000: sget-object v1, L-$$Lambda$TestFile4$1;.INSTANCE:L-$$Lambda$TestFile4$1; // field@0001 |0002: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0005: sget-object v1, L-$$Lambda$TestFile4$2;.INSTANCE:L-$$Lambda$TestFile4$2; // field@0000 |0007: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |000a: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |000c: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |000f: new-instance v0, L-$$Lambda$3; // type@0002 |0011: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0014: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0017: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |0019: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |001c: new-instance v0, L-$$Lambda$3; // type@0002 |001e: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0021: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0024: return-void
76. |[000440] TestFile4.main:([Ljava/lang/String;)V |0000: sget-object v1, L-$$Lambda$TestFile4$1;.INSTANCE:L-$$Lambda$TestFile4$1; // field@0001 |0002: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0005: sget-object v1, L-$$Lambda$TestFile4$2;.INSTANCE:L-$$Lambda$TestFile4$2; // field@0000 |0007: invoke-static {v1}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |000a: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |000c: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |000f: new-instance v0, L-$$Lambda$3; // type@0002 |0011: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0014: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0017: sget-object v1, Ljava/lang/System;.out:Ljava/io/PrintStream; // field@0003 |0019: invoke-virtual {v1}, Ljava/lang/Object;.getClass:()Ljava/lang/Class; // method@0010 |001c: new-instance v0, L-$$Lambda$3; // type@0002 |001e: invoke-direct {v0, v1}, L-$$Lambda$3;.<init>:(Ljava/io/PrintStream;)V // method@0006 |0021: invoke-static {v0}, LTestFile4;.sendMessage:(LTestFile4$PostMan;)V // method@000d |0024: return-void
77. Obfuscation in Action TestFile5
78. Quick Note: ProGuard vs. R8 ByteCode TestFile5.class Transformers DeSugar ProGuard(Shrinking, Obfuscation, Optimization Source TestFile5.java (Java/Kotlin) Compilers Javac Kotlinc Dex Compilers DX, D8(DeSugar) Dalvik Bytecode classes.dex ProGuard R8 ByteCode TestFile5.class Transformers DeSugar Source TestFile5.java (Java/Kotlin) Compilers Javac Kotlinc Dex Compilers DX, D8(DeSugar) R8(DeSugar, Shrinking, Obfuscation, Optimization) Dalvik Bytecode classes.dex <--- We will be using this
79. Compiling with R8... Swapping over to APK’s - TestFile5
80. Swapping over to Android Studio
81. Different Keep Rules - Be as defined as possible (point to a specific package) - Rarely should you ever use just “-keep”. Use a variant: - keep(doesn’t run shrinking/obfuscation on class or it’s members) - Keepclassmembers (still shrinks/optimizes/obfuscates class, but keeps the members) - Keepnames(still shrinks/optimizes classes and members, but doesn’t obfuscate) - Keepclassmembernames (still shrinks/optimizes classes and members, obfuscates classes, but does not obfuscate members) - Check out https://jebware.com/blog/?p=418 for a guide on this
82. State of Android Compilation Native Languages Java Kotlin Run Time VM’s Dalvik ART Run Time Compilation JIT AOT AOT & JIT Package Types OBB APK Dex Compilers Jack/Jill DX D8(With DeSugar R8(DeSugar/Shrinking/Obfuscation/Optimization) Compilers Javac Kotlinc Transformers DeSugar ProGuard/DexGuard (Shrinking/Optimization/Obfuscation) Custom
83. References https://software.intel.com/en-us/blogs/2014/06/18/art-vs-dalvik-introducing-the-new-android-x86-runtime https://www.journaldev.com/23464/android-runtime-dvm-vs-art-aot-vs-jit Jake Wharton Blogs/Presentations ProGuards Website @DanLowe0x00 @DustNSummers
84. Questions?