De-Obfuscating ProGuard at GDG DevFest Sydney 2018

ProGuard is a mystery/blackbox for most of the Android developers. This talk digs a bit deeper on the ProGuard DSL and the Android specific DSL that developers should know about.

GDG DevFest Sydney 2018
https://www.devfest.org.au/
https://twitter.com/gdg_sydney

Transcript

1. De-obfuscating ProGuard
2. @gfg5tek grevolution grevolution
3. De-obfuscating ProGuard
4. What is ProGuard?
5. What is ProGuard?
6. What is ProGuard?
7. What is ProGuard?
8. What is ProGuard?
9. Enable/Use ProGuard buildTypes { release { } }
10. Enable/Use ProGuard buildTypes { release { minifyEnabled true } }
11. Enable/Use ProGuard buildTypes { release { minifyEnabled true proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro' } }
12. Enable/Use ProGuard buildTypes { release { minifyEnabled true proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' } }
13. Enable/Use ProGuard buildTypes { release { minifyEnabled true proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' } } project-dir/build/intermediates/proguard-files/<typeFlavor>/aapt_rules.txt
14. Initialisation ProGuard, version 5.3.3 Reading input""... Reading program jar [/Users/shanulhaq/"".../758e6f0e11f6ae1b2e554f7997746672/jars/classes.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../f69e924e49df5f472d3f9711870e1566/jars/classes.jar] (filtered) . . . Reading program jar [/Users/shanulhaq/"".../207a6efae6a3555e326de41f76bdadd9a239cbce/common-1.1.1.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../e55b70d1f5620db124b3e85a7f4bdc7bd48d9f95/common-1.1.1.jar] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/intermediates/classes/release] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/tmp/kotlin-classes/release] (filtered) Reading library jar [/opt/android-sdk/platforms/android-28/android.jar] Reading library jar [/opt/android-sdk/platforms/android-28/optional/org.apache.http.legacy.jar] . . . Note: duplicate definition of library class [org.apache.http.params.CoreConnectionPNames] Note: duplicate definition of library class [org.apache.http.params.HttpConnectionParams] Note: duplicate definition of library class [org.apache.http.params.HttpParams] Note: there were 11 duplicate class definitions. (http:"//proguard.sourceforge.net/manual/troubleshooting.html#duplicateclass) Initializing""...
15. Initialisation ProGuard, version 5.3.3 Reading input""... Reading program jar [/Users/shanulhaq/"".../758e6f0e11f6ae1b2e554f7997746672/jars/classes.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../f69e924e49df5f472d3f9711870e1566/jars/classes.jar] (filtered) . . . Reading program jar [/Users/shanulhaq/"".../207a6efae6a3555e326de41f76bdadd9a239cbce/common-1.1.1.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../e55b70d1f5620db124b3e85a7f4bdc7bd48d9f95/common-1.1.1.jar] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/intermediates/classes/release] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/tmp/kotlin-classes/release] (filtered) Reading library jar [/opt/android-sdk/platforms/android-28/android.jar] Reading library jar [/opt/android-sdk/platforms/android-28/optional/org.apache.http.legacy.jar] . . . Note: duplicate definition of library class [org.apache.http.params.CoreConnectionPNames] Note: duplicate definition of library class [org.apache.http.params.HttpConnectionParams] Note: duplicate definition of library class [org.apache.http.params.HttpParams] Note: there were 11 duplicate class definitions. (http:"//proguard.sourceforge.net/manual/troubleshooting.html#duplicateclass) Initializing""...
16. Initialisation ProGuard, version 5.3.3 Reading input""... Reading program jar [/Users/shanulhaq/"".../758e6f0e11f6ae1b2e554f7997746672/jars/classes.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../f69e924e49df5f472d3f9711870e1566/jars/classes.jar] (filtered) . . . Reading program jar [/Users/shanulhaq/"".../207a6efae6a3555e326de41f76bdadd9a239cbce/common-1.1.1.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../e55b70d1f5620db124b3e85a7f4bdc7bd48d9f95/common-1.1.1.jar] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/intermediates/classes/release] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/tmp/kotlin-classes/release] (filtered) Reading library jar [/opt/android-sdk/platforms/android-28/android.jar] Reading library jar [/opt/android-sdk/platforms/android-28/optional/org.apache.http.legacy.jar] . . . Note: duplicate definition of library class [org.apache.http.params.CoreConnectionPNames] Note: duplicate definition of library class [org.apache.http.params.HttpConnectionParams] Note: duplicate definition of library class [org.apache.http.params.HttpParams] Note: there were 11 duplicate class definitions. (http:"//proguard.sourceforge.net/manual/troubleshooting.html#duplicateclass) Initializing""...
17. Initialisation ProGuard, version 5.3.3 Reading input""... Reading program jar [/Users/shanulhaq/"".../758e6f0e11f6ae1b2e554f7997746672/jars/classes.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../f69e924e49df5f472d3f9711870e1566/jars/classes.jar] (filtered) . . . Reading program jar [/Users/shanulhaq/"".../207a6efae6a3555e326de41f76bdadd9a239cbce/common-1.1.1.jar] (filtered) Reading program jar [/Users/shanulhaq/"".../e55b70d1f5620db124b3e85a7f4bdc7bd48d9f95/common-1.1.1.jar] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/intermediates/classes/release] (filtered) Reading program directory [/Users/shanulhaq/"".../app/build/tmp/kotlin-classes/release] (filtered) Reading library jar [/opt/android-sdk/platforms/android-28/android.jar] Reading library jar [/opt/android-sdk/platforms/android-28/optional/org.apache.http.legacy.jar] . . . Note: duplicate definition of library class [org.apache.http.params.CoreConnectionPNames] Note: duplicate definition of library class [org.apache.http.params.HttpConnectionParams] Note: duplicate definition of library class [org.apache.http.params.HttpParams] Note: there were 11 duplicate class definitions. (http:"//proguard.sourceforge.net/manual/troubleshooting.html#duplicateclass) Initializing""...
18. Initialisation Note: duplicate definition of library class [org.apache.http.params.CoreConnectionPNames] Note: duplicate definition of library class [org.apache.http.params.HttpConnectionParams] Note: duplicate definition of library class [org.apache.http.params.HttpParams] Note: there were 11 duplicate class definitions. (http:"//proguard.sourceforge.net/manual/troubleshooting.html#duplicateclass)
19. Initialisation Note: duplicate definition of library class [org.apache.http.params.CoreConnectionPNames] Note: duplicate definition of library class [org.apache.http.params.HttpConnectionParams] Note: duplicate definition of library class [org.apache.http.params.HttpParams] Note: there were 11 duplicate class definitions. (http:"//proguard.sourceforge.net/manual/troubleshooting.html#duplicateclass) -dontnote
20. Initialisation ProGuard, version 5.3.3 Reading input""... . . . . . . Initializing… Ignoring unused library classes""... Original number of library classes: 4801 Final number of library classes: 1292 Printing kept classes, fields, and methods""...
21. Initialisation seeds.txt ProGuard, version 5.3.3 Reading input""... . . . . . . Initializing… Ignoring unused library classes""... Original number of library classes: 4801 Final number of library classes: 1292 Printing kept classes, fields, and methods""...
22. Initialisation seeds.txt usage.txt ProGuard, version 5.3.3 Reading input""... . . . . . . Initializing… Ignoring unused library classes""... Original number of library classes: 4801 Final number of library classes: 1292 Printing kept classes, fields, and methods""... Shrinking""... Printing usage to [/Users/shanulhaq/…/mapping/release/usage.txt]
23. Initialisation seeds.txt . . android.support.v7.widget.ActionBarOverlayLayout android.support.v7.widget.ActionBarOverlayLayout: ActionBarOverlayLayout(android.content.Context) android.support.v7.widget.ActionBarOverlayLayout: ActionBarOverlayLayout(android.content.Context,android.util.AttributeSet) android.support.v7.widget.ActionBarOverlayLayout: void setActionBarVisibilityCallback(android.support.v7.widget.ActionBarOverlayLayout$ActionBarVisibilityCallback) android.support.v7.widget.ActionBarOverlayLayout: void setOverlayMode(boolean) android.support.v7.widget.ActionBarOverlayLayout: void setHasNonEmbeddedTabs(boolean) android.support.v7.widget.ActionBarOverlayLayout: void setShowingForActionMode(boolean) android.support.v7.widget.ActionBarOverlayLayout: int getNestedScrollAxes() android.support.v7.widget.ActionBarOverlayLayout: void setHideOnContentScrollEnabled(boolean) android.support.v7.widget.ActionBarOverlayLayout: int getActionBarHideOffset() android.support.v7.widget.ActionBarOverlayLayout: void setActionBarHideOffset(int) android.support.v7.widget.ActionBarOverlayLayout: void setWindowCallback(android.view.Window$Callback) android.support.v7.widget.ActionBarOverlayLayout: void setWindowTitle(java.lang.CharSequence) android.support.v7.widget.ActionBarOverlayLayout: java.lang.CharSequence getTitle() android.support.v7.widget.ActionBarOverlayLayout: void setUiOptions(int) android.support.v7.widget.ActionBarOverlayLayout: void setIcon(int) android.support.v7.widget.ActionBarOverlayLayout: void setIcon(android.graphics.drawable.Drawable) android.support.v7.widget.ActionBarOverlayLayout: void setLogo(int) . .
24. Initialisation usage.txt . . android.support.v7.widget.ActionBarOverlayLayout: private static final java.lang.String TAG private static final int ACTION_BAR_ANIMATE_DELAY 565:565:public boolean isHideOnContentScrollEnabled() 654:655:public boolean hasIcon() 660:661:public boolean hasLogo() 726:728:public void saveToolbarHierarchyState(android.util.SparseArray) 732:734:public void restoreToolbarHierarchyState(android.util.SparseArray) android.support.v7.widget.ActionBarOverlayLayout$LayoutParams: 756:757:public ActionBarOverlayLayout$LayoutParams(android.view.ViewGroup$MarginLayoutParams) . .
25. Shrinking options -dontshrink -whyareyoukeeping <class_specification>
26. Shrinking options -dontshrink -whyareyoukeeping <class_specification>
27. -whyareyoukeeping android.support.v7.view.ViewPropertyAnimatorCompatSet$1 is invoked by android.support.v7.view.ViewPropertyAnimatorCompatSet: ViewPropertyAnimatorCompatSet() (42:120) is invoked by android.support.v7.app.WindowDecorActionBar: void doShow(boolean) (788:834) is invoked by android.support.v7.app.WindowDecorActionBar: void updateVisibility(boolean) (771:785) is invoked by android.support.v7.app.WindowDecorActionBar: void showForSystem() (697:701) implements android.support.v7.widget.ActionBarOverlayLayout$ActionBarVisibilityCallback: void showForSystem() is invoked by android.support.v7.widget.ActionBarOverlayLayout: void onWindowSystemUiVisibilityChanged(int) (230:251) is invoked by android.support.v7.widget.ActionBarOverlayLayout: void setActionBarVisibilityCallback(android.support.v7.widget.ActionBarOverlayLayout$ActionBarVisibilit yCallback) (167:178) is kept by a directive in the configuration.
28. -whyareyoukeeping android.support.v7.view.ViewPropertyAnimatorCompatSet$1 is invoked by android.support.v7.view.ViewPropertyAnimatorCompatSet: ViewPropertyAnimatorCompatSet() (42:120) is invoked by android.support.v7.app.WindowDecorActionBar: void doShow(boolean) (788:834) is invoked by android.support.v7.app.WindowDecorActionBar: void updateVisibility(boolean) (771:785) is invoked by android.support.v7.app.WindowDecorActionBar: void showForSystem() (697:701) implements android.support.v7.widget.ActionBarOverlayLayout$ActionBarVisibilityCallback: void showForSystem() is invoked by android.support.v7.widget.ActionBarOverlayLayout: void onWindowSystemUiVisibilityChanged(int) (230:251) is invoked by android.support.v7.widget.ActionBarOverlayLayout: void setActionBarVisibilityCallback(android.support.v7.widget.ActionBarOverlayLayout$ActionBarVisibilit yCallback) (167:178) is kept by a directive in the configuration.
29. Keep options -keep class_specification -keepclassmembers class_specification -keepclasseswithmembers class_specification -keepnames class_specification -keepclassmembernames class_specification -keepclasswithmembernames class_specification
30. Keep options Keep From being removed or renamed From being renamed Classes and class members -keep -keepnames Class members only -keepclassmembers -keepclassmembernames Classes and class members, if class members present -keepclasseswithmembers -keepclasseswithmembernames
31. Class specification [@annotationtype] [[!]public|final|abstract|@ ""...] [!]interface|class|enum classname [extends|implements [@annotationtype] classname] [{ [@annotationtype] [[!]public|private|protected|static|volatile|transient ""...] <fields> | (fieldtype fieldname); [@annotationtype] [[!]public|private|protected|static|synchronized|native|abstract|strictfp ""...] <methods> | <init>(argumenttype,""...) | classname(argumenttype,""...) | (returntype methodname(argumenttype,""...)); [@annotationtype] [[!]public|private|protected|static ""... ] *; ""... }]
32. Class specification [@annotationtype] [[!]public|final|abstract|@ ""...] [!]interface|class|enum classname [extends|implements [@annotationtype] classname] [{ [@annotationtype] [[!]public|private|protected|static|volatile|transient ""...] <fields> | (fieldtype fieldname); [@annotationtype] [[!]public|private|protected|static|synchronized|native|abstract|strictfp ""...] <methods> | <init>(argumenttype,""...) | classname(argumenttype,""...) | (returntype methodname(argumenttype,""...)); [@annotationtype] [[!]public|private|protected|static ""... ] *; ""... }]
33. Class specification [@annotationtype] [[!]public|final|abstract|@ ""...] [!]interface|class|enum classname [extends|implements [@annotationtype] classname] [{ [@annotationtype] [[!]public|private|protected|static|volatile|transient ""...] <fields> | (fieldtype fieldname); [@annotationtype] [[!]public|private|protected|static|synchronized|native|abstract|strictfp ""...] <methods> | <init>(argumenttype,""...) | classname(argumenttype,""...) | (returntype methodname(argumenttype,""...)); [@annotationtype] [[!]public|private|protected|static ""... ] *; ""... }]
34. Class specification [@annotationtype] [[!]public|final|abstract|@ ""...] [!]interface|class|enum classname [extends|implements [@annotationtype] classname] [{ [@annotationtype] [[!]public|private|protected|static|volatile|transient ""...] <fields> | (fieldtype fieldname); [@annotationtype] [[!]public|private|protected|static|synchronized|native|abstract|strictfp ""...] <methods> | <init>(argumenttype,""...) | classname(argumenttype,""...) | (returntype methodname(argumenttype,""...)); [@annotationtype] [[!]public|private|protected|static ""... ] *; ""... }]
35. Class specification [@annotationtype] [[!]public|final|abstract|@ ""...] [!]interface|class|enum classname [extends|implements [@annotationtype] classname] [{ [@annotationtype] [[!]public|private|protected|static|volatile|transient ""...] <fields> | (fieldtype fieldname); [@annotationtype] [[!]public|private|protected|static|synchronized|native|abstract|strictfp ""...] <methods> | <init>(argumenttype,""...) | classname(argumenttype,""...) | (returntype methodname(argumenttype,""...)); [@annotationtype] [[!]public|private|protected|static ""... ] *; ""... }]
36. -keep<X> example -keepclassmembers public class * extends android.view.View { void set*(""***); ""*** get*(); }
37. -whyareyoukeeping android.support.v7.view.ViewPropertyAnimatorCompatSet$1 is invoked by android.support.v7.view.ViewPropertyAnimatorCompatSet: ViewPropertyAnimatorCompatSet() (42:120) is invoked by android.support.v7.app.WindowDecorActionBar: void doShow(boolean) (788:834) is invoked by android.support.v7.app.WindowDecorActionBar: void updateVisibility(boolean) (771:785) is invoked by android.support.v7.app.WindowDecorActionBar: void showForSystem() (697:701) implements android.support.v7.widget.ActionBarOverlayLayout$ActionBarVisibilityCallback: void showForSystem() is invoked by android.support.v7.widget.ActionBarOverlayLayout: void onWindowSystemUiVisibilityChanged(int) (230:251) is invoked by android.support.v7.widget.ActionBarOverlayLayout: void setActionBarVisibilityCallback(android.support.v7.widget.ActionBarOverlayLayout$ActionBarVisibilit yCallback) (167:178) is kept by a directive in the configuration.
38. -keep<X> example -keepclassmembers public class * extends android.view.View { void set*(""***); ""*** get*(); }
39. -keep<X> example -keepclassmembers public class * extends android.view.View { void set*(""***); ""*** get*(); } Keep all the setters and getters of every public class that extends android.view.View class
40. -keep<X> example -keepclassmembers class * extends android.app.Activity { public void *(android.view.View); } Keep all methods that take android.view.View as argument of every class that extends android.app.Activity
41. -keep<X> example -keepclassmembers enum * { public static "**[] values(); public static "** valueOf(java.lang.String); } Keep values() and valueOf() method of all enums
42. -keep<X> example -keepclassmembers class "**.R$* { public static <fields>; } Keep all static fields of all subclasses of every .R class in every package
43. -keep<X> example -keep class android.support.annotation.Keep -keep @android.support.annotation.Keep class * {*;} -keepclasseswithmembers class * { @android.support.annotation.Keep <methods>;} -keepclasseswithmembers class * { @android.support.annotation.Keep <fields>;} -keepclasseswithmembers class * { @android.support.annotation.Keep <init>(""...);} keep the android.support.annotation.Keep class keep every class that has @Keep annotation keep every method in all classes that has @Keep annotation keep every field in all classes that has @Keep annotation keep all constructors in all classes that has @Keep annotation
44. Obfuscation package au.com.proguardexample.model.Person public class Person { public String name; public int age; public Person[] friends; public Person(String name){} public Person(){ } public Person(String name, int age){} public void printName(){} public void printAge(){} public void printFriends(){} } package au.com.proguardexample.a.a public class a { public String a; public int b; public a[] c; public a(String a){} public a(){ } public a(String a, int b){} public void a(){} public void b(){} public void c(){} }
45. Obfuscation - mapping.txt au.com.example.proguard.proguardexample.model.Person "-> au.com.example.proguard.proguardexample.a.a: java.lang.String name "-> a int age "-> b au.com.example.proguard.proguardexample.model.Person[] friends "-> c void <init>(java.lang.String) "-> <init> void <init>() "-> <init> void <init>(java.lang.String,int) "-> <init> void printName() "-> a void printAge() "-> b void printFriends() "-> c
46. Obfuscation options -dontusemixedcaseclassnames -dontskipnonpubliclibraryclasses -keepattributes <attribute_filter>
47. Optimization Optimizing""... Number of finalized classes: 558 Number of unboxed enum classes: 0 Number of vertically merged classes: 0 (disabled) Number of horizontally merged classes: 0 (disabled) Number of removed write-only fields: 0 (disabled) Number of privatized fields: 0 (disabled) Number of inlined constant fields: 0 (disabled) Number of privatized methods: 596 Number of staticized methods: 337 Number of finalized methods: 4175 Number of removed method parameters: 421 Number of inlined constant parameters: 125 Number of inlined constant return values: 59 Number of inlined short method calls: 1104 Number of inlined unique method calls: 1883 Number of inlined tail recursion calls: 14 Number of merged code blocks: 88 Number of variable peephole optimizations: 6128 Number of arithmetic peephole optimizations: 0 (disabled) Number of cast peephole optimizations: 0 (disabled) Number of field peephole optimizations: 46 Number of branch peephole optimizations: 1800 Number of string peephole optimizations: 376 Number of simplified instructions: 664 Number of removed instructions: 3262 Number of removed local variables: 256 Number of removed exception blocks: 54 Number of optimized local variable frames: 2052 Shrinking""... Removing unused program classes and class elements""... Original number of program classes: 1118 Final number of program classes: 1069 Optimizing""... Number of finalized classes: 0 Number of unboxed enum classes: 0 Number of vertically merged classes: 0 (disabled) Number of horizontally merged classes: 0 (disabled) Number of removed write-only fields: 0 (disabled) Number of privatized fields: 0 (disabled) Number of inlined constant fields: 0 (disabled) Number of privatized methods: 0 Number of staticized methods: 0 Number of finalized methods: 0 Number of removed method parameters: 4 Number of inlined constant parameters: 6 Number of inlined constant return values: 16 Number of inlined short method calls: 0 Number of inlined unique method calls: 0 Number of inlined tail recursion calls: 0 Number of merged code blocks: 0 Number of variable peephole optimizations: 9 Number of arithmetic peephole optimizations: 0 (disabled) Number of cast peephole optimizations: 0 (disabled) Number of field peephole optimizations: 0 Number of branch peephole optimizations: 0 Number of string peephole optimizations: 0 Number of simplified instructions: 1 Number of removed instructions: 16 Number of removed local variables: 0 Number of removed exception blocks: 0 Number of optimized local variable frames: 0 Shrinking""... Removing unused program classes and class elements""... Original number of program classes: 1065 Final number of program classes: 1065 n = 5
48. proguard-android-optimize.txt -optimizations !code/simplification/arithmetic,!code/simplification/cast,!field"/*,!class/merging"/* -optimizationpasses 5 -allowaccessmodification
49. Optimization options -dontoptimize -optimizations <optimization_filter> -optimizationpasses <n> -assumenosideeffects <class_specification>
50. Optimization options -dontoptimize -optimizations <optimization_filter> -optimizationpasses <n> -assumenosideeffects <class_specification>
51. Optimization options -dontoptimize -optimizations <optimization_filter> -optimizationpasses <n> -assumenosideeffects <class_specification>
52. Optimization options -dontoptimize -optimizations <optimization_filter> -optimizationpasses <n> -assumenosideeffects <class_specification>
53. Optimization options -dontoptimize -optimizations <optimization_filter> -optimizationpasses <n> -assumenosideeffects <class_specification>
54. -assumenosideeffects <class_specification> -assumenosideeffects class android.util.Log { public static boolean isLoggable(java.lang.String, int); public static int v(""...); public static int i(""...); public static int w(""...); public static int d(""...); public static int e(""...); }
55. More to learn • Resource Shrinking and resources.txt • Class structure using -dump directive and dump.txt • Further options for keep, shrinking, obfuscation, general. • Optimization filters • Code attributes
56. That’s All