Write your own ProGuard Edward Dale @scompt

Write your own ProGuard Edward Dale @scompt R8

ProGuard: Purpose ProGuard is the most popular optimizer for Java bytecode. It makes your Java and Android 
 applications up to 90% smaller and up to 
 20% faster. ProGuard also provides minimal 
 protection against reverse engineering by 
 obfuscating the names of classes, fields and 
 methods. https://www.guardsquare.com/en/proguard

ProGuard: Purpose ProGuard is the most popular optimizer for Java bytecode. It makes your Java and Android 
 applications up to 90% smaller and up to 
 20% faster. ProGuard also provides minimal 
 protection against reverse engineering by 
 obfuscating the names of classes, fields and 
 methods. https://www.guardsquare.com/en/proguard

ProGuard: Highlights • ProGuard is a command-line tool with an optional graphical user interface. • ProGuard is easy to configure. A few intuitive command line options or a simple configuration file is all it takes. All available options are detailed in the user manual. • ProGuard is fast. It processes small Android applications and entire run-time libraries in seconds. • ProGuard is the default tool in development environments like Oracle’s Wireless Toolkit, NetBeans, EclipseME, Intel’s TXE SDK and Google’s Android SDK.

ProGuard: Highlights • ProGuard is easy to configure. A few intuitive command line options or a simple configuration file is all it takes. All available options are detailed in the user manual. • ProGuard is fast. It processes small Android applications and entire runtime libraries in seconds.

ProGuard Steps Shrink Optimize Obfuscate Preverify

ProGuard Steps Shrink Optimize Obfuscate Preverify

Transform API Build Steps S O O P Compile Assemble Process .java .xml Dex .apk

Build Steps Compile Assemble Transform API Process .java .xml Dex .apk

Transform API The goal of this API is to simplify injecting custom class manipulations without having to deal with tasks, and to offer more flexibility on what is manipulated. http://tools.android.com/tech-docs/new-build-system/transform-api

Transform API Compiled Bytecode Resources Libraries Other Files } {Compiled Bytecode Resources Libraries Other Files Transform

Transform API Compiled Bytecode Resources Libraries Other Files } {Compiled Bytecode Resources Libraries Other Files Compiled Bytecode

Transform API Examples JacocoTransform https://goo.gl/jNoA2w Instruments code for measuring coverage MultiDexTransform https://goo.gl/MxnuiX Gathers information about the project to configure MultiDex in a later stage ProGuardTransform https://goo.gl/h7CGb4 Runs ProGuard…

Transform API 1 class MyPlugin implements Plugin { 2 @Override 3 void apply(Project p) { 4 def t = new WriteYourOwnProGuardTransform(p) 5 p.android.registerTransform(t) 6 } 7 } 1 apply plugin: MyPlugin build.gradle MyPlugin.groovy

1 class WriteYourOwnProGuardTransform(val project: Project) 2 : Transform() { 3 4 override fun isIncremental() = false 5 6 override fun getName() = "writeYourOwnProGuard" 7 8 @Throws(TransformException::class, 9 InterruptedException::class, 10 IOException::class) 11 override fun transform(invocation: TransformInvocation) { WriteYourOwnProGuardTransform.kotlin

1 override fun getInputTypes() = 2 setOf(QualifiedContent.DefaultContentType.CLASSES, 3 QualifiedContent.DefaultContentType.RESOURCES) 4 5 override fun getScopes() = 6 setOf(QualifiedContent.Scope.PROJECT, 7 QualifiedContent.Scope.SUB_PROJECTS, 8 QualifiedContent.Scope.EXTERNAL_LIBRARIES) WriteYourOwnProGuardTransform.kotlin

1 public interface TransformInvocation { 2 Context getContext(); 3 4 Collection getInputs(); 5 6 Collection getReferencedInputs(); 7 8 Collection getSecondaryInputs(); 9 10 TransformOutputProvider getOutputProvider(); 11 12 boolean isIncremental(); 13 } 1 public interface TransformInput { 2 Collection getJarInputs(); 3 4 Collection getDirectoryInputs(); 5 }

1 ImmutableJarInput{name=com.android.support:appcompat-v7:26.1.0, 2 file=/Users/scompt/.gradle/caches/ 3 transforms-1/files-1.1/ 4 appcompat-v7-26.1.0.aar/ 5 3cba64e049f7e363d1d5aa6ed4ca7a3f/ 6 jars/classes.jar, 7 contentTypes=CLASSES, 8 scopes=EXTERNAL_LIBRARIES, 9 status=NOTCHANGED} 1 ImmutableDirectoryInput{name=aa2bd2bBIGHASH, 2 file=/Users/scompt/projects/ 3 WriteYourOwnProGuard/app/build/ 4 intermediates/classes/debug, 5 contentTypes=CLASSES, 6 scopes=PROJECT, 7 changedFiles={}}

.jar .class 1 ImmutableJarInput{name=com.android.support:appcompat-v7:26.1.0, 2 file=/Users/scompt/.gradle/caches/ 3 transforms-1/files-1.1/ 4 appcompat-v7-26.1.0.aar/ 5 3cba64e049f7e363d1d5aa6ed4ca7a3f/ 6 jars/classes.jar, 7 contentTypes=CLASSES, 8 scopes=EXTERNAL_LIBRARIES, 9 status=NOTCHANGED} 1 ImmutableDirectoryInput{name=aa2bd2bBIGHASH, 2 file=/Users/scompt/projects/ 3 WriteYourOwnProGuard/app/build/ 4 intermediates/classes/debug, 5 contentTypes=CLASSES, 6 scopes=PROJECT, 7 changedFiles={}}

Bytecode Manipulation Apache Commons BCEL (Byte Code Engineering Library) https://commons.apache.org/proper/commons-bcel/ JBoss Javassist http://jboss-javassist.github.io/javassist/ OW2 Consortium ASM http://asm.ow2.org/

Bytecode Manipulation Apache Commons BCEL (Byte Code Engineering Library) https://commons.apache.org/proper/commons-bcel/ JBoss Javassist http://jboss-javassist.github.io/javassist/ OW2 Consortium ASM http://asm.ow2.org/

Transform API Build Steps S O O P Compile Assemble Process .java .xml Dex .apk

ProGuard Steps Shrink Optimize Obfuscate Preverify

Shrink • Removes all classes, methods, resources not reachable from an entry point (seeds) • Dynamically referenced classes/methods need to be "kept" using -keep or -keepclasseswithmembers

Shrink FeedActivity FeedApi UserManager UserApi LoginActivity OldUserManager

Shrink FeedActivity FeedApi UserManager UserApi LoginActivity OldUserManager seeds = []

Shrink FeedActivity FeedApi UserManager UserApi LoginActivity OldUserManager seeds = [FeedActivity, LoginActivity]

Shrink 1. Find all classes in the program 2. Find all connections between classes in the program 3. Find all “seed” classes 4. Find all classes reachable from seeds 5. Remove all unreachable classes

Shrink 1. Find all classes in the program 1 for (directoryInput in input.directoryInputs) { 2 val root = directoryInput.file.toPath() 3 Files.walkFileTree(root, /* Accumulate class name */) 1 val classPool = ClassPool.getDefault() 2 for (input in transformInvocation.inputs) { 3 for (jarInput in input.jarInputs) { 4 val stream = Files.newInputStream(jarInput.file.toPath()) 5 JarInputStream(stream).use { 6 var entry: JarEntry? = it.nextJarEntry 7 while (entry != null) { 8 val klass = classPool.makeClass(stream) 9 // Accumulate class name

Shrink 2. Find all connections between classes in the program 1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames

Shrink 2. Find all connections between classes in the program 1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames constPool

“For each type it loads, a Java virtual machine must store a constant pool. A constant pool is an ordered set of constants used by the type, including literals (string, integer, and floating point constants) and symbolic references to types, fields, and methods. Entries in the constant pool are referenced by index, much like the elements of an array. Because it holds symbolic references to all types, fields, and methods used by a type, the constant pool plays a central role in the dynamic linking of Java programs” –Inside the Java Virtual Machine by Bill Venners

Shrink 2. Find all connections between classes in the program 1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames

Shrink 3. Find all “seed” classes 1 val isSeedClass = klass.name.endsWith(“Activity")

Shrink 4. Find all classes reachable from seeds 1 private fun buildGraph(deps: Multimap) 2 : Graph { 3 4 val graph = DefaultDirectedGraph() 5 deps.keySet() 6 .filterNot { it.startsWith("java.lang") } 7 .forEach { graph.addVertex(it) } 8 deps.values() 9 .filterNot { it.startsWith("java.lang") } 10 .forEach { graph.addVertex(it) } 11 deps.entries() 12 .filterNot { it.key.startsWith("java.lang") } 13 .filterNot { it.value.startsWith("java.lang") } 14 .filterNot { it.key == it.value } 15 .forEach { graph.addEdge(it.key, it.value) } 16 return graph 17 }

Shrink 4. Find all classes reachable from seeds 1 graph.vertexSet() 2 .filter { it.endsWith("Activity") } 3 .map { DepthFirstIterator(graph, it).asSequence().toSet() } 4 .flatten()

Shrink 5. Remove all unreachable classes 1 override fun transform(invocation: TransformInvocation) { 2 val classNames = getAllClassNames(invocation) 3 val connections = findConnections(classNames) 4 val classGraph = buildGraph(connections) 5 val reachableClasses = findReachableClasses(classGraph) 6 7 // for-each JAR file 8 // remove unreachable classes 9 // repackage JAR file 10 // copy JAR file in output 11 12 // for-each class file 13 // if reachable copy to output

No content

No content

No content

No content

ProGuard Steps Shrink Optimize Obfuscate Preverify

Optimize • Performs lots of different bytecode-level optimizations to the code • For concrete example, see Jake Wharton’s recent “Sinking Your Teeth Into Bytecode” presentation https://goo.gl/8hnRzP

Optimize • Marks methods as final, whenever possible. • Removes unused method parameters. • Propagates the values of method parameters from method invocations to the invoked methods. • Propagates the values of method return values from methods to their invocations. • Inlines short methods. • Inlines methods that are only called once.

ProGuard Steps Shrink Optimize Obfuscate Preverify

Obfuscate • Classes and class members receive new short random names, except for the ones listed by the various -keep options • Internal attributes that are useful for debugging are removed

Obfuscate FeedActivity FeedApi UserManager UserApi LoginActivity OldUserManager

Obfuscate FeedActivity A B C LoginActivity OldUserManager

Obfuscate

Resources • Example ProGuard-like transform – Edward Dale
 https://github.com/scompt/WriteYourOwnProGuard • “Sinking Your Teeth Into Bytecode” – Jake Wharton
 http://jakewharton.com/sinking-your-teeth-into-bytecode/ • Gephi – The Open Graph Viz Platform
 https://gephi.org/ • JGraphT
 http://jgrapht.org/

Image Citations https://emojipedia.org https://freeletics.com https://gephi.org http://www.cuelogic.com http://knowyourmeme.com https://imgflip.com https://www.flaticon.com https://www.jgrapht.org

Questions? Edward Dale @scompt