Share
Embed
Write your own ProGuard
ProGuard: We all use it, but how much do we really know about it? Obviously, it's something that Google recommends that we use. And somewhere you read that it makes the code smaller. But what does it really do?
In this presentation, we'll answer that question together by developing a replacement for it. Like the real ProGuard, it will be able to shrink, optimize, and obfuscate code. The difference from ProGuard will be that we understand every step.
Edward Dale
December 01, 2017
Transcript
53 slides
-
ProGuard: Purpose ProGuard is the most popular optimizer for Java
bytecode. It makes your Java and Android applications up to 90% smaller and up to 20% faster. ProGuard also provides minimal protection against reverse engineering by obfuscating the names of classes, fields and methods. https://www.guardsquare.com/en/proguard -
ProGuard: Purpose ProGuard is the most popular optimizer for Java
bytecode. It makes your Java and Android applications up to 90% smaller and up to 20% faster. ProGuard also provides minimal protection against reverse engineering by obfuscating the names of classes, fields and methods. https://www.guardsquare.com/en/proguard -
ProGuard: Highlights • ProGuard is a command-line tool with an
optional graphical user interface. • ProGuard is easy to configure. A few intuitive command line options or a simple configuration file is all it takes. All available options are detailed in the user manual. • ProGuard is fast. It processes small Android applications and entire run-time libraries in seconds. • ProGuard is the default tool in development environments like Oracle’s Wireless Toolkit, NetBeans, EclipseME, Intel’s TXE SDK and Google’s Android SDK. -
ProGuard: Highlights • ProGuard is easy to configure. A few
intuitive command line options or a simple configuration file is all it takes. All available options are detailed in the user manual. • ProGuard is fast. It processes small Android applications and entire runtime libraries in seconds. -
Transform API Build Steps S O O P Compile Assemble
Process .java .xml Dex .apk -
Transform API The goal of this API is to simplify
injecting custom class manipulations without having to deal with tasks, and to offer more flexibility on what is manipulated. http://tools.android.com/tech-docs/new-build-system/transform-api -
Transform API Compiled Bytecode Resources Libraries Other Files } {Compiled
Bytecode Resources Libraries Other Files Transform -
Transform API Compiled Bytecode Resources Libraries Other Files } {Compiled
Bytecode Resources Libraries Other Files Compiled Bytecode -
Transform API Examples JacocoTransform https://goo.gl/jNoA2w Instruments code for measuring coverage
MultiDexTransform https://goo.gl/MxnuiX Gathers information about the project to configure MultiDex in a later stage ProGuardTransform https://goo.gl/h7CGb4 Runs ProGuard… -
Transform API 1 class MyPlugin implements Plugin<Project> { 2 @Override
3 void apply(Project p) { 4 def t = new WriteYourOwnProGuardTransform(p) 5 p.android.registerTransform(t) 6 } 7 } 1 apply plugin: MyPlugin build.gradle MyPlugin.groovy -
1 class WriteYourOwnProGuardTransform(val project: Project) 2 : Transform() { 3
4 override fun isIncremental() = false 5 6 override fun getName() = "writeYourOwnProGuard" 7 8 @Throws(TransformException::class, 9 InterruptedException::class, 10 IOException::class) 11 override fun transform(invocation: TransformInvocation) { WriteYourOwnProGuardTransform.kotlin -
1 override fun getInputTypes() = 2 setOf(QualifiedContent.DefaultContentType.CLASSES, 3 QualifiedContent.DefaultContentType.RESOURCES) 4
5 override fun getScopes() = 6 setOf(QualifiedContent.Scope.PROJECT, 7 QualifiedContent.Scope.SUB_PROJECTS, 8 QualifiedContent.Scope.EXTERNAL_LIBRARIES) WriteYourOwnProGuardTransform.kotlin -
1 public interface TransformInvocation { 2 Context getContext(); 3 4
Collection<TransformInput> getInputs(); 5 6 Collection<TransformInput> getReferencedInputs(); 7 8 Collection<SecondaryInput> getSecondaryInputs(); 9 10 TransformOutputProvider getOutputProvider(); 11 12 boolean isIncremental(); 13 } 1 public interface TransformInput { 2 Collection<JarInput> getJarInputs(); 3 4 Collection<DirectoryInput> getDirectoryInputs(); 5 } -
1 ImmutableJarInput{name=com.android.support:appcompat-v7:26.1.0, 2 file=/Users/scompt/.gradle/caches/ 3 transforms-1/files-1.1/ 4 appcompat-v7-26.1.0.aar/ 5 3cba64e049f7e363d1d5aa6ed4ca7a3f/
6 jars/classes.jar, 7 contentTypes=CLASSES, 8 scopes=EXTERNAL_LIBRARIES, 9 status=NOTCHANGED} 1 ImmutableDirectoryInput{name=aa2bd2bBIGHASH, 2 file=/Users/scompt/projects/ 3 WriteYourOwnProGuard/app/build/ 4 intermediates/classes/debug, 5 contentTypes=CLASSES, 6 scopes=PROJECT, 7 changedFiles={}} -
.jar .class 1 ImmutableJarInput{name=com.android.support:appcompat-v7:26.1.0, 2 file=/Users/scompt/.gradle/caches/ 3 transforms-1/files-1.1/ 4 appcompat-v7-26.1.0.aar/
5 3cba64e049f7e363d1d5aa6ed4ca7a3f/ 6 jars/classes.jar, 7 contentTypes=CLASSES, 8 scopes=EXTERNAL_LIBRARIES, 9 status=NOTCHANGED} 1 ImmutableDirectoryInput{name=aa2bd2bBIGHASH, 2 file=/Users/scompt/projects/ 3 WriteYourOwnProGuard/app/build/ 4 intermediates/classes/debug, 5 contentTypes=CLASSES, 6 scopes=PROJECT, 7 changedFiles={}} -
Bytecode Manipulation Apache Commons BCEL (Byte Code Engineering Library) https://commons.apache.org/proper/commons-bcel/
JBoss Javassist http://jboss-javassist.github.io/javassist/ OW2 Consortium ASM http://asm.ow2.org/ -
Bytecode Manipulation Apache Commons BCEL (Byte Code Engineering Library) https://commons.apache.org/proper/commons-bcel/
JBoss Javassist http://jboss-javassist.github.io/javassist/ OW2 Consortium ASM http://asm.ow2.org/ -
Transform API Build Steps S O O P Compile Assemble
Process .java .xml Dex .apk -
Shrink • Removes all classes, methods, resources not reachable from
an entry point (seeds) • Dynamically referenced classes/methods need to be "kept" using -keep or -keepclasseswithmembers -
Shrink 1. Find all classes in the program 2. Find
all connections between classes in the program 3. Find all “seed” classes 4. Find all classes reachable from seeds 5. Remove all unreachable classes -
Shrink 1. Find all classes in the program 1 for
(directoryInput in input.directoryInputs) { 2 val root = directoryInput.file.toPath() 3 Files.walkFileTree(root, /* Accumulate class name */) 1 val classPool = ClassPool.getDefault() 2 for (input in transformInvocation.inputs) { 3 for (jarInput in input.jarInputs) { 4 val stream = Files.newInputStream(jarInput.file.toPath()) 5 JarInputStream(stream).use { 6 var entry: JarEntry? = it.nextJarEntry 7 while (entry != null) { 8 val klass = classPool.makeClass(stream) 9 // Accumulate class name -
Shrink 2. Find all connections between classes in the program
1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames -
Shrink 2. Find all connections between classes in the program
1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames constPool -
“For each type it loads, a Java virtual machine must
store a constant pool. A constant pool is an ordered set of constants used by the type, including literals (string, integer, and floating point constants) and symbolic references to types, fields, and methods. Entries in the constant pool are referenced by index, much like the elements of an array. Because it holds symbolic references to all types, fields, and methods used by a type, the constant pool plays a central role in the dynamic linking of Java programs” –Inside the Java Virtual Machine by Bill Venners -
Shrink 2. Find all connections between classes in the program
1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames -
Shrink 3. Find all “seed” classes 1 val isSeedClass =
klass.name.endsWith(“Activity") -
Shrink 4. Find all classes reachable from seeds 1 private
fun buildGraph(deps: Multimap<String, String>) 2 : Graph<String, DefaultEdge> { 3 4 val graph = DefaultDirectedGraph() 5 deps.keySet() 6 .filterNot { it.startsWith("java.lang") } 7 .forEach { graph.addVertex(it) } 8 deps.values() 9 .filterNot { it.startsWith("java.lang") } 10 .forEach { graph.addVertex(it) } 11 deps.entries() 12 .filterNot { it.key.startsWith("java.lang") } 13 .filterNot { it.value.startsWith("java.lang") } 14 .filterNot { it.key == it.value } 15 .forEach { graph.addEdge(it.key, it.value) } 16 return graph 17 } -
Shrink 4. Find all classes reachable from seeds 1 graph.vertexSet()
2 .filter { it.endsWith("Activity") } 3 .map { DepthFirstIterator(graph, it).asSequence().toSet() } 4 .flatten() -
Shrink 5. Remove all unreachable classes 1 override fun transform(invocation:
TransformInvocation) { 2 val classNames = getAllClassNames(invocation) 3 val connections = findConnections(classNames) 4 val classGraph = buildGraph(connections) 5 val reachableClasses = findReachableClasses(classGraph) 6 7 // for-each JAR file 8 // remove unreachable classes 9 // repackage JAR file 10 // copy JAR file in output 11 12 // for-each class file 13 // if reachable copy to output -
Optimize • Performs lots of different bytecode-level optimizations to the
code • For concrete example, see Jake Wharton’s recent “Sinking Your Teeth Into Bytecode” presentation https://goo.gl/8hnRzP -
Optimize • Marks methods as final, whenever possible. • Removes
unused method parameters. • Propagates the values of method parameters from method invocations to the invoked methods. • Propagates the values of method return values from methods to their invocations. • Inlines short methods. • Inlines methods that are only called once. -
Obfuscate • Classes and class members receive new short random
names, except for the ones listed by the various -keep options • Internal attributes that are useful for debugging are removed -
Resources • Example ProGuard-like transform – Edward Dale https://github.com/scompt/WriteYourOwnProGuard •
“Sinking Your Teeth Into Bytecode” – Jake Wharton http://jakewharton.com/sinking-your-teeth-into-bytecode/ • Gephi – The Open Graph Viz Platform https://gephi.org/ • JGraphT http://jgrapht.org/