Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Write your own ProGuard

Edward Dale
December 01, 2017

Write your own ProGuard

ProGuard: We all use it, but how much do we really know about it? Obviously, it's something that Google recommends that we use. And somewhere you read that it makes the code smaller. But what does it really do?

In this presentation, we'll answer that question together by developing a replacement for it. Like the real ProGuard, it will be able to shrink, optimize, and obfuscate code. The difference from ProGuard will be that we understand every step.

Edward Dale

December 01, 2017
Tweet

More Decks by Edward Dale

Other Decks in Programming

Transcript

  1. ProGuard: Purpose ProGuard is the most popular optimizer for Java

    bytecode. It makes your Java and Android 
 applications up to 90% smaller and up to 
 20% faster. ProGuard also provides minimal 
 protection against reverse engineering by 
 obfuscating the names of classes, fields and 
 methods. https://www.guardsquare.com/en/proguard
  2. ProGuard: Purpose ProGuard is the most popular optimizer for Java

    bytecode. It makes your Java and Android 
 applications up to 90% smaller and up to 
 20% faster. ProGuard also provides minimal 
 protection against reverse engineering by 
 obfuscating the names of classes, fields and 
 methods. https://www.guardsquare.com/en/proguard
  3. ProGuard: Highlights • ProGuard is a command-line tool with an

    optional graphical user interface. • ProGuard is easy to configure. A few intuitive command line options or a simple configuration file is all it takes. All available options are detailed in the user manual. • ProGuard is fast. It processes small Android applications and entire run-time libraries in seconds. • ProGuard is the default tool in development environments like Oracle’s Wireless Toolkit, NetBeans, EclipseME, Intel’s TXE SDK and Google’s Android SDK.
  4. ProGuard: Highlights • ProGuard is easy to configure. A few

    intuitive command line options or a simple configuration file is all it takes. All available options are detailed in the user manual. • ProGuard is fast. It processes small Android applications and entire runtime libraries in seconds.
  5. Transform API The goal of this API is to simplify

    injecting custom class manipulations without having to deal with tasks, and to offer more flexibility on what is manipulated. http://tools.android.com/tech-docs/new-build-system/transform-api
  6. Transform API Compiled Bytecode Resources Libraries Other Files } {Compiled

    Bytecode Resources Libraries Other Files Transform
  7. Transform API Compiled Bytecode Resources Libraries Other Files } {Compiled

    Bytecode Resources Libraries Other Files Compiled Bytecode
  8. Transform API Examples JacocoTransform https://goo.gl/jNoA2w Instruments code for measuring coverage

    MultiDexTransform https://goo.gl/MxnuiX Gathers information about the project to configure MultiDex in a later stage ProGuardTransform https://goo.gl/h7CGb4 Runs ProGuard…
  9. Transform API 1 class MyPlugin implements Plugin<Project> { 2 @Override

    3 void apply(Project p) { 4 def t = new WriteYourOwnProGuardTransform(p) 5 p.android.registerTransform(t) 6 } 7 } 1 apply plugin: MyPlugin build.gradle MyPlugin.groovy
  10. 1 class WriteYourOwnProGuardTransform(val project: Project) 2 : Transform() { 3

    4 override fun isIncremental() = false 5 6 override fun getName() = "writeYourOwnProGuard" 7 8 @Throws(TransformException::class, 9 InterruptedException::class, 10 IOException::class) 11 override fun transform(invocation: TransformInvocation) { WriteYourOwnProGuardTransform.kotlin
  11. 1 override fun getInputTypes() = 2 setOf(QualifiedContent.DefaultContentType.CLASSES, 3 QualifiedContent.DefaultContentType.RESOURCES) 4

    5 override fun getScopes() = 6 setOf(QualifiedContent.Scope.PROJECT, 7 QualifiedContent.Scope.SUB_PROJECTS, 8 QualifiedContent.Scope.EXTERNAL_LIBRARIES) WriteYourOwnProGuardTransform.kotlin
  12. 1 public interface TransformInvocation { 2 Context getContext(); 3 4

    Collection<TransformInput> getInputs(); 5 6 Collection<TransformInput> getReferencedInputs(); 7 8 Collection<SecondaryInput> getSecondaryInputs(); 9 10 TransformOutputProvider getOutputProvider(); 11 12 boolean isIncremental(); 13 } 1 public interface TransformInput { 2 Collection<JarInput> getJarInputs(); 3 4 Collection<DirectoryInput> getDirectoryInputs(); 5 }
  13. 1 ImmutableJarInput{name=com.android.support:appcompat-v7:26.1.0, 2 file=/Users/scompt/.gradle/caches/ 3 transforms-1/files-1.1/ 4 appcompat-v7-26.1.0.aar/ 5 3cba64e049f7e363d1d5aa6ed4ca7a3f/

    6 jars/classes.jar, 7 contentTypes=CLASSES, 8 scopes=EXTERNAL_LIBRARIES, 9 status=NOTCHANGED} 1 ImmutableDirectoryInput{name=aa2bd2bBIGHASH, 2 file=/Users/scompt/projects/ 3 WriteYourOwnProGuard/app/build/ 4 intermediates/classes/debug, 5 contentTypes=CLASSES, 6 scopes=PROJECT, 7 changedFiles={}}
  14. .jar .class 1 ImmutableJarInput{name=com.android.support:appcompat-v7:26.1.0, 2 file=/Users/scompt/.gradle/caches/ 3 transforms-1/files-1.1/ 4 appcompat-v7-26.1.0.aar/

    5 3cba64e049f7e363d1d5aa6ed4ca7a3f/ 6 jars/classes.jar, 7 contentTypes=CLASSES, 8 scopes=EXTERNAL_LIBRARIES, 9 status=NOTCHANGED} 1 ImmutableDirectoryInput{name=aa2bd2bBIGHASH, 2 file=/Users/scompt/projects/ 3 WriteYourOwnProGuard/app/build/ 4 intermediates/classes/debug, 5 contentTypes=CLASSES, 6 scopes=PROJECT, 7 changedFiles={}}
  15. Bytecode Manipulation Apache Commons BCEL (Byte Code Engineering Library) https://commons.apache.org/proper/commons-bcel/

    JBoss Javassist http://jboss-javassist.github.io/javassist/ OW2 Consortium ASM http://asm.ow2.org/
  16. Bytecode Manipulation Apache Commons BCEL (Byte Code Engineering Library) https://commons.apache.org/proper/commons-bcel/

    JBoss Javassist http://jboss-javassist.github.io/javassist/ OW2 Consortium ASM http://asm.ow2.org/
  17. Shrink • Removes all classes, methods, resources not reachable from

    an entry point (seeds) • Dynamically referenced classes/methods need to be "kept" using -keep or -keepclasseswithmembers
  18. Shrink 1. Find all classes in the program 2. Find

    all connections between classes in the program 3. Find all “seed” classes 4. Find all classes reachable from seeds 5. Remove all unreachable classes
  19. Shrink 1. Find all classes in the program 1 for

    (directoryInput in input.directoryInputs) { 2 val root = directoryInput.file.toPath() 3 Files.walkFileTree(root, /* Accumulate class name */) 1 val classPool = ClassPool.getDefault() 2 for (input in transformInvocation.inputs) { 3 for (jarInput in input.jarInputs) { 4 val stream = Files.newInputStream(jarInput.file.toPath()) 5 JarInputStream(stream).use { 6 var entry: JarEntry? = it.nextJarEntry 7 while (entry != null) { 8 val klass = classPool.makeClass(stream) 9 // Accumulate class name
  20. Shrink 2. Find all connections between classes in the program

    1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames
  21. Shrink 2. Find all connections between classes in the program

    1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames constPool
  22. “For each type it loads, a Java virtual machine must

    store a constant pool. A constant pool is an ordered set of constants used by the type, including literals (string, integer, and floating point constants) and symbolic references to types, fields, and methods. Entries in the constant pool are referenced by index, much like the elements of an array. Because it holds symbolic references to all types, fields, and methods used by a type, the constant pool plays a central role in the dynamic linking of Java programs” –Inside the Java Virtual Machine by Bill Venners
  23. Shrink 2. Find all connections between classes in the program

    1 val classPool = ClassPool.getDefault() 2 for (pathToClass in classPaths) { 3 FileInputStream(pathToClass.toFile()).use { 4 val klass = classPool.makeClassIfNew(it) 5 val sourceClassName = klass.name 6 val depClassNames = klass.classFile 7 .constPool 8 .classNames
  24. Shrink 4. Find all classes reachable from seeds 1 private

    fun buildGraph(deps: Multimap<String, String>) 2 : Graph<String, DefaultEdge> { 3 4 val graph = DefaultDirectedGraph() 5 deps.keySet() 6 .filterNot { it.startsWith("java.lang") } 7 .forEach { graph.addVertex(it) } 8 deps.values() 9 .filterNot { it.startsWith("java.lang") } 10 .forEach { graph.addVertex(it) } 11 deps.entries() 12 .filterNot { it.key.startsWith("java.lang") } 13 .filterNot { it.value.startsWith("java.lang") } 14 .filterNot { it.key == it.value } 15 .forEach { graph.addEdge(it.key, it.value) } 16 return graph 17 }
  25. Shrink 4. Find all classes reachable from seeds 1 graph.vertexSet()

    2 .filter { it.endsWith("Activity") } 3 .map { DepthFirstIterator(graph, it).asSequence().toSet() } 4 .flatten()
  26. Shrink 5. Remove all unreachable classes 1 override fun transform(invocation:

    TransformInvocation) { 2 val classNames = getAllClassNames(invocation) 3 val connections = findConnections(classNames) 4 val classGraph = buildGraph(connections) 5 val reachableClasses = findReachableClasses(classGraph) 6 7 // for-each JAR file 8 // remove unreachable classes 9 // repackage JAR file 10 // copy JAR file in output 11 12 // for-each class file 13 // if reachable copy to output
  27. Optimize • Performs lots of different bytecode-level optimizations to the

    code • For concrete example, see Jake Wharton’s recent “Sinking Your Teeth Into Bytecode” presentation https://goo.gl/8hnRzP
  28. Optimize • Marks methods as final, whenever possible. • Removes

    unused method parameters. • Propagates the values of method parameters from method invocations to the invoked methods. • Propagates the values of method return values from methods to their invocations. • Inlines short methods. • Inlines methods that are only called once.
  29. Obfuscate • Classes and class members receive new short random

    names, except for the ones listed by the various -keep options • Internal attributes that are useful for debugging are removed
  30. Resources • Example ProGuard-like transform – Edward Dale
 https://github.com/scompt/WriteYourOwnProGuard •

    “Sinking Your Teeth Into Bytecode” – Jake Wharton
 http://jakewharton.com/sinking-your-teeth-into-bytecode/ • Gephi – The Open Graph Viz Platform
 https://gephi.org/ • JGraphT
 http://jgrapht.org/