Introduction to Bytecode Instrumentation with AGP 8

Talk given at DroidKaigi 2023.

Transcript

1. Marcel Schnelle @marcelschnelle Introduction to Bytecode Instrumentation with AGP 8
2. // HelloWorld.kt fun main(args: Array<String>) { val name = args.firstOrNull() ?: "Guest" println("Hello $name!") }
3. // ================HelloWorldKt.class ================= // class version 52.0 (52) // access flags 0x31 public final class HelloWorldKt { // access flags 0x19 public final static main([Ljava/lang/String;)V @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0 L0 ALOAD 0 LDC "args" INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkNotNullParameter (Ljava/lang/Object;Ljava/lang/String;)V L1 LINENUMBER 2 L1 ALOAD 0 INVOKESTATIC kotlin/collections/ArraysKt.firstOrNull ([Ljava/lang/Object;)Ljava/lang/Object; CHECKCAST java/lang/String DUP IFNONNULL L2 L3 LINENUMBER 2 L3 POP LDC “Guest" (…) // HelloWorld.kt fun main(args: Array<String>) { val name = args.firstOrNull() ?: "Guest" println("Hello $name!") }
4. // ================HelloWorldKt.class ================= // class version 52.0 (52) // access flags 0x31 public final class HelloWorldKt { // access flags 0x19 public final static main([Ljava/lang/String;)V @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0 L0 ALOAD 0 LDC "args" INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkNotNullParameter (Ljava/lang/Object;Ljava/lang/String;)V L1 LINENUMBER 2 L1 ALOAD 0 INVOKESTATIC kotlin/collections/ArraysKt.firstOrNull ([Ljava/lang/Object;)Ljava/lang/Object; CHECKCAST java/lang/String DUP IFNONNULL L2 L3 LINENUMBER 2 L3 POP LDC “Guest" (…) // HelloWorld.kt fun main(args: Array<String>) { val name = args.firstOrNull() ?: "Guest" println("Hello $name!") } Source code
5. // ================HelloWorldKt.class ================= // class version 52.0 (52) // access flags 0x31 public final class HelloWorldKt { // access flags 0x19 public final static main([Ljava/lang/String;)V @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0 L0 ALOAD 0 LDC "args" INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkNotNullParameter (Ljava/lang/Object;Ljava/lang/String;)V L1 LINENUMBER 2 L1 ALOAD 0 INVOKESTATIC kotlin/collections/ArraysKt.firstOrNull ([Ljava/lang/Object;)Ljava/lang/Object; CHECKCAST java/lang/String DUP IFNONNULL L2 L3 LINENUMBER 2 L3 POP LDC “Guest" (…) // HelloWorld.kt fun main(args: Array<String>) { val name = args.firstOrNull() ?: "Guest" println("Hello $name!") } Java Bytecode
6. Bytecode Instrumentation?
7. // ================HelloWorldKt.class ================= // class version 52.0 (52) // access flags 0x31 public final class HelloWorldKt { // access flags 0x19 public final static main([Ljava/lang/String;)V @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0 L0 ALOAD 0 LDC "args" INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkNotNullParameter (Ljava/lang/Object;Ljava/ lang/String;)V L1 LINENUMBER 2 L1 ALOAD 0 INVOKESTATIC kotlin/collections/ArraysKt.firstOrNull ([Ljava/lang/Object;)Ljava/lang/Object; CHECKCAST java/lang/String DUP IFNONNULL L2 L3 LINENUMBER 2 L3 POP LDC “Guest" (…)
8. // ================HelloWorldKt.class ================= // class version 52.0 (52) // access flags 0x31 public final class HelloWorldKt { // access flags 0x19 public final static main([Ljava/lang/String;)V @Lorg/jetbrains/annotations/Nullable;() // invisible, parameter 0 L0 ALOAD 0 LDC "args" INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkNotNullParameter (Ljava/lang/Object;Ljava/ lang/String;)V L1 LINENUMBER 2 L1 ALOAD 0 INVOKESTATIC kotlin/collections/ArraysKt.firstOrNull ([Ljava/lang/Object;)Ljava/lang/Object; CHECKCAST java/lang/String DUP IFNONNULL L2 L3 LINENUMBER 2 L3 POP LDC “Admin” (…)
9. Why though? • Desugaring (Dx) • Code optimization & shrinking (ProGuard) • Library-specific features (Sentry, Realm)
10. D8/R8 Instrumentation API Android compilation pipeline Source 
 (.kt) (simplified) Java bytecode 
 (.class) Dalvik executable 
 (.dex)
12. @Deprecated (kind of)
13. • Introduced in AGP 7.0, “official” since AGP 8.0 • Bridge between AGP and ASM, a popular Java decomposition library • Analyze and modify Java bytecode with a low level API • Higher level abstractions exist, such as Javassist (→ DK’19 talk) New Bytecode Instrumentation API
14. androidComponents.onVariants { variant -> variant.instrumentation.transformClassesWith( classVisitorFactoryImplClass = MyClassVisitorFactory::class.java, scope = InstrumentationScope.PROJECT, instrumentationParamsConfig = {}, ) } Entry point for bytecode instrumentation // build.gradle.kts Android Components
15. androidComponents.onVariants { variant -> variant.instrumentation.transformClassesWith( classVisitorFactoryImplClass = MyClassVisitorFactory::class.java, scope = InstrumentationScope.PROJECT, instrumentationParamsConfig = {}, ) } • debug • release • build flavors // build.gradle.kts Android Components Entry point for bytecode instrumentation
16. androidComponents.onVariants { variant -> variant.instrumentation.transformClassesWith( classVisitorFactoryImplClass = MyClassVisitorFactory::class.java, scope = InstrumentationScope.PROJECT, instrumentationParamsConfig = {}, ) } • Unrelated to UI tests • Hook for transforms // build.gradle.kts Android Components Entry point for bytecode instrumentation
17. androidComponents.onVariants { variant -> variant.instrumentation.transformClassesWith( classVisitorFactoryImplClass = MyClassVisitorFactory::class.java, scope = InstrumentationScope.PROJECT, instrumentationParamsConfig = {}, ) } • Implementation of AsmClassVisitorFactory • Connection to ASM API // build.gradle.kts Android Components Entry point for bytecode instrumentation
18. androidComponents.onVariants { variant -> variant.instrumentation.transformClassesWith( classVisitorFactoryImplClass = MyClassVisitorFactory::class.java, scope = InstrumentationScope.PROJECT, instrumentationParamsConfig = {}, ) } • Which classes should the transform deal with? • PROJECT or ALL // build.gradle.kts Android Components Entry point for bytecode instrumentation
19. androidComponents.onVariants { variant -> variant.instrumentation.transformClassesWith( classVisitorFactoryImplClass = MyClassVisitorFactory::class.java, scope = InstrumentationScope.PROJECT, instrumentationParamsConfig = {}, ) } • Custom configuration settings • Provided via Gradle plugin • Optional // build.gradle.kts Android Components Entry point for bytecode instrumentation
20. • val parameters: Property<T> • val instrumentationContext: InstrumentationContext • fun isInstrumentable(ClassData): Boolean • fun createClassVisitor(ClassContext, ClassVisitor): ClassVisitor AsmClassVisitorFactory<T> Android Gradle Plugin ObjectWeb ASM
21. • AnnotationVisitor visitAnnotation(String, boolean) • void visitOuterClass(String, String, String) • void visitInnerClass(String, String, String, int) • FieldVisitor visitField(int, String, String, String, Object) • MethodVisitor visitMethod(int, String, String, String, String[]) • ... • void visitEnd() ClassVisitor Android Gradle Plugin ObjectWeb ASM
22. ʮSensitive Data Redactorʯ An exemplary Transform implementation
23. The idea data class User( val name: String, val age: Int, val password: String, val luckyNumber: Int, ) User(name=Hoge, age=33, password=droidsaregreat, luckyNumber=1337) User.toString()
24. The idea data class User( val name: String, val age: Int, @Redacted val password: String, @Redacted val luckyNumber: Int, ) User(name=Hoge, age=33, password=********, luckyNumber=********) User.toString()
25. public toString()Ljava/lang/String; @Lorg/jetbrains/annotations/NotNull;() // invisible NEW java/lang/StringBuilder DUP INVOKESPECIAL java/lang/StringBuilder.<init> ()V LDC "User(name=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 GETFIELD de/mannodermaus/dk23/models/User.name : Ljava/lang/String; INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB LDC ", age=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 GETFIELD de/mannodermaus/dk23/models/User.age : I INVOKEVIRTUAL java/lang/StringBuilder.append (I)Ljava/lang/StringBuilder; LDC ", password=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 GETFIELD de/mannodermaus/dk23/models/User.password : Ljava/lang/String; INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB LDC ", luckyNumber=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 GETFIELD de/mannodermaus/dk23/models/User.luckyNumber : I INVOKEVIRTUAL java/lang/StringBuilder.append (I)Ljava/lang/StringBuilder; LDC ")" The idea
26. public toString()Ljava/lang/String; @Lorg/jetbrains/annotations/NotNull;() // invisible NEW java/lang/StringBuilder DUP INVOKESPECIAL java/lang/StringBuilder.<init> ()V LDC "User(name=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 GETFIELD de/mannodermaus/dk23/models/User.name : Ljava/lang/String; INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB LDC ", age=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 GETFIELD de/mannodermaus/dk23/models/User.age : I INVOKEVIRTUAL java/lang/StringBuilder.append (I)Ljava/lang/StringBuilder; LDC ", password=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 LDC “********” GETFIELD de/mannodermaus/dk23/models/User.password : Ljava/lang/String; INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB LDC ", luckyNumber=" INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB ALOAD 0 LDC “********” GETFIELD de/mannodermaus/dk23/models/User.luckyNumber : I INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringB LDC ")" The idea
27. • Determine if it is a Kotlin data class • Collect constructor parameters annotated with @Redacted • Rewrite toString() bytecode to redact info about those parameters • Annotate the class with @RedactedMarker afterwards Proposed Workflow “For each class,…”
28. Demonstration https://github.com/mannodermaus/dk23-bytecode
29. Sources & Resources • https://java-decompiler.github.io • https://github.com/romainguy/kotlin-explorer • https://github.com/getsentry/sentry-android-gradle-plugin • https://jakewharton.com/digging-into-d8-and-r8 • Title image by Michel Didier Joomun 
 (https://unsplash.com/photos/TXZV_xE9ZZI)
30. Marcel Schnelle @marcelschnelle Introduction to Bytecode Instrumentation with AGP 8