Slide 1
Slide 1 text
Manipulation de bytecode :
démocratisons la magie noire !
1
Julien Ponge
@jponge
Frédéric Le Mouël
@flemouel
Slide 2
Slide 2 text
2
@jponge @flemouel
Slide 3
Slide 3 text
3
Nous faisons du middleware.
dynamique
pervasif
adaptatif
reloading
multi-tenancy
resource-control
dynamic aspects
(...)
(...)
Slide 4
Slide 4 text
4
Conteneurs (Java EE, Spring)
Android liblayout
Play! Framework
JRebel
Hibernate
Langages sur JVM
Terracotta
FindBugs
Slide 5
Slide 5 text
5
java.lang.reflect
... parfois il faut “plus”
Slide 6
Slide 6 text
6
JVM+Bytecode
ASM
AspectJ
Byteman
JooFlux
... en fait c’est facile !
écrire et transformer du bytecode
faire simplement des modifications
statiques courantes avec des aspects
aspects dynamiques, application à
l’amélioration des couvertures de tests
aspects dynamiques via invokedynamic
Slide 7
Slide 7 text
JVM + Bytecode
crash course
7
Slide 8
Slide 8 text
8
“JVM Bytecode for Dummies”
— Charles Nutter @headius
Clair, complet et didactique !
Slide 9
Slide 9 text
9
java/lang/String
Nom de classe
Slide 10
Slide 10 text
10
V void
Z boolean
C char
B byte
S short
I int
F float
J long
D double
Types primitifs
Slide 11
Slide 11 text
11
Ljava/lang/String;
Type objet : L;
Slide 12
Slide 12 text
12
[I
[Ljava/lang/String;
Tableaux : [
Slide 13
Slide 13 text
13
(I,I)I
int foo(int, int)
(Ljava/lang/String;)V
void foo(String)
Slide 14
Slide 14 text
14
Constant pool
(nombres, String, types)
Variables locales
Machine à pile
OpCodes
invocation de méthode
manipulation de pile
get / set champs
arithmétique
allocation
sauts
lock
(...)
0 java/lang/String
1 “Foo”
2 666
... ...
0 this (sauf static)
1 arg1
2 arg2
3 int i
... ...
Slide 15
Slide 15 text
15
public class HelloWorld {
public static void main(String[] args) {
int i = 1;
int j = 2;
int k = 100;
System.out.println(">>> " + (i + j + k));
}
}
Slide 16
Slide 16 text
16
$ javap -c HelloWorld
Compiled from "HelloWorld.java"
public class HelloWorld extends java.lang.Object{
public HelloWorld();
Code:
0: aload_0
1: invokespecial #1; //Method java/lang/Object."":()V
4: return
(...)
Slide 17
Slide 17 text
16
$ javap -c HelloWorld
Compiled from "HelloWorld.java"
public class HelloWorld extends java.lang.Object{
public HelloWorld();
Code:
0: aload_0
1: invokespecial #1; //Method java/lang/Object."":()V
4: return
(...)
invocation private et constructeur
+ invoke{static, interface, virtual, dynamic}
décompilateur
constructeur
+ ,
this
dans le constant pool
Slide 18
Slide 18 text
17
public static void main(java.lang.String[]);
Code:
0: iconst_1
1: istore_1
2: iconst_2
3: istore_2
4: bipush 100
6: istore_3
7: getstatic #2; //Field java/lang/System.out:Ljava/io/PrintStream;
10: new #3; //class java/lang/StringBuilder
13: dup
14: invokespecial #4; //Method java/lang/StringBuilder."":()V
17: ldc #5; //String >>>
19: invokevirtual #6; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuil
22: iload_1
23: iload_2
24: iadd
25: iload_3
26: iadd
27: invokevirtual #7; //Method java/lang/StringBuilder.append:(I)Ljava/lang/StringBuilder;
30: invokevirtual #8; //Method java/lang/StringBuilder.toString:()Ljava/lang/String;
33: invokevirtual #9; //Method java/io/PrintStream.println:(Ljava/lang/String;)V
36: return
? ? ?
Slide 19
Slide 19 text
18
1
iconst_1
0 args
1
2
3
variables
Slide 20
Slide 20 text
19
istore_1
0 args
1 1
2
3
variables
Slide 21
Slide 21 text
20
2
iconst_2
0 args
1 1
2
3
variables
Slide 22
Slide 22 text
21
istore_2
0 args
1 1
2 2
3
variables
Slide 23
Slide 23 text
22
100
bipush 100
0 args
1 1
2 2
3
variables
Slide 24
Slide 24 text
23
istore_3
0 args
1 1
2 2
3 100
variables
Slide 25
Slide 25 text
24
out
getstatic
System.out
0 args
1 1
2 2
3 100
variables
Slide 26
Slide 26 text
25
builder
out
new
StringBuilder
0 args
1 1
2 2
3 100
variables
Slide 27
Slide 27 text
26
builder
builder
out
dup
0 args
1 1
2 2
3 100
variables
Slide 28
Slide 28 text
27
builder
out
invokespecial
StringBuilder.
0 args
1 1
2 2
3 100
variables
Slide 29
Slide 29 text
28
“>>> “
builder
out
ldc
“>>> ”
0 args
1 1
2 2
3 100
variables
Slide 30
Slide 30 text
29
builder
out
invokevirtual
append(String)
0 args
1 1
2 2
3 100
variables
Slide 31
Slide 31 text
30
1
builder
out
iload_1
0 args
1 1
2 2
3 100
variables
Slide 32
Slide 32 text
31
2
1
builder
out
iload_2
0 args
1 1
2 2
3 100
variables
Slide 33
Slide 33 text
32
3
builder
out
iadd
0 args
1 1
2 2
3 100
variables
Slide 34
Slide 34 text
33
100
3
builder
out
iload_3
0 args
1 1
2 2
3 100
variables
Slide 35
Slide 35 text
34
103
builder
out
iadd
0 args
1 1
2 2
3 100
variables
Slide 36
Slide 36 text
35
builder
out
invokevirtual
append(int)
0 args
1 1
2 2
3 100
variables
Slide 37
Slide 37 text
36
“>>> 103”
out
invokevirtual
toString()
0 args
1 1
2 2
3 100
variables
Slide 38
Slide 38 text
37
invokevirtual
println(String)
0 args
1 1
2 2
3 100
variables
Slide 39
Slide 39 text
38
return
0 args
1 1
2 2
3 100
variables
Slide 40
Slide 40 text
ASM
39
Slide 41
Slide 41 text
40
package my;
public class Main {
public static void main(String[] args) {
for (int i = 0; i < 10; i++) {
System.out.println("Hello Devoxx France!");
}
}
}
Générons ceci sans javac !
Slide 42
Slide 42 text
41
i: int [start, end]
start:
i = 0
loop:
println(System.out, “Hello Devoxx France!”)
incr(i, 1)
jump(loop) if (i < 10)
end:
return
goto isn’t harmful!
Slide 43
Slide 43 text
42
ClassWriter writer = new ClassWriter(COMPUTE_MAXS | COMPUTE_FRAMES);
writer.visit(V1_6, ACC_PUBLIC, "my/Main", null, "java/lang/Object", null);
writer.visitSource("Main.java", null);
MethodVisitor mv = writer.visitMethod(ACC_STATIC | ACC_PUBLIC, "main",
"([Ljava/lang/String;)V", null, null);
mv.visitCode();
Classe + main(String...)
Slide 44
Slide 44 text
43
Label start = new Label();
Label loop = new Label();
Label end = new Label();
mv.visitLabel(start);
final int COUNTER_VAR = 1;
mv.visitLocalVariable("counter", "I", null, start, end, COUNTER_VAR);
mv.visitInsn(ICONST_0);
mv.visitVarInsn(ISTORE, COUNTER_VAR);
i = 0
Slide 45
Slide 45 text
44
mv.visitLabel(loop);
mv.visitFieldInsn(GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;");
mv.visitLdcInsn("Hello Devoxx France!");
mv.visitMethodInsn(INVOKEVIRTUAL, "java/io/PrintStream", "println",
"(Ljava/lang/String;)V");
println()
Slide 46
Slide 46 text
45
mv.visitIincInsn(COUNTER_VAR, 1);
mv.visitVarInsn(ILOAD, COUNTER_VAR);
mv.visitLdcInsn(10);
mv.visitJumpInsn(IF_ICMPLT, loop);
mv.visitLabel(end);
mv.visitInsn(RETURN);
mv.visitMaxs(666, 666);
mv.visitEnd();
writer.visitEnd(); i++, (i < 10)?, goto
Slide 47
Slide 47 text
46
File outDir = new File("gen/my");
outDir.mkdirs();
FileOutputStream out = new FileOutputStream(new File(outDir, "Main.class"));
out.write(writer.toByteArray());
out.close();
.class !
Slide 48
Slide 48 text
47
(réalisé sans trucages)
Slide 49
Slide 49 text
48
profiling de chaque
méthode utilisée
par un programme
Slide 50
Slide 50 text
Manifest-Version: 1.0
Premain-Class: profiler.ProfilerTransformer
Can-Redefine-Classes: true
Can-Retransform-Classes: true
Can-Set-Native-Method-Prefix: false
java -javaagent:profiler.jar some.Main
Slide 51
Slide 51 text
50
public class ProfilerTransformer implements ClassFileTransformer {
public static void premain(String agentArgs, Instrumentation inst) {
inst.addTransformer(new ProfilerTransformer());
}
public byte[] transform(ClassLoader classLoader, String s, Class> aClass,
ProtectionDomain protectionDomain, byte[] bytes)
throws IllegalClassFormatException {
ClassReader reader = new ClassReader(bytes);
ClassWriter writer = new ClassWriter(reader, COMPUTE_FRAMES | COMPUTE_MAXS);
ProfilerClassVisitor profiler = new ProfilerClassVisitor(ASM4, writer);
reader.accept(profiler, 0);
return writer.toByteArray();
}
}
Slide 52
Slide 52 text
51
@Override
public void visitCode() {
super.visitCode();
startVarId = newLocal(Type.getType("J"));
mv.visitMethodInsn(INVOKESTATIC, "java/lang/System", "nanoTime", "()J");
mv.visitVarInsn(LSTORE, startVarId);
}
Slide 53
Slide 53 text
52
@Override
public void visitInsn(int opcode) {
if ((opcode >= IRETURN && opcode <= RETURN) || opcode == ATHROW) {
(...)
mv.visitMethodInsn(INVOKESTATIC, "java/lang/System", "nanoTime", "()J");
mv.visitVarInsn(LLOAD, startVarId);
mv.visitInsn(LSUB);
mv.visitMethodInsn(INVOKEVIRTUAL, "java/io/PrintStream", "println", "(J)V");
}
super.visitInsn(opcode);
}
Slide 54
Slide 54 text
53
➟ adapters, inliner, optimizers, ...
➟ modèle objet
➟ analyse statique de code
➟ tracing, retro-engineering, ...
org.objectweb.asm.commons
org.objectweb.asm.tree
org.objectweb.asm.tree.analysis
org.objectweb.asm.util
Slide 55
Slide 55 text
54
“Ok, et pour les
cas plus simples ?”
Slide 56
Slide 56 text
55
Slide 57
Slide 57 text
56
WTF Layer
WTF Layer
WTF Layer
Security, validation,
logging, transactions, ...
Slide 58
Slide 58 text
57
public class UserController {
// (...)
public void update(UserProfile profile)
throws ControllerException {
validator.check(profile);
if (logger.isLevelEnabled(DEBUG)) {
logger.debug("Updating: " + profile);
}
dao.save(profile);
redirect("/home");
}
// (...)
}
“cross-cutting concerns”
“business logic”
Slide 59
Slide 59 text
58
Pointcuts Advices
Inter-type declarations
Où ? Quoi ?
+ méthodes + attributs + hiérarchie
+ vérifications + ordre
Slide 60
Slide 60 text
59
(demo)
*
* en cas de plantage, ne jettez pas de tomates, merci.
Slide 61
Slide 61 text
60
aspect ArgsTracer {
before(): target(HelloWorld) && call(void run(String...)) {
System.out.println("Calling run()");
}
}
Que se passe t-il avec un before ?
Slide 62
Slide 62 text
61
--- HelloWorld.bytecode.original 2012-04-11 22:36:22.000000000 +0200
+++ HelloWorld.bytecode.weaved 2012-04-11 22:45:51.000000000 +0200
@@ -19,8 +19,16 @@
3: dup
4: invokespecial #34; //Method "":()V
7: aload_0
- 8: invokevirtual #35; //Method run:([Ljava/lang/String;)V
- 11: return
+ 8: astore_1
+ 9: astore_2
+ 10: invokestatic #44; //Method ArgsTracer.aspectOf:()LArgsTracer;
+ 13: aload_2
+ 14: aload_1
+ 15: invokevirtual #48; //Method ArgsTracer.ajc$before$ArgsTracer$1$66cd2654:(LHelloWorld;
[Ljava/lang/String;)V
+ 18: aload_2
+ 19: aload_1
+ 20: invokevirtual #35; //Method run:([Ljava/lang/String;)V
+ 23: return
}
Slide 63
Slide 63 text
62
aspect MakePersonComparable {
declare parents: Person implements Comparable;
public int Person.compareTo(Person other) {
return getName().compareTo(other.getName());
}
}
Que se passe t-il avec des
déclarations inter-types ?
Slide 64
Slide 64 text
63
$ javap -c -private Person
Compiled from "Person.java"
public class Person extends java.lang.Object implements java.lang.Comparable{
(...)
public int compareTo(java.lang.Object);
Code:
0: aload_0
1: aload_1
2: checkcast #1; //class Person
5: invokevirtual #50; //Method compareTo:(LPerson;)I
8: ireturn
public int compareTo(Person);
Code:
0: aload_0
1: aload_1
2: invokestatic #60; //Method MakePersonComparable.ajc$interMethod
$MakePersonComparable$Person$compareTo:(LPerson;LPerson;)I
5: ireturn
}
Slide 65
Slide 65 text
64
$ javap -c -private MakePersonComparable
Compiled from "MakePersonComparable.aj"
public class MakePersonComparable extends java.lang.Object{
(...)
public static int ajc$interMethod$MakePersonComparable$Person$compareTo(Person, Person);
Code:
0: aload_0
1: invokevirtual #44; //Method Person.getName:()Ljava/lang/String;
4: aload_1
5: invokevirtual #44; //Method Person.getName:()Ljava/lang/String;
8: invokevirtual #49; //Method java/lang/String.compareTo:(Ljava/lang/String;)I
11: ireturn
(...)
}
Slide 66
Slide 66 text
No content
Slide 67
Slide 67 text
No content
Slide 68
Slide 68 text
67
AspectJ est puissant et expressif
AspectJ est pratique pour des modifications statiques
AspectJ est à utiliser avec modération
↪ Spring Roo
Slide 69
Slide 69 text
68
Slide 70
Slide 70 text
69
Règles dynamiques !
tests : injection de fautes
traces d’exécution
suivi de ressources
(...)
Slide 71
Slide 71 text
70
Event
Condition
Action
“100+ instances ?”
“Jetter une exception !”
new java.io.FileInputStream(...);
Slide 72
Slide 72 text
71
Byteman agent rule1.btm
rule2.btm
rule3.btm
JVM
void redefineClasses(ClassDefinition... definitions)
throws ClassNotFoundException,
UnmodifiableClassException
classes
bmsubmit
bminstall
java -javagent:lib/byteman.jar=...
Slide 73
Slide 73 text
72
RULE calling print in HelloWorld
CLASS HelloWorld
METHOD print(String)
IF true
DO traceln("calling print() with str = " + $1)
ENDRULE
Slide 74
Slide 74 text
73
java.lang.String
^java.lang.Object
^java.util.List
java.util.List
Types
CLASS
INTERFACE
CLASS
INTERFACE
Slide 75
Slide 75 text
74
AT
ENTRY
EXIT
THROW (count)
AT, AFTER
INVOKE (count)
SYNCHRONIZE (count)
AT, AFTER
LINE
READ (count)
READ $ (count)
WRITE (count)
WRITE $ (count)
Spécifier une cible
Slide 76
Slide 76 text
75
debug(msg)
traceln(msg)
traceOpen(id, filename)
traceln(id, msg)
traceClose(id)
traceStack()
traceStack(id)
killThread()
killJVM()
killJVM(code)
Waiters
Rendez-vous
Joiners
Countdowns
Flags
Counters
Timers
Helpers
actions + conditions
+ les vôtres
Slide 77
Slide 77 text
76
(demo)
*
* en cas de plantage, ne jettez pas de tomates, merci.
Slide 78
Slide 78 text
No content
Slide 79
Slide 79 text
foo.txt BlobStore.java LICENSE COPYING (...)
work/index
work/e6a92ec2fe5fba022c31c32c97ea455cee4b2736
work/a9a1282b23431aab89c9a4647fd88569eb9a389d
work/6a55a0ac5df9e1edce33995a8ac59f17a04a64c9
Slide 80
Slide 80 text
79
maximum testable humainement
≃
Slide 81
Slide 81 text
File.renameTo()
File.renameTo()
Lecture flux
+ File.exists()
+ File.delete()
Lecture flux
+ File.exists()
+ File.delete()
Lecture flux
+ corruption GZIP
Lecture flux
+ corruption GZIP
80
Difficile à
reproduire ...
Slide 82
Slide 82 text
81
@RunWith(BMUnitRunner.class)
public class BlobStoreTest
{ ... }
Slide 83
Slide 83 text
82
@BMScript(value = "create_but_fail_to_mkdir", dir = BYTEMAN_SCRIPTS)
@Test(expected = BlobStoreException.class)
public void create_but_fail_to_mkdir() throws IOException {
new BlobStore(new File(temporaryFolder.getRoot(), "missing"));
}
RULE inject a fault into BlobStore to fail on mkdirs()
CLASS blob.store.BlobStore
METHOD ensureValidWorkingDirectory(File)
AFTER INVOKE java.io.File.mkdirs()
IF true
DO $! = false
ENDRULE
Slide 84
Slide 84 text
RULE create a countdown when entering BlobStore#rewriteIndex()
CLASS blob.store.BlobStore
METHOD rewriteIndex
AT ENTRY
IF true
DO createCountDown("KillIndex", 1)
ENDRULE
RULE inject a fault into BlobStore#rewriteIndex() to fail appending to the index
CLASS com.google.common.io.Files
METHOD append
AT ENTRY
IF countDown("KillIndex")
DO throw new java.io.IOException("Simulating a I/O error")
ENDRULE
private void rewriteIndex() {
if (!indexFile.delete()) {
throw new BlobStoreException("Could not delete " + indexFile);
}
try {
for (String key : index.keySet()) {
append(indexLineFor(key, index.get(key)), indexFile, UTF_8);
}
} catch (IOException e) {
throw new BlobStoreException(e);
}
}
#fail à la 2ème
itération
Slide 85
Slide 85 text
84
versus
Slide 86
Slide 86 text
Méconnu
Magique
Simple
Utile !
85
Slide 87
Slide 87 text
86
JooFlux
(research work in progress)
Slide 88
Slide 88 text
87
invokedynamic (Java 7+)
pour les langages dynamiques
lambda Java 8, Nashorn (JS)
→ talk de Rémi Forax
Slide 89
Slide 89 text
(...)
iconst_1
iconst_2
invokedynamic add(I,I)I + bootstrap
(...)
fonction typée,
mais cible résolue
à l’exécution
renvoie un
java.lang.invoke.CallSite
via un invokestatic ou autre
Slide 90
Slide 90 text
89
CallSite method handle
chaîne de
combinateurs
méthode cible
cible
(modifiable)
asType()
asCollector()
dropArguments()
guardWithTest()
(...)
(JIT-friendly)
Slide 91
Slide 91 text
90
JooFlux invokeXYZ 㱺 invokedynamic
agent JVM
agent JMX
remplacement de méthode
rechargement de méthode
ajout d’aspect dynamique
impact minimal coté JIT
Slide 92
Slide 92 text
91
(demo)
*
* en cas de plantage, ne jettez pas de tomates, merci.
Slide 93
Slide 93 text
92
Avons-nous démocratisé
la magie noire ?
dites-le nous !
Slide 94
Slide 94 text
93
@jponge @flemouel