Taming Javaagents [BCN JUG 2015]

Taming Javaagents [BCN JUG 2015]

Java agents were an awesome addition to the JVM as it opened a lot of opportunities for tool designers and changed Java tooling landscape quite drastically. In conjunction with Java bytecode manipulation libraries it is now possible to do amazing things to Java classes: we can experiment with programming models, redefine classes in runtime, record
execution flow, etc. In this session we will give an overview of Java agents’ functionality along with the usage examples and real world experiences. We will learn, how to implement an agent and apply Instrumentation API in combination with bytecode manipulation libraries to solve interesting tasks.

5d01eb7205b787b5991db85a11ee5e68?s=128

Oleg Šelajev

April 27, 2015
Tweet

Transcript

  1. Oleg Šelajev @shelajev ZeroTurnaround Taming Javaagents

  2. whoami @shelajev

  3. None
  4. Plan •Java agents •Instrumentation API •Javassist •Challenges

  5. Java Tooling • IDEs • Profilers • Code coverage tools

    • Debuggers • Monitoring agents • JRebel-like tools
  6. None
  7. Java Agent $> java -javaagent:agent.jar Main

  8. Real World Problem

  9. Real World Solution

  10. public static void main(String[] args) {
 for (String arg :

    args) {
 new Thread(new Runnable() {
 public void run() {
 //.....
 }
 }).start(); Java app
  11. public static void main(String[] args) {
 for (String arg :

    args) {
 new Thread(new Runnable() {
 public void run() {
 //.....
 }
 }).start(); RULE trace thread start CLASS java.lang.Thread METHOD start() IF true DO traceln("*** start for thread: "+ $0.getName()) ENDRULE Java app Byteman rule
  12. public static void main(String[] args) {
 for (String arg :

    args) {
 new Thread(new Runnable() {
 public void run() {
 //.....
 }
 }).start(); RULE trace thread start CLASS java.lang.Thread METHOD start() IF true DO traceln("*** start for thread: "+ $0.getName()) ENDRULE > java -javaagent:byteman.jar=script:thread.btm,boot:byteman.jar \ -Dorg.jboss.byteman.transform.all org.my.AppMain2 foo bar baz *** start for thread: foo foo *** start for thread: bar bar *** start for thread: baz baz Java app Byteman rule
  13. Java Agent $> java -javaagent:agent.jar Main

  14. Java Agent Overview

  15. import java.lang.instrument.ClassFileTransformer; import java.lang.instrument.Instrumentation; public class Agent { public static

    void premain(String args, Instrumentation inst) { inst.addTransformer(new ClassFileTransformer { … }); } public static void agentmain(String args, Instrumentation inst) { premain(args, inst); } }
  16. • java -javaagent:<jarfile>[=options] • META-INF/MANIFEST.MF is required public static void

    premain(String args, Instrumentation inst)
  17. • Executed when the agent attaches to the running JVM

    • META-INF/MANIFEST.MF is required • Requires support for loading agents in JVM • Allows adding the code to JVM post-factum public static void agentmain(String args, Instrumentation inst)
  18. Manifest.mf

  19. Manifest.mf Manifest-Version: 1.0 Ant-Version: Apache Ant 1.8.1 Created-By: 1.7.0_15-b03 (Oracle

    Corporation) Premain-Class: com.zeroturnaround.javarebel.java5.AgentInstall Can-Redefine-Classes: true Boot-Class-Path: jrebel.jar Main-Class: com.zeroturnaround.javarebel.java4.Install Implementation-Title: JRebel Specification-Title: jrebel Specification-Version: 5.4.0 Implementation-Version: 201309021535
  20. import com.sun.tools.attach.VirtualMachine; //attach to target VM VirtualMachine vm = VirtualMachine.attach("2177");

    //get system properties in the target VM Properties props = vm.getSystemProperties(); //load agent into the VM vm.loadAgent("agent.jar", "arg1=x,arg2=y"); //detach from VM vm.detach(); http://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/com/sun/tools/attach/VirtualMachine.html
  21. JVM JVM Process 1 Process 2 Agent -cp tools.jar

  22. Instrumentation API

  23. java.lang.instrument.Instrumentation void addTransformer(ClassFileTransformer transformer, boolean canRetransform); void appendToBootstrapClassLoaderSearch(JarFile jarfile); void

    appendToSystemClassLoaderSearch(JarFile jarfile); Class[] getAllLoadedClasses(); Class[] getInitiatedClasses(ClassLoader loader); void redefineClasses(ClassDefinition... classes); void retransformClasses(Class<?>... classes);
  24. java.lang.instrument.ClassFileTransformer byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain pd,

    byte[] classfileBuffer);
  25. java.lang.instrument.ClassFileTransformer public class MyTransformer implements ClassFileTransformer { public void byte[]

    transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain pd, byte[] classfileBuffer){ ClassPool cp = ClassPool.getDefault(); CtClass ct = cp.makeClass(new ByteArrayInputStream(classfileBuffer)); // transform the bytes as required, // for instance - with ASM / Javassist return ct.toBytecode(); } }
  26. java.lang.instrument.ClassFileTransformer public class MyTransformer implements ClassFileTransformer { public void byte[]

    transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain pd, byte[] classfileBuffer){ ClassPool cp = ClassPool.getDefault(); CtClass ct = cp.makeClass(new ByteArrayInputStream(classfileBuffer)); // transform the bytes as required, // for instance - with ASM / Javassist return ct.toBytecode(); } }
  27. Class loading Instrumentation

  28. Class loading ClassFileTransformer ClassFileTransformer ClassFileTransformer Instrumentation

  29. Class loading ClassFileTransformer ClassFileTransformer ClassFileTransformer Javassist ASM Instrumentation

  30. Javassist • Bytecode manipulation made easy • Source-level and bytecode-level

    API • Uses the vocabulary of Java language • On-the-fly compilation of the injected code • http://www.jboss.org/javassist
  31. Finding Resource Leaks

  32. JRebel & XRebel -javaagent:jrebel.jar -javaagent:xrebel.jar

  33. JRebel Reload classes

  34. JRebel notify of changes

  35. JRebel reload conf

  36. JRebel javassist

  37. Challenge Javaagent

  38. oleg@zeroturnaround.com @shelajev github.com/shelajev/ResourceLeakAgent Contact me

  39. Going Native • Java Virtual Machine Tool Interface (JVM TI)

    • Java Native Interface (JNI) • java.lang.instrument package
  40. Going Native • Java code • native / JNI call

    • C / C++ code • clang -shared -undefined dynamic_lookup -o agent.so -I /Library/ Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/include/ -I / Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/ include/darwin native-agent.cpp • JVMTI call • Great power!
  41. Going Native • Java code • native / JNI call

    • C / C++ code • clang -shared -undefined dynamic_lookup -o agent.so -I /Library/ Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/include/ -I / Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/ include/darwin native-agent.cpp • JVMTI call • Great power! handwaving
  42. Going Native • Memory management • Thread / ThreadGroup •

    Heap • Local variables / Early returns / Stack frames • Breakpoints • Timers / Event Management / Monitors
  43. None