Java Debugging Internals

Transcript

1. Java Debugging Internals @yardenlaif
2. 2 Who Am I? • Software Engineer at Rookout • Background in low level C programming in linux IOT environments • Java, Go, Ruby • C#, Python, JavaScript, C++
3. Non Breaking Breakpoints 3
4. Non Breaking Breakpoints 4
5. jdb -attach 8000 java -agentlib:jdwp= transport=dt_socket,server=y, suspend=n,address=*:8000 myApp JDB - The Java Debugger 5 JVM MyApp Debugger CLI
6. JPDA - Java Platform Debugging Architecture The Java Platform Debugger Architecture (JPDA) consists of three interfaces designed for use by debuggers in development environments for desktop systems. - Oracle Documentation 6
7. JPDA - Java Platform Debugging Architecture • JDI - Java Debug Interface • JDWP - Java Debug Wire Protocol • JVMTI - Java Virtual Machine Tool Interface 7
8. JDI - Java Debug Interface • High level Java API • Inspect & Control • Connect remotely 8
9. JDI - Java Debug Interface • Add breakpoint ◦ createBreakpointRequest • Collect data ◦ visibleVariables ◦ getArgumentValues ◦ frames • Continue program ◦ resume 9
10. Using JDI • No availability during collection • Zero control or visibility • Easy to make mistakes • Single instance debugging 10
11. JPDA - Java Platform Debugging Architecture • JDI - Java Debug Interface • JDWP - Java Debug Wire Protocol • JVMTI - Java Virtual Machine Tool Interface 11
12. JDWP - Java Debug Wire Protocol • A protocol • Used for communication between a debugger and the JVM which it debugs 12 JVM MyApp Debugger CLI JDWP
13. Using JDWP • No availability during collection • Zero control or visibility • Easy to make mistakes • Single instance debugging • Managing multiple clients and connecting to multiple servers 13
14. JPDA - Java Platform Debugging Architecture • JDI - Java Debug Interface • JDWP - Java Debug Wire Protocol • JVMTI - Java Virtual Machine Tool Interface 14
15. JVMTI - Java Virtual Machine Tool Interface • Native programming interface • Interacts directly with the JVM • Inspect & Control 15
16. JVMTI - Java Virtual Machine Tool Interface • SetBreakpoint • SetEventNotificationMode ◦ Breakpoint • GetLocalObject ◦ jobject 16
17. JVMTI - Get Stack Trace /* Get Stack Trace */ err = (*jvmti)->GetStackTrace(jvmti, thr, 0, 5, &frames, &count); if (err != JVMTI_ERROR_NONE) { printf("(GetThreadInfo) Error expected: %d, got: %d\n", JVMTI_ERROR_NONE, err); describe(err); printf("\n"); } printf("Number of records filled: %d\n", count); if (err == JVMTI_ERROR_NONE && count >=1) { char *methodName; methodName = "yet_to_call()"; char *declaringClassName; jclass declaring_class; int i=0; printf("Exception Stack Trace\n"); printf("=====================\n"); printf("Stack Trace Depth: %d\n", count); for ( i=0; i < count; i++) { err = (*jvmti)->GetMethodName(jvmti, frames[i].method, &methodName, NULL, NULL); if (err == JVMTI_ERROR_NONE) { err = (*jvmti)->GetMethodDeclaringClass(jvmti, frames[i].method, &declaring_class); err = (*jvmti)->GetClassSignature(jvmti, declaring_class, &declaringClassName, NULL); if (err == JVMTI_ERROR_NONE) { printf("at method %s() in class %s\n", methodName, declaringClassName); } } } } 17 *A lot of code*
18. Java - Get Stack Trace Thread.currentThread().getStackTrace() 18
19. Using JVMTI • Incredibly Complex • Native vulnerabilities • Not portable • Bugs are catastrophic 19
20. 20 Newer? Simpler? Full control? In-process? Unbreakable?
21. Requirements 21 • Don’t break • Predictability • Availability • Performance • Security and compliance • Multi-instance debugging
22. Thread 2: Client Connection public Response Process(Request req) { String id = req.GetID(); if (isExistingUser(id)) { return new Response("User exists!"); } // Create new user... } Requirements 22 • Don’t break • Availability Thread 1: Server Client Connection Thread 3: Client Connection public Response Process(Request req) { String id = req.GetID(); if (isExistingUser(id)) { return new Response("User exists!"); } // Create new user... } Thread 2: Client Connection public Response Process(Request req) { String id = req.GetID(); if (isExistingUser(id)) { return new Response("User exists!"); } // Create new user... }
23. -agentlib:jdwp address=*:8000 transport=dt_socket server=y suspend=n Break It Down java -agentlib:jdwp=transport=dt_socket,server=y, suspend=n,address=*:8000 myApp 23
24. Java Agents 24
25. Java Agents 25 JVM MyApp Java Agent java.lang.Instrument API
26. Sample Program 26 1| public class HelloWorld { 2| public static void main(String[] args) { 3| while(true) { 4| System.out.println("Hello, World!"); 5| } 6| } 7| }
27. Sample Program 27 • Build HelloWorld to JAR • Run: java -jar HelloWorld.jar • Output: Hello, World! Hello, World! ...
28. Java Agent - Premain 28 public class JavaAgent { public static void premain(String agentArgs, Instrumentation instrumentation) throws Exception { System.out.println("Hello, JavaSummit!"); } }
29. Java Agent - Premain 29 public class JavaAgent { public static void premain(String agentArgs, Instrumentation instrumentation) throws Exception { System.out.println("Hello, JavaSummit!"); } }
30. Java Agent - Premain 30 public class JavaAgent { public static void premain(String agentArgs, Instrumentation instrumentation) throws Exception { System.out.println("Hello, JavaSummit!"); } }
31. Java Agent - Premain 31 public class JavaAgent { public static void premain(String agentArgs, Instrumentation instrumentation) throws Exception { System.out.println("Hello, JavaSummit!"); } }
32. Java Agent - Premain 32 • Build JavaAgent to JAR • Run with java agent: java -javaagent:JavaAgent.jar -jar HelloWorld.jar • Output: Hello, JavaSummit! Hello, World! ...
33. Java Agent - ClassFileTransformer 33 JVM Compiler Code Bytecode
34. Java Agent - ClassFileTransformer 34 JVM 32 | … 33 | new Item() 34 | … Class Loader Get Item class bytecode Return Item class bytecode
35. Java Agent - ClassFileTransformer 35 JVM 32 | … 33 | new Item() 34 | … Class Loader Get Item class bytecode Return Item class bytecode Class File Transformer Return transformed Item class bytecode
36. Java Agent - ClassFileTransformer 36 public class JavaAgent { public static void premain(String agentArgs, Instrumentation instrumentation) throws Exception { System.out.println("Hello, JavaSummit!"); } } • void addTransformer(ClassFileTransformer transformer);
37. Java Agent - ClassFileTransformer 37 public class DebugTransformer implements ClassFileTransformer { @Override public byte[] transform(ClassLoader classLoader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) {
38. Java Agent - ClassFileTransformer 38 public class DebugTransformer implements ClassFileTransformer { @Override public byte[] transform(ClassLoader classLoader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) {
39. Java Agent - ClassFileTransformer 39 public class DebugTransformer implements ClassFileTransformer { @Override public byte[] transform(ClassLoader classLoader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) {
40. Open Source Bytecode Manipulation Tools BCEL ASM 40
41. Bytecode Abstraction 41 • AbstractInsnNode ◦ JumpInsnNode ◦ MethodInsnNode ◦ VarInsnNode • InsnList instructions
42. Bytecode Manipulation 42 byte[] addPrintCall(byte[] classfileBuffer) { // Add call to System.out.println("Transformed!") // Return transformed bytecode }
43. Java Agent - ClassFileTransformer 43 public class DebugTransformer implements ClassFileTransformer { @Override public byte[] transform(ClassLoader classLoader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) { if (className.contains("HelloWorld")) { return addPrintCall(classfileBuffer); } return classfileBuffer; } }
44. Java Agent - ClassFileTransformer 44 public class DebugTransformer implements ClassFileTransformer { @Override public byte[] transform(ClassLoader classLoader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) { if (className.contains("HelloWorld")) { return addPrintCall(classfileBuffer); } return classfileBuffer; } }
45. Java Agent - Premain 45 public class JavaAgent { public static void premain(String agentArgs, Instrumentation instrumentation) throws Exception { DebugTransformer transformer = new DebugTransformer(); instrumentation.addTransformer(transformer); } }
46. Java Agent - Premain 46 public class JavaAgent { public static void premain(String agentArgs, Instrumentation instrumentation) throws Exception { DebugTransformer transformer = new DebugTransformer(); instrumentation.addTransformer(transformer); } }
47. Java Agent - Premain 47 • Build JavaAgent to JAR • Run with java agent: java -javaagent:JavaAgent.jar -jar HelloWorld.jar • Output: Transformed! Hello, World! Hello, World! ...
48. Non-Breaking Breakpoints!
49. Profiling Interpreter C2 C1 Profiling Profiling Interpreter C2 C1 Profiling JVM Execution Engine 49
50. Add Breakpoint 50 public void Breakpoint(Object[] locals) { System.out.println("Breakpoint hit!"); for (Object local : locals) { System.out.println(local); } }
51. Add Breakpoint 51 byte[] addBreakpoint(byte[] classfileBuffer, int lineno) { // Find where to insert Breakpoint call // Push local variables onto stack // Insert call to Breakpoint // Return transformed bytecode }
52. Add Breakpoint 52 byte[] addBreakpoint(byte[] classfileBuffer, int lineno) { // Find where to insert Breakpoint call // Push local variables onto stack // Insert call to Breakpoint // Return transformed bytecode }
53. Add Breakpoint 53 byte[] addBreakpoint(byte[] classfileBuffer, int lineno) { // Find where to insert Breakpoint call // Push local variables onto stack // Insert call to Breakpoint // Return transformed bytecode }
54. Add Breakpoint 54 byte[] addBreakpoint(byte[] classfileBuffer, int lineno) { // Find where to insert Breakpoint call // Push local variables onto stack // Insert call to Breakpoint // Return transformed bytecode }
55. Add Breakpoint 55 byte[] addBreakpoint(byte[] classfileBuffer, int lineno) { // Find where to insert Breakpoint call // Push local variables onto stack // Insert call to Breakpoint // Return transformed bytecode }
56. Sample Program 56 1| public class HelloWorld { 2| public static void main(String[] args) { 3| while(true) { 4| System.out.println("Hello, World!"); 5| } 6| } 7| }
57. Java Agent - ClassFileTransformer 57 public class DebugTransformer implements ClassFileTransformer { @Override public byte[] transform(ClassLoader classLoader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) { if (className.contains("HelloWorld")) { return addBreakpoint(classfileBuffer, 4); } return classfileBuffer; } }
58. Java Agent - Premain 58 • Build JavaAgent to JAR • Run with java agent: java -javaagent:JavaAgent.jar -jar HelloWorld.jar • Output: Breakpoint hit! [Ljava.lang.Object;@3d075dc0 Hello, World! ...
59. 59 What if the class has already been loaded?
60. Magic Instrumentation 60 void redefineClasses(ClassDefinition... definitions) void retransformClasses(Class<?>... classes)
61. Putting it all together 61 • Transform on demand • Add call to breakpoint function • Access locals as objects
62. 62 Putting it all together
63. Thank You @yardenlaif rookout.com