Hitchhiker’s Guide to Java Mission Control Alexey Ragozin alexey.ragozin@gmail.com

Profilers are Different Instrumentation built into JVM Java Visual VM JVMTI Instrumentation JMX

Java Mission Control • Session Profiling – classic • JVM Startup Profiling • Continuous Profiling

Java Mission Control

Java Mission Control Do not Panic!

Java Mission Control Most useful views  Method Profiling  Allocation Profiling  Contention & Monitors Information  Thread Execution Timeline  Exception Log  GC Events  JVM Configuration Details  IO Events

Method Profiling

Allocation Profiling

Contention & Monitors

Contention & Monitors

Thread & GC Timeline

Exception Log

Garbage Collection Log

Garbage Collection Log

JVM Configuration Details

JVM Configuration Details

IO Events

Profiling Session Configuration  Garbage Collector  Complier  Method Sampling  Thread Dump  Exceptions  Synchronization Threshold  File I/O Threshold  Socket I/O Threshold  Heap Statistics  Class Loading  Allocation Profiling Enforces Safepoints May be slow if there are many exceptions Set low threshold for profiling Enforces Full GC

Startup Profiling Prepare & Export configuration

Startup Profiling Start JVM with options -XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording= delay=1s,duration=60s,name=MyRecording, filename=myrecording.jfr, settings=C:\PathTo\MyProfiling.jfc

Method Sampling Tracing  Instrument and record every interesting change in observed system Sampling  Capture periodic snapshots of observed system Task.run() wait() process() getBody() read() parse() handle() load() read() Time encode() wait() process() getBody() read() parse() handle() load() read() encode() wait() process() getBody() read() parse() handle() encode()

Method Sampling Task.run() wait() process() getBody() read() parse() handle() load() read() Time encode() wait() process() getBody() read() parse() handle() load() read() encode() wait() process() getBody() read() parse() handle() encode() 10 Task.run 7 3 process() 4 handle() 3 encode() 2 ... 1 ... 1 load() geBody() 2 1 read() parse() 1 ... 3 wait()

Flame Graph Task.run() wait() process() getBody() read() parse() handle() load() read() Time encode() wait() process() getBody() read() parse() handle() load() read() encode() wait() process() getBody() read() parse() handle() encode() Task.run() process() wait() getBody() handle() read() parse() load() encode() SJK - https://github.com/aragozin/jvm-tools

Method Sampling Traditional Profilers  Dump all threads periodically – Stop The World  JVM can suspend thread on in discrete points of execution Flight Recorder  Threads sampled during execution of Java code  No Safepoints / No Stop The World  No samples is thread is not running!

Method Sampling

Method Sampling Bias Problem – Frequent JNI calls into InflaterInputStream Java Flight Recorder SJK (JMX sampling) perf * * https://github.com/jvm-profiling-tools/perf-map-agent Total samples 5653 100.00% ZIP 2659 47.04% Other 2994 52.96% Total samples 2905 100.00% ZIP 2870 98.80% Other 35 1.20% Total samples 59896 100.00% ZIP 54771 91.44% Other 5125 8.56%

Method Sampling Bias Problem – Frequent JNI calls into InflaterInputStream Fix - 92% (565ms -> 45ms) Java Flight Recorder SJK (JMX sampling) perf * * https://github.com/jvm-profiling-tools/perf-map-agent Total samples 5653 100.00% ZIP 2659 47.04% Other 2994 52.96% Total samples 2905 100.00% ZIP 2870 98.80% Other 35 1.20% Total samples 59896 100.00% ZIP 54771 91.44% Other 5125 8.56%

Method Sampling Bias Sampling via JMX Flight Recorder Sampling Unsafe.park()

Method Sampling Bias Sampling via JMX Filter out Unsafe.park() Flight Recorder Sampling

Method Sampling Bias Flight Recorder Sampling Flight Recorder Sampling -XX:+UnlockDiagnosticVMOptions - XX:+DebugNonSafepoints

Execution Time Optimization  Prove what you spend time in code not in wait  Double check with Visual VM or other profiler  Use contention, events and thread timeline views  Set thresholds to 0

Allocation Sampling Instrumentation – Normal Profilers  Instrument every new instruction  Expensive!  Interfere JVM optimizations Flight Recorder  Fast path – Object allocated from TLAB  Slow path – new TLAB have to be allocated  Sample on slow path (every few hundreds of kB)  Non uniform sampling (TLAB size is dynamic) Sampled

Summary Method sampling Method tracing  Generic  IO Allocation sampling Heap dump inspection Startup profiling Continuous profiling Flight Recorder Only Java code Low overhead No Built-in events Low overhead No Yes May require license Classic Java Profiler On Safepoints High overhead Yes - High overhead Yes ? ?

KEEP CALM AND MAKE YOUR JAVA FAST AGAIN Alexey Ragozin alexey.ragozin@gmail.com http://blog.ragozin.info