Slide 1

Slide 1 text

Does My Profiler Tell The Truth? Fabian Lange

Slide 2

Slide 2 text

Profilers are Measurement Tools

Slide 3

Slide 3 text

–Grady Booch „A fool with a tool is still a fool.“

Slide 4

Slide 4 text

–Christian Bauer „Zitat hier eingeben.“

Slide 5

Slide 5 text

Available Tools Mission Control YourKit Solaris Studio VisualVM AppDynamics NetBeans Profiler New Relic JProfiler Honest Profiler JProbe Dynatrace Satoris

Slide 6

Slide 6 text

Source: RebelLabs Developer Productivity Report, Page 22: 0t.ee/javaperformance15

Slide 7

Slide 7 text

Java Virtual Machine Tool Interface

Slide 8

Slide 8 text

JSR-163 Java Platform Profiling Architecture defines APIs:
 
 java.lang.instrument ClassFileTransformer java.lang.management MemoryMXBean ThreadMXBean

Slide 9

Slide 9 text

How To Measure Code? Native Agent written in C inserted into the JVM using -agentpath Java Agent using java.lang.instrument package loaded using -javaagent

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

–Werner Heisenberg, 1927 „One cannot measure Java code without interfering with the JVM“

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

Error Sources

Slide 14

Slide 14 text

Overhead

Slide 15

Slide 15 text

Overhead Runtime Delay CPU Consumption Thread Scheduling Memory Consumption Network Saturation Diskspace Usage

Slide 16

Slide 16 text

Accuracy

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Accuracy System.currentTimeMillis() /** * Returns the current time in milliseconds. Note that while the unit of time of the return value is a millisecond, * the granularity of the value depends on the underlying operating system and may be larger. For example, many * operating systems measure time in units of tens of milliseconds. * * See the description of the class Date for a discussion of slight discrepancies that may arise between * "computer time" and coordinated universal time (UTC). */ System.nanoTime() /** * Returns the current value of the running Java Virtual Machine's high-resolution time source, in nanoseconds. * * This method can only be used to measure elapsed time and is not related to any other notion of system or wall-clock * time. The value returned represents nanoseconds since some fixed but arbitrary origin time (perhaps in the * future, so values may be negative). The same origin is used by all invocations of this method in an instance of a * Java virtual machine; other virtual machine instances are likely to use a different origin. * * This method provides nanosecond precision, but not necessarily nanosecond resolution (that is, how frequently the * value changes) - no guarantees are made except that the resolution is at least as good as that of * {@link #currentTimeMillis()}. * * The values returned by this method become meaningful only when the difference between two such values, obtained * within the same instance of a Java virtual machine, is computed. */

Slide 19

Slide 19 text

Time

Slide 20

Slide 20 text

Time Wall-Clock Time “Real” time which has passed since start. Measurable with a clock on the wall. CPU Time Time the CPU was busy. Measurable but questionable.

Slide 21

Slide 21 text

Lots of Data

Slide 22

Slide 22 text

Data Collection

Slide 23

Slide 23 text

Data Collection (Stack) Sampling Checking JVM activity in regular intervals. Instrumentation Injection of measurement code. Sampling II Reducing data by omission.

Slide 24

Slide 24 text

Sampling t main() foo() bar() baz()

Slide 25

Slide 25 text

Sampling - Great t main() foo() bar() baz()

Slide 26

Slide 26 text

Sampling - Not So Great t main() foo() bar()

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

Instrumentation

Slide 29

Slide 29 text

Instrumentation t main() foo() bar() baz()

Slide 30

Slide 30 text

Sampling
 vs
 Instrumentation DEMO

Slide 31

Slide 31 text

Code on Github https://github.com/CodingFabian/SamplingVsInstrumentation

Slide 32

Slide 32 text

Using JProfiler DEMO

Slide 33

Slide 33 text

Using HProf DEMO

Slide 34

Slide 34 text

Using Honest Profiler DEMO

Slide 35

Slide 35 text

Safepoints

Slide 36

Slide 36 text

Safepoints Sampling thread has to wait for steady state to interrogate other threads. Safepoints are in-between code, so conceptually sampling never sees running code. Honest Profiler uses JVMTI AsyncGetCallTrace which does not wait for safepoints. github.com/RichardWarburton/honest-profiler jeremymanson.blogspot.co.uk/2013/07/lightweight-asynchronous-sampling.html

Slide 37

Slide 37 text

From My Daily Work

Slide 38

Slide 38 text

Performance Tuning Guide Start off with Sampling Do not take results to serious Look for bottlenecks < 10 ms is most of the time irrelevant when profiling Get better results from benchmarking Check code, bytecode, assembly

Slide 39

Slide 39 text

How Much Influence has Instrumentation Code? DEMO

Slide 40

Slide 40 text

Does My Profiler Tell The Truth? NO

Slide 41

Slide 41 text

Use JMH for Benchmarks

Slide 42

Slide 42 text

Further Reading Dapper, a Large-Scale Distributed Systems Tracing Infrastructure static.googleusercontent.com/media/research.google.com/de//pubs/archive/36356.pdf Evaluating the Accuracy of Java Profilers www-plan.cs.colorado.edu/klipto/mytkowicz-pldi10.pdf How to Measure Java Performance blog.codecentric.de/en/2011/10/measure-java-performance-sampling-or-instrumentation/ Java Microbenchmark Harness openjdk.java.net/projects/code-tools/jmh/ Richard Warbutons Honest Profiler github.com/RichardWarburton/honest-profiler

Slide 43

Slide 43 text

Want to know more? @CodingFabian [email protected] lj speakerdeck.com/CodingFabian github.com/CodingFabian