Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
Avoiding Benchmarking Pitfalls on the JVM Julien Ponge
Slide 2
Slide 2 text
@jponge ! Maître de Conférences INSA/Telecom ! CITI-INRIA Laboratory DynaMid team Associate Director (industry) ! Golo, IzPack, Maven Mojo, GlassFish, Devoxx, etc ! Oracle Java Magazine
Slide 3
Slide 3 text
Why are we here today?
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
Simple language Simple to use & hack
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
Function/closures calls Method dispatch (hard!) Arithmetic operations (…)
Slide 8
Slide 8 text
Runtime Bytecode JVM
Slide 9
Slide 9 text
“Do stuff” start stop “repeat, rinse” Theory…
Slide 10
Slide 10 text
“It doesn’t make (any) sense at all…” Practice…
Slide 11
Slide 11 text
package main ! import ( "fmt" "time" ) ! func Fib(n int64) int64 { if (n <= 2) { return n } else { return Fib(n - 1) + Fib(n - 2) } } ! func main() { for i := 0; i < 10; i++ { start := time.Now() Fib(44) fmt.Println(time.Since(start)) } } 4.738337584s 4.477160663s 4.408593662s 4.398906292s 4.533431874s 4.57529716s 4.519544937s 4.673349279s 4.945866449s 4.886524219s “Looks ok”
Slide 12
Slide 12 text
object FibApp extends App { ! def fib(n: Long): Long = if (n <= 2) n else fib(n - 1) + fib(n - 2) ! def howfast[T](block: => T): T = { val start = System.currentTimeMillis() val res = block println((System.currentTimeMillis() - start) + "ms") res } ! for (i <- 1 to 10) { howfast { fib(44) } } } 2787ms 3068ms 2857ms 2832ms 2797ms 2797ms 2839ms 2778ms 2896ms 2752ms “Looks ok”
Slide 13
Slide 13 text
“Do stuff” start stop “repeat, rinse” You could be “ok”: I/O, threads, complex call graphs, …
Slide 14
Slide 14 text
Static compilation Guess tricks at compilation time (C, C++, Go, etc)
Slide 15
Slide 15 text
Interpretation Just interpret from bytecode / AST (Python, Ruby, etc)
Slide 16
Slide 16 text
Dynamic compilation Guess tricks during execution (Java, Scala, JavaScript/V8, etc) ! ! Hybrid interpreter + JIT (HotSpot) “Just JIT” (V8, JRockit)
Slide 17
Slide 17 text
Dynamic compilation …is a (micro)benchmark killer.
Slide 18
Slide 18 text
Pitfalls in action
Slide 19
Slide 19 text
static double distance(double x1, double y1, double x2, double y2) { double dx = x2 - x1; double dy = y2 - y1; return Math.sqrt((dx * dx) + (dy * dy)); } How fast?
Slide 20
Slide 20 text
Demo (a not so naïve framework)
Slide 21
Slide 21 text
Dead-code elimination Constant folding Loop unrolling On-stack replacement Inlining Locks removal (…) https://wiki.openjdk.java.net/display/HotSpot/PerformanceTechniques
Slide 22
Slide 22 text
Caliper Google ! JUnitBenchmarks CarrotLabs ! JMH OpenJDK ✔ ⨉ ⨉
Slide 23
Slide 23 text
JMH to the rescue
Slide 24
Slide 24 text
“JMH is a Java harness for building, running, and analysing nano/micro/milli/ macro benchmarks written in Java and other languages targeting the JVM.”
Slide 25
Slide 25 text
Helps defeating unwanted optimisations.
Slide 26
Slide 26 text
Demo (JMH in anger)
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
A few examples Micro-benchmarks from Golo
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
JMH is great, but: ! • doubt • peer review • beware HotSpot • results don’t compose • read all Aleksey’s samples
Slide 37
Slide 37 text
(the end)
Slide 38
Slide 38 text
Julien Ponge @jponge ! https://julien.ponge.org/ http://dynamid.citi-lab.fr/ http://golo-lang.org/