JVM Performance Magic Tricks

HotSpot, the JVM we all know and love, is the brain in which our Java and Scala juices flow. At its core lies the JIT (“Just-In-Time”) compiler, whose sole purpose is to make your code run fast. Here are some of the more interesting optimizations performed by it.

Transcript

2. HotSpot • The brain in which our Java and Scala juices flow. • Its execution speed and efficiency is nearing that of native compiled code. • At its core: The JIT compiler.
3. So... The JIT compiler? • Information is gathered at runtime. ◦ Which paths in the code are traveled often? ◦ Which methods are called the most, or, where are the hot spots?
4. So... The JIT compiler? • Once enough information about a hot method is collected... ◦ The compiler kicks in. ◦ Compiles the bytecode into a lean and efficient native version of itself. ◦ May re-compile later due to over-optimism.
5. Some standard optimizations • Simple method inlining. • Dead code removal. • Native math ops instead of library calls. • Invariant hoisting.
6. Some are extra awesome!
7. Divide and conquer How many times have you used the following pattern? StringBuilder sb = new StringBuilder("Ingredients: "); for (int i = 0; i < ingredients.length; i++) { if (i > 0) { sb.append(", "); } sb.append(ingredients[i]); } return sb.toString();
8. Divide and conquer ...or perhaps this one? boolean nemoFound = false; for (int i = 0; i < fish.length; i++) { String curFish = fish[i]; if (!nemoFound) { if (curFish.equals("Nemo")) { System.out.println("Nemo! There you are!"); nemoFound = true; continue; } } if (nemoFound) { System.out.println("We already found Nemo!"); } else { System.out.println("We still haven't found Nemo :("); } }
9. Divide and conquer • Both loops do one thing for a while, • Then another thing from a certain point on. • The compiler can spot these patterns. ◦ Split the loops into cases. ◦ “Peel” several iterations.
10. Divide and conquer • The condition: if (i > 0) ◦ false once, ◦ true thereafter. ◦ Peel one iteration! StringBuilder sb = new StringBuilder("Ingredients: "); for (int i = 0; i < ingredients.length; i++) { if (i > 0) { sb.append(", "); } sb.append(ingredients[i]); } return sb.toString();
11. Divide and conquer ...will compile as if it were written like so: StringBuilder sb = new StringBuilder("Ingredients: "); if (ingredients.length > 0) { sb.append(ingredients[0]); for (int i = 1; i < ingredients.length; i++) { sb.append(", "); sb.append(ingredients[i]); } } return sb.toString(); First iteration All other iterations
12. Living on the edge • Null checks are bread-and-butter. • Sometimes null is a valid value: ◦ Missing values ◦ Error indication • Sometimes we check just to be on the safe side.
13. Living on the edge Some checks may be practically redundant. If your code behaves well, the assertion may never fail. public static String l33tify(String phrase) { if (phrase == null) { throw new IllegalArgumentException("Null bad!"); } return phrase.replace('e', '3'); }
14. Living on the edge • Code runs many, many times. • The assertion never fails. • The JIT compiler is optimistic. ...assumes the check is unnecessary!
15. Living on the edge The compiler may drop the check altogether, and compile it as if it were written like so: public static String l33tify(String phrase) { if (phrase == null) { throw new IllegalArgumentException("Null bad!"); } return phrase.replace('e', '3'); }
16. Living on the edge Wait... What if that happy-path assumption eventually proves to be wrong?
17. Living on the edge • The JVM is now executing native code. ◦ A null reference would not result in a fuzzy NullPointerException. ...but rather in a real, harsh memory access violation.
18. Living on the edge • The JVM intercepts the SIGSEGV (and recovers) • Follows-up with a de-optimization. ...Method is recompiled, this time with the null check in place.
19. Virtual insanity The JIT compiler has dynamic runtime data on which it can rely when making decisions.
20. Virtual insanity Method inlining: Step 1: Take invoked method. Step 2: Take invoker method. Step 3: Embed former in latter.
21. Virtual insanity Method inlining: ◦ Useful when trying to avoid costly invocations. ◦ Tricky when dealing with dynamic dispatch.
22. Virtual insanity public class Main { public static void perform(Song s) { s.sing(); } } public interface Song { public void sing(); }
23. Virtual insanity public class GangnamStyle implements Song { @Override public void sing() { println("Oppan gangnam style!"); } } public class Baby implements Song { @Override public void sing() { println("And I was like baby, baby, baby, oh"); } } public class BohemianRhapsody implements Song { @Override public void sing() { println("Thunderbolt and lightning, very very frightening me"); } }
24. Virtual insanity • perform might run millions of times. • Each time, sing is invoked. This is a co$tly dynamic dispatch!
25. Virtual insanity Inlining polymorphic calls is not so simple... ...in a static compiler.
26. Virtual insanity The JIT compiler is dynamic. Take advantage of runtime information!
27. Virtual insanity The JVM might decide, according to the statistics it gathered, that 95% of the invocations target an instance of GangnamStyle.
28. Virtual insanity The compiler can perform an optimistic optimization: Eliminate the virtual calls to sing. ...or most of them anyway.
29. Virtual insanity Optimized compiled code will behave like so: public static void perform(Song s) { if (s fastnativeinstanceof GangnamStyle) { println("Oppan gangnam style!"); } else { s.sing(); } }
30. Can I help? • The JIT compiler is built to optimize: ◦ Straightforward, simple code. ◦ Common patterns. ◦ No nonsense.
31. Can I help? The best way to help your compiler is to not try so hard to help it. Just write your code as you otherwise would!