Unleash the Power of Open Source Java Profilers

Profilers help to analyze performance bottlenecks of your application - if you know which to use and how to work with them.

There are many open-source profilers, like async-profiler or JMC. This talk will give you insights into these tools, focusing on:

- Understanding the basic concepts of profiling like flame graphs, ...
- Usage of async-profiler and JMC
- Advantages and disadvantages of the different tools

Slides for my JavaZone 2023 talk, similar to the slides for other conferences. You can find a recording at vimeo and more information on my profiling talks page.

Transcript

1. Unleash the Power of Open Source Java Profilers Johannes Bechberger The best JDKTM Krzysztof Ślusarski
3. Client Admin Server Question? Set Question JSON
4. Premature Optimization We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. — Donald Knuth “ Source: Jacob Appelbaum, Wikipedia
5. So the event came... ... and the server was overwhelmed
6. Premature Optimization We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. — Donald Knuth “ Source: Jacob Appelbaum, Wikipedia
7. Profilers to the rescue
9. Source: ARD Degeto/WistingSesong/Viaplay
10. main serverLoop handleQuestionRequest currentQuestion isQuestionEnabled fun main() { ... serverLoop() } fun serverLoop() { while (true) { req = ... if (req.isQuestionRequest) handleQuestionRequest(req) ... } } fun handleQuestionRequest(req) { if (isQuestionEnabled()) { emit(currentQuestion().json) } else { emit({}) } } parseJSON parseJSON
12. main serverLoop handleQuestionRequest fun main() { ... serverLoop() } fun serverLoop() { while (true) { req = ... if (req.isQuestionRequest) handleQuestionRequest(req) ... } } fun handleQuestionRequest(req) { if (isQuestionEnabled()) { emit(currentQuestion().json) } else { emit({}) } }
13. Debugging Testing Profiling Toolbox
16. Who of you used a profiler?
17. yes no JetBrains 2022 survey
19. What is profiling? A report on the amounts of time spent in each routine of a program, used to find and tune away the hot spots in it. — The Jargon File “
20. Different Profilers
21. Instrumenting Profilers Inserting instructions into the code automatically fun serverLoop() { logEntry(“serverLoop”) while (true) { req = ... if (req.isQuestionRequest) handleQuestionRequest(req) ... } logExit(“serverLoop”) } Profiled Code ≠ Production Code long time = time(); println(“serverLoop: “ + (time() – start)) on class load
22. Sampling Profilers fun serverLoop() { while (true) { req = ... if (req.isQuestionRequest) handleQuestionRequest(req) ... } } Profiled Code = Production Code
23. Profilers aren’t rocket science
24. So let’s write our own https://mostlynerdless.de
25. Main agentmain(args) premain(args) Profiler sample sleep Store start store
26. Flamegraphs would be great main serverLoop handleQuestionRequest currentQuestion isQuestionEnabled parseJSON parseJSON
27. Turning traces into flamegraphs parseJSON currentQuestion handleQR serverLoop main private static class Node { final String method; final Map<String, Node> children; long samples = 0; void addTrace(List<String> trace) }
28. Turning traces into flamegraphs parseJSON currentQuestion handleQR serverLoop main main serverLoop handleQR currentQuestion parseJSON 1 1 1 1 1
29. Flamegraphs would be great main serverLoop handleQuestionRequest currentQuestion isQuestionEnabled parseJSON parseJSON
30. Turning traces into flamegraphs parseJSON currentQuestion handleQR serverLoop main main serverLoop handleQR currentQuestion parseJSON 2 2 2 2 2
31. Flamegraphs would be great main serverLoop handleQuestionRequest currentQuestion isQuestionEnabled parseJSON parseJSON
32. Turning traces into flamegraphs currentQuestion handleQR serverLoop main main serverLoop handleQR currentQuestion parseJSON 3 3 3 3 2
33. Flamegraphs would be great main serverLoop handleQuestionRequest currentQuestion isQuestionEnabled parseJSON parseJSON
34. Turning traces into flamegraphs isQuestionEnabled handleQR serverLoop main main serverLoop handleQR currentQuestion parseJSON 4 4 4 3 2 parseJSON isQuestionEnabled parseJSON 1 1
35. Turning traces into flamegraphs main serverLoop handleQR currentQuestion parseJSON 4 4 4 3 2 isQuestionEnabled parseJSON 1 1 main serverLoop handleQuestionRequest currentQuestion isQEnabled parseJSON parseJSON
36. With the power of d3-flame-graph
37. Reality
38. Java/JDK Mission Control Application Performance Monitors, ... Sampling Profiler External VisualVM Netbeans async-profiler Sync Async Forte Analyzer Built-In Java/JDK Flight Recorder 2016 1991 ASGCT 2002 2012 2018 Open Source 2010
39. Don’t trust them
41. AsyncGetCallTrace
42. Tests in the OpenJDK?
44. That’s my work
45. Take profiles with a grain of salt
46. Profiling Loop Model Hypothesis Evaluation
47. Obtaining a profile
48. Async-Profiler java \ -agentpath:libasyncProfiler.so=start,\ event=cpu,\ file=flame.html,flamegraph \ arguments java –jar ap-loader.jar profiler PID ... Download from GitHub file=file.jfr,jfr,jfrsync,alloc \
49. Some of its features • Many events can trigger sampling • locks • perf-events like cache-misses • methods (via instrumentation) • Embeddable • via ap-loader • Hackable Async-profiler - manual by use cases by Krzysztof Ślusarski
50. JDK Flight Recorder (JFR) java \ -XX:+UnlockDiagnosticVMOptions \ -XX:+DebugNonSafepoints \ -XX:+FlightRecorder \ -XX:StartFlightRecording=filename=file.jfr \ arguments jcmd PID JFR.start jcmd PID JFR.dump filename=file.jfr jcmd PID JFR.stop Already included in your JDK 8+
51. https://sapmachine.io/jfrevents
52. Fibonacci Server maas.com/fib/{n}
53. Custom JFR Events class SessionEvent extends jdk.jfr.Event { int sessionId; int n; .... } var event = new SessionEvent(sessionId, n); event.begin(); ctx.result("fibonacci: " + fib(n)); event.commit();
56. Inspecting a profile
57. JDK Mission Control
62. JFR Plugin for IntelliJ Still a prototype
68. Impact on performance in some benchmark, 48s, 8 cores, 4% standard deviation Overhead JFR (reduced setting) 0 - 5%, typically < 2% JFR (profiling setting) 1 - 8%, typically < 5% async-profiler 3 - 6%, typically < 2% async-profiler with jfrsync 3 - 10%
69. Continuous Profiling
70. asprof -e wall \ --loop 1m -f profile-%t.jfr <PID>
71. Pyroscope
74. Parca
76. ... and many more
77. Debugging Testing Profiling Toolbox
78. @parttimen3rd on Twitter parttimenerd on GitHub mostlynerdless.de @SweetSapMachine sapmachine.io