Continuous Performance Regression
Testing With JfrUnit
Gunnar Morling
Software Engineer, Red Hat
@gunnarmorling
Slide 2
Slide 2 text
#JfrUnit @gunnarmorling
Today’s Objective: Learn about...
Implementing performance regression tests with
JDK Flight Recorder and JfrUnit
Analyzing performance regressions in JDK Mission
Control
Emitting JFR events from 3rd party libraries
1
2
3
Slide 3
Slide 3 text
#JfrUnit @gunnarmorling
Gunnar Morling
● Open source software engineer at Red Hat
○ Debezium
○ Quarkus, Hibernate
● Spec Lead for Bean Validation 2.0
● Java Champion
● @gunnarmorling
Slide 4
Slide 4 text
#JfrUnit @gunnarmorling
Challenges of Performance Tests
● Tests based on throughput or latency are dependent on
specifics of the environment
○ Require production-like set-up
○ Impacted by unrelated concurrent load (e.g. on CI server)
Slide 5
Slide 5 text
#JfrUnit @gunnarmorling
● Assert proxy metrics like memory
allocation or IO
● Failures may indicate a
performance regression
● Plain unit tests
● Analyse in JDK Mission Control
JfrUnit — Performance Regression Tests
“Unit Testing, for Performance”
#JfrUnit @gunnarmorling
JEP 349
● Processing JFR events in “real-time”
● Use Cases
○ Insight into live data
○ Export to monitoring services
○ Predictive analytics
○ JfrUnit 😎
JFR Event Streaming
Slide 15
Slide 15 text
#JfrUnit @gunnarmorling
API
JFR Event Streaming
Slide 16
Slide 16 text
#JfrUnit @gunnarmorling
● Control, visualize, and
analyze JFR recordings
● And more:
○ JMX console
○ heap dump analyzer
○ etc.
OpenJDK Mission Control
#JfrUnit @gunnarmorling
● Emit JFR events from
existing 3rd party code
● Attaches to the JVM
and instruments classes
● Configured via XML
or JMC Agent Plug-in
JMC Agent
#JfrUnit @gunnarmorling
● An exploration for querying JFR using SQL
○ Pull-based (e.g. for automated analyses)
○ Streaming queries (e.g. for predictive
analytics)
● Work-in-progress at
https://github.com/moditect/jfr-analytics
JFR Analytics
#JfrUnit @gunnarmorling
● Robust tests with very fast feedback cycle
○ Independent from environment
○ Proactive identification of issues
● Metrics need solid understanding; e.g. increased allocation may
or may not be a problem
● Cannot identify all issues, e.g. locks showing up only under load
● Won’t help with issues elsewhere, e.g. a bad query execution plan
JfrUnit — Discussion and Limitations
Slide 24
Slide 24 text
#JfrUnit @gunnarmorling
● Asserting proxy metrics allows for reliable performance
regression tests
● JfrUnit: a tool in the performance testing box
○ Complement - no replacement - for other tests
● JMC Agent: produce events from code not
under your control
● Move beyond Java 1.8
Take Aways
Slide 25
Slide 25 text
#JfrUnit @gunnarmorling
https://horreum.hyperfoil.io/
Outlook — Keeping Track of Historic Values
Slide 26
Slide 26 text
#JfrUnit @gunnarmorling
● JfrUnit
https://github.com/moditect/jfrunit
● Introduction to JfrUnit
https://www.morling.dev/blog/towards-continuous-performance-regression
-testing/
● JVM performance monitoring with JMC Agent
https://developers.redhat.com/articles/2021/11/16/jvm-performance-monito
ring-jmc-agent
● Profiling Java inside containers with ContainerJFR
https://developers.redhat.com/devnation/tech-talks/containerjfr
Resources