Continuous Performance Regression Testing With JfrUnit Gunnar Morling Software Engineer, Red Hat @gunnarmorling

#JfrUnit @gunnarmorling Today’s Objective: Learn about... Implementing performance regression tests with JDK Flight Recorder and JfrUnit Analyzing performance regressions in JDK Mission Control Emitting JFR events from 3rd party libraries 1 2 3

#JfrUnit @gunnarmorling Gunnar Morling ● Open source software engineer at Red Hat ○ Debezium ○ Quarkus, Hibernate ● Spec Lead for Bean Validation 2.0 ● Java Champion ● @gunnarmorling

#JfrUnit @gunnarmorling Challenges of Performance Tests ● Tests based on throughput or latency are dependent on specifics of the environment ○ Require production-like set-up ○ Impacted by unrelated concurrent load (e.g. on CI server)

#JfrUnit @gunnarmorling ● Assert proxy metrics like memory allocation or IO ● Failures may indicate a performance regression ● Plain unit tests ● Analyse in JDK Mission Control JfrUnit — Performance Regression Tests “Unit Testing, for Performance”

#JfrUnit @gunnarmorling Hello, JfrUnit!

#JfrUnit @gunnarmorling Hello, JfrUnit!

#JfrUnit @gunnarmorling Hello, JfrUnit!

#JfrUnit @gunnarmorling Hello, JfrUnit!

#JfrUnit @gunnarmorling Hello, JfrUnit!

#JfrUnit @gunnarmorling Tools © James Petts https://flic.kr/p/LgibJR (CC BY-SA 2.0)

#JfrUnit @gunnarmorling ● A JVM “troubleshooting, monitoring, and profiling framework” ● 150+ built-in event types; custom ones ● Open-source since Java 11 ● Low overhead, meant for “always-on” OpenJDK Flight Recorder

#JfrUnit @gunnarmorling https://bestsolution-at.github.io/jfr-doc/ JFR Event Types

#JfrUnit @gunnarmorling JEP 349 ● Processing JFR events in “real-time” ● Use Cases ○ Insight into live data ○ Export to monitoring services ○ Predictive analytics ○ JfrUnit 😎 JFR Event Streaming

#JfrUnit @gunnarmorling API JFR Event Streaming

#JfrUnit @gunnarmorling ● Control, visualize, and analyze JFR recordings ● And more: ○ JMX console ○ heap dump analyzer ○ etc. OpenJDK Mission Control

#JfrUnit @gunnarmorling Demo © Wall Boat https://flic.kr/p/Y6zkmX (Public Domain)

#JfrUnit @gunnarmorling JMC Agent © Fitz1187 https://flic.kr/p/6pGGe2 (CC BY-ND 2.0)

#JfrUnit @gunnarmorling ● Emit JFR events from existing 3rd party code ● Attaches to the JVM and instruments classes ● Configured via XML or JMC Agent Plug-in JMC Agent

#JfrUnit @gunnarmorling Bonus: JFR Analytics © Steve Jurvetson https://flic.kr/p/Vv7m (CC BY 2.0)

#JfrUnit @gunnarmorling ● An exploration for querying JFR using SQL ○ Pull-based (e.g. for automated analyses) ○ Streaming queries (e.g. for predictive analytics) ● Work-in-progress at https://github.com/moditect/jfr-analytics JFR Analytics

#JfrUnit @gunnarmorling Wrap-Up © Maria Eklind https://flic.kr/p/251HmVw (CC BY-SA 2.0)

#JfrUnit @gunnarmorling ● Robust tests with very fast feedback cycle ○ Independent from environment ○ Proactive identification of issues ● Metrics need solid understanding; e.g. increased allocation may or may not be a problem ● Cannot identify all issues, e.g. locks showing up only under load ● Won’t help with issues elsewhere, e.g. a bad query execution plan JfrUnit — Discussion and Limitations

#JfrUnit @gunnarmorling ● Asserting proxy metrics allows for reliable performance regression tests ● JfrUnit: a tool in the performance testing box ○ Complement - no replacement - for other tests ● JMC Agent: produce events from code not under your control ● Move beyond Java 1.8 Take Aways

#JfrUnit @gunnarmorling https://horreum.hyperfoil.io/ Outlook — Keeping Track of Historic Values

#JfrUnit @gunnarmorling ● JfrUnit https://github.com/moditect/jfrunit ● Introduction to JfrUnit https://www.morling.dev/blog/towards-continuous-performance-regression -testing/ ● JVM performance monitoring with JMC Agent https://developers.redhat.com/articles/2021/11/16/jvm-performance-monito ring-jmc-agent ● Profiling Java inside containers with ContainerJFR https://developers.redhat.com/devnation/tech-talks/containerjfr Resources

#JfrUnit @gunnarmorling Q&A gunnar@hibernate.org @gunnarmorling 📧

No content