Brought to you by Continuous Performance Regression Testing with JfrUnit Gunnar Morling Principal Software Engineer at

Today’s Objective: Learn About... ■ Implementing performance regression tests with JDK Flight Recorder and JfrUnit ■ Analyzing performance regressions in JDK Mission Control ■ Emitting JFR events from 3rd party libraries

Gunnar Morling Principal Software Engineer at Red Hat ■ Debezium ■ Quarkus ■ JfrUnit, ModiTect, kcctl ■ Spec Lead for Bean Validation 2.0 ■ Java Champion ■ @gunnarmorling

Challenges of Performance Tests ■ Tests based on throughput or latency are dependent on specifics of the environment ● Require production-like set-up ● Impacted by unrelated concurrent load (e.g. on CI server)

JfrUnit — Assert Performance Impacting Metrics ■ Assert proxy metrics like memory allocation or IO ● Based on JDK Flight Recorder events ● Failures may indicate a performance regression ■ Plain unit tests ■ Analyse in JDK Mission Control

Hello, JfrUnit!

Tools

OpenJDK Flight Recorder ■ “Troubleshooting, monitoring, and profiling framework embedded into the JVM” ■ 150+ built-in event types; custom ones ■ Open-source since Java 11 (backported to 8) ■ Low overhead, meant for “always-on” ■ Accessible via recording files and streaming

JFR Event Types https://bestsolution-at.github.io/jfr-doc/

OpenJDK Mission Control ■ Control, visualize, and analyze JFR recordings ■ And more: ● JMX console ● heap dump analyzer ● etc.

Demo Time!

JfrUnit — Discussion and Limitations ■ Robust tests with very fast feedback cycle ● Independent from environment ● Proactive identification of issues ■ Metrics need solid understanding; e.g. increased allocation may or may not be a problem ■ Cannot identify all issues, e.g. locks showing up only under load ■ Won’t help with issues elsewhere, e.g. a bad query execution plan

Wrap-Up

Take Aways ■ JfrUnit: a tool in the performance testing box ● Complement - no replacement - for other tests ■ JMC Agent: produce events from code not under your control ■ Move beyond Java 1.8 🚀

Outlook — Keeping Track of Historic Values https://horreum.hyperfoil.io/

Resources ■ JfrUnit https://github.com/moditect/jfrunit ■ Introduction to JfrUnit https://www.morling.dev/blog/towards-continuous-performance-regression-t esting/ ■ Introduction to JMC Agent https://developers.redhat.com/blog/2020/10/29/collect-jdk-flight-recorder-ev ents-at-runtime-with-jmc-agent/ ■ Profiling Java inside containers with ContainerJFR https://developers.redhat.com/devnation/tech-talks/containerjfr

Brought to you by Gunnar Morling [email protected] @gunnarmorling