Slide 1

Slide 1 text

Continuous Performance Regression Testing With JfrUnit Gunnar Morling Software Engineer, Red Hat @gunnarmorling

Slide 2

Slide 2 text

#JfrUnit @gunnarmorling Today’s Objective: Learn about... Implementing performance regression tests with JDK Flight Recorder and JfrUnit Analyzing performance regressions in JDK Mission Control Emitting JFR events from 3rd party libraries 1 2 3

Slide 3

Slide 3 text

#JfrUnit @gunnarmorling Gunnar Morling ● Open source software engineer at Red Hat ○ Debezium ○ Quarkus, Hibernate ● Spec Lead for Bean Validation 2.0 ● Java Champion ● @gunnarmorling

Slide 4

Slide 4 text

#JfrUnit @gunnarmorling Challenges of Performance Tests ● Tests based on throughput or latency are dependent on specifics of the environment ○ Require production-like set-up ○ Impacted by unrelated concurrent load (e.g. on CI server)

Slide 5

Slide 5 text

#JfrUnit @gunnarmorling ● Assert proxy metrics like memory allocation or IO ● Failures may indicate a performance regression ● Plain unit tests ● Analyse in JDK Mission Control JfrUnit — Performance Regression Tests “Unit Testing, for Performance”

Slide 6

Slide 6 text

#JfrUnit @gunnarmorling Hello, JfrUnit!

Slide 7

Slide 7 text

#JfrUnit @gunnarmorling Hello, JfrUnit!

Slide 8

Slide 8 text

#JfrUnit @gunnarmorling Hello, JfrUnit!

Slide 9

Slide 9 text

#JfrUnit @gunnarmorling Hello, JfrUnit!

Slide 10

Slide 10 text

#JfrUnit @gunnarmorling Hello, JfrUnit!

Slide 11

Slide 11 text

#JfrUnit @gunnarmorling Tools © James Petts https://flic.kr/p/LgibJR (CC BY-SA 2.0)

Slide 12

Slide 12 text

#JfrUnit @gunnarmorling ● A JVM “troubleshooting, monitoring, and profiling framework” ● 150+ built-in event types; custom ones ● Open-source since Java 11 ● Low overhead, meant for “always-on” OpenJDK Flight Recorder

Slide 13

Slide 13 text

#JfrUnit @gunnarmorling https://bestsolution-at.github.io/jfr-doc/ JFR Event Types

Slide 14

Slide 14 text

#JfrUnit @gunnarmorling JEP 349 ● Processing JFR events in “real-time” ● Use Cases ○ Insight into live data ○ Export to monitoring services ○ Predictive analytics ○ JfrUnit 😎 JFR Event Streaming

Slide 15

Slide 15 text

#JfrUnit @gunnarmorling API JFR Event Streaming

Slide 16

Slide 16 text

#JfrUnit @gunnarmorling ● Control, visualize, and analyze JFR recordings ● And more: ○ JMX console ○ heap dump analyzer ○ etc. OpenJDK Mission Control

Slide 17

Slide 17 text

#JfrUnit @gunnarmorling Demo © Wall Boat https://flic.kr/p/Y6zkmX (Public Domain)

Slide 18

Slide 18 text

#JfrUnit @gunnarmorling JMC Agent © Fitz1187 https://flic.kr/p/6pGGe2 (CC BY-ND 2.0)

Slide 19

Slide 19 text

#JfrUnit @gunnarmorling ● Emit JFR events from existing 3rd party code ● Attaches to the JVM and instruments classes ● Configured via XML or JMC Agent Plug-in JMC Agent

Slide 20

Slide 20 text

#JfrUnit @gunnarmorling Bonus: JFR Analytics © Steve Jurvetson https://flic.kr/p/Vv7m (CC BY 2.0)

Slide 21

Slide 21 text

#JfrUnit @gunnarmorling ● An exploration for querying JFR using SQL ○ Pull-based (e.g. for automated analyses) ○ Streaming queries (e.g. for predictive analytics) ● Work-in-progress at https://github.com/moditect/jfr-analytics JFR Analytics

Slide 22

Slide 22 text

#JfrUnit @gunnarmorling Wrap-Up © Maria Eklind https://flic.kr/p/251HmVw (CC BY-SA 2.0)

Slide 23

Slide 23 text

#JfrUnit @gunnarmorling ● Robust tests with very fast feedback cycle ○ Independent from environment ○ Proactive identification of issues ● Metrics need solid understanding; e.g. increased allocation may or may not be a problem ● Cannot identify all issues, e.g. locks showing up only under load ● Won’t help with issues elsewhere, e.g. a bad query execution plan JfrUnit — Discussion and Limitations

Slide 24

Slide 24 text

#JfrUnit @gunnarmorling ● Asserting proxy metrics allows for reliable performance regression tests ● JfrUnit: a tool in the performance testing box ○ Complement - no replacement - for other tests ● JMC Agent: produce events from code not under your control ● Move beyond Java 1.8 Take Aways

Slide 25

Slide 25 text

#JfrUnit @gunnarmorling https://horreum.hyperfoil.io/ Outlook — Keeping Track of Historic Values

Slide 26

Slide 26 text

#JfrUnit @gunnarmorling ● JfrUnit https://github.com/moditect/jfrunit ● Introduction to JfrUnit https://www.morling.dev/blog/towards-continuous-performance-regression -testing/ ● JVM performance monitoring with JMC Agent https://developers.redhat.com/articles/2021/11/16/jvm-performance-monito ring-jmc-agent ● Profiling Java inside containers with ContainerJFR https://developers.redhat.com/devnation/tech-talks/containerjfr Resources

Slide 27

Slide 27 text

#JfrUnit @gunnarmorling Q&A [email protected] @gunnarmorling 📧

Slide 28

Slide 28 text

No content