KCD Austria: Modern Java CI/CD observability with Gradle/Quarkus

Well-functioning Continuous Integration, Delivery, and Deployment pipelines are instrumental to the stability of cloud-native applications, including disaster recovery when things go wrong. Hence CI/CD observability is instrumental and might be as important as one for production environments. Many tools provide integrations with OTel, and initiatives like OTEP-0223 target standardizing the reporting formats across the ecosystem.

We will focus on a modern Java stack for cloud-native applications: Quarkus, Gradle, Java Flight Recorder (JFR), recent Java versions, and GitHub Actions.

P.S: If you are interested in the subject, join the #cicd-o11y channel on the CNCF Slack!

Transcript

1. KCD Austria 2024 PRESENTER Modern Java app CI/CD observability with OTel, Quarkus and Gradle #otel-cicd on the CNCF Slack Oleg Nenashev Gradle Inc.
2. Oleg Nenashev @oleg_nenashev
3. CNCF Slack, #otel-cicd SIG speakerdeck.com/onenashev
4. > whoami @oleg_nenashev oleg-nenashev Dr. Nenashev / Mr. Jenkins Community builder Developer Tools Hacker #RussiansAgainstPutin #StandWithUkraine 1.x
5. My first observability project, 2009 Oil flow meter and monitoring system Now: tundrasolutions.ca/products/neftemer
6. * parody content
8. 8 logz.io/blog/continuous-observability-cicd-pipelines/ changelog.com/shipit/27
9. OTel in Jenkins • Traces can go through different tools, not just a single one • Jenkins + Maven • Maven + Jenkins Source: plugins.jenkins.io/opentelemetry/
10. 10 Source: plugins.jenkins.io/opentelemetry/ Source reddit.com/r/Program merHumor/comment s/7zfgwg/frontend_vs _backend/ Analytics / Processing Data Ingestion
11. CI/CD Observability Data Processing 1
12. 12 CI/CD Observability. Data Ingestion
13. 13 And Java?
15. 15 Java All the Way Down Your Pipeline(s) Your Build Tool Your Framework Your JVM Your System Test Framework & JVM Java the stack up all the way
16. All things need 011y 16
17. Demo Time! 17 github.com/oleg-nenashev/opentelemetry-gradle-quarkus-demo BEWARE: I decided to do a GitHub Demo
18. 18 Java All the Way Down Your Pipeline(s) Your Build Tool Your Framework Your JVM Your System Test Framework & JVM Java the stack up all the way
19. 19 See also: https://www.lucavall.in/blog/ci-cd-ob servability-on-github-actions-and-th e-role-of-opentelemetry GitHub Actions
20. Sounds Promising? 20
22. In the demo • honeycombio/gha-buildevents • krzko/setup-telemetry • krzko/export-job-telemetry • inception-health/otel-export-trace-action 22 Demo: github.com/oleg-nenashev/opentelemetry-gradle-quarkus-demo
23. 23 https://twitter.com/oleg_nenashev/status/1741858998624665653
24. 24 Java All the Way Down Your Pipeline(s) Your Build Tool Your Framework Your JVM Your System Test Framework & JVM Java the stack up all the way
26. 26 gradle --scan Gradle Profiler Java Tools and IDEs Gradle Observability
27. 27 OpenTelemetry plugin for Gradle • Injecting traces and spans to the Gradle execution • Helps to find the slowest parts of your build with the Gradle OpenTelemetry build plugin. • Created by Craig Atkinson • craigatk/opentelemetry-gradle-plugin
28. Gradle Build Scan & Develocity
29. 29 Gradle tasks tracing github.com/craigatk/ opentelemetry-gradle-plugin
30. Tests traces too! github.com/craigatk/opentelemetry-gradle-plugin
31. 31 Java All the Way Down Your Pipeline(s) Your Build Tool Your Framework Your JVM Your System Test Framework & JVM Java the stack up all the way
32. Source: https://dev.to/rogervinas/top-5-server-side-frameworks-for-kotlin-in-2022-quarkus-2bp
33. 33 quarkus.io/guides/opentelemetry
34. In the demo • Test Reporting from Quarkus • Profiles for developer and test environments • Manual: quarkusDev mode 34 Demo: github.com/oleg-nenashev/opentelemetry-gradle-quarkus-demo
35. 35 Java All the Way Down Your Pipeline(s) Your Build Tool Your Framework Your JVM Your System Test Framework & JVM Java the stack up all the way
36. And then the JVM? • OpenTelemetry Java Agent • JMX Java Flight Recorder • Automatic data ingestion for metrics, logs and (some spans) 36 Demo: github.com/oleg-nenashev/opentelemetry-gradle-quarkus-demo
37. Java Flight Recorder (Java 17+ for LTS) 37 OTel Java Agent
38. 38 Java All the Way Down Your Pipeline(s) Your Build Tool Your Framework Your JVM Your System Test Framework & JVM Java the stack up all the way
39. 39 Did we say “modern Java”? Localstack Cloud Native Architecture
40. Observability of Distributed Systems * the same rabbithole
41. 41 And it is not even CD… • Relationships between Pipelines are even more complex • Standardization is needed • CDEvents project in the Continuous Delivery Foundation • OTEP-0223 by Dotan Horowitz for adoption in OTel traces
42. Demo Time! 42 github.com/oleg-nenashev/opentelemetry-gradle-quarkus-demo BEWARE: I decided to do a GitHub Demo
43. 43 Source: plugins.jenkins.io/opentelemetry/ Source reddit.com/r/Program merHumor/comment s/7zfgwg/frontend_vs _backend/ Analytics / Processing Data Ingestion
44. Learnings 44 • Each tool has different OTel configuration • Env propagation is fun • Span/Traces propagation is partial • Good luck with maintaining it in universal way :( Source reddit.com/r/Program merHumor/comment s/7zfgwg/frontend_vs _backend/
47. 47 Adopt OTel and Open Standards in YOUR projects and libraries
48. 48 Protip: Combine Tools for different Info Levels
49. + • Quarkus is supported by Gradle Inc. We provide caching, build scans and analytics for the project • Public Instance: https://ge.quarkus.io/ gradle/develocity-oss-projects
50. 50 Protip: Combine Tools for different Info Levels Pipeline / Steps Build Tool and App specifics
51. Takeaways • Java is Observable • Multiple Extra layers, you need to observe all • Build Tools CAN be observable • You’re responsible for the glue • Integrated trace is a massive endeavour 51
52. References • oleg-nenashev/opentelemetry-gradle-quarkus-demo • github.com/craigatk/opentelemetry-gradle-plugin • opentelemetry.io/docs/zero-code/java/agent • quarkus.io/guides/opentelemetry • plugins.jenkins.io/opentelemetry 52
53. GitHub: oleg-nenashev Twitter: @oleg_nenashev CNCF Slack, #otel-cicd SIG Questions? CNCF Project Booths
54. v Thank you!