Java: Containerized, Serverless, Cloud-native

Transcript

3. Timeline 03
9. Definitions 09
10. Cloud-native Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, micro-services, immutable infrastructure, and declarative APIs exemplify this approach. “ 10
11. Cloud-native These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. “ 11
12. Serverless I don't want to care about servers. “ 12
24. Make JAR, not WAR 24
25. Fat-less app 25
26. Container-awareness Inside Linux containers, OpenJDK versions 8 and later can correctly detect the container-limited number of CPU cores and available RAM. For all currently supported OpenJDK versions this is turned on by default. “ 26
30. Container support Container support is enabled by default since Java 10 Some settings and parameters are back-ported to Java 8 Can be disabled with -XX:-UseContainerSupport • • • 30
31. Container support -XX:ActiveProcessorCount=count -XX:InitialRAMPercentage=mem -XX:MaxRAM=mem -XX:MaxRAMPercentage=pct -XX:MinRAMPercentage=pct • • • • • 31
32. GC 32
33. 1791 33
34. Image size 34
37. Tangle 37
42. Log4shell 42
43. Observability In software, observability is the ability to ask new questions of the health of your running services without deploying new instrumentation. “ 43
44. The three pillars Metrics Logs Traces • • • 44
45. Prometheus 45
46. Prometheus architecture 46
47. Exporters 47
48. JMX exporter 48
49. JMX exporter https://github.com/prometheus/jmx_exporter 49
50. JMX exporter java -javaagent:./jmx_prometheus_javaagent-0.16.1.jar=8080:config. 01. 50
51. Spring Actuator 51
52. Akka 52
53. Prometheus Client for Java import io.prometheus.client.Counter; class YourClass { static final Counter requests = Counter.build() .name("requests_total").help("Total requests.").register(); void processRequest() { requests.inc(); // Your code here. } } 01. 53
54. Traces 54
55. Definition A trace is a data/execution path through the system, and can be thought of as a directed acyclic graph of spans. 55
56. Trace/Span 56
57. Jaeger 57
58. Jaeger architecture 58
59. Instrumentation 59
60. Get tracer Tracer tracer = Configuration.fromEnv().getTracer(); 01. 60
61. Access span Span span = tracer.scopeManager().activeSpan(); if (span != null) { span.log("..."); } 01. 02. 03. 04. 61
62. Create span Span span = tracer.buildSpan("someWork").start(); try (Scope scope = tracer.scopeManager().activate(span)) { // Do things. } catch(Exception ex) { Tags.ERROR.set(span, true); span.log(Map.of(Fields.EVENT, "error", Fields.ERROR_OBJECT, ex, Fields.MESSAGE, ex.getMessage())); } finally { span.finish(); } 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 62
63. Ignore parent span Span span = tracer. buildSpan("someWork"). ignoreActiveSpan(). start(); 01. 02. 03. 04. 63
64. JFR 64
65. Java Flight Recorder It was first introduced in JRockit. Many features of JRockit including JFR were merged into Oracle HotSpot at version 8. Till version 11, JFR/JMC was considered a commercial feature ( - XX:+UnlockCommercialFeatures -XX:+FlightRecorder ). In 11, JFR became free, but JMC (Mission Control UI) was removed from JDK, but remained a separate utility. • • • • 65
66. jcmd jcmd <PID> JFR.start duration=60s filename=recording.jfr jcmd <PID> JFR.start jcmd <PID> JFR.dump name=1 filename=recording.jfr jcmd <PID> JFR.stop 01. 02. 03. 04. 66
67. jfr jfr print --events CPULoad,GarbageCollection recording.jfr jfr print --categories "GC,JVM,Java*" recording.jfr jfr summary recording.jfr jfr metadata recording.jfr 01. 02. 03. 04. 67
68. Custom events import jdk.jfr.Event; public class RestCallEvent extends Event { public String path; public String key; public long dataSize; } 01. 02. 03. 04. 05. 06. 07. 68
69. Custom events event.begin(); // do something event.key = key; event.dataSize = val.length(); // do something event.end(); event.commit(); 01. 02. 03. 04. 05. 06. 07. 69
70. JFR Streaming (Java 14) try (var rs = new RecordingStream()) { rs.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(1)); rs.enable("jdk.JavaMonitorEnter").withThreshold(Duration.ofMilli rs.onEvent("jdk.CPULoad", event -> { System.out.println(event.getFloat("machineTotal")); }); rs.onEvent("jdk.JavaMonitorEnter", event -> { System.out.println(event.getClass("monitorClass")); }); rs.start(); 01. 02. 03. 04. 05. 06. 07. 08. 09. 10. 70
71. JFR Streaming (Java 14) Configuration c = Configuration.getConfiguration("default"); try (var rs = new RecordingStream(c)) { rs.onEvent("jdk.GarbageCollection", System.out::println); rs.onEvent("jdk.CPULoad", System.out::println); rs.onEvent("jdk.JVMInformation", System.out::println); rs.start(); } } 01. 02. 03. 04. 05. 06. 07. 08. 71
72. AWS CodeGuru 72
73. AWS X-ray 73
74. Serverless 74
75. If it were 2005... Jetty/Tomcat/Ruby/PHP (on a server) MySQL, PostgreSQL, HSQL, Sqlite (on a server) File system (on a server) • • • 75
76. If it were 2015... Jetty/Tomcat/Ruby/PHP (in a container on a server) or PaaS in the cloud MySQL, PostgreSQL, HSQL, Sqlite (on a server or in a container on a server) or DaaS in the cloud File system (in a volume on a server) or object storage in the cloud • • • 76
77. In 2022... CDN + FaaS + LB in the cloud DaaS in the cloud Object storage in the cloud • • • 77
78. FaaS JVM choices 78
79. GCP 79
80. GCP (512m) -XX:MaxRAM=512m -XX:MaxRAMPercentage=70 01. 02. 80
81. GCP (512m) Copy MarkSweepCompact • • 81
82. GCP (4096m) -XX:MaxRAM=4096m -XX:MaxRAMPercentage=70 01. 02. 82
83. GCP (4096m) G1 Young G1 Old • • 83
84. Azure 84
85. Azure -XX:+TieredCompilation -XX:TieredStopAtLevel=1 -Xverify:none -Djava.net.preferIPv4Stack=true 01. 02. 03. 04. 85
86. GC PS Scavenge PS MarkSweep • • 86
87. AWS (512) -XX:MaxHeapSize=445645k -XX:MaxMetaspaceSize=52429k -XX:ReservedCodeCacheSize=26214k -XX:+UseSerialGC 01. 02. 03. 04. 87
88. AWS (4096) -XX:MaxHeapSize=3948544k -XX:MaxMetaspaceSize=163840k -XX:ReservedCodeCacheSize=81920k -XX:+UseSerialGC 01. 02. 03. 04. 88
89. AWS -javaagent:/var/runtime/amzn-log4j-security-jdk11-0.1alpha.jar -Xshare:on -XX:SharedArchiveFile=/var/lang/lib/server/runtime.jsa -XX:-TieredCompilation -Djava.net.preferIPv4Stack=true 01. 02. 03. 04. 05. 89
90. CDS Class Data Sharing It contains 1300+ core library classes loaded by the bootstrap class loader It is stored in a format that can be loaded very quickly, compared to loading from a JAR file • • • 90
91. Dynamic CDS -XX:ArchiveClassesAtExit=cds.jsa 01. 91
92. JVM FaaS on clouds AWS: Serial, only JIT C2, shared class data, memory set explicitly GCP: Serial or G1, auto-tuning memory pools Azure: Parallel, only JIT C1 • • • 92
93. JIT 93
94. JVMCI 94
95. Graal Compiler Graal Compiler = JIT Compiler written in Java Added in Java 10 Removed in Java 17 • • • 95
96. Graal VM OpenJDK with Graal compiler Truffle framework Tooling for other languages (Python, Node.js, Ruby etc.) Native image + Substrate VM • • • • 96
97. Modes JVM Native • • 97
98. Native 98
99. Nice and shiny? Requires extra tooling (platform image, C++ compiler) Compilation time could be quite long for larger code bases "Closed-world" approach requires extra configuration for handling reflection, dynamic proxies etc. Some runtime tooling is not available (no way to get a memory dump) • • • • 99
100. Reflection native-image -H:ReflectionConfigurationFiles=r.json ... 01. 100
101. Reflection { { "name": "java.lang.String$CaseInsensitiveComparator", "queriedMethods": [ { "name": "compare" } ] } ] 01. 02. 03. 04. 05. 06. 07. 08. 09. 101
102. Hmmm... Long compilation times Extra configuration or code changes are required Not all Java functionality is supported • • • 102
103. Why bother? improved startup time (= faster scaling) reduced memory usage (= cheaper) reduced image size better performance (?) better security (?) ideal for serverless/ML workloads • • • • • • 103
104. Community! Spring Native (native image support for Spring/Spring Boot) Quarkus (RedHat's baby, lots of integrations, community work, build-time code generation) Micronaut (no reflection, build-time code generation) Helidon (Oracle's baby, support for jlink) • • • • 104
105. Quarkus 105
106. Helidon 106
107. Infra-as-code 107
108. CDK 108
109. Infra-as-Java Bucket bucket = Bucket.Builder .create(this, targetBucket).build(); PolicyStatement statement1 = PolicyStatement.Builder.create() .effect(Effect.ALLOW) .actions(asList("s3:GetBucket", "s3:PutObject")) .resources(asList("arn:aws:s3:::" + bucket.getBucketName() + "/* .build(); 01. 02. 03. 04. 05. 06. 07. 08. 109
110. Infra-as-Java Vpc vpc = new Vpc(this, "VPC"); AutoScalingGroup asg = AutoScalingGroup.Builder .create(this,"ASG") .vpc(vpc) .instanceType(InstanceType.of(BURSTABLE2, MICRO)) .machineImage(new AmazonLinuxImage()) .build(); 01. 02. 03. 04. 05. 06. 07. 08. 110
111. Infra-as-Java HealthCheck.Builder healthCheckBuilder = new HealthCheck.Builder(); HealthCheck healthCheck = healthCheckBuilder.port(80).build(); LoadBalancer lb = LoadBalancer.Builder.create(this,"LB") .vpc(vpc) .internetFacing(Boolean.TRUE) .healthCheck(healthCheck) .build(); 01. 02. 03. 04. 05. 06. 07. 08. 111
112. Conclusion JVM ecosystem has many options for different load types. It makes it useful for any cloud-native/serverless/containerized environment. Community is vibrant and responsive. • • • 112
114. Future 114
115. Project Loom Thread.startVirtualThread( () -> { System.out.println("Hello World"); } ); 01. 02. 03. 04. 05. 115
116. Alibaba's Dragonwell https://dragonwell-jdk.io/ 116
117. Project Leyden The primary goal of this Project is to address the long-term pain points of Java's slow startup time, slow time to peak performance, and large footprint. “ 117
118. Thank you! 118
119. Questions? 119