Java: Containerized, Serverless, Cloud-native

Transcript

1. View Slide

2. View Slide

3. Timeline
   03
   

   View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. View Slide

9. Definitions
   09
   

   View Slide

10. Cloud-native
    Cloud native technologies empower organizations to build and run scalable
    applications in modern, dynamic environments such as public, private, and
    hybrid clouds. Containers, service meshes, micro-services, immutable
    infrastructure, and declarative APIs exemplify this approach.
    “
    10
    

    View Slide

11. Cloud-native
    These techniques enable loosely coupled systems that are resilient,
    manageable, and observable. Combined with robust automation, they allow
    engineers to make high-impact changes frequently and predictably with
    minimal toil.
    “
    11
    

    View Slide

12. Serverless
    I don't want to care about servers.
    “
    12
    

    View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. View Slide

22. View Slide

23. View Slide

24. Make JAR, not
    WAR
    24
    

    View Slide

25. Fat-less app
    25
    

    View Slide

26. Container-awareness
    Inside Linux containers, OpenJDK versions 8 and later can correctly detect
    the container-limited number of CPU cores and available RAM. For all
    currently supported OpenJDK versions this is turned on by default.
    “
    26
    

    View Slide

27. View Slide

28. View Slide

29. View Slide

30. Container support
    Container support is enabled by default since Java 10
    Some settings and parameters are back-ported to Java 8
    Can be disabled with -XX:-UseContainerSupport
    •
    •
    •
    30
    

    View Slide

31. Container support
    -XX:ActiveProcessorCount=count
    -XX:InitialRAMPercentage=mem
    -XX:MaxRAM=mem
    -XX:MaxRAMPercentage=pct
    -XX:MinRAMPercentage=pct
    •
    •
    •
    •
    •
    31
    

    View Slide

32. GC
    32
    

    View Slide

33. View Slide

34. Image size
    34
    

    View Slide

35. View Slide

36. View Slide

37. Tangle
    37
    

    View Slide

38. View Slide

39. View Slide

40. View Slide

41. View Slide

42. Log4shell
    42
    

    View Slide

43. Observability
    In software, observability is the ability to ask new questions of the health of
    your running services without deploying new instrumentation.
    “
    43
    

    View Slide

44. The three pillars
    Metrics
    Logs
    Traces
    •
    •
    •
    44
    

    View Slide

45. Prometheus
    45
    

    View Slide

46. Prometheus architecture
    46
    

    View Slide

47. Exporters
    47
    

    View Slide

48. JMX exporter
    48
    

    View Slide

49. JMX exporter
    https://github.com/prometheus/jmx_exporter
    49
    

    View Slide

50. JMX exporter
    java -javaagent:./jmx_prometheus_javaagent-0.16.1.jar=8080:config.
    01.
    50
    

    View Slide

51. Spring Actuator
    51
    

    View Slide

52. Akka
    52
    

    View Slide

53. Prometheus Client for Java
    import io.prometheus.client.Counter;
    
    class YourClass {
    
    static final Counter requests = Counter.build()
    
    .name("requests_total").help("Total requests.").register();
    
    void processRequest() {
    
    requests.inc();
    
    // Your code here.
    
    }
    
    }
    
    01.
    53
    

    View Slide

54. Traces
    54
    

    View Slide

55. Definition
    A trace is a data/execution path through the system, and can be thought
    of as a directed acyclic graph of spans.
    55
    

    View Slide

56. Trace/Span
    56
    

    View Slide

57. Jaeger
    57
    

    View Slide

58. Jaeger architecture
    58
    

    View Slide

59. Instrumentation
    59
    

    View Slide

60. Get tracer
    Tracer tracer = Configuration.fromEnv().getTracer();
    01.
    60
    

    View Slide

61. Access span
    Span span = tracer.scopeManager().activeSpan();
    if (span != null) {
    span.log("...");
    }
    01.
    02.
    03.
    04.
    61
    

    View Slide

62. Create span
    Span span = tracer.buildSpan("someWork").start();
    try (Scope scope = tracer.scopeManager().activate(span)) {
    // Do things.
    } catch(Exception ex) {
    Tags.ERROR.set(span, true);
    span.log(Map.of(Fields.EVENT, "error", Fields.ERROR_OBJECT,
    ex, Fields.MESSAGE, ex.getMessage()));
    } finally {
    span.finish();
    }
    01.
    02.
    03.
    04.
    05.
    06.
    07.
    08.
    09.
    10.
    62
    

    View Slide

63. Ignore parent span
    Span span = tracer.
    buildSpan("someWork").
    ignoreActiveSpan().
    start();
    01.
    02.
    03.
    04.
    63
    

    View Slide

64. JFR
    64
    

    View Slide

65. Java Flight Recorder
    It was first introduced in JRockit.
    Many features of JRockit including JFR were merged into Oracle HotSpot
    at version 8.
    Till version 11, JFR/JMC was considered a commercial feature ( -
    XX:+UnlockCommercialFeatures -XX:+FlightRecorder ).
    In 11, JFR became free, but JMC (Mission Control UI) was removed from
    JDK, but remained a separate utility.
    •
    •
    •
    •
    65
    

    View Slide

66. jcmd
    jcmd JFR.start duration=60s filename=recording.jfr
    jcmd JFR.start
    jcmd JFR.dump name=1 filename=recording.jfr
    jcmd JFR.stop
    01.
    02.
    03.
    04.
    66
    

    View Slide

67. jfr
    jfr print --events CPULoad,GarbageCollection recording.jfr
    jfr print --categories "GC,JVM,Java*" recording.jfr
    jfr summary recording.jfr
    jfr metadata recording.jfr
    01.
    02.
    03.
    04.
    67
    

    View Slide

68. Custom events
    import jdk.jfr.Event;
    public class RestCallEvent extends Event {
    public String path;
    public String key;
    public long dataSize;
    }
    01.
    02.
    03.
    04.
    05.
    06.
    07.
    68
    

    View Slide

69. Custom events
    event.begin();
    // do something
    event.key = key;
    event.dataSize = val.length();
    // do something
    event.end();
    event.commit();
    01.
    02.
    03.
    04.
    05.
    06.
    07.
    69
    

    View Slide

70. JFR Streaming (Java 14)
    try (var rs = new RecordingStream()) {
    rs.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(1));
    rs.enable("jdk.JavaMonitorEnter").withThreshold(Duration.ofMilli
    rs.onEvent("jdk.CPULoad", event -> {
    System.out.println(event.getFloat("machineTotal"));
    });
    rs.onEvent("jdk.JavaMonitorEnter", event -> {
    System.out.println(event.getClass("monitorClass"));
    });
    rs.start();
    01.
    02.
    03.
    04.
    05.
    06.
    07.
    08.
    09.
    10.
    70
    

    View Slide

71. JFR Streaming (Java 14)
    Configuration c = Configuration.getConfiguration("default");
    try (var rs = new RecordingStream(c)) {
    rs.onEvent("jdk.GarbageCollection", System.out::println);
    rs.onEvent("jdk.CPULoad", System.out::println);
    rs.onEvent("jdk.JVMInformation", System.out::println);
    rs.start();
    }
    }
    01.
    02.
    03.
    04.
    05.
    06.
    07.
    08.
    71
    

    View Slide

72. AWS CodeGuru
    72
    

    View Slide

73. AWS X-ray
    73
    

    View Slide

74. Serverless
    74
    

    View Slide

75. If it were 2005...
    Jetty/Tomcat/Ruby/PHP (on a server)
    MySQL, PostgreSQL, HSQL, Sqlite (on a server)
    File system (on a server)
    •
    •
    •
    75
    

    View Slide

76. If it were 2015...
    Jetty/Tomcat/Ruby/PHP (in a container on a server) or PaaS in the cloud
    MySQL, PostgreSQL, HSQL, Sqlite (on a server or in a container on a
    server) or DaaS in the cloud
    File system (in a volume on a server) or object storage in the cloud
    •
    •
    •
    76
    

    View Slide

77. In 2022...
    CDN + FaaS + LB in the cloud
    DaaS in the cloud
    Object storage in the cloud
    •
    •
    •
    77
    

    View Slide

78. FaaS JVM choices
    78
    

    View Slide

79. GCP
    79
    

    View Slide

80. GCP (512m)
    -XX:MaxRAM=512m
    -XX:MaxRAMPercentage=70
    01.
    02.
    80
    

    View Slide

81. GCP (512m)
    Copy
    MarkSweepCompact
    •
    •
    81
    

    View Slide

82. GCP (4096m)
    -XX:MaxRAM=4096m
    -XX:MaxRAMPercentage=70
    01.
    02.
    82
    

    View Slide

83. GCP (4096m)
    G1 Young
    G1 Old
    •
    •
    83
    

    View Slide

84. Azure
    84
    

    View Slide

85. Azure
    -XX:+TieredCompilation
    -XX:TieredStopAtLevel=1
    -Xverify:none
    -Djava.net.preferIPv4Stack=true
    01.
    02.
    03.
    04.
    85
    

    View Slide

86. GC
    PS Scavenge
    PS MarkSweep
    •
    •
    86
    

    View Slide

87. AWS (512)
    -XX:MaxHeapSize=445645k
    -XX:MaxMetaspaceSize=52429k
    -XX:ReservedCodeCacheSize=26214k
    -XX:+UseSerialGC
    01.
    02.
    03.
    04.
    87
    

    View Slide

88. AWS (4096)
    -XX:MaxHeapSize=3948544k
    -XX:MaxMetaspaceSize=163840k
    -XX:ReservedCodeCacheSize=81920k
    -XX:+UseSerialGC
    01.
    02.
    03.
    04.
    88
    

    View Slide

89. AWS
    -javaagent:/var/runtime/amzn-log4j-security-jdk11-0.1alpha.jar
    -Xshare:on
    -XX:SharedArchiveFile=/var/lang/lib/server/runtime.jsa
    -XX:-TieredCompilation
    -Djava.net.preferIPv4Stack=true
    01.
    02.
    03.
    04.
    05.
    89
    

    View Slide

90. CDS
    Class Data Sharing
    It contains 1300+ core library classes loaded by the bootstrap class
    loader
    It is stored in a format that can be loaded very quickly, compared to
    loading from a JAR file
    •
    •
    •
    90
    

    View Slide

91. Dynamic CDS
    -XX:ArchiveClassesAtExit=cds.jsa
    01.
    91
    

    View Slide

92. JVM FaaS on clouds
    AWS: Serial, only JIT C2, shared class data, memory set explicitly
    GCP: Serial or G1, auto-tuning memory pools
    Azure: Parallel, only JIT C1
    •
    •
    •
    92
    

    View Slide

93. JIT
    93
    

    View Slide

94. JVMCI
    94
    

    View Slide

95. Graal Compiler
    Graal Compiler = JIT Compiler written in Java
    Added in Java 10
    Removed in Java 17
    •
    •
    •
    95
    

    View Slide

96. Graal VM
    OpenJDK with Graal compiler
    Truffle framework
    Tooling for other languages (Python, Node.js, Ruby etc.)
    Native image + Substrate VM
    •
    •
    •
    •
    96
    

    View Slide

97. Modes
    JVM
    Native
    •
    •
    97
    

    View Slide

98. Native
    98
    

    View Slide

99. Nice and shiny?
    Requires extra tooling (platform image, C++ compiler)
    Compilation time could be quite long for larger code bases
    "Closed-world" approach requires extra configuration for handling
    reflection, dynamic proxies etc.
    Some runtime tooling is not available (no way to get a memory dump)
    •
    •
    •
    •
    99
    

    View Slide

100. Reflection
     native-image -H:ReflectionConfigurationFiles=r.json ...
     01.
     100
     

     View Slide

101. Reflection
     {
     {
     "name":
     "java.lang.String$CaseInsensitiveComparator",
     "queriedMethods": [
     { "name": "compare" }
     ]
     }
     ]
     01.
     02.
     03.
     04.
     05.
     06.
     07.
     08.
     09.
     101
     

     View Slide

102. Hmmm...
     Long compilation times
     Extra configuration or code changes are required
     Not all Java functionality is supported
     •
     •
     •
     102
     

     View Slide

103. Why bother?
     improved startup time (= faster scaling)
     reduced memory usage (= cheaper)
     reduced image size
     better performance (?)
     better security (?)
     ideal for serverless/ML workloads
     •
     •
     •
     •
     •
     •
     103
     

     View Slide

104. Community!
     Spring Native (native image support for Spring/Spring Boot)
     Quarkus (RedHat's baby, lots of integrations, community work, build-time
     code generation)
     Micronaut (no reflection, build-time code generation)
     Helidon (Oracle's baby, support for jlink)
     •
     •
     •
     •
     104
     

     View Slide

105. Quarkus
     105
     

     View Slide

106. Helidon
     106
     

     View Slide

107. Infra-as-code
     107
     

     View Slide

108. CDK
     108
     

     View Slide

109. Infra-as-Java
     Bucket bucket = Bucket.Builder
     .create(this, targetBucket).build();
     PolicyStatement statement1 = PolicyStatement.Builder.create()
     .effect(Effect.ALLOW)
     .actions(asList("s3:GetBucket", "s3:PutObject"))
     .resources(asList("arn:aws:s3:::" + bucket.getBucketName() + "/*
     .build();
     01.
     02.
     03.
     04.
     05.
     06.
     07.
     08.
     109
     

     View Slide

110. Infra-as-Java
     Vpc vpc = new Vpc(this, "VPC");
     AutoScalingGroup asg = AutoScalingGroup.Builder
     .create(this,"ASG")
     .vpc(vpc)
     .instanceType(InstanceType.of(BURSTABLE2, MICRO))
     .machineImage(new AmazonLinuxImage())
     .build();
     01.
     02.
     03.
     04.
     05.
     06.
     07.
     08.
     110
     

     View Slide

111. Infra-as-Java
     HealthCheck.Builder healthCheckBuilder =
     new HealthCheck.Builder();
     HealthCheck healthCheck = healthCheckBuilder.port(80).build();
     LoadBalancer lb = LoadBalancer.Builder.create(this,"LB")
     .vpc(vpc)
     .internetFacing(Boolean.TRUE)
     .healthCheck(healthCheck)
     .build();
     01.
     02.
     03.
     04.
     05.
     06.
     07.
     08.
     111
     

     View Slide

112. Conclusion
     JVM ecosystem has many options for different load types.
     It makes it useful for any cloud-native/serverless/containerized
     environment.
     Community is vibrant and responsive.
     •
     •
     •
     112
     

     View Slide

113. View Slide

114. Future
     114
     

     View Slide

115. Project Loom
     Thread.startVirtualThread(
     () -> {
     System.out.println("Hello World");
     }
     );
     01.
     02.
     03.
     04.
     05.
     115
     

     View Slide

116. Alibaba's Dragonwell
     https://dragonwell-jdk.io/
     116
     

     View Slide

117. Project Leyden
     The primary goal of this Project is to address the long-term pain points of
     Java's slow startup time, slow time to peak performance, and large footprint.
     “
     117
     

     View Slide

118. Thank you!
     118
     

     View Slide

119. Questions?
     119
     

     View Slide

120. View Slide