Java Performance on Kubernetes

Many organisations are migrating their existing Java applications or developing new ones with Kubernetes as the target platform. However, sooner or later they find out there is a lot of fine tuning on production.

Many teams are confronted with some issues that keep their Java applications from running smoothly on Kubernetes like high memory consumption, high latency and slow startup times. In this session I would like to talk about some of these issues and how to tackle them based on real project experience.

Furthermore I would like to talk about the advantages of choosing Java Frameworks that are better suited for the cloud. I will show you a small demo to highlight the difference among three very popular Java frameworks: Spring Boot, Quarkus and Micronaut.

Transcript

1. Java Performance on Kubernetes The small print.
2. Who am I? 2 Carlos Barragan Chief Technologist @ Novatec 👨💻 Software developer 🏠 Architect 🗣 Speaker @barraganc carlosbarragan-hdz
3. This talk is about useful information I have found over the years & experience in several projects. 3
4. Do you run Java Applications on K8s? 4
5. Do you run Java Applications with 5 2 CPUs or less
6. Number of Cores – New Relic Survey 2022 6 https://newrelic.com/resources/report/2022-state-of-java-ecosystem 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 1 2 3 4 5 6 8 Number of Cores
7. Do you run Java Applications with 7 512 MB Memory Limit or less
8. Do you run Java Applications with 8 1 GB Memory Limit or less
9. Heap Memory– New Relic Survey 2022 9 https://newrelic.com/resources/report/2022-state-of-java-ecosystem 512 MB 49% 1 GB 17% 2 GB 16% 3 GB 14% 4 GB 4% MEMORY
10. Which GC and how much max heap is your app using with 1 GB and 1 CPU? 10
11. Hint: the JVM only considers SerialGC or Parallel GC by default. 11 Which GC and how much max heap is your app using with 1 GB and 1 CPU?
12. 12 Which GC and how much max heap is your app using with 1 GB and 1 CPU? 256 MB Serial GC Max Heap
13. 13 Which GC and how much max heap is your app using with 1 GB and 2 CPU? 256 MB Serial GC Max Heap
14. 14 https://github.com/openjdk/jdk/blob/master/src/hotspot/share/gc/shared/gcConfig.cpp
15. 15 https://github.com/openjdk/jdk/blob/master/src/hotspot/share/runtime/os.cpp At least 2 CPUs At least ~1792 MB RAM
16. § Longer stop-the-world (STW) collections pauses (than G1) § In general, less efficient than G1. What happens with SerialGC? 16
17. 17 https://learn.microsoft.com/en-us/azure/developer/java/containers/overview
18. 18 https://learn.microsoft.com/en-us/azure/developer/java/containers/overview 1GB = 256MB heap
19. Don’t rely on the JVM to set your app’s heap size and GC! 19
20. § https://bugs.openjdk.org/browse/JDK-8230305 (Fixed in JDK 11) - cgroups V2 memory limits ignored - Fix came out in July 2022! § https://bugs.java.com/bugdatabase/view_bug.do?bug_id=82815 71 (JDK 11,17,18) - don’t use cpu.shares by default - If neither "CPU requests" or "CPU limits" are set, Kubernetes runs the container with no (upper) CPU limit, and minimal CPU Shares. Before this CSR, the JVM will limit itself to a single CPU. - Fix came out in February 2022 JDK Bugs 20
21. The JVM, as many frameworks, relies on the number of CPUs to size its thread pools. 21
22. § commonPool is a static thread-pool used inside the JDK by CompletableFuture and Parallel Streams. § The commonPool parallelism is determined as follows: - Parallelism = available CPUs – 1 - Example: 4 CPUs à Parallelism 3 - Example: 2 CPUs à Parallelism 1 commonPool in ForkJoinPool 22 CompletableFuture parallelism § Parallelism = 1 - new Thread is created per Task § Parallelism > 2 - #Threads = #CPUs - 1
25. 25 CPUs Reported Parallelism Threads Duration 6 CPUs 6 5 5 10s 2 CPUs 2 1 50 1s 1 CPU 1 1 50 1s
26. 26 java.util.concurrent.ForkJoinPool.common.parallelism java.util.concurrent.ForkJoinPool.common.threadFactory -XX:ActiveProcessorCount=4
27. 27 Event Loop 1 IO Thread 1 Event Loop 2 IO Thread 2
28. 28 Event Loop 1 IO Thread 1 Event Loop 2 IO Thread 2 Available #CPUs
29. 29 Event Loop 1 IO Thread 1 Event Loop 2 IO Thread 2 Available #CPUs Spring Reactive
30. Don’t neglect quick start up times! 30
31. 31 Kubernetes Cluster Ready Starting Killed
32. 32 Kubernetes Cluster Ready Starting Killed
33. 33 Kubernetes Cluster Ready Starting Killed
34. 34 Kubernetes Cluster Ready Starting Killed
35. 35 Kubernetes Cluster Ready Starting Killed
36. 36 Kubernetes Cluster Ready Starting Killed
37. 37 Kubernetes Cluster Ready Starting Killed CTS (Christmas Tree State)
38. Demo 38 https://github.com/carlosbarragan/java-containers-demo
40. Horizontal scaling solves unexpected load surges. 40 It shouldn’t solve an app’s poor performance
41. Cluster sizing. 41
42. 42 Node Pod - 1 CPU - 1GB Node - 2 CPUs - 4 GB Node Node Node Node Node
43. 43 Pod - 2 CPU - 2 GB Node - 4 CPUs - 8 GB Node Node Node
44. Most Java apps scale better up than out. 44
45. Recommendations 45
46. java –jar myapp.jar 46
47. Good starting settings 47 Container CPU * Container Memory Heap Size GC 2 4 GB 75% Parallel java –XX:+UseParallelGC –XX:MaxRAMPercentage=75 … -jar my-app.jar https://learn.microsoft.com/en-us/azure/developer/java/containers/overview
48. Set CPU Requests. Don’t set CPU Limits. 48
49. 49 https://home.robusta.dev/blog/stop-using-cpu-limits
50. Test. Observe. Adapt. 50
51. 51 Thanks!