Optimizing Java apps for Serverless...

2 @saturnism @gcpcloud Mostly startup speed & memory usage

3 @saturnism @gcpcloud It's a trade off though… Faster startup != more efficient

4 @saturnism @gcpcloud JVM traditionally optimize for... Long Running, High throughput, High concurrency

5 @saturnism @gcpcloud So we… Cache, Connection Pool, JIT

6 @saturnism @gcpcloud Minimize Memory Usages Minimize JVM startup time Minimize Framework init time Minimize Application init time

Container Image

8 @saturnism @gcpcloud Use Jib Jib produces optimized container image for fast reads Trade off - almost none Some frameworks require additional configuration Works out of the box with Spring Boot though

JVM

10 @saturnism @gcpcloud Java -XX:+TieredCompilation -XX:TieredStopAtLevel=1 Turn off optimization compiler Trade off - no optimization for hot paths Turn it off means no speed up...

11 @saturnism @gcpcloud java -noverify Don't verify classes Trade off - undetected malicious edits to bytecode JVM might crash Worse, you might get hacked

12 @saturnism @gcpcloud java -Xss256k Reduce thread stack size, and thus non-heap memory Trade off - StackOverflowError Need to find optimal value

13 @saturnism @gcpcloud Use OpenJDK 8u142 or above Container awareness Trade off - none, just do it! If you don't... See wrong # of CPUs, use more than you should See wrong amount of Memory, get OOMKilled

14 @saturnism @gcpcloud Dealing with OOMKilled Need to understand Memory usage Heap + Non-Heap = Heap + (Metaspace + Code Cache + Thread stack + ...)

15 @saturnism @gcpcloud Native Memory Tracking -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics (Doesn't work when set via JAVA_TOOL_OPTIONS - must be part of the argument)

16 @saturnism @gcpcloud Use Memory Calculator Non-heap usage estimated O(classes) https://github.com/cloudfoundry/java-buildpack-memory-calculator

17 @saturnism @gcpcloud Understand Memory Usage and Accounting JVM Heap Non-Heap tmpfs Write filesystem Read Memory Accounting - JVM - Heap - Non-Heap - Code Cache - Thread - GC - … - tmpfs (/tmp/) - Filesystem buffers/cache - What system reads - JVM, JDK, .so lib, ...

Threads

19 @saturnism @gcpcloud Reduce # of Threads Tomcat defaults to 10 min spare threads, 200 max threads More threads = more memory too (stack size, -Xss) Trade off - lower throughput/concurrency Cloud Run max at 80 concurrent reqs per instance anyways...

20 @saturnism @gcpcloud Pending Requests Most thread-per-connection apps queues up requests/connections that it cannot immediately serve. Understand what a single app instance is capable of Reduce this number

21 @saturnism @gcpcloud Use Reactive / Webflux Asynchronous, non-blocking, high concurrency Defaults to CPU cores * 2 threads Use less memory Trade off - if you block... Then your exhaust the few threads Worse, if new worker threads are created in unbounded pool...

22 @saturnism @gcpcloud Avoid Background Tasks Cloud Run throttles CPU if instances has no active requests Something to be aware of... What's running in the background? Scheduled tasks, @Timer, @Async, message receivers Metrics senders, trace senders, ...

23 @saturnism @gcpcloud Connection Pools Set Max Instances Cloud Run Max Instances x Connections per Instance < Max Connections Allowed Something to be aware of... Connection eviction may not happen if CPU throttled

Application / Framework

25 @saturnism @gcpcloud Use Spring Boot 2.2.x or up spring.main.lazy-initialization=true Trade off No tradeoffs. Spring Boot 2.2.x implemented a number of optimizations For < 2.2, consider disabling JMX, actuator, ...

26 @saturnism @gcpcloud Use 2-CPU gcloud run deploy … --cpu=2 Trade off - Cost Spring Boot utilizes multiple threads for initialization Single CPU allocation may cause contention

27 @saturnism @gcpcloud Spring Boot Lazy Initialization spring.main.lazy-initialization=true Trade off - initialization on first request First request may be slower Use if there is a readiness check (GKE, App Engine)

28 @saturnism @gcpcloud Use Context Indexer org.springframework spring-context-indexer true Trade off - slower if few components May help in the case of deep component/bean graphs

29 @saturnism @gcpcloud Make sure Dev Tool is not in Production Trade off - no trade offs Spring Boot Dev Tool helps w/ local development, but is not needed in Production If using Jib, make sure this is not included

30 @saturnism @gcpcloud Application Lifecycle Trade off - no trade offs In App Engine, implement /_ah/health and /_ah/warmup Prevents app receiving requests prematurely

Native Image

32 @saturnism @gcpcloud To Graal or not to Graal? More accurately, to native image or not?

33 @saturnism @gcpcloud Trade Offs! Compile time GC JIT Throughput

34 @saturnism @gcpcloud Frameworks support for Native Image Quarkus Micronaut Spring Boot (Experimental) More than framework - libraries need to support them too

35 @saturnism @gcpcloud Additional Resources Cloud Run Java Optimization Guide https://cloud.google.com/run/docs/tips/java Spring Boot in a Container (Dave Syer) https://spring.io/blog/2018/11/08/spring-boot-in-a-container How Fast is Spring? (Dave Syer) https://spring.io/blog/2018/12/12/how-fast-is-spring OpenJDK and Containers (Christine Flood) https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/ Running Spring Boot App in GraalVM (Sebastian Deluze) https://www.youtube.com/watch?v=3eoAxphAUIg Spring to Kubernetes Faster and Easier (Ray Tsang) https://saturnism.me/talk/kubernetes-spring-java-best-practices/

Thanks! @saturnism | saturnism.me