Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Optimizing Java Applications for Serverless

Ray Tsang
September 09, 2020

Optimizing Java Applications for Serverless

This deck captures the different knobs to turn for a JVM application and the trade offs to make.

Full documentation here: https://cloud.google.com/run/docs/tips/java

Ray Tsang

September 09, 2020

More Decks by Ray Tsang

Other Decks in Technology


  1. 6 @saturnism @gcpcloud Minimize Memory Usages Minimize JVM startup time

    Minimize Framework init time Minimize Application init time
  2. 8 @saturnism @gcpcloud Use Jib Jib produces optimized container image

    for fast reads Trade off - almost none Some frameworks require additional configuration Works out of the box with Spring Boot though
  3. JVM

  4. 10 @saturnism @gcpcloud Java -XX:+TieredCompilation -XX:TieredStopAtLevel=1 Turn off optimization compiler

    Trade off - no optimization for hot paths Turn it off means no speed up...
  5. 11 @saturnism @gcpcloud java -noverify Don't verify classes Trade off

    - undetected malicious edits to bytecode JVM might crash Worse, you might get hacked
  6. 12 @saturnism @gcpcloud java -Xss256k Reduce thread stack size, and

    thus non-heap memory Trade off - StackOverflowError Need to find optimal value
  7. 13 @saturnism @gcpcloud Use OpenJDK 8u142 or above Container awareness

    Trade off - none, just do it! If you don't... See wrong # of CPUs, use more than you should See wrong amount of Memory, get OOMKilled
  8. 14 @saturnism @gcpcloud Dealing with OOMKilled Need to understand Memory

    usage Heap + Non-Heap = Heap + (Metaspace + Code Cache + Thread stack + ...)
  9. 16 @saturnism @gcpcloud Use Memory Calculator Non-heap usage estimated O(classes)

  10. 17 @saturnism @gcpcloud Understand Memory Usage and Accounting JVM Heap

    Non-Heap tmpfs Write filesystem Read Memory Accounting - JVM - Heap - Non-Heap - Code Cache - Thread - GC - … - tmpfs (/tmp/) - Filesystem buffers/cache - What system reads - JVM, JDK, .so lib, ...
  11. 19 @saturnism @gcpcloud Reduce # of Threads Tomcat defaults to

    10 min spare threads, 200 max threads More threads = more memory too (stack size, -Xss) Trade off - lower throughput/concurrency Cloud Run max at 80 concurrent reqs per instance anyways...
  12. 20 @saturnism @gcpcloud Pending Requests Most thread-per-connection apps queues up

    requests/connections that it cannot immediately serve. Understand what a single app instance is capable of Reduce this number
  13. 21 @saturnism @gcpcloud Use Reactive / Webflux Asynchronous, non-blocking, high

    concurrency Defaults to CPU cores * 2 threads Use less memory Trade off - if you block... Then your exhaust the few threads Worse, if new worker threads are created in unbounded pool...
  14. 22 @saturnism @gcpcloud Avoid Background Tasks Cloud Run throttles CPU

    if instances has no active requests Something to be aware of... What's running in the background? Scheduled tasks, @Timer, @Async, message receivers Metrics senders, trace senders, ...
  15. 23 @saturnism @gcpcloud Connection Pools Set Max Instances Cloud Run

    Max Instances x Connections per Instance < Max Connections Allowed Something to be aware of... Connection eviction may not happen if CPU throttled
  16. 25 @saturnism @gcpcloud Use Spring Boot 2.2.x or up spring.main.lazy-initialization=true

    Trade off No tradeoffs. Spring Boot 2.2.x implemented a number of optimizations For < 2.2, consider disabling JMX, actuator, ...
  17. 26 @saturnism @gcpcloud Use 2-CPU gcloud run deploy … --cpu=2

    Trade off - Cost Spring Boot utilizes multiple threads for initialization Single CPU allocation may cause contention
  18. 27 @saturnism @gcpcloud Spring Boot Lazy Initialization spring.main.lazy-initialization=true Trade off

    - initialization on first request First request may be slower Use if there is a readiness check (GKE, App Engine)
  19. 29 @saturnism @gcpcloud Make sure Dev Tool is not in

    Production Trade off - no trade offs Spring Boot Dev Tool helps w/ local development, but is not needed in Production If using Jib, make sure this is not included
  20. 30 @saturnism @gcpcloud Application Lifecycle Trade off - no trade

    offs In App Engine, implement /_ah/health and /_ah/warmup Prevents app receiving requests prematurely
  21. 32 @saturnism @gcpcloud To Graal or not to Graal? More

    accurately, to native image or not?
  22. 34 @saturnism @gcpcloud Frameworks support for Native Image Quarkus Micronaut

    Spring Boot (Experimental) More than framework - libraries need to support them too
  23. 35 @saturnism @gcpcloud Additional Resources Cloud Run Java Optimization Guide

    https://cloud.google.com/run/docs/tips/java Spring Boot in a Container (Dave Syer) https://spring.io/blog/2018/11/08/spring-boot-in-a-container How Fast is Spring? (Dave Syer) https://spring.io/blog/2018/12/12/how-fast-is-spring OpenJDK and Containers (Christine Flood) https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/ Running Spring Boot App in GraalVM (Sebastian Deluze) https://www.youtube.com/watch?v=3eoAxphAUIg Spring to Kubernetes Faster and Easier (Ray Tsang) https://saturnism.me/talk/kubernetes-spring-java-best-practices/