Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Optimizing Java Applications for Serverless

5e0c801ac1a5d0512bb9774ab158d06d?s=47 Ray Tsang
September 09, 2020

Optimizing Java Applications for Serverless

This deck captures the different knobs to turn for a JVM application and the trade offs to make.

Full documentation here: https://cloud.google.com/run/docs/tips/java


Ray Tsang

September 09, 2020


  1. Optimizing Java apps for Serverless...

  2. 2 @saturnism @gcpcloud Mostly startup speed & memory usage

  3. 3 @saturnism @gcpcloud It's a trade off though… Faster startup

    != more efficient
  4. 4 @saturnism @gcpcloud JVM traditionally optimize for... Long Running, High

    throughput, High concurrency
  5. 5 @saturnism @gcpcloud So we… Cache, Connection Pool, JIT

  6. 6 @saturnism @gcpcloud Minimize Memory Usages Minimize JVM startup time

    Minimize Framework init time Minimize Application init time
  7. Container Image

  8. 8 @saturnism @gcpcloud Use Jib Jib produces optimized container image

    for fast reads Trade off - almost none Some frameworks require additional configuration Works out of the box with Spring Boot though
  9. JVM

  10. 10 @saturnism @gcpcloud Java -XX:+TieredCompilation -XX:TieredStopAtLevel=1 Turn off optimization compiler

    Trade off - no optimization for hot paths Turn it off means no speed up...
  11. 11 @saturnism @gcpcloud java -noverify Don't verify classes Trade off

    - undetected malicious edits to bytecode JVM might crash Worse, you might get hacked
  12. 12 @saturnism @gcpcloud java -Xss256k Reduce thread stack size, and

    thus non-heap memory Trade off - StackOverflowError Need to find optimal value
  13. 13 @saturnism @gcpcloud Use OpenJDK 8u142 or above Container awareness

    Trade off - none, just do it! If you don't... See wrong # of CPUs, use more than you should See wrong amount of Memory, get OOMKilled
  14. 14 @saturnism @gcpcloud Dealing with OOMKilled Need to understand Memory

    usage Heap + Non-Heap = Heap + (Metaspace + Code Cache + Thread stack + ...)
  15. 15 @saturnism @gcpcloud Native Memory Tracking -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics (Doesn't work

    when set via JAVA_TOOL_OPTIONS - must be part of the argument)
  16. 16 @saturnism @gcpcloud Use Memory Calculator Non-heap usage estimated O(classes)

  17. 17 @saturnism @gcpcloud Understand Memory Usage and Accounting JVM Heap

    Non-Heap tmpfs Write filesystem Read Memory Accounting - JVM - Heap - Non-Heap - Code Cache - Thread - GC - … - tmpfs (/tmp/) - Filesystem buffers/cache - What system reads - JVM, JDK, .so lib, ...
  18. Threads

  19. 19 @saturnism @gcpcloud Reduce # of Threads Tomcat defaults to

    10 min spare threads, 200 max threads More threads = more memory too (stack size, -Xss) Trade off - lower throughput/concurrency Cloud Run max at 80 concurrent reqs per instance anyways...
  20. 20 @saturnism @gcpcloud Pending Requests Most thread-per-connection apps queues up

    requests/connections that it cannot immediately serve. Understand what a single app instance is capable of Reduce this number
  21. 21 @saturnism @gcpcloud Use Reactive / Webflux Asynchronous, non-blocking, high

    concurrency Defaults to CPU cores * 2 threads Use less memory Trade off - if you block... Then your exhaust the few threads Worse, if new worker threads are created in unbounded pool...
  22. 22 @saturnism @gcpcloud Avoid Background Tasks Cloud Run throttles CPU

    if instances has no active requests Something to be aware of... What's running in the background? Scheduled tasks, @Timer, @Async, message receivers Metrics senders, trace senders, ...
  23. 23 @saturnism @gcpcloud Connection Pools Set Max Instances Cloud Run

    Max Instances x Connections per Instance < Max Connections Allowed Something to be aware of... Connection eviction may not happen if CPU throttled
  24. Application / Framework

  25. 25 @saturnism @gcpcloud Use Spring Boot 2.2.x or up spring.main.lazy-initialization=true

    Trade off No tradeoffs. Spring Boot 2.2.x implemented a number of optimizations For < 2.2, consider disabling JMX, actuator, ...
  26. 26 @saturnism @gcpcloud Use 2-CPU gcloud run deploy … --cpu=2

    Trade off - Cost Spring Boot utilizes multiple threads for initialization Single CPU allocation may cause contention
  27. 27 @saturnism @gcpcloud Spring Boot Lazy Initialization spring.main.lazy-initialization=true Trade off

    - initialization on first request First request may be slower Use if there is a readiness check (GKE, App Engine)
  28. 28 @saturnism @gcpcloud Use Context Indexer <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context-indexer</artifactId> <optional>true</optional>

    </dependency> Trade off - slower if few components May help in the case of deep component/bean graphs
  29. 29 @saturnism @gcpcloud Make sure Dev Tool is not in

    Production Trade off - no trade offs Spring Boot Dev Tool helps w/ local development, but is not needed in Production If using Jib, make sure this is not included
  30. 30 @saturnism @gcpcloud Application Lifecycle Trade off - no trade

    offs In App Engine, implement /_ah/health and /_ah/warmup Prevents app receiving requests prematurely
  31. Native Image

  32. 32 @saturnism @gcpcloud To Graal or not to Graal? More

    accurately, to native image or not?
  33. 33 @saturnism @gcpcloud Trade Offs! Compile time GC JIT Throughput

  34. 34 @saturnism @gcpcloud Frameworks support for Native Image Quarkus Micronaut

    Spring Boot (Experimental) More than framework - libraries need to support them too
  35. 35 @saturnism @gcpcloud Additional Resources Cloud Run Java Optimization Guide

    https://cloud.google.com/run/docs/tips/java Spring Boot in a Container (Dave Syer) https://spring.io/blog/2018/11/08/spring-boot-in-a-container How Fast is Spring? (Dave Syer) https://spring.io/blog/2018/12/12/how-fast-is-spring OpenJDK and Containers (Christine Flood) https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/ Running Spring Boot App in GraalVM (Sebastian Deluze) https://www.youtube.com/watch?v=3eoAxphAUIg Spring to Kubernetes Faster and Easier (Ray Tsang) https://saturnism.me/talk/kubernetes-spring-java-best-practices/
  36. Thanks! @saturnism | saturnism.me