Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Optimizing Java Applications for Serverless

Ray Tsang
September 09, 2020

Optimizing Java Applications for Serverless

This deck captures the different knobs to turn for a JVM application and the trade offs to make.

Full documentation here: https://cloud.google.com/run/docs/tips/java

Ray Tsang

September 09, 2020
Tweet

More Decks by Ray Tsang

Other Decks in Technology

Transcript

  1. Optimizing Java apps for Serverless...

    View Slide

  2. 2
    @saturnism @gcpcloud
    Mostly startup speed & memory usage

    View Slide

  3. 3
    @saturnism @gcpcloud
    It's a trade off though…
    Faster startup != more efficient

    View Slide

  4. 4
    @saturnism @gcpcloud
    JVM traditionally optimize for...
    Long Running, High throughput, High concurrency

    View Slide

  5. 5
    @saturnism @gcpcloud
    So we…
    Cache, Connection Pool, JIT

    View Slide

  6. 6
    @saturnism @gcpcloud
    Minimize Memory Usages
    Minimize JVM startup time
    Minimize Framework init time
    Minimize Application init time

    View Slide

  7. Container Image

    View Slide

  8. 8
    @saturnism @gcpcloud
    Use Jib
    Jib produces optimized container image for fast reads
    Trade off - almost none
    Some frameworks require additional configuration
    Works out of the box with Spring Boot though

    View Slide

  9. JVM

    View Slide

  10. 10
    @saturnism @gcpcloud
    Java -XX:+TieredCompilation -XX:TieredStopAtLevel=1
    Turn off optimization compiler
    Trade off - no optimization for hot paths
    Turn it off means no speed up...

    View Slide

  11. 11
    @saturnism @gcpcloud
    java -noverify
    Don't verify classes
    Trade off - undetected malicious edits to bytecode
    JVM might crash
    Worse, you might get hacked

    View Slide

  12. 12
    @saturnism @gcpcloud
    java -Xss256k
    Reduce thread stack size, and thus non-heap memory
    Trade off - StackOverflowError
    Need to find optimal value

    View Slide

  13. 13
    @saturnism @gcpcloud
    Use OpenJDK 8u142 or above
    Container awareness
    Trade off - none, just do it! If you don't...
    See wrong # of CPUs, use more than you should
    See wrong amount of Memory, get OOMKilled

    View Slide

  14. 14
    @saturnism @gcpcloud
    Dealing with OOMKilled
    Need to understand Memory usage
    Heap + Non-Heap =
    Heap + (Metaspace + Code Cache + Thread stack + ...)

    View Slide

  15. 15
    @saturnism @gcpcloud
    Native Memory Tracking
    -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics
    (Doesn't work when set via JAVA_TOOL_OPTIONS - must be part of the argument)

    View Slide

  16. 16
    @saturnism @gcpcloud
    Use Memory Calculator
    Non-heap usage estimated O(classes)
    https://github.com/cloudfoundry/java-buildpack-memory-calculator

    View Slide

  17. 17
    @saturnism @gcpcloud
    Understand Memory Usage and Accounting
    JVM
    Heap
    Non-Heap
    tmpfs
    Write
    filesystem
    Read
    Memory Accounting
    - JVM
    - Heap
    - Non-Heap
    - Code Cache
    - Thread
    - GC
    - …
    - tmpfs (/tmp/)
    - Filesystem buffers/cache
    - What system reads - JVM, JDK, .so
    lib, ...

    View Slide

  18. Threads

    View Slide

  19. 19
    @saturnism @gcpcloud
    Reduce # of Threads
    Tomcat defaults to 10 min spare threads, 200 max threads
    More threads = more memory too (stack size, -Xss)
    Trade off - lower throughput/concurrency
    Cloud Run max at 80 concurrent reqs per instance anyways...

    View Slide

  20. 20
    @saturnism @gcpcloud
    Pending Requests
    Most thread-per-connection apps queues up
    requests/connections that it cannot immediately serve.
    Understand what a single app instance is capable of
    Reduce this number

    View Slide

  21. 21
    @saturnism @gcpcloud
    Use Reactive / Webflux
    Asynchronous, non-blocking, high concurrency
    Defaults to CPU cores * 2 threads
    Use less memory
    Trade off - if you block...
    Then your exhaust the few threads
    Worse, if new worker threads are created in unbounded pool...

    View Slide

  22. 22
    @saturnism @gcpcloud
    Avoid Background Tasks
    Cloud Run throttles CPU if instances has no active requests
    Something to be aware of...
    What's running in the background?
    Scheduled tasks, @Timer, @Async, message receivers
    Metrics senders, trace senders, ...

    View Slide

  23. 23
    @saturnism @gcpcloud
    Connection Pools
    Set Max Instances
    Cloud Run Max Instances x Connections per Instance < Max Connections Allowed
    Something to be aware of...
    Connection eviction may not happen if CPU throttled

    View Slide

  24. Application / Framework

    View Slide

  25. 25
    @saturnism @gcpcloud
    Use Spring Boot 2.2.x or up
    spring.main.lazy-initialization=true
    Trade off
    No tradeoffs. Spring Boot 2.2.x implemented a number of optimizations
    For < 2.2, consider disabling JMX, actuator, ...

    View Slide

  26. 26
    @saturnism @gcpcloud
    Use 2-CPU
    gcloud run deploy … --cpu=2
    Trade off - Cost
    Spring Boot utilizes multiple threads for initialization
    Single CPU allocation may cause contention

    View Slide

  27. 27
    @saturnism @gcpcloud
    Spring Boot Lazy Initialization
    spring.main.lazy-initialization=true
    Trade off - initialization on first request
    First request may be slower
    Use if there is a readiness check (GKE, App Engine)

    View Slide

  28. 28
    @saturnism @gcpcloud
    Use Context Indexer

    org.springframework
    spring-context-indexer
    true

    Trade off - slower if few components
    May help in the case of deep component/bean graphs

    View Slide

  29. 29
    @saturnism @gcpcloud
    Make sure Dev Tool is not in Production
    Trade off - no trade offs
    Spring Boot Dev Tool helps w/ local development, but is not
    needed in Production
    If using Jib, make sure this is not included

    View Slide

  30. 30
    @saturnism @gcpcloud
    Application Lifecycle
    Trade off - no trade offs
    In App Engine, implement /_ah/health and /_ah/warmup
    Prevents app receiving requests prematurely

    View Slide

  31. Native Image

    View Slide

  32. 32
    @saturnism @gcpcloud
    To Graal or not to Graal?
    More accurately, to native image or not?

    View Slide

  33. 33
    @saturnism @gcpcloud
    Trade Offs!
    Compile time
    GC
    JIT
    Throughput

    View Slide

  34. 34
    @saturnism @gcpcloud
    Frameworks support for Native Image
    Quarkus
    Micronaut
    Spring Boot (Experimental)
    More than framework - libraries need to support them too

    View Slide

  35. 35
    @saturnism @gcpcloud
    Additional Resources
    Cloud Run Java Optimization Guide https://cloud.google.com/run/docs/tips/java
    Spring Boot in a Container (Dave Syer) https://spring.io/blog/2018/11/08/spring-boot-in-a-container
    How Fast is Spring? (Dave Syer) https://spring.io/blog/2018/12/12/how-fast-is-spring
    OpenJDK and Containers (Christine Flood) https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/
    Running Spring Boot App in GraalVM (Sebastian Deluze) https://www.youtube.com/watch?v=3eoAxphAUIg
    Spring to Kubernetes Faster and Easier (Ray Tsang) https://saturnism.me/talk/kubernetes-spring-java-best-practices/

    View Slide

  36. Thanks!
    @saturnism | saturnism.me

    View Slide