Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Performance-Tweaks für Java-Serverless-Anwendun...

Performance-Tweaks für Java-Serverless-Anwendungen (IT-Tage)

Serverless-Funktionen erfreuen sich zunehmender Beliebtheit. Der Fokus auf den Funktionscode (und nicht auf die darunterliegende Serverinfrastruktur) sowie die dynamische Anpassung an den aktuellen Ressourcenbedarf sind einige Gründe dafür. Doch nicht selten orientieren sich erfahrene Java-Entwickler und -Architekten in diesem Zuge neu und schwenken auf alternative Programmiersprachen wie Node.js oder Python. Hintergrund ist die Abrechnung von Serverless-Funktionen basierend auf Ausführungsdauer (CPU-Zeit) und Speicherverbrauch. Java wird in diesem Zuge immer noch als speicher-hungrig und langsam (Cold starts) eingeordnet.

Der Vortrag zeigt auf, welche Stellschrauben bei Java in Verbindung mit Serverless-Anwendungen betrachtet werden sollten. Dabei werden sowohl Parameter der JVM (wie Heap Size, Garbage Collection und Tiered Compilation) betrachtet, als auch ein Überblick über Frameworks und Tools (wie GraalVM, Quarkus, Micronaut und Spring Native) gegeben. Zuletzt wird auf Möglichkeiten des CRaC (Coordinated Restore at Checkpoint) Projekts eingegangen.

Dennis Kieselhorst

December 12, 2023
Tweet

More Decks by Dennis Kieselhorst

Other Decks in Programming

Transcript

  1. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. #ittage Dennis Kieselhorst Maximilian Schellhorn Performance-Tweaks für Java-Serverless-Anwendungen
  2. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Serverful architecture Application Load Balancer Web app container Container target group HTTP server Framework Application code Spring Boot application
  3. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Event Changes in data state Requests to endpoints Changes in resource state Serverless architecture Application code AWS Lambda function
  4. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Functions are invoked by events • AWS Lambda is invoked via events (API Event, S3 Object Event, SQS Event ..) • Events follow a certain structure • This is different from accepting a HTTP connection Amazon API Gateway AWS Lambda POST /v1/pets HTTP/2 Host: x.execute-api.eu-west-1.. User-Agent: curl/7.64.1 Accept: */* Content-Type: application/json Content-Length: 39 Body: {“data:“ : “test“} { "body": “{“data“: “test“}“ , "resource": "/{proxy+}", "path": "/path/to/resource", "httpMethod": "POST", "isBase64Encoded": true, "headers": { "Accept-Encoding": "gzip", "Accept-Language": "en-US,en;q=0.8" }, }, "requestContext": { "accountId": "123456789012", "requestId": "c6af9ac6-7b61-..", ... } }
  5. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Lambda Execution Model Amazon SNS Amazon S3 reqs Amazon Kinesis changes AWS Lambda service Amazon API Gateway Lambda function Lambda function Synchronous Asynchronous Poll Amazon SQS AWS Lambda Function URLs
  6. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. What are Lambda functions? Handler function • Function executed on invocation • Java 8, 11, 17, 21 supported Event • Invocation data sent to function • JSON event payload differs by source (API, State change, Queue) Context • Additional information from the service • Example: Request ID
  7. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Tune the serverless function's resources… AWS Lambda exposes only a memory control, with the % of CPU core and network capacity allocated to a function proportionally. Is your code CPU, network or memory-bound? If so, it could be cheaper to increase memory. > Memory, > vCPU, > Network 10240 MB 128 MB
  8. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. …with Power Tuning https://github.com/alexcasalboni/aws-lambda-power-tuning
  9. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Leverage multithreading I/O bound workloads will likely see gains by multithreading. < 1 vCPU: CPU bound workloads won't see gains > 1 vCPU: you can leverage multiple vCPUs 10240 MB 128 MB 6 vCPU 1 vCPU 1769 MB
  10. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. 1 2 Request Request Initialize Application Running application 3 Request … • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually Java in a container or a VM Initialization
  11. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Execution Environment ready? Download Code Start Runtime Initialize Function Code Code execution Request Code execution No Yes Create Execution Environment Execution Environment Cold start Warm start AWS Lambda request handling
  12. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution Java with Serverless functions using AWS Lambda • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started – simpler programming model • Scales to zero if there is no request – Managed auto scaling
  13. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Performance Tweaks
  14. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. 0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9 Tiered Compilation Compile Time frameworks No framework No optimization Lightw. deps Function handler GraalVM Effort to modernize Cold-start (seconds) Provisioned Concurrency Performance optimizations (before 2022) Default with Java 17 & 21 on AWS Lambda
  15. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. AWS Lambda Provisioned Concurrency • Sets floor on minimum number of execution environments • Pre-warm execution environments to reduce cold-start impact • Can save costs in certain situations
  16. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Tiered Compilation Setting Set JVM tiered compilation level to 1 and use the C1 compiler. JAVA_TOOL_OPTIONS= -XX:+TieredCompilation -XX:TieredStopAtLevel=1 C1 compiler quickly produces native code. No profiling overhead, but no benefits from more aggressive C2 compilation. related blogpost Enabled by default in AWS Lambda Java 17 & 21 runtime!
  17. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Dependencies • Trade-off between feature richness & performance • Examples § jackson-jr for serializiation § SimpleLogger for logging § Spring FunctionHandler<Event> instead of RestController (directly handle API-Gateway events instead of HTTP) • Be careful with dependency injection: Annotation scanning adds more class loading.
  18. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. AWS SDK and Common Runtime Client • As of today there are two version of the AWS SDK for Java (AWS SDK for Java v1 and AWS SDK for Java v2 ). • We recommend to use the latest version AWS SDK for Java v2 because it has been designed to improve the performance and usability. • The AWS Common Runtime (CRT) HTTP client is a new HTTP client you can use with the AWS SDK for Java v2. The CRT-based HTTP client is an asynchronous, non-blocking HTTP client built on top of the Java bindings of the AWS Common Runtime.
  19. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart Up to 10x faster startup performance
  20. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart • Takes a snapshot of the memory and disk state • Snapshot encryption & caching for low-latency access • Creates new Lambda environments from cached snapshot • Fully Managed microVM snapshot technology
  21. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart - 1. Publication Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke
  22. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Invoke Resume Invoke Resume Invoke Resume Invoke Resume Invoke Resume AWS Lambda SnapStart – 2. Invocation Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke Invoke Resume Invoke […]
  23. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Code execution Invoke Resume Resume Snapshot Post Snapshot Hook (optional) Pre Snapshot Hook (optional) Create Execution Environment Init during deployment Download Code Start Runtime Initialize Function Code Create Snapshot first request How AWS Lambda SnapStart works
  24. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Considerations Network connections Ephemeral data Uniqueness
  25. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Runtime Hooks • Runtime hooks are available as part of the open-source Coordinated Restore at Checkpoint (CRaC) project. • Include dependency (e.g. Maven) <dependency> <groupId>org.crac</groupId> <artifactId>crac</artifactId> <version>1.4.0</version> </dependency>
  26. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. CRaC interface for post- and pre-hooks examples - beforeCheckpoint - priming - class loading - dependency injection - afterRestore - unique ID for execution environment - re-establish connections
  27. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke After Restore() Before Snapshot() Load and initialize handler Resize Heap
  28. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle Run handler code Compile Package Deploy Configure API clients Reflection and Class Loading Dependency Injection JIT Compilation Build and Deploy Handler Invocation Initialize After Restore() Before Snapshot() JVM start Download code Load and initialize handler Resize Heap Publish Invoke
  29. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart benefits • Start your functions from pre-initialized snapshots • No additional cost • Minimal to no code changes • Customization via Runtime Hooks Without SnapStart With SnapStart 6.725 - 80 % 1.15 Full Duration Results will vary based on your application
  30. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. 0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9 Tiered Compilation Compile Time frameworks No framework No optimization Lightw. deps Function handler GraalVM Effort to modernize Cold-start (seconds) Provisioned Concurrency AWS Lambda performance optimizations SnapStart Default with Java 17 & 21 on AWS Lambda
  31. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Compile your application ahead-of-time (AOT) • GraalVM creates a native image of your application at build-time. • Frameworks like Quarkus, Micronaut, and Spring Native integrate GraalVM and add further benefits like build-time dependency injection. • Some JVM features not supported. • Application startup time reduced of more than 90%
  32. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. GraalVM native images on AWS Lambda • Create a native binary and deploy as AWS Lambda custom runtime • Sub-second application startup and low memory footprint • Multiple frameworks provide built-in support (Micronaut, Quarkus, Spring) bootstrap
  33. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Additional considerations
  34. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Helpful frameworks Spring • Well-known framework for Java based Enterprise applications • Spring Boot 3.2 now supports CRaC • Spring Cloud for AWS: awspring.io • Major Contributor: VMware (now Broadcom) Quarkus • Container first (Kube native) but also has extensions for AWS and other Cloud providers • Fast boot time and low footprint • Jakarta EE standard compatible • Major Contributor: Red Hat Micronaut • Majority of work during compile-time • Eliminates reflection, runtime proxies, dynamic class loading • Smaller startup time and memory footprint • AWS CDK and SDK support • Major Contributor: Oracle
  35. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Custom AWS silicon with 64-bit Arm cores Targeted optimizations to reduce costs Rapidly innovate, build, and iterate Range of instance types Same AWS Nitro building blocks Best performance per watt of energy use in Amazon EC2 AWS Graviton2, Graviton3 & Graviton4 X86 vs. ARM – AWS Graviton processor Graviton
  36. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Monitor and tune the Garbage Collection Monitor the Garbage Collection logs, try a fixed heap size specific to your application. Example shows 100 calls with heap size 400 MB: • 5% call duration reduction • Only minor garbage collection activities
  37. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Database connection pooling • Database connection pools (e.g. HikariCP) initialize multiple database connections during startup • Since an AWS Lambda environment handles a single request at a time you only need 1 connection per environment • Ensure to stay within database connection pool limits • Solution: RDS Proxy or Reserved Concurrency (Limit connections) … Container
  38. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Powertools for AWS Lambda J A V A L I B R A R Y T O S I M P L I F Y T R A C I N G , S T R U C T U R E D L O G G I N G A N D C U S T O M M E T R I C S • Facilitate best practice adoption • Increase developer velocity • Optional modules: • Parameter handling • Idempotency • SQS Large Message and Batching Options • Serialization, Validation and custom resource support
  39. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Future • The Java ecosystem is adapting and introduced several new projects to foster innovation. • Project Leyden will address startup time, memory footprint and peak performance by introducing a concept of static images to the Java Platform, and to the JDK. • The changes to the release cycle mean that innovation will be delivered faster.
  40. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Deep Dive: Java on AWS Lambda Workshop https://catalog.workshops.aws/java-on-aws-lambda
  41. IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. Thank you! © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.