Performance-Tweaks für Java-Serverless-Anwendungen (IT-Tage)

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web
Services, Inc. or its affiliates. All rights reserved. #ittage Dennis Kieselhorst Maximilian Schellhorn Performance-Tweaks für Java-Serverless-Anwendungen

Services, Inc. or its affiliates. All rights reserved. Serverful architecture Application Load Balancer Web app container Container target group HTTP server Framework Application code Spring Boot application

Services, Inc. or its affiliates. All rights reserved. Functions are invoked by events • AWS Lambda is invoked via events (API Event, S3 Object Event, SQS Event ..) • Events follow a certain structure • This is different from accepting a HTTP connection Amazon API Gateway AWS Lambda POST /v1/pets HTTP/2 Host: x.execute-api.eu-west-1.. User-Agent: curl/7.64.1 Accept: */* Content-Type: application/json Content-Length: 39 Body: {“data:“ : “test“} { "body": “{“data“: “test“}“ , "resource": "/{proxy+}", "path": "/path/to/resource", "httpMethod": "POST", "isBase64Encoded": true, "headers": { "Accept-Encoding": "gzip", "Accept-Language": "en-US,en;q=0.8" }, }, "requestContext": { "accountId": "123456789012", "requestId": "c6af9ac6-7b61-..", ... } }

Services, Inc. or its affiliates. All rights reserved. Lambda Execution Model Amazon SNS Amazon S3 reqs Amazon Kinesis changes AWS Lambda service Amazon API Gateway Lambda function Lambda function Synchronous Asynchronous Poll Amazon SQS AWS Lambda Function URLs

Services, Inc. or its affiliates. All rights reserved. What are Lambda functions? Handler function • Function executed on invocation • Java 8, 11, 17, 21 supported Event • Invocation data sent to function • JSON event payload differs by source (API, State change, Queue) Context • Additional information from the service • Example: Request ID

Services, Inc. or its affiliates. All rights reserved. Tune the serverless function's resources… AWS Lambda exposes only a memory control, with the % of CPU core and network capacity allocated to a function proportionally. Is your code CPU, network or memory-bound? If so, it could be cheaper to increase memory. > Memory, > vCPU, > Network 10240 MB 128 MB

Services, Inc. or its affiliates. All rights reserved. Leverage multithreading I/O bound workloads will likely see gains by multithreading. < 1 vCPU: CPU bound workloads won't see gains > 1 vCPU: you can leverage multiple vCPUs 10240 MB 128 MB 6 vCPU 1 vCPU 1769 MB

Services, Inc. or its affiliates. All rights reserved. 1 2 Request Request Initialize Application Running application 3 Request … • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually Java in a container or a VM Initialization

Services, Inc. or its affiliates. All rights reserved. Execution Environment ready? Download Code Start Runtime Initialize Function Code Code execution Request Code execution No Yes Create Execution Environment Execution Environment Cold start Warm start AWS Lambda request handling

Services, Inc. or its affiliates. All rights reserved. 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution Java with Serverless functions using AWS Lambda • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started – simpler programming model • Scales to zero if there is no request – Managed auto scaling

Services, Inc. or its affiliates. All rights reserved. 0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9 Tiered Compilation Compile Time frameworks No framework No optimization Lightw. deps Function handler GraalVM Effort to modernize Cold-start (seconds) Provisioned Concurrency Performance optimizations (before 2022) Default with Java 17 & 21 on AWS Lambda

Services, Inc. or its affiliates. All rights reserved. AWS Lambda Provisioned Concurrency • Sets floor on minimum number of execution environments • Pre-warm execution environments to reduce cold-start impact • Can save costs in certain situations

Services, Inc. or its affiliates. All rights reserved. Tiered Compilation Setting Set JVM tiered compilation level to 1 and use the C1 compiler. JAVA_TOOL_OPTIONS= -XX:+TieredCompilation -XX:TieredStopAtLevel=1 C1 compiler quickly produces native code. No profiling overhead, but no benefits from more aggressive C2 compilation. related blogpost Enabled by default in AWS Lambda Java 17 & 21 runtime!

Services, Inc. or its affiliates. All rights reserved. Dependencies • Trade-off between feature richness & performance • Examples § jackson-jr for serializiation § SimpleLogger for logging § Spring FunctionHandler<Event> instead of RestController (directly handle API-Gateway events instead of HTTP) • Be careful with dependency injection: Annotation scanning adds more class loading.

Services, Inc. or its affiliates. All rights reserved. AWS SDK and Common Runtime Client • As of today there are two version of the AWS SDK for Java (AWS SDK for Java v1 and AWS SDK for Java v2 ). • We recommend to use the latest version AWS SDK for Java v2 because it has been designed to improve the performance and usability. • The AWS Common Runtime (CRT) HTTP client is a new HTTP client you can use with the AWS SDK for Java v2. The CRT-based HTTP client is an asynchronous, non-blocking HTTP client built on top of the Java bindings of the AWS Common Runtime.

Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart • Takes a snapshot of the memory and disk state • Snapshot encryption & caching for low-latency access • Creates new Lambda environments from cached snapshot • Fully Managed microVM snapshot technology

Services, Inc. or its affiliates. All rights reserved. Invoke Resume Invoke Resume Invoke Resume Invoke Resume Invoke Resume AWS Lambda SnapStart – 2. Invocation Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke Invoke Resume Invoke […]

Services, Inc. or its affiliates. All rights reserved. Code execution Invoke Resume Resume Snapshot Post Snapshot Hook (optional) Pre Snapshot Hook (optional) Create Execution Environment Init during deployment Download Code Start Runtime Initialize Function Code Create Snapshot first request How AWS Lambda SnapStart works

Services, Inc. or its affiliates. All rights reserved. Runtime Hooks • Runtime hooks are available as part of the open-source Coordinated Restore at Checkpoint (CRaC) project. • Include dependency (e.g. Maven) <dependency> <groupId>org.crac</groupId> <artifactId>crac</artifactId> <version>1.4.0</version> </dependency>

Services, Inc. or its affiliates. All rights reserved. CRaC interface for post- and pre-hooks examples - beforeCheckpoint - priming - class loading - dependency injection - afterRestore - unique ID for execution environment - re-establish connections

Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke After Restore() Before Snapshot() Load and initialize handler Resize Heap

Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle Run handler code Compile Package Deploy Configure API clients Reflection and Class Loading Dependency Injection JIT Compilation Build and Deploy Handler Invocation Initialize After Restore() Before Snapshot() JVM start Download code Load and initialize handler Resize Heap Publish Invoke

Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart benefits • Start your functions from pre-initialized snapshots • No additional cost • Minimal to no code changes • Customization via Runtime Hooks Without SnapStart With SnapStart 6.725 - 80 % 1.15 Full Duration Results will vary based on your application

Services, Inc. or its affiliates. All rights reserved. 0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9 Tiered Compilation Compile Time frameworks No framework No optimization Lightw. deps Function handler GraalVM Effort to modernize Cold-start (seconds) Provisioned Concurrency AWS Lambda performance optimizations SnapStart Default with Java 17 & 21 on AWS Lambda

Services, Inc. or its affiliates. All rights reserved. Compile your application ahead-of-time (AOT) • GraalVM creates a native image of your application at build-time. • Frameworks like Quarkus, Micronaut, and Spring Native integrate GraalVM and add further benefits like build-time dependency injection. • Some JVM features not supported. • Application startup time reduced of more than 90%

Services, Inc. or its affiliates. All rights reserved. GraalVM native images on AWS Lambda • Create a native binary and deploy as AWS Lambda custom runtime • Sub-second application startup and low memory footprint • Multiple frameworks provide built-in support (Micronaut, Quarkus, Spring) bootstrap

Services, Inc. or its affiliates. All rights reserved. Helpful frameworks Spring • Well-known framework for Java based Enterprise applications • Spring Boot 3.2 now supports CRaC • Spring Cloud for AWS: awspring.io • Major Contributor: VMware (now Broadcom) Quarkus • Container first (Kube native) but also has extensions for AWS and other Cloud providers • Fast boot time and low footprint • Jakarta EE standard compatible • Major Contributor: Red Hat Micronaut • Majority of work during compile-time • Eliminates reflection, runtime proxies, dynamic class loading • Smaller startup time and memory footprint • AWS CDK and SDK support • Major Contributor: Oracle

Services, Inc. or its affiliates. All rights reserved. Custom AWS silicon with 64-bit Arm cores Targeted optimizations to reduce costs Rapidly innovate, build, and iterate Range of instance types Same AWS Nitro building blocks Best performance per watt of energy use in Amazon EC2 AWS Graviton2, Graviton3 & Graviton4 X86 vs. ARM – AWS Graviton processor Graviton

Services, Inc. or its affiliates. All rights reserved. Monitor and tune the Garbage Collection Monitor the Garbage Collection logs, try a fixed heap size specific to your application. Example shows 100 calls with heap size 400 MB: • 5% call duration reduction • Only minor garbage collection activities

Services, Inc. or its affiliates. All rights reserved. Database connection pooling • Database connection pools (e.g. HikariCP) initialize multiple database connections during startup • Since an AWS Lambda environment handles a single request at a time you only need 1 connection per environment • Ensure to stay within database connection pool limits • Solution: RDS Proxy or Reserved Concurrency (Limit connections) … Container

Services, Inc. or its affiliates. All rights reserved. Powertools for AWS Lambda J A V A L I B R A R Y T O S I M P L I F Y T R A C I N G , S T R U C T U R E D L O G G I N G A N D C U S T O M M E T R I C S • Facilitate best practice adoption • Increase developer velocity • Optional modules: • Parameter handling • Idempotency • SQS Large Message and Batching Options • Serialization, Validation and custom resource support

Services, Inc. or its affiliates. All rights reserved. Future • The Java ecosystem is adapting and introduced several new projects to foster innovation. • Project Leyden will address startup time, memory footprint and peak performance by introducing a concept of static images to the Java Platform, and to the JDK. • The changes to the release cycle mean that innovation will be delivered faster.

Performance-Tweaks für Java-Serverless-Anwendungen (IT-Tage)

Performance-Tweaks für Java-Serverless-Anwendungen (IT-Tage)

More Decks by Dennis Kieselhorst

Other Decks in Programming

Featured

Transcript