Performance-Tweaks für Java-Serverless-Anwendungen (Java Forum Nord 2023)

JAVA FORUM NORD 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023,
Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Performance-Tweaks für Java-Serverless- Anwendungen Dennis Kieselhorst Principal Solutions Architect Amazon Web Services

Amazon Web Services, Inc. or its affiliates. All rights reserved. 5 1 2 Request Request Initialize application Running application 3 Request … • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually Java in a container or a VM Initialization

Amazon Web Services, Inc. or its affiliates. All rights reserved. 6 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution Initialize Application Initialize Application Java with Serverless functions • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started – simpler programming model • Scales to zero if there is no request – Managed auto scaling

Amazon Web Services, Inc. or its affiliates. All rights reserved. What are AWS Lambda functions? 7 Handler function • Function executed on invocation • Processes incoming event Event • Invocation data sent to function • JSON event payload differs by source (API, State change, Queue) Context • Additional information from the service • Example: Request ID

Amazon Web Services, Inc. or its affiliates. All rights reserved. Tune the serverless function's resources… AWS Lambda exposes only a memory control, with the % of CPU core and network capacity allocated to a function proportionally. Is your code CPU, network or memory-bound? If so, it could be cheaper to increase memory. > Memory, > vCPU, > Network 10240 MB 128 MB

Amazon Web Services, Inc. or its affiliates. All rights reserved. Leverage multithreading I/O bound workloads will likely see gains by multithreading. < 1 vCPU: CPU bound workloads won't see gains > 1 vCPU: you can leverage multiple vCPUs 10240 MB 128 MB 6 vCPU 1 vCPU 1769 MB

Amazon Web Services, Inc. or its affiliates. All rights reserved. Custom AWS silicon with 64-bit Arm cores Targeted optimizations to reduce costs Rapidly innovate, build, and iterate Range of instance types Same AWS Nitro building blocks Best performance per watt of energy use in Amazon EC2 AWS Graviton2 & Graviton3 X86 vs. ARM – AWS Graviton processor

Amazon Web Services, Inc. or its affiliates. All rights reserved. The lifecycle of your AWS Lambda function Time Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Build and deploy Initialize Handler invocation

Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Cold start Download code Warm start Build and deploy Handler invocation Initialize

Amazon Web Services, Inc. or its affiliates. All rights reserved. Provisioned Concurrency • Managed solution to avoid „cold starts“ • You set provisioned concurrency for individual functions • No code changes required • Integrated with AWS Auto Scaling • Additional costs 14

Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Boosted host CPU access Build and deploy Handler invocation Initialize up to 10 seconds

Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Build and deploy Handler invocation Initialize up to 10 seconds Avoid: normal Lambda charges incur

Amazon Web Services, Inc. or its affiliates. All rights reserved. Monitor and tune the Garbage Collection Monitor the Garbage Collection logs, try a fixed heap size specific to your application. Example shows 100 calls with heap size 400 MB: • 5% call duration reduction • Only minor garbage collection activities

Amazon Web Services, Inc. or its affiliates. All rights reserved. Tiered Compilation Setting Set JVM tiered compilation level to 1 and use the C1 compiler. JAVA_TOOL_OPTIONS= -XX:+TieredCompilation -XX:TieredStopAtLevel=1 C1 compiler quickly produces native code. No profiling overhead, but no benefits from more aggressive C2 compilation. related blogpost Enabled by default in AWS Lambda Java 17 runtime!

Amazon Web Services, Inc. or its affiliates. All rights reserved. Dependencies • Trade-off between feature richness & performance • Examples ▪ jackson-jr for serializiation ▪ SimpleLogger for logging ▪ Spring FunctionHandler<Event> instead of RestController (directly handle API-Gateway events instead of HTTP) • Be careful with dependency injection: Annotation scanning adds more class loading. 20

Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS SDK and Common Runtime Client • As of today there are two version of the AWS SDK for Java (AWS SDK for Java v1 and AWS SDK for Java v2 ). • We recommend to use the latest version AWS SDK for Java v2 because it has been designed to improve the performance and usability. • The AWS Common Runtime (CRT) HTTP client is a new HTTP client you can use with the AWS SDK for Java v2. The CRT-based HTTP client is an asynchronous, non-blocking HTTP client built on top of the Java bindings of the AWS Common Runtime. 21

Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart Up to 10x faster startup performance NEW

Amazon Web Services, Inc. or its affiliates. All rights reserved. Invoke Resume Invoke Resume Invoke Resume Invoke Resume Invoke Resume The AWS Lambda SnapStart lifecycle Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke Invoke Resume Invoke […]

Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke Load and initialize handler Resize Heap

Amazon Web Services, Inc. or its affiliates. All rights reserved. Runtime Hooks • Runtime hooks are available as part of the open-source Coordinated Restore at Checkpoint (CRaC) project. • Include dependency (e.g. Maven) <dependency> <groupId>io.github.crac</groupId> <artifactId>org-crac</artifactId> <version>0.1.3</version> </dependency> 27

Amazon Web Services, Inc. or its affiliates. All rights reserved. CRaC interface for post- and pre-hooks examples - beforeCheckpoint - priming - class loading - dependency injection - afterRestore - unique ID for execution environment - re-establish connections

Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke After Restore() Before Snapshot() Load and initialize handler Resize Heap

Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle Run handler code Compile Package Deploy Configure API clients Reflection and Class Loading Dependency Injection JIT Compilation Build and Deploy Handler Invocation Initialize After Restore() Before Snapshot() JVM start Download code Load and initialize handler Resize Heap Publish Invoke

Amazon Web Services, Inc. or its affiliates. All rights reserved. SnapStart benefits • Start your functions from pre-initialized snapshots • No additional cost • Minimal to no code changes • Customization via Runtime Hooks 31 Without SnapStart With SnapStart 6.725 - 80 % 1.15 Full Duration Results will vary based on your application

Amazon Web Services, Inc. or its affiliates. All rights reserved. 32 Database connection pooling • Database connection pools (e.g. HikariCP) initialize multiple database connections during startup • Since an AWS Lambda environment handles a single request at a time you only need 1 connection per environment • Ensure to stay within database connection pool limits • Solution: RDS Proxy or Reserved Concurrency (Limit connections) … Container

Amazon Web Services, Inc. or its affiliates. All rights reserved. Powertools for AWS Lambda 33 J A V A L I B R A R Y T O S I M P L I F Y T R A C I N G , S T R U C T U R E D L O G G I N G A N D C U S T O M M E T R I C S • Facilitate best practice adoption • Increase developer velocity • Optional modules: • Parameter handling • Idempotency • SQS Large Message and Batching Optiongs • Serialization, Validation and custom resource support

Amazon Web Services, Inc. or its affiliates. All rights reserved. Compile your application ahead-of-time (AOT) • GraalVM creates a native image of your application at build-time. • Frameworks like Quarkus, Micronaut, and Spring Native integrate GraalVM and add further benefits like build-time dependency injection. • Some JVM features not supported. • Application startup time reduced of more than 90%

Amazon Web Services, Inc. or its affiliates. All rights reserved. GraalVM native images on AWS Lambda • Create a native binary and deploy as AWS Lambda custom runtime • Sub-second application startup and low memory footprint • Multiple frameworks provide built-in support (Micronaut, Quarkus, Spring) 35 bootstrap

Amazon Web Services, Inc. or its affiliates. All rights reserved. Future • The Java ecosystem is adapting and introduced several new projects to foster innovation. • Project Leyden will address startup time, memory footprint and peak performance by introducing a concept of static images to the Java Platform, and to the JDK. • The changes to the release cycle mean that innovation will be delivered faster. 37

Performance-Tweaks für Java-Serverless-Anwendun...

Performance-Tweaks für Java-Serverless-Anwendungen (Java Forum Nord 2023)

More Decks by Dennis Kieselhorst

Other Decks in Programming

Featured

Transcript