Slide 1

Slide 1 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. #ittage Dennis Kieselhorst Maximilian Schellhorn Performance-Tweaks für Java-Serverless-Anwendungen

Slide 2

Slide 2 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Serverful architecture Application Load Balancer Web app container Container target group HTTP server Framework Application code Spring Boot application

Slide 3

Slide 3 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Event Changes in data state Requests to endpoints Changes in resource state Serverless architecture Application code AWS Lambda function

Slide 4

Slide 4 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Functions are invoked by events • AWS Lambda is invoked via events (API Event, S3 Object Event, SQS Event ..) • Events follow a certain structure • This is different from accepting a HTTP connection Amazon API Gateway AWS Lambda POST /v1/pets HTTP/2 Host: x.execute-api.eu-west-1.. User-Agent: curl/7.64.1 Accept: */* Content-Type: application/json Content-Length: 39 Body: {“data:“ : “test“} { "body": “{“data“: “test“}“ , "resource": "/{proxy+}", "path": "/path/to/resource", "httpMethod": "POST", "isBase64Encoded": true, "headers": { "Accept-Encoding": "gzip", "Accept-Language": "en-US,en;q=0.8" }, }, "requestContext": { "accountId": "123456789012", "requestId": "c6af9ac6-7b61-..", ... } }

Slide 5

Slide 5 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Execution Model Amazon SNS Amazon S3 reqs Amazon Kinesis changes AWS Lambda service Amazon API Gateway Lambda function Lambda function Synchronous Asynchronous Poll Amazon SQS AWS Lambda Function URLs

Slide 6

Slide 6 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. What are Lambda functions? Handler function • Function executed on invocation • Java 8, 11, 17, 21 supported Event • Invocation data sent to function • JSON event payload differs by source (API, State change, Queue) Context • Additional information from the service • Example: Request ID

Slide 7

Slide 7 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tune the serverless function's resources… AWS Lambda exposes only a memory control, with the % of CPU core and network capacity allocated to a function proportionally. Is your code CPU, network or memory-bound? If so, it could be cheaper to increase memory. > Memory, > vCPU, > Network 10240 MB 128 MB

Slide 8

Slide 8 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. …with Power Tuning https://github.com/alexcasalboni/aws-lambda-power-tuning

Slide 9

Slide 9 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Leverage multithreading I/O bound workloads will likely see gains by multithreading. < 1 vCPU: CPU bound workloads won't see gains > 1 vCPU: you can leverage multiple vCPUs 10240 MB 128 MB 6 vCPU 1 vCPU 1769 MB

Slide 10

Slide 10 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1 2 Request Request Initialize Application Running application 3 Request … • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually Java in a container or a VM Initialization

Slide 11

Slide 11 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Execution Environment ready? Download Code Start Runtime Initialize Function Code Code execution Request Code execution No Yes Create Execution Environment Execution Environment Cold start Warm start AWS Lambda request handling

Slide 12

Slide 12 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution Java with Serverless functions using AWS Lambda • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started – simpler programming model • Scales to zero if there is no request – Managed auto scaling

Slide 13

Slide 13 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Lambda Performance Tweaks

Slide 14

Slide 14 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9 Tiered Compilation Compile Time frameworks No framework No optimization Lightw. deps Function handler GraalVM Effort to modernize Cold-start (seconds) Provisioned Concurrency Performance optimizations (before 2022) Default with Java 17 & 21 on AWS Lambda

Slide 15

Slide 15 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda Provisioned Concurrency • Sets floor on minimum number of execution environments • Pre-warm execution environments to reduce cold-start impact • Can save costs in certain situations

Slide 16

Slide 16 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tiered Compilation Setting Set JVM tiered compilation level to 1 and use the C1 compiler. JAVA_TOOL_OPTIONS= -XX:+TieredCompilation -XX:TieredStopAtLevel=1 C1 compiler quickly produces native code. No profiling overhead, but no benefits from more aggressive C2 compilation. related blogpost Enabled by default in AWS Lambda Java 17 & 21 runtime!

Slide 17

Slide 17 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Dependencies • Trade-off between feature richness & performance • Examples § jackson-jr for serializiation § SimpleLogger for logging § Spring FunctionHandler instead of RestController (directly handle API-Gateway events instead of HTTP) • Be careful with dependency injection: Annotation scanning adds more class loading.

Slide 18

Slide 18 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS SDK and Common Runtime Client • As of today there are two version of the AWS SDK for Java (AWS SDK for Java v1 and AWS SDK for Java v2 ). • We recommend to use the latest version AWS SDK for Java v2 because it has been designed to improve the performance and usability. • The AWS Common Runtime (CRT) HTTP client is a new HTTP client you can use with the AWS SDK for Java v2. The CRT-based HTTP client is an asynchronous, non-blocking HTTP client built on top of the Java bindings of the AWS Common Runtime.

Slide 19

Slide 19 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart Up to 10x faster startup performance

Slide 20

Slide 20 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart • Takes a snapshot of the memory and disk state • Snapshot encryption & caching for low-latency access • Creates new Lambda environments from cached snapshot • Fully Managed microVM snapshot technology

Slide 21

Slide 21 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart - 1. Publication Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke

Slide 22

Slide 22 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Invoke Resume Invoke Resume Invoke Resume Invoke Resume Invoke Resume AWS Lambda SnapStart – 2. Invocation Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke Invoke Resume Invoke […]

Slide 23

Slide 23 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Code execution Invoke Resume Resume Snapshot Post Snapshot Hook (optional) Pre Snapshot Hook (optional) Create Execution Environment Init during deployment Download Code Start Runtime Initialize Function Code Create Snapshot first request How AWS Lambda SnapStart works

Slide 24

Slide 24 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Considerations Network connections Ephemeral data Uniqueness

Slide 25

Slide 25 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Runtime Hooks • Runtime hooks are available as part of the open-source Coordinated Restore at Checkpoint (CRaC) project. • Include dependency (e.g. Maven) org.crac crac 1.4.0

Slide 26

Slide 26 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. CRaC interface for post- and pre-hooks examples - beforeCheckpoint - priming - class loading - dependency injection - afterRestore - unique ID for execution environment - re-establish connections

Slide 27

Slide 27 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke After Restore() Before Snapshot() Load and initialize handler Resize Heap

Slide 28

Slide 28 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle Run handler code Compile Package Deploy Configure API clients Reflection and Class Loading Dependency Injection JIT Compilation Build and Deploy Handler Invocation Initialize After Restore() Before Snapshot() JVM start Download code Load and initialize handler Resize Heap Publish Invoke

Slide 29

Slide 29 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart benefits • Start your functions from pre-initialized snapshots • No additional cost • Minimal to no code changes • Customization via Runtime Hooks Without SnapStart With SnapStart 6.725 - 80 % 1.15 Full Duration Results will vary based on your application

Slide 30

Slide 30 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9 Tiered Compilation Compile Time frameworks No framework No optimization Lightw. deps Function handler GraalVM Effort to modernize Cold-start (seconds) Provisioned Concurrency AWS Lambda performance optimizations SnapStart Default with Java 17 & 21 on AWS Lambda

Slide 31

Slide 31 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compile your application ahead-of-time (AOT) • GraalVM creates a native image of your application at build-time. • Frameworks like Quarkus, Micronaut, and Spring Native integrate GraalVM and add further benefits like build-time dependency injection. • Some JVM features not supported. • Application startup time reduced of more than 90%

Slide 32

Slide 32 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. GraalVM native images on AWS Lambda • Create a native binary and deploy as AWS Lambda custom runtime • Sub-second application startup and low memory footprint • Multiple frameworks provide built-in support (Micronaut, Quarkus, Spring) bootstrap

Slide 33

Slide 33 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Additional considerations

Slide 34

Slide 34 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Helpful frameworks Spring • Well-known framework for Java based Enterprise applications • Spring Boot 3.2 now supports CRaC • Spring Cloud for AWS: awspring.io • Major Contributor: VMware (now Broadcom) Quarkus • Container first (Kube native) but also has extensions for AWS and other Cloud providers • Fast boot time and low footprint • Jakarta EE standard compatible • Major Contributor: Red Hat Micronaut • Majority of work during compile-time • Eliminates reflection, runtime proxies, dynamic class loading • Smaller startup time and memory footprint • AWS CDK and SDK support • Major Contributor: Oracle

Slide 35

Slide 35 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Custom AWS silicon with 64-bit Arm cores Targeted optimizations to reduce costs Rapidly innovate, build, and iterate Range of instance types Same AWS Nitro building blocks Best performance per watt of energy use in Amazon EC2 AWS Graviton2, Graviton3 & Graviton4 X86 vs. ARM – AWS Graviton processor Graviton

Slide 36

Slide 36 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Monitor and tune the Garbage Collection Monitor the Garbage Collection logs, try a fixed heap size specific to your application. Example shows 100 calls with heap size 400 MB: • 5% call duration reduction • Only minor garbage collection activities

Slide 37

Slide 37 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Database connection pooling • Database connection pools (e.g. HikariCP) initialize multiple database connections during startup • Since an AWS Lambda environment handles a single request at a time you only need 1 connection per environment • Ensure to stay within database connection pool limits • Solution: RDS Proxy or Reserved Concurrency (Limit connections) … Container

Slide 38

Slide 38 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Powertools for AWS Lambda J A V A L I B R A R Y T O S I M P L I F Y T R A C I N G , S T R U C T U R E D L O G G I N G A N D C U S T O M M E T R I C S • Facilitate best practice adoption • Increase developer velocity • Optional modules: • Parameter handling • Idempotency • SQS Large Message and Batching Options • Serialization, Validation and custom resource support

Slide 39

Slide 39 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Future • The Java ecosystem is adapting and introduced several new projects to foster innovation. • Project Leyden will address startup time, memory footprint and peak performance by introducing a concept of static images to the Java Platform, and to the JDK. • The changes to the release cycle mean that innovation will be delivered faster.

Slide 40

Slide 40 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Deep Dive: Java on AWS Lambda Workshop https://catalog.workshops.aws/java-on-aws-lambda

Slide 41

Slide 41 text

IT-TAGE 2023 – PERFORMANCE-TWEAKS FÜR JAVA-SERVERLESS-ANWENDUNGEN © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.