Effectively Using Java on Serverless and Containers

Slide 1

Slide 1 text

Slide 2

Slide 2 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Effectively Using Java on Serverless and Containers Sascha Möllering D O P 3 0 5 (he/ him) Principal Solutions Architect, Containers AWS Dennis Kieselhorst (he/ him) Principal Solutions Architect AWS

Slide 3

Slide 3 text

Slide 4

Slide 4 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why are we here? • Lots of Java applications and talented people around • Large ecosystem in terms of tooling and libraries • Java Community Process (JCP) helps to continously evolve the language Source: https://pypl.github.io

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tune your function's resources % of CPU core and network capacity are allocated to a function proportionally Is your code CPU, network or memory-bound? If so, it could be cheaper to increase memory 10240 MB 128 MB AWS Lambda exposes a memory control in 1 MB increments

Slide 9

Slide 9 text

Slide 10

Slide 10 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Leverage multithreading I/O bound workloads will likely see gains by multithreading < 1 vCPU: CPU bound workloads won't see gains > 1 vCPU: you can leverage multiple vCPUs 10240 MB 128 MB 6 vCPU 1 vCPU 1769 MB

Slide 11

Slide 11 text

Slide 12

Slide 12 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build and deploy Time The lifecycle of your AWS Lambda function Handler invocation Initialize Load and initialize handler JVM start Run handler code Compile Package Deploy Download code

Slide 13

Slide 13 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started – simpler programming model • Scales to zero if there is no request – Managed auto scaling 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution The lifecycle of your AWS Lambda function Env #1 Env #2

Slide 14

Slide 14 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Cold start Download code Warm start Build and deploy Handler invocation Initialize

Slide 15

Slide 15 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Boosted host CPU access Build and deploy Handler invocation Initialize up to 10 seconds

Slide 16

Slide 16 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Build and deploy Handler invocation Initialize up to 10 seconds Avoid: normal Lambda charges incur

Slide 17

Slide 17 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Build and deploy Handler invocation Initialize

Slide 18

Slide 18 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configure clients at build-time or at initialization • Optimize metadata discovery • Move client code to constructor or static initializers

Slide 19

Slide 19 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS SDK for Java 2.0 • Rewrite of version 1.0 • Non-blocking I/O • Strong focus on consistency, immutability, and ease of use • Pluggable HTTP client implementations: • NettyNioAsyncClient • ApacheHttpClient • HttpUrlConnectionClient • AWS CRT HTTP Client

Slide 20

Slide 20 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Build and deploy Handler invocation Initialize Dependency Injection

Slide 21

Slide 21 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Avoid dependency injection • Dependency injection adds dependencies Annotation scanning adds more class loading • If required, move dependency injection to the initialize phase. The aws-serverless-java-container project provides wrappers for lift-and- shift migrations of Spring and other frameworks • Use build-time dependency injection, e.g. Dagger

Slide 22

Slide 22 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Reflection and Class Loading Dependency Injection Build and deploy Handler invocation Initialize

Slide 23

Slide 23 text

Slide 24

Slide 24 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart overview Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke microVM snapshot technology

Slide 25

Slide 25 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart applies to published versions AWS Lambda Amazon API Gateway arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring:prod Alias arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring:1 Version arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring $LATEST

Slide 26

Slide 26 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Enable AWS Lambda SnapStart (e.g. in AWS SAM) Transform: AWS::Serverless-2016-10-31 Description: Lambda Function with SnapStart and versions Resources: HelloWorldFunction: Type: AWS::Serverless::Function Properties: CodeUri: Handler: Runtime: java21 AutoPublishAlias: live SnapStart: ApplyOn: PublishedVersions

Slide 27

Slide 27 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Invoke Resume Invoke Resume Invoke Resume Invoke Resume Invoke Resume The AWS Lambda SnapStart lifecycle Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke Invoke Resume Invoke […]

Slide 28

Slide 28 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke Load and initialize handler

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle Run handler code Compile Package Deploy Configure API clients Reflection and Class Loading Dependency Injection JIT Compilation Build and Deploy Handler Invocation Initialize After Restore() Before Snapshot() JVM start Download code Load and initialize handler Publish Invoke

Slide 32

Slide 32 text

Slide 33

Slide 33 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Common issues • Applies to Amazon ECS and Amazon EKS • Slow to start up • High CPU consumption • OOM killed due to high memory usage

Slide 34

Slide 34 text

Slide 35

Slide 35 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1 2 Request Request Running application 3 Request … • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually Java in a container Initialization

Slide 36

Slide 36 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Topics for today • Reduce container image size • Ahead-of-time compilation • Coordinated Restore at Checkpoint (CRaC) • Optimize price-performance with AWS Graviton Kernel Base Image Image Image Container w ritable Add site Add app AM ZN Linux references parent image

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compress image using zstd • Container images distributed as a bundle of layers • Container image builders use gzip • Alternate compression: zstd • zstd is a compression algorithm developed by Meta

Slide 40

Slide 40 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compress image using zstd • Internal testing with AWS Fargate • Up to a 27% reduction in startup times • Larger container images, greatest improvement

Slide 41

Slide 41 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Container image optimization with Jib • Tool to build optimized OCI images • No container runtime • Maven or Gradle-plugin, Java library as well • Jib separates Java application into multiple layers

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Slide 44

Slide 44 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compile your application ahead-of-time (AOT) • GraalVM creates a native image of your application at build- time • Frameworks like Quarkus, Helidon, and Micronaut integrate GraalVM and add further benefits like build-time dependency injection • Some JVM features not supported • Application startup time reduced of more than 90%

Slide 45

Slide 45 text

Slide 46

Slide 46 text

Slide 47

Slide 47 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 50 What is CRaC? • Coordinated Restore at Checkpoint (CRaC), OpenJDK project by Azul • Capture a checkpoint when warmed up, then uses it to launch • AWS Lambda SnapStart uses CRaC API for runtime hooks

Slide 48

Slide 48 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. What about CRaC? 1. Create a container image 2. Start application 3. Create the checkpoint 4. Create container image with checkpoint files 5. Run the container from the checkpoint

Slide 49

Slide 49 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build AWS CodeCommit AWS CodePipeline Amazon ECR Amazon EKS 2 Source Action 1 4 kubectl apply … (CD/GitOps) Build JAR Create container image Push to ECR Checkout source code With CRaC Checkpoint Run container Warm up & capture checkpoint Create container image with CRaC checkpoint files Push to ECR AWS CodePipeline 3 What about CRaC?

Slide 50

Slide 50 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. CRaC performance results Deployment Checkpoint files size (MB, uncompressed) Image size in Amazon ECR (MB) Time to download Checkpoint files (Seconds) Startup time (Seconds) Total startup time (Seconds) No CRaC – 354.3 – 19 19 CRaC – Container image 184 389.91 (contains CRaC files) – 2.5 2.5 CRaC – EFS 184 354.3 – 4.2 4.2 CRaC – S3 CLI 184 467.71 (contains AWS CLI) 7.5 2.5 10

Slide 51

Slide 51 text

Slide 52

Slide 52 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-arch build pipeline Graviton EC2 instance/CodeBuild x86 EC2 instance/CodeBuild Graviton EC2 instance/CodeBuild Code repo Build and push (x86) Build and push (arm64) Build/push multi- arch manifest Deploy Test (x86) Test (arm64) AWS CodeCommit AWS CodePipeline Jenkins Cirrus CI Azure Pipelines Jenkins logo by Frontside / CC BY-SA 3.0 Travis CI

Slide 53

Slide 53 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Map of Container runtime optimizations Effectiveness Easiness Ahead of Time Compilation Container Image size reduction ARM64 Application optimizations CRaC

Slide 54

Slide 54 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Map of AWS Lambda runtime optimizations Effectiveness Easiness Monitoring Power Tuning Ahead of Time Compilation Garbage Collection Monitoring/Tuning API Client Configuration Tiered Compilation Setting Multithreading AWS SDK v2 AWS Lambda SnapStart

Slide 55

Slide 55 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Q Developer Reimagines the experience across the entire software delivery lifecycle Helps developers and IT professionals build and manage secure, scalable, and highly available applications Helps you write, debug, test, optimize, and upgrade your code faster Converses with you to explore new AWS capabilities, learn unfamiliar technologies, and architect solutions Amazon Q is built with security and privacy in mind from the start, making it easier for organizations to use generative AI safely.

Slide 56

Slide 56 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build and test (Java 8) Apply knowledge base Build and test (Java 17) Fix errors Upgraded code Source code M O D E R N I Z E L A N G U A G E V E R S I O N S I N A F R A C T I O N O F T H E T I M E Maintain and modernize: Amazon Q code transformation Knowledge DB

Slide 57

Slide 57 text

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

© 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! Please complete the session survey in the mobile app Sascha Möllering smoell@amazon.de @sascha242 linkedin.com/in/smoell Dennis Kieselhorst dkieselh@amazon.de linkedin.com/in/kieselhorst