Effectively Using Java on Serverless and Containers

2020 marked the 25th anniversary of the Java programming language. One of the most widely used programming languages, Java is used as the server-side language for many back-end development projects. This session is intended for Java developers who like to optimize their applications for AWS Lambda and containers. We will discuss various tweaks and recommendations for each architecture to improve all phases of the development lifecycle – during build-, deploy-, and runtime. For this, we will revisit best practices from the past and look at new ones to reduce the cold start time, speed up execution, and improve monitoring. Before we finish the session, we will demonstrate how you can leverage GraalVM to optimize the application even further.

Transcript

1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. B E R L I N | 1 5 + 1 6 M A Y 2 0 2 4
2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Effectively Using Java on Serverless and Containers Sascha Möllering D O P 3 0 5 (he/ him) Principal Solutions Architect, Containers AWS Dennis Kieselhorst (he/ him) Principal Solutions Architect AWS
3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Java Containers AWS Lambda This Session
4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why are we here? • Lots of Java applications and talented people around • Large ecosystem in terms of tooling and libraries • Java Community Process (JCP) helps to continously evolve the language Source: https://pypl.github.io
5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Measure. Then optimize. There‘s no silver bullet.
6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimize AWS Lambda
7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Java 21 Develop AWS Lambda functions with the Amazon Corretto distribution of Java 21 NEW
8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tune your function's resources % of CPU core and network capacity are allocated to a function proportionally Is your code CPU, network or memory-bound? If so, it could be cheaper to increase memory 10240 MB 128 MB AWS Lambda exposes a memory control in 1 MB increments
9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Get best memory from Lambda Power Tuning
10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Leverage multithreading I/O bound workloads will likely see gains by multithreading < 1 vCPU: CPU bound workloads won't see gains > 1 vCPU: you can leverage multiple vCPUs 10240 MB 128 MB 6 vCPU 1 vCPU 1769 MB
11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimize your code
12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build and deploy Time The lifecycle of your AWS Lambda function Handler invocation Initialize Load and initialize handler JVM start Run handler code Compile Package Deploy Download code
13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started – simpler programming model • Scales to zero if there is no request – Managed auto scaling 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution The lifecycle of your AWS Lambda function Env #1 Env #2
14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Cold start Download code Warm start Build and deploy Handler invocation Initialize
15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Boosted host CPU access Build and deploy Handler invocation Initialize up to 10 seconds
16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Build and deploy Handler invocation Initialize up to 10 seconds Avoid: normal Lambda charges incur
17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Build and deploy Handler invocation Initialize
18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Configure clients at build-time or at initialization • Optimize metadata discovery • Move client code to constructor or static initializers
19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS SDK for Java 2.0 • Rewrite of version 1.0 • Non-blocking I/O • Strong focus on consistency, immutability, and ease of use • Pluggable HTTP client implementations: • NettyNioAsyncClient • ApacheHttpClient • HttpUrlConnectionClient • AWS CRT HTTP Client
20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Build and deploy Handler invocation Initialize Dependency Injection
21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Avoid dependency injection • Dependency injection adds dependencies Annotation scanning adds more class loading • If required, move dependency injection to the initialize phase. The aws-serverless-java-container project provides wrappers for lift-and- shift migrations of Spring and other frameworks • Use build-time dependency injection, e.g. Dagger
22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Reflection and Class Loading Dependency Injection Build and deploy Handler invocation Initialize
23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart Up to 10x faster startup performance
24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart overview Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke microVM snapshot technology
25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda SnapStart applies to published versions AWS Lambda Amazon API Gateway arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring:prod Alias arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring:1 Version arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring $LATEST
26. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Enable AWS Lambda SnapStart (e.g. in AWS SAM) Transform: AWS::Serverless-2016-10-31 Description: Lambda Function with SnapStart and versions Resources: HelloWorldFunction: Type: AWS::Serverless::Function Properties: CodeUri: <code-location> Handler: <handler> Runtime: java21 AutoPublishAlias: live SnapStart: ApplyOn: PublishedVersions
27. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Invoke Resume Invoke Resume Invoke Resume Invoke Resume Invoke Resume The AWS Lambda SnapStart lifecycle Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke Invoke Resume Invoke […]
28. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke Load and initialize handler
29. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Considerations Network connections Ephemeral data Uniqueness
30. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke After Restore() Before Snapshot() Load and initialize handler
31. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Time The AWS Lambda SnapStart lifecycle Run handler code Compile Package Deploy Configure API clients Reflection and Class Loading Dependency Injection JIT Compilation Build and Deploy Handler Invocation Initialize After Restore() Before Snapshot() JVM start Download code Load and initialize handler Publish Invoke
32. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers
33. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Common issues • Applies to Amazon ECS and Amazon EKS • Slow to start up • High CPU consumption • OOM killed due to high memory usage
34. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. What is initialization? Start JVM Initialize Context and Classes Retrieve Container Image Run method Run method …
35. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 1 2 Request Request Running application 3 Request … • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually Java in a container Initialization
36. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Topics for today • Reduce container image size • Ahead-of-time compilation • Coordinated Restore at Checkpoint (CRaC) • Optimize price-performance with AWS Graviton Kernel Base Image Image Image Container w ritable Add site Add app AM ZN Linux references parent image
37. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reduce container image size
38. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimizing your container • Optimize for smaller size • Use a minimalist operating system • Not all runtimes are equal!
39. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compress image using zstd • Container images distributed as a bundle of layers • Container image builders use gzip • Alternate compression: zstd • zstd is a compression algorithm developed by Meta
40. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compress image using zstd • Internal testing with AWS Fargate • Up to a 27% reduction in startup times • Larger container images, greatest improvement
41. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Container image optimization with Jib • Tool to build optimized OCI images • No container runtime • Maven or Gradle-plugin, Java library as well • Jib separates Java application into multiple layers
42. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Container image optimization with Jib
43. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Ahead-of-time compilation
44. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compile your application ahead-of-time (AOT) • GraalVM creates a native image of your application at build- time • Frameworks like Quarkus, Helidon, and Micronaut integrate GraalVM and add further benefits like build-time dependency injection • Some JVM features not supported • Application startup time reduced of more than 90%
45. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Spring Boot 3 example 1 2
46. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Coordinated Restore at Checkpoint (CRaC)
47. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 50 What is CRaC? • Coordinated Restore at Checkpoint (CRaC), OpenJDK project by Azul • Capture a checkpoint when warmed up, then uses it to launch • AWS Lambda SnapStart uses CRaC API for runtime hooks
48. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. What about CRaC? 1. Create a container image 2. Start application 3. Create the checkpoint 4. Create container image with checkpoint files 5. Run the container from the checkpoint
49. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build AWS CodeCommit AWS CodePipeline Amazon ECR Amazon EKS 2 Source Action 1 4 kubectl apply … (CD/GitOps) Build JAR Create container image Push to ECR Checkout source code With CRaC Checkpoint Run container Warm up & capture checkpoint Create container image with CRaC checkpoint files Push to ECR AWS CodePipeline 3 What about CRaC?
50. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. CRaC performance results Deployment Checkpoint files size (MB, uncompressed) Image size in Amazon ECR (MB) Time to download Checkpoint files (Seconds) Startup time (Seconds) Total startup time (Seconds) No CRaC – 354.3 – 19 19 CRaC – Container image 184 389.91 (contains CRaC files) – 2.5 2.5 CRaC – EFS 184 354.3 – 4.2 4.2 CRaC – S3 CLI 184 467.71 (contains AWS CLI) 7.5 2.5 10
51. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimize price-performance with AWS Graviton
52. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Multi-arch build pipeline Graviton EC2 instance/CodeBuild x86 EC2 instance/CodeBuild Graviton EC2 instance/CodeBuild Code repo Build and push (x86) Build and push (arm64) Build/push multi- arch manifest Deploy Test (x86) Test (arm64) AWS CodeCommit AWS CodePipeline Jenkins Cirrus CI Azure Pipelines Jenkins logo by Frontside / CC BY-SA 3.0 Travis CI
53. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Map of Container runtime optimizations Effectiveness Easiness Ahead of Time Compilation Container Image size reduction ARM64 Application optimizations CRaC
54. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Map of AWS Lambda runtime optimizations Effectiveness Easiness Monitoring Power Tuning Ahead of Time Compilation Garbage Collection Monitoring/Tuning API Client Configuration Tiered Compilation Setting Multithreading AWS SDK v2 AWS Lambda SnapStart
55. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Q Developer Reimagines the experience across the entire software delivery lifecycle Helps developers and IT professionals build and manage secure, scalable, and highly available applications Helps you write, debug, test, optimize, and upgrade your code faster Converses with you to explore new AWS capabilities, learn unfamiliar technologies, and architect solutions Amazon Q is built with security and privacy in mind from the start, making it easier for organizations to use generative AI safely.
56. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Build and test (Java 8) Apply knowledge base Build and test (Java 17) Fix errors Upgraded code Source code M O D E R N I Z E L A N G U A G E V E R S I O N S I N A F R A C T I O N O F T H E T I M E Maintain and modernize: Amazon Q code transformation Knowledge DB
57. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Q Code Transformation Automates the end-to-end process of upgrading and transforming code
58. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Java applications Minutes each, on average Days Amazon Q Code Transformation
59. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. If you want to learn more …
60. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! Please complete the session survey in the mobile app Sascha Möllering smoell@amazon.de @sascha242 linkedin.com/in/smoell Dennis Kieselhorst dkieselh@amazon.de linkedin.com/in/kieselhorst