Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Effectively Using Java on Serverless and Containers

Effectively Using Java on Serverless and Containers

2020 marked the 25th anniversary of the Java programming language. One of the most widely used programming languages, Java is used as the server-side language for many back-end development projects. This session is intended for Java developers who like to optimize their applications for AWS Lambda and containers. We will discuss various tweaks and recommendations for each architecture to improve all phases of the development lifecycle – during build-, deploy-, and runtime. For this, we will revisit best practices from the past and look at new ones to reduce the cold start time, speed up execution, and improve monitoring. Before we finish the session, we will demonstrate how you can leverage GraalVM to optimize the application even further.

Dennis Kieselhorst

May 15, 2024
Tweet

More Decks by Dennis Kieselhorst

Other Decks in Programming

Transcript

  1. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. B E R L I N | 1 5 + 1 6 M A Y 2 0 2 4
  2. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Effectively Using Java on Serverless and Containers Sascha Möllering D O P 3 0 5 (he/ him) Principal Solutions Architect, Containers AWS Dennis Kieselhorst (he/ him) Principal Solutions Architect AWS
  3. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Java Containers AWS Lambda This Session
  4. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Why are we here? • Lots of Java applications and talented people around • Large ecosystem in terms of tooling and libraries • Java Community Process (JCP) helps to continously evolve the language Source: https://pypl.github.io
  5. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Measure. Then optimize. There‘s no silver bullet.
  6. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Optimize AWS Lambda
  7. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Java 21 Develop AWS Lambda functions with the Amazon Corretto distribution of Java 21 NEW
  8. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Tune your function's resources % of CPU core and network capacity are allocated to a function proportionally Is your code CPU, network or memory-bound? If so, it could be cheaper to increase memory 10240 MB 128 MB AWS Lambda exposes a memory control in 1 MB increments
  9. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Get best memory from Lambda Power Tuning
  10. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Leverage multithreading I/O bound workloads will likely see gains by multithreading < 1 vCPU: CPU bound workloads won't see gains > 1 vCPU: you can leverage multiple vCPUs 10240 MB 128 MB 6 vCPU 1 vCPU 1769 MB
  11. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Optimize your code
  12. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Build and deploy Time The lifecycle of your AWS Lambda function Handler invocation Initialize Load and initialize handler JVM start Run handler code Compile Package Deploy Download code
  13. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started – simpler programming model • Scales to zero if there is no request – Managed auto scaling 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution The lifecycle of your AWS Lambda function Env #1 Env #2
  14. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Cold start Download code Warm start Build and deploy Handler invocation Initialize
  15. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time The lifecycle of your AWS Lambda function Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Boosted host CPU access Build and deploy Handler invocation Initialize up to 10 seconds
  16. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Build and deploy Handler invocation Initialize up to 10 seconds Avoid: normal Lambda charges incur
  17. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Build and deploy Handler invocation Initialize
  18. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Configure clients at build-time or at initialization • Optimize metadata discovery • Move client code to constructor or static initializers
  19. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS SDK for Java 2.0 • Rewrite of version 1.0 • Non-blocking I/O • Strong focus on consistency, immutability, and ease of use • Pluggable HTTP client implementations: • NettyNioAsyncClient • ApacheHttpClient • HttpUrlConnectionClient • AWS CRT HTTP Client
  20. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Build and deploy Handler invocation Initialize Dependency Injection
  21. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Avoid dependency injection • Dependency injection adds dependencies Annotation scanning adds more class loading • If required, move dependency injection to the initialize phase. The aws-serverless-java-container project provides wrappers for lift-and- shift migrations of Spring and other frameworks • Use build-time dependency injection, e.g. Dagger
  22. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time Eliminate or move activities to earlier phases Load and initialize handler JVM start Run handler code Compile Package Deploy Download code Configure API clients Reflection and Class Loading Dependency Injection Build and deploy Handler invocation Initialize
  23. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Lambda SnapStart Up to 10x faster startup performance
  24. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Lambda SnapStart overview Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke microVM snapshot technology
  25. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS Lambda SnapStart applies to published versions AWS Lambda Amazon API Gateway arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring:prod Alias arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring:1 Version arn:aws:lambda:eu-west-1:123456789012:function:unicorn-store-spring $LATEST
  26. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Enable AWS Lambda SnapStart (e.g. in AWS SAM) Transform: AWS::Serverless-2016-10-31 Description: Lambda Function with SnapStart and versions Resources: HelloWorldFunction: Type: AWS::Serverless::Function Properties: CodeUri: <code-location> Handler: <handler> Runtime: java21 AutoPublishAlias: live SnapStart: ApplyOn: PublishedVersions
  27. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Invoke Resume Invoke Resume Invoke Resume Invoke Resume Invoke Resume The AWS Lambda SnapStart lifecycle Publish Init Encrypted snapshot stored Tiered low- latency cache Invoke Invoke Resume Invoke […]
  28. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke Load and initialize handler
  29. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Considerations Network connections Ephemeral data Uniqueness
  30. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time The AWS Lambda SnapStart lifecycle JVM start Run handler code Compile Package Deploy Download code Build and Deploy Handler Invocation Initialize Publish Invoke After Restore() Before Snapshot() Load and initialize handler
  31. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Time The AWS Lambda SnapStart lifecycle Run handler code Compile Package Deploy Configure API clients Reflection and Class Loading Dependency Injection JIT Compilation Build and Deploy Handler Invocation Initialize After Restore() Before Snapshot() JVM start Download code Load and initialize handler Publish Invoke
  32. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Common issues • Applies to Amazon ECS and Amazon EKS • Slow to start up • High CPU consumption • OOM killed due to high memory usage
  33. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. What is initialization? Start JVM Initialize Context and Classes Retrieve Container Image Run method Run method …
  34. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 1 2 Request Request Running application 3 Request … • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually Java in a container Initialization
  35. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Topics for today • Reduce container image size • Ahead-of-time compilation • Coordinated Restore at Checkpoint (CRaC) • Optimize price-performance with AWS Graviton Kernel Base Image Image Image Container w ritable Add site Add app AM ZN Linux references parent image
  36. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Reduce container image size
  37. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Optimizing your container • Optimize for smaller size • Use a minimalist operating system • Not all runtimes are equal!
  38. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Compress image using zstd • Container images distributed as a bundle of layers • Container image builders use gzip • Alternate compression: zstd • zstd is a compression algorithm developed by Meta
  39. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Compress image using zstd • Internal testing with AWS Fargate • Up to a 27% reduction in startup times • Larger container images, greatest improvement
  40. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Container image optimization with Jib • Tool to build optimized OCI images • No container runtime • Maven or Gradle-plugin, Java library as well • Jib separates Java application into multiple layers
  41. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Container image optimization with Jib
  42. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Ahead-of-time compilation
  43. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Compile your application ahead-of-time (AOT) • GraalVM creates a native image of your application at build- time • Frameworks like Quarkus, Helidon, and Micronaut integrate GraalVM and add further benefits like build-time dependency injection • Some JVM features not supported • Application startup time reduced of more than 90%
  44. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Spring Boot 3 example 1 2
  45. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Coordinated Restore at Checkpoint (CRaC)
  46. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 50 What is CRaC? • Coordinated Restore at Checkpoint (CRaC), OpenJDK project by Azul • Capture a checkpoint when warmed up, then uses it to launch • AWS Lambda SnapStart uses CRaC API for runtime hooks
  47. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. What about CRaC? 1. Create a container image 2. Start application 3. Create the checkpoint 4. Create container image with checkpoint files 5. Run the container from the checkpoint
  48. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Build AWS CodeCommit AWS CodePipeline Amazon ECR Amazon EKS 2 Source Action 1 4 kubectl apply … (CD/GitOps) Build JAR Create container image Push to ECR Checkout source code With CRaC Checkpoint Run container Warm up & capture checkpoint Create container image with CRaC checkpoint files Push to ECR AWS CodePipeline 3 What about CRaC?
  49. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. CRaC performance results Deployment Checkpoint files size (MB, uncompressed) Image size in Amazon ECR (MB) Time to download Checkpoint files (Seconds) Startup time (Seconds) Total startup time (Seconds) No CRaC – 354.3 – 19 19 CRaC – Container image 184 389.91 (contains CRaC files) – 2.5 2.5 CRaC – EFS 184 354.3 – 4.2 4.2 CRaC – S3 CLI 184 467.71 (contains AWS CLI) 7.5 2.5 10
  50. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Optimize price-performance with AWS Graviton
  51. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Multi-arch build pipeline Graviton EC2 instance/CodeBuild x86 EC2 instance/CodeBuild Graviton EC2 instance/CodeBuild Code repo Build and push (x86) Build and push (arm64) Build/push multi- arch manifest Deploy Test (x86) Test (arm64) AWS CodeCommit AWS CodePipeline Jenkins Cirrus CI Azure Pipelines Jenkins logo by Frontside / CC BY-SA 3.0 Travis CI
  52. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Map of Container runtime optimizations Effectiveness Easiness Ahead of Time Compilation Container Image size reduction ARM64 Application optimizations CRaC
  53. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Map of AWS Lambda runtime optimizations Effectiveness Easiness Monitoring Power Tuning Ahead of Time Compilation Garbage Collection Monitoring/Tuning API Client Configuration Tiered Compilation Setting Multithreading AWS SDK v2 AWS Lambda SnapStart
  54. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Q Developer Reimagines the experience across the entire software delivery lifecycle Helps developers and IT professionals build and manage secure, scalable, and highly available applications Helps you write, debug, test, optimize, and upgrade your code faster Converses with you to explore new AWS capabilities, learn unfamiliar technologies, and architect solutions Amazon Q is built with security and privacy in mind from the start, making it easier for organizations to use generative AI safely.
  55. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Build and test (Java 8) Apply knowledge base Build and test (Java 17) Fix errors Upgraded code Source code M O D E R N I Z E L A N G U A G E V E R S I O N S I N A F R A C T I O N O F T H E T I M E Maintain and modernize: Amazon Q code transformation Knowledge DB
  56. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Amazon Q Code Transformation Automates the end-to-end process of upgrading and transforming code
  57. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Java applications Minutes each, on average Days Amazon Q Code Transformation
  58. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. If you want to learn more …
  59. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Thank you! Please complete the session survey in the mobile app Sascha Möllering [email protected] @sascha242 linkedin.com/in/smoell Dennis Kieselhorst [email protected] linkedin.com/in/kieselhorst