Java on AWS Lambda: Differences and challenges

© 2022, Amazon Web Services, Inc. or its aﬃliates. All
rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Java on AWS Lambda: Diﬀerences and challenges Jerome Van Der Linden Sr Solutions Architect Builder @AWS 23/05/2023

rights reserved. A standard Serverful Java / Spring Application Client Application Load Balancer Instances HTTP server Spring Framework Application code Spring Boot application Dispatcher Servlet Handler Mapping Handler Adapter Controller Service Repository You take care of + networking + security + autoscaling + multi-AZ + patching + upgrades + …

rights reserved. No infrastructure provisioning, no management Automatic scaling Pay for value Highly available and secure What is serverless?

rights reserved. AWS Lambda F U L L Y M A N A G E D S E R V E R L E S S C O M P U T E Auto Scaling group Availability Zone A Availability Zone B Availability Zone C Region Instances Instances Instances AWS Lambda ≈ Load Balancer EC2 Instance FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM

rights reserved. AWS Lambda F U L L Y M A N A G E D S E R V E R L E S S C O M P U T E Auto Scaling group Availability Zone A Availability Zone B Availability Zone C Region Instances Instances Instances AWS Lambda ≈ Load Balancer EC2 Instance FireCracker MicroVM Execution Environment Language Runtime Application code You take care of FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM

rights reserved. A standard Serverful Java / Spring Application Client Application Load Balancer Instances HTTP server Spring Framework Application code Spring Boot application Dispatcher Servlet Handler Mapping Handler Adapter Controller Service Repository Always running Always accepting requests

rights reserved. AWS Lambda Auto Scaling group Availability Zone A Availability Zone B Availability Zone C Region Instances Instances Instances AWS Lambda ≈ Load Balancer EC2 Instance FireCracker MicroVM Execution Environment Language Runtime Application code FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM NOT always running Accept 1 request at a time

rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution Env. #1

rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution Execution environment is blocked or busy for this entire time Env. #1

rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution 2 Initialization Execution Env. #1 Env. #2 3 Initialization Execution Env. #3

rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution 2 Initialization Execution Env. #1 Env. #2 3 Initialization Execution Env. #3 Execution 4

rights reserved. AWS Lambda E V E N T - D R I V E N F U N C T I O N S Event source Services (anything) Database AWS Service 3rd party service Function Changes in data state Changes in resource state Request to endpoints Node.js Python C# Go Ruby Runtime API 8 / 11 / 17

rights reserved. AWS Lambda E V E N T - D R I V E N F U N C T I O N S Event source Services(anything) Function com.amazonaws:aws-lambda-java-core:1.2.2 com.amazonaws:aws-lambda-java-events:3.11.1 software.amazon.awssdk:dynamodb:2.20.56 software.amazon.awssdk:s3:2.20.56 software.amazon.awssdk:* Or any Java library you’d like to use

rights reserved. AWS Lambda A N A T O M Y O F A F U N C T I O N public class App implements RequestHandler<Event, Response>{ public Response handleRequest(final Event event, final Context context) { // do some stuff return response; } } • Handler: function that will be triggered for each event received • Event: the incoming event that triggered the function (in JSON format) • Context: informations about the function configuration and invocation • Your code, anything, … • Input type • Output type

rights reserved. AWS Lambda C O M M O N U S E C A S E S Amazon API Gateway AWS Lambda AWS Lambda AWS AppSync APIs / microservices (REST / GraphQL) AWS Lambda AWS Step Functions Orchestration AWS Lambda AWS Config Operations/ Remediation AWS Lambda Amazon SNS AWS Lambda AWS Lambda Amazon EventBridge Amazon SQS Event-driven architectures File processing Stream processing Data processing (transformation , cleansing, …) Analytics IoT backend …

rights reserved. AWS Lambda C O M M O N U S E C A S E : H A N D L I N G H T T P / R E S T R E Q U E S T S Amazon API Gateway AWS Lambda public class App implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> { public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { String name = input.getQueryStringParameters().get("name"); Map<String, String> headers = new HashMap<>(); headers.put("Content-Type", "application/json"); APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent() .withHeaders(headers); try { String output = String.format("{ \"message\": \"hello %s\" }", name); return response .withStatusCode(200) .withBody(output); } catch (IOException e) { return response .withBody("{}") .withStatusCode(500); } } }

rights reserved. Single-purpose functions do “x” do “y” do “z” • Beneﬁts: • Easier to debug, trace errors • Enforce least privilege, one role per function • Simpler to test • Favor reusability • Less surface area • Tips: • Use naming conventions and tags to enhance discoverability • Don’t go nano

rights reserved. Testing • Unit / Integration tests with aws-lambda-java-tests @ParameterizedTest @Event(value = "sqs/sqs_event.json", type = SQSEvent.class) public void testInjectSQSEvent(SQSEvent event) { // test your handleRequest method with this event as parameter } • Local invocation with SAM: sam local invoke MyFunction --event local-event.json

rights reserved. AWS Lambda Supports either Zip or Jar format • Gradle: using Zip build type • Maven: using Shade plugin / Assembly plugin or Dependency plugin P A C K A G I N G https://docs.aws.amazon.com/lambda/latest/dg/java-package.html

rights reserved. AWS Lambda Supports container images • AWS Images for Lambda (lambda/java) • Bring your own image (ex if you want Java 20) • Run and test images locally P A C K A G I N G $ docker build -t docker-image:test . $ docker run –p 9000:8080 docker-image:test $ curl –XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}' https://docs.aws.amazon.com/lambda/latest/dg/java-image.html

rights reserved. AWS Lambda • Using the AWS CLI: aws lambda create-function ... D E P L O Y M E N T

rights reserved. AWS Lambda • Using the AWS CLI: aws lambda create-function ... • Using Infrastructure as code: § AWS SAM D E P L O Y M E N T https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html

rights reserved. AWS Lambda • Using the AWS CLI • Using Infrastructure as code: § AWS SAM § AWS CDK D E P L O Y M E N T https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html

rights reserved. AWS Lambda • Using the AWS CLI • Using Infrastructure as code: § AWS SAM § AWS CDK § Terraform § … D E P L O Y M E N T https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function

rights reserved. Soft Limits C A N B E I N C R E A S E D Concurrency = (average requests per second) * (average request duration in seconds) è (average requests per second) = Concurrency / (average request duration in seconds) Ex for a function that takes 250ms: 1000 / 0.25 ≈ 4000 req / sec https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html

rights reserved. Hard limits AWS Step Functions Amazon S3 Container image Amazon EFS (NFS) No GPU

rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Cold start Warm start Time 1 Initialization Execution Env. #1 Execution 4

rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function You can optimize this AWS optimizes this

rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function You can optimize this

rights reserved. Reducing Cold Starts § Use AWS Java SDK v2 and use only the required dependencies (s3, dynamodb, …) § Remove unnecessary jar dependencies § Use lighweight dependencies (jackson-jr vs jackson, slf4j simple logger vs log4j, …) R E D U C E A S M U C H A S P O S S I B L E T H E S I Z E O F T H E P A C K A G E

rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function You can optimize this

rights reserved. Reducing Cold Starts Use the AWS CRT HTTP Client • faster SDK startup time • smaller memory footprint • reduced latency time U S E T H E A W S C R T H T T P C L I E N T <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>aws-crt-client</artifactId> </dependency> S3AsyncClient .builder() .httpClientBuilder( AwsCrtAsyncHttpClient.builder().maxConcurrency(100)) .build(); https://aws.amazon.com/blogs/developer/announcing-availability-of-the-aws-crt-http-client-in-the-aws-sdk-for-java-2-x/

rights reserved. Reducing Cold Starts § Objects that’s expected to be reused across multiple invocations should move its respective initialization code to either the static initializer or constructor. ⚠ Don’t store secrets that must not be reused § When using the AWS SDK, pre-conﬁgure speciﬁc items during client initialization. For example: region, the credentials provider, and endpoint. I N I T I A L I Z E B E F O R E T H E H A N D L E R

rights reserved. Reducing Cold Starts § C1 is a lightweight & fast compiler (no profiling) § C2 is using the profiler to detect “hot” code and produce native code, optimised, but it’s heavier and slower – Also in the Lambda context (not always running), it’s not very useful § Set Tiered Compilation level to 1 with an environment variable U S E T I E R E D C O M P I L A T I O N JAVA_TOOL_OPTIONS=-XX:+TieredCompilation -XX:TieredStopAtLevel=1 https://aws.amazon.com/blogs/compute/increasing-performance-of-java-aws-lambda-functions-using-tiered-compilation/

rights reserved. Reducing Cold Starts • Avoid runtime dependency injection & reflection as much as possible (eg. Spring, Spring Boot) • Use compile-time dependency injection frameworks (eg. Dagger 2, Micronaut, Quarkus). A V O I D D I A N D R E F L E C T I O N https://github.com/aws-samples/serverless-java-frameworks-samples

rights reserved. Reducing Cold Starts Compile your application ahead-of-time (AOT) and create Native Images with • https://www.graalvmonlambda.com/ • Requires a custom Lambda Runtime • May not work for all Java applications B U I L D N A T I V E I M A G E S 1 10 100 1000 10000 100000 1 2 3 4 5 6 7 8 9 10 Iteration JVM Native image Same Lambda function with Quarkus invocation savings of more than 95%. https://aws.amazon.com/blogs/architecture/field-notes-optimize-your-java-application-for-aws-lambda-with-quarkus/

rights reserved. Reducing Cold Starts Right size the memory with Lambda Power Tuning A D J U S T M E M O R Y ( A N D C P U ) https://github.com/alexcasalboni/aws-lambda-power-tuning & use multi-threading

rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function AWS optimizes this

rights reserved. Reducing Cold Starts Use Provisioned Concurrency to warm-up ahead of time. Especially important if you’re expecting a big spike in traﬃc on a short amount of time. P R O V I S I O N E D C O N C U R R E N C Y https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html

rights reserved. Reducing Cold Starts S N A P S T A R T Code execution Invoke Resume Resume Snapshot Post Snapshot Hook (optional) Pre Snapshot Hook (optional) Create Execution Environment Init during deployment Download Code Start Runtime Initialize Function Code Create Snapshot first request https://aws.amazon.com/blogs/compute/starting-up-faster-with-aws-lambda-snapstart/

rights reserved. Want to try? W O R K S H O P J A V A O N A W S L A M B D A https://s12d.com/java-workshop

rights reserved. Thank you! Jerome Van Der Linden @jeromevdl

Java on AWS Lambda: Differences and challenges

Java on AWS Lambda: Differences and challenges

More Decks by Jérôme Van Der Linden

Other Decks in Technology

Featured

Transcript