Java on AWS Lambda: Differences and challenges

For more than 25 years, Java has played an essential role in building enterprise applications. The traditional approach of deploying more or less monolithic applications on application servers is completely challenged in the cloud, especially with Serverless and Lambda functions. During this presentation, we will see the main paradigm differences as well as the challenges of this new approach: with among others the invocation and execution model, the development model (what about the usual frameworks), how to package and deploy, cold starts and optimizations, use cases...

Transcript

1. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Java on AWS Lambda: Differences and challenges Jerome Van Der Linden Sr Solutions Architect Builder @AWS 23/05/2023
2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Difference #1: Scope of responsibility
3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. A standard Serverful Java / Spring Application Client Application Load Balancer Instances HTTP server Spring Framework Application code Spring Boot application Dispatcher Servlet Handler Mapping Handler Adapter Controller Service Repository You take care of + networking + security + autoscaling + multi-AZ + patching + upgrades + …
4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. No infrastructure provisioning, no management Automatic scaling Pay for value Highly available and secure What is serverless?
5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda F U L L Y M A N A G E D S E R V E R L E S S C O M P U T E Auto Scaling group Availability Zone A Availability Zone B Availability Zone C Region Instances Instances Instances AWS Lambda ≈ Load Balancer EC2 Instance FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM
6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda F U L L Y M A N A G E D S E R V E R L E S S C O M P U T E Auto Scaling group Availability Zone A Availability Zone B Availability Zone C Region Instances Instances Instances AWS Lambda ≈ Load Balancer EC2 Instance FireCracker MicroVM Execution Environment Language Runtime Application code You take care of FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM
7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Difference #2: Execution model
8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. A standard Serverful Java / Spring Application Client Application Load Balancer Instances HTTP server Spring Framework Application code Spring Boot application Dispatcher Servlet Handler Mapping Handler Adapter Controller Service Repository Always running Always accepting requests
9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda Auto Scaling group Availability Zone A Availability Zone B Availability Zone C Region Instances Instances Instances AWS Lambda ≈ Load Balancer EC2 Instance FireCracker MicroVM Execution Environment Language Runtime Application code FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM FireCracker MicroVM NOT always running Accept 1 request at a time
10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution Env. #1
11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution Execution environment is blocked or busy for this entire time Env. #1
12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution 2 Initialization Execution Env. #1 Env. #2 3 Initialization Execution Env. #3
13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda E X E C U T I O N E N V I R O N M E N T S Time 1 Initialization Execution 2 Initialization Execution Env. #1 Env. #2 3 Initialization Execution Env. #3 Execution 4
14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Difference #3: Invocation model
15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda E V E N T - D R I V E N F U N C T I O N S Event source Services (anything) Database AWS Service 3rd party service Function Changes in data state Changes in resource state Request to endpoints Node.js Python C# Go Ruby Runtime API 8 / 11 / 17
16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda E V E N T - D R I V E N F U N C T I O N S Event source Services(anything) Function com.amazonaws:aws-lambda-java-core:1.2.2 com.amazonaws:aws-lambda-java-events:3.11.1 software.amazon.awssdk:dynamodb:2.20.56 software.amazon.awssdk:s3:2.20.56 software.amazon.awssdk:* Or any Java library you’d like to use
17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda A N A T O M Y O F A F U N C T I O N public class App implements RequestHandler<Event, Response>{ public Response handleRequest(final Event event, final Context context) { // do some stuff return response; } } • Handler: function that will be triggered for each event received • Event: the incoming event that triggered the function (in JSON format) • Context: informations about the function configuration and invocation • Your code, anything, … • Input type • Output type
18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda C O M M O N U S E C A S E S Amazon API Gateway AWS Lambda AWS Lambda AWS AppSync APIs / microservices (REST / GraphQL) AWS Lambda AWS Step Functions Orchestration AWS Lambda AWS Config Operations/ Remediation AWS Lambda Amazon SNS AWS Lambda AWS Lambda Amazon EventBridge Amazon SQS Event-driven architectures File processing Stream processing Data processing (transformation , cleansing, …) Analytics IoT backend …
19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda C O M M O N U S E C A S E : H A N D L I N G H T T P / R E S T R E Q U E S T S Amazon API Gateway AWS Lambda public class App implements RequestHandler<APIGatewayProxyRequestEvent, APIGatewayProxyResponseEvent> { public APIGatewayProxyResponseEvent handleRequest(final APIGatewayProxyRequestEvent input, final Context context) { String name = input.getQueryStringParameters().get("name"); Map<String, String> headers = new HashMap<>(); headers.put("Content-Type", "application/json"); APIGatewayProxyResponseEvent response = new APIGatewayProxyResponseEvent() .withHeaders(headers); try { String output = String.format("{ \"message\": \"hello %s\" }", name); return response .withStatusCode(200) .withBody(output); } catch (IOException e) { return response .withBody("{}") .withStatusCode(500); } } }
20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Difference #4: Development model
21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Single-purpose functions do “x” do “y” do “z” • Benefits: • Easier to debug, trace errors • Enforce least privilege, one role per function • Simpler to test • Favor reusability • Less surface area • Tips: • Use naming conventions and tags to enhance discoverability • Don’t go nano
22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Testing • Unit / Integration tests with aws-lambda-java-tests @ParameterizedTest @Event(value = "sqs/sqs_event.json", type = SQSEvent.class) public void testInjectSQSEvent(SQSEvent event) { // test your handleRequest method with this event as parameter } • Local invocation with SAM: sam local invoke MyFunction --event local-event.json
23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Difference #5: Deployment model
24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda Supports either Zip or Jar format • Gradle: using Zip build type • Maven: using Shade plugin / Assembly plugin or Dependency plugin P A C K A G I N G https://docs.aws.amazon.com/lambda/latest/dg/java-package.html
25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda Supports container images • AWS Images for Lambda (lambda/java) • Bring your own image (ex if you want Java 20) • Run and test images locally P A C K A G I N G $ docker build -t docker-image:test . $ docker run –p 9000:8080 docker-image:test $ curl –XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}' https://docs.aws.amazon.com/lambda/latest/dg/java-image.html
26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda • Using the AWS CLI: aws lambda create-function ... D E P L O Y M E N T
27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda • Using the AWS CLI: aws lambda create-function ... • Using Infrastructure as code: § AWS SAM D E P L O Y M E N T https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html
28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda • Using the AWS CLI • Using Infrastructure as code: § AWS SAM § AWS CDK D E P L O Y M E N T https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html
29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda • Using the AWS CLI • Using Infrastructure as code: § AWS SAM § AWS CDK § Terraform § … D E P L O Y M E N T https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function
30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Challenge #1: Lambda Limits
31. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Soft Limits C A N B E I N C R E A S E D Concurrency = (average requests per second) * (average request duration in seconds) è (average requests per second) = Concurrency / (average request duration in seconds) Ex for a function that takes 250ms: 1000 / 0.25 ≈ 4000 req / sec https://docs.aws.amazon.com/lambda/latest/dg/lambda-concurrency.html
32. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Hard limits AWS Step Functions Amazon S3 Container image Amazon EFS (NFS) No GPU
33. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Challenge #2: Cold Starts
34. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Cold start Warm start Time 1 Initialization Execution Env. #1 Execution 4
35. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function You can optimize this AWS optimizes this
36. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function You can optimize this
37. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts § Use AWS Java SDK v2 and use only the required dependencies (s3, dynamodb, …) § Remove unnecessary jar dependencies § Use lighweight dependencies (jackson-jr vs jackson, slf4j simple logger vs log4j, …) R E D U C E A S M U C H A S P O S S I B L E T H E S I Z E O F T H E P A C K A G E
38. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function You can optimize this
39. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts Use the AWS CRT HTTP Client • faster SDK startup time • smaller memory footprint • reduced latency time U S E T H E A W S C R T H T T P C L I E N T <dependency> <groupId>software.amazon.awssdk</groupId> <artifactId>aws-crt-client</artifactId> </dependency> S3AsyncClient .builder() .httpClientBuilder( AwsCrtAsyncHttpClient.builder().maxConcurrency(100)) .build(); https://aws.amazon.com/blogs/developer/announcing-availability-of-the-aws-crt-http-client-in-the-aws-sdk-for-java-2-x/
40. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts § Objects that’s expected to be reused across multiple invocations should move its respective initialization code to either the static initializer or constructor. ⚠ Don’t store secrets that must not be reused § When using the AWS SDK, pre-configure specific items during client initialization. For example: region, the credentials provider, and endpoint. I N I T I A L I Z E B E F O R E T H E H A N D L E R
41. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts § C1 is a lightweight & fast compiler (no profiling) § C2 is using the profiler to detect “hot” code and produce native code, optimised, but it’s heavier and slower – Also in the Lambda context (not always running), it’s not very useful § Set Tiered Compilation level to 1 with an environment variable U S E T I E R E D C O M P I L A T I O N JAVA_TOOL_OPTIONS=-XX:+TieredCompilation -XX:TieredStopAtLevel=1 https://aws.amazon.com/blogs/compute/increasing-performance-of-java-aws-lambda-functions-using-tiered-compilation/
42. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts • Avoid runtime dependency injection & reflection as much as possible (eg. Spring, Spring Boot) • Use compile-time dependency injection frameworks (eg. Dagger 2, Micronaut, Quarkus). A V O I D D I A N D R E F L E C T I O N https://github.com/aws-samples/serverless-java-frameworks-samples
43. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts Compile your application ahead-of-time (AOT) and create Native Images with • https://www.graalvmonlambda.com/ • Requires a custom Lambda Runtime • May not work for all Java applications B U I L D N A T I V E I M A G E S 1 10 100 1000 10000 100000 1 2 3 4 5 6 7 8 9 10 Iteration JVM Native image Same Lambda function with Quarkus invocation savings of more than 95%. https://aws.amazon.com/blogs/architecture/field-notes-optimize-your-java-application-for-aws-lambda-with-quarkus/
44. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts Right size the memory with Lambda Power Tuning A D J U S T M E M O R Y ( A N D C P U ) https://github.com/alexcasalboni/aws-lambda-power-tuning & use multi-threading
45. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda I N I T I A L I Z A T I O N O F T H E E X E C U T I O N E N V I R O N M E N T Time 1 Initialization Execution Env. #1 Execution 4 Create Env Download code Start Runtime Initialize Function AWS optimizes this
46. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts Use Provisioned Concurrency to warm-up ahead of time. Especially important if you’re expecting a big spike in traffic on a short amount of time. P R O V I S I O N E D C O N C U R R E N C Y https://docs.aws.amazon.com/lambda/latest/dg/provisioned-concurrency.html
47. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reducing Cold Starts S N A P S T A R T Code execution Invoke Resume Resume Snapshot Post Snapshot Hook (optional) Pre Snapshot Hook (optional) Create Execution Environment Init during deployment Download Code Start Runtime Initialize Function Code Create Snapshot first request https://aws.amazon.com/blogs/compute/starting-up-faster-with-aws-lambda-snapstart/
48. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Going further…
49. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Want to try? W O R K S H O P J A V A O N A W S L A M B D A https://s12d.com/java-workshop
50. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! Jerome Van Der Linden @jeromevdl