Designing and delivering cloud-optimized Java applications

BOA327 Designing and delivering cloud-optimized Java applications
re:Invent 2022

Java is a mature and popular language that has been used for over 25 years. Common questions for Java users are how ready Java is for the cloud era and what best practices are for developing cloud-native applications. This chalk talk begins by reviewing the Java language, standards, and culture, as well as the ecosystem that surrounds it. It then dives deep into how to architect Java applications to maintain developer productivity, providing best practices for modern frameworks, integration and delivery scenarios, and overall modernization of the application lifecycle.

Transcript

1. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Designing and delivering cloud-optimized Java applications Senior Developer Advocate Amazon Web Services Viktor Vedmich Senior Developer Advocate Amazon Web Services Mohammed Fazalullah
3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Java – 25 years and going Apache Software Lines of code 1996-2020. Source: The Apache Software Foundation - Annual Report - FY2020
4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda Java ecosystem overview Spring journey to AWS Lambda Accelerate with modern frameworks Other considerations Summary
5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Motivation Java plays an essential role when building reliable software architectures Spring Boot is the most popular Java Application Framework with +60%* of Java developers reportedly using it Broad ecosystem (libraries, tools and resources) & Developer experience Over the past years companies invested in these technologies New feature development: Spring 3.0 with GraalVM Support, Project Leyden, Java release cycle * Source: https://www.jrebel.com/blog/2021-java-technology-report
6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Challenges with Java & Spring in Serverless Java initially has been designed around factors that differ from modern cloud-native requirements There is a paradigm shift from always-on serverful applications to short-lived serverless functions Large enterprise frameworks often perform expensive operations during initial JVM startup up time Gathering metrics from JVM based usage – combining them across service traces
7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Spring Boot on a server • Initialize the environment once • Handle multiple request with the same instance (concurrently) • Instance keeps running after request processing • Scaling based on metrics or manually 1 2 Request Request Initialize full Spring Context Running application 3 Request … Initialization
8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compute options on AWS Amazon EC2 • Virtual Machines (Linux/Windows) AWS Lambda • Execute code without provisioning or managing servers • Synchronous (request/response) or Asynchronous execution models Containers • Broad range of fully managed, do-it-yourself and serverless container options H/W OS Runtime Level of abstraction VM Task Function Unit of Consumption
9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers common issues • Slow to start up • High CPU consumption • OOM killed due to high memory usage
10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers recommendations • Check your container logs • Use at least Java 8u191 with UseContainerSupport flag, or better use Java 10 • Kubernetes probes (livenessProbe, readinessProbe, startupProbe) • HPA (Horizontal Pod Autoscaler) and VPA (Vertical Pod Autoscaler) for scaling
11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimizing your containers • Optimize for smaller size • Use a minimalist operating system • Not all runtimes are equal! REPOSITORY SIZE node:latest 674MB java:latest 643MB node:slim 184MB ubuntu:latest 85.8MB alpine:latest 4.41MB busybox:latest 1.15MB
12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda Event Databases AWS Services Etc. Lambda Function API Request State change
13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Spring Boot on AWS Lambda • Initialize execution environment on first invoke • An execution environment handles a single request at a time • If an execution environment is busy with a request – another environment is started • Scales to zero if there is no request – Managed auto scaling 1 Initialization Execution Execution 2 Execution 3 Initialization 4 Execution Initialize full Spring Context Initialize full Spring Context
14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Why optimize Java on AWS Lambda You are charged based on execution duration, rounded up to the nearest 100ms. The price depends on the amount of memory you allocate to your function. Each % in performance optimization can make your customers happier and save money! Photo by Katie Harp on Unsplash
15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. A Spring journey to AWS Lambda 1. Migrate 2. Optimize 3. Refactor 4. Accelerate
16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migrate HTTP request AWS Lambda Event Amazon API Gateway
17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Serverless Java Container • Transforms events so that Spring can handle them as if it was an HTTP Request • Routing via RestController, POJO serialization, HTTP status codes • Add the library to your code and provide a configuration class
18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Optimize • Right size your applications memory setting with AWS Lambda Power Tuning • Use Tiered Compilation to reduce cold-starts • Use AWS SDK & Initialization best practices • Pre-warm execution environments with Provisioned Concurrency
19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Lambda Power Tuning Source: https://github.com/alexcasalboni/aws-lambda-power-tuning
20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Tiered compilation • Use optimized C1 compiler for faster application start up • Add an environment variable to change the compiler level Without Tiered With Tiered 11.378 - 40 % 6.725 Init Duration Results will vary based on your application
21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Follow general AWS Java best practices • Use the new AWS SDK version (v2) with proper configuration and optimized HTTP clients • Initialize expensive operations outside the handler code
22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Provisioned Concurrency • Pre-Provision a given number of AWS Lambda execution environments • No initialization time for X concurrent requests • Pay for memory you reserve whether used or not. When used, cheaper than on-demand price. Time 1 Initialization Execution 2 Initialization Execution PC = 2
23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Refactor • Further improve performance and memory footprint by refactoring your application • Using light-weight dependencies • Move to a functional approach
24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Using lightweight dependencies • Trade-off between feature richness & performance • spring-data-jpa vs. spring-data-jdbc • jackson vs. jackon-jr, Log4J2 vs. SLF4J SimpleLogger Spring Data JPA Spring Data JDBC 6.725 - 25 % 5.052 Duration Results will vary based on your application #YMMV
25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Spring cloud functions • Moving to a functional approach • Avoid overhead of providing an own HTTP server (API Gateway takes care of this) • Use a functional approach and directly handle the event from API Gateway • RestController vs. FunctionHandler<Event>
26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Accelerate • GraalVM Native image with Spring Native • Create a native binary - usage with AWS Lambda custom runtime • Sub-second application startup • Multiple frameworks provide built-in support (Micronaut, Quarkus) • Use alternative compile time frameworks on the JVM (Micronaut, Quarkus) • Cloud-Native frameworks – Built for the cloud • Allows you to shift certain tasks to build time (e.g. Dependency Injection) • Can further improve application start-up and memory footprint
27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Results of using Native Image (Slide 1 of 2) Source https://www.graalvm.org/why-graalvm/
28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Results of using Native Image (Slide 2 of 2) Source https://www.graalvm.org/why-graalvm/
29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Compile your application ahead-of-time (AOT) GraalVM creates a native image of your application at build-time. Frameworks like Quarkus, and Micronaut integrate GraalVM and add further benefits like build-time dependency injection. Some JVM features not supported.
30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. AOT compilation Static analysis Native executable GraalVM and native image Application Libraries JDK Substrate VM Reachable methods, fields and classes Machine code Image Heap
31. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Micronaut • Annotation driven programming model • Declarative reactive compile-time HTTP clients • Cloud native (service discovery, distributed tracing, cloud support) • Easy data-acess layer • Micronaut + GraalVM • No reflection, no runtime proxies, no bytecode generation, no dynamic class loading • Instantaneously spin up servers & clients for unit tests
32. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Starting with Micronaut Launch
33. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Starting with Micronaut CLI
34. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Building with Micronaut In Maven In Gradle
35. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
36. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Other considerations • Function granularity and least privilege • Connection pooling • Endpoint based functionality (Spring Actuator, Metrics scraping) • File-System logging • Caching
37. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Summary 1. Motivations and challenges for modernizing Java applications 2. Migrate your application to run on AWS Lambda with the provided tools & libraries 3. Optimize your application & follow AWS Lambda best practises 4. Refactor your application to further reduce dependencies and improve performance 5. Accelerate your application by using alternative VMs or frameworks
38. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Resources • Micronaut https://micronaut.io/ • GraalVM https://www.graalvm.org/ • Serverless Java Container https://github.com/awslabs/aws- serverless-java-container • AWS Lambda Power Tuning https://docs.aws.amazon.com/lambda/latest/operatorguide/p rofile-functions.html • SVS 310 From Serverful to Serverless Java with AWS Lambda https://serverlessland.com/reinvent2022/svs310
39. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Mohammed Fazalullah Viktor Vedmich