Serverless Java hacks: Think different and leave old habits behind

In this presentation, we will delve into the details of developing serverless applications in Java, emphasizing the need for a paradigm shift in coding practices and architectural considerations. I will explain key aspects of serverless Java, including source code structuring, framework choices (many of them based on Jakarta EE), deployment strategies tailored for serverless environments, and runtime optimizations with GraalVM native compilation and tools like CRaC and AWS Snapstart.

Serverless applications require precise attention to startup speed optimizations and efficient resource initialization. We will discuss strategies for resource management, advocating for lean connection and thread pools. Additionally, we will examine the importance of external configuration, statelessness, and the efficient storage of state data in external services or databases within the context of serverless architectures.

As part of our technical exploration, I will highlight the advantages of using specialized client libraries optimized for startup speed, in contrast to traditional libraries designed for high throughput. By the end of this session, you will gain a deep understanding of the technical nuances involved in serverless Java development, equipping you with the tools and insights needed to optimize performance and seamlessly adapt to serverless computing environments.

Transcript

1. Ondro Mihályi @OndroMih Serverless Java hacks Think different and leave old habits behind
2. Ondro Mihályi @OndroMih Understanding Serverless Computing ➢ Focus on running code not maintaining infrastructure ➢ Lower costs, pay only when code is running ➢ Scale to zero – when code runs rarely ➢ Scale to "Infinity" – during peak loads
3. Ondro Mihályi @OndroMih Extensions Runtime Function Request Extensions Runtime INIT INVOKE SHUTDOWN At cold start Every invocation → Leads back to a cold start AWS Lambda Lifecycle
4. Ondro Mihályi @OndroMih What to avoid ➢ Keeping state in functions ➢ Synchronous I/O operations ➢ Resource pooling ➢ Prematured concurrency ➢ Large function package sizes
5. Then, what to do instead?
6. Ondro Mihályi @OndroMih 1. External State Storage and Caching ➢ Functions are ephemeral – data is not shared and can be lost ➢ Keep persistent data in external storage ➢ Cache data in a global variable for subsequent executions
7. Ondro Mihályi @OndroMih Caching data in a function static Map<String, Person> personCache = new HashMap<>(); public APIGatewayProxyResponseEvent handleRequest( APIGatewayProxyRequestEvent req, Context context) { String id = req.getQueryStringParameters().get("id"); Person p = personCache.get(id); if (p == null) { p = em.find(Person.class, id); PersonCache.put(id, p);
8. Ondro Mihályi @OndroMih 2. Keep things Event-Driven ➢ Synchronous calls are costly ➢ Function1 → Function 2 → Function 3 ➔While Function 3 is running, you pay for Function 1 and Function 2 execution too
9. Ondro Mihályi @OndroMih Keep things Event-Driven ➢ Functions are designed to be asynchronous (everything is an event) ➢ Function1 → event for Function 2 ➢ Function 2 → event for Function 3 ➢ Function 3 → action ➢Synchronous calls only when needed or fast
10. Ondro Mihályi @OndroMih Cost of synchronous calls Result Function 1 Function 2 Function 3 Invoke Invoke
11. Ondro Mihályi @OndroMih Save cost with asynchronous calls Event with a result Function 1 Function 2 Function 3
12. Ondro Mihályi @OndroMih Save cost with emited events Event with a result Function 1 Function 2 Function 3 Save to Dynamo DB / S3 Bucket Event
13. Ondro Mihályi @OndroMih 3. Pooling resources Just don’t!
14. Ondro Mihályi @OndroMih Cache resources instead ➢ Just cache resources ➢ Create resources only when you need them ➢ An individual function always serves a single event ➢ You’ll only need one connection pool or thread (mostly) ➔Only cache between invocations
15. Ondro Mihályi @OndroMih Pre-create resources in background ➢Delay resource creation, cache created resources ➢Is that all we can do? Computation Connect to DB Main thread Work with DB
16. Ondro Mihályi @OndroMih Pre-create resources in background ➢We can predict and prepare in a daemon thread ➢If we don’t need it, throw it away (like branch prediction) Computation Connect to DB Main thread Daemon thread Work with DB
17. Ondro Mihályi @OndroMih 4. Use fast libraries ➢ Most widely used libraries may be designed for optimal execution but longer startup ➢ Functions often execute fast ➢ Choose lighter libraries optimized for fast startup ➔e.g. a fast HTTP client, libraries with native code
18. Ondro Mihályi @OndroMih 5. Infrastructure as a code ➢ You don’t maintain but still manage infra – even more, with many small functions ➢Automate it with Java ➔e.g. AWS CDK, Pulumi
19. Ondro Mihályi @OndroMih Infrastructure as a Java code (CDK) Function myFunction = new Function(this, "MyFunctionID", FunctionProps.builder() .functionName("MyFunction") .handler("com.MyFunction::handleRequest") .runtime(Runtime.JAVA_17) .code(Code.fromAsset("my-function-package.jar")) .memorySize(3_000) .logRetention(RetentionDays.ONE_DAY) .build());
20. What about cold starts?
21. Ondro Mihályi @OndroMih Techniques to reduce cold starts ➢ Optimize JVM for fast startup (- XX:TieredStopAtLevel=1) ➢ Restore from a checkpoint ➢ Build-time optimizations ➢ Native compilation
22. Ondro Mihályi @OndroMih Restore from a checkpoint ➢ AWS Snapstart, OpenJDK CRaC, OpenJ9 CRIU ➔ Start app ➔ Store checkpoint of the JVM before serving requests ➔ Restore from checkpoint during cold start
23. Ondro Mihályi @OndroMih Init in a static block before checkpoint public APIGatewayProxyResponseEvent handleRequest( APIGatewayProxyRequestEvent req, Context context) { MyApp app = new MyApp(); app.initialize(config); return app.process(req); }
24. Ondro Mihályi @OndroMih Init in a static block before checkpoint static MyApp app; static { app = new MyApp(); app.initialize(config); } public APIGatewayProxyResponseEvent handleRequest(… return app.process( request );
25. Ondro Mihályi @OndroMih Build-time optimizations ➢ Generate config or bytecode during build ➢ Compile to native code ➔ Quarkus framework supports both
26. Ondro Mihályi @OmniFish Questions
27. Ondro Mihályi @OmniFishEE OmniFish - Jakarta EE Consulting & Support Jakarta EE Consultancy (migrations, tuning) Jakarta EE Application Development GlassFish Server Support omnifish.ee Jakarta EE Training