Ondro Mihályi @OndroMih Serverless Java hacks Think different and leave old habits behind

Ondro Mihályi @OndroMih Understanding Serverless Computing ➢ Focus on running code not maintaining infrastructure ➢ Lower costs, pay only when code is running ➢ Scale to zero – when code runs rarely ➢ Scale to "Infinity" – during peak loads

Ondro Mihályi @OndroMih Extensions Runtime Function Request Extensions Runtime INIT INVOKE SHUTDOWN At cold start Every invocation → Leads back to a cold start AWS Lambda Lifecycle

Ondro Mihályi @OndroMih What to avoid ➢ Keeping state in functions ➢ Synchronous I/O operations ➢ Resource pooling ➢ Prematured concurrency ➢ Large function package sizes

Then, what to do instead?

Ondro Mihályi @OndroMih 1. External State Storage and Caching ➢ Functions are ephemeral – data is not shared and can be lost ➢ Keep persistent data in external storage ➢ Cache data in a global variable for subsequent executions

Ondro Mihályi @OndroMih Caching data in a function static Map personCache = new HashMap<>(); public APIGatewayProxyResponseEvent handleRequest( APIGatewayProxyRequestEvent req, Context context) { String id = req.getQueryStringParameters().get("id"); Person p = personCache.get(id); if (p == null) { p = em.find(Person.class, id); PersonCache.put(id, p);

Ondro Mihályi @OndroMih 2. Keep things Event-Driven ➢ Synchronous calls are costly ➢ Function1 → Function 2 → Function 3 ➔While Function 3 is running, you pay for Function 1 and Function 2 execution too

Ondro Mihályi @OndroMih Keep things Event-Driven ➢ Functions are designed to be asynchronous (everything is an event) ➢ Function1 → event for Function 2 ➢ Function 2 → event for Function 3 ➢ Function 3 → action ➢Synchronous calls only when needed or fast

Ondro Mihályi @OndroMih Cost of synchronous calls Result Function 1 Function 2 Function 3 Invoke Invoke

Ondro Mihályi @OndroMih Save cost with asynchronous calls Event with a result Function 1 Function 2 Function 3

Ondro Mihályi @OndroMih Save cost with emited events Event with a result Function 1 Function 2 Function 3 Save to Dynamo DB / S3 Bucket Event

Ondro Mihályi @OndroMih 3. Pooling resources Just don’t!

Ondro Mihályi @OndroMih Cache resources instead ➢ Just cache resources ➢ Create resources only when you need them ➢ An individual function always serves a single event ➢ You’ll only need one connection pool or thread (mostly) ➔Only cache between invocations

Ondro Mihályi @OndroMih Pre-create resources in background ➢Delay resource creation, cache created resources ➢Is that all we can do? Computation Connect to DB Main thread Work with DB

Ondro Mihályi @OndroMih Pre-create resources in background ➢We can predict and prepare in a daemon thread ➢If we don’t need it, throw it away (like branch prediction) Computation Connect to DB Main thread Daemon thread Work with DB

Ondro Mihályi @OndroMih 4. Use fast libraries ➢ Most widely used libraries may be designed for optimal execution but longer startup ➢ Functions often execute fast ➢ Choose lighter libraries optimized for fast startup ➔e.g. a fast HTTP client, libraries with native code

Ondro Mihályi @OndroMih 5. Infrastructure as a code ➢ You don’t maintain but still manage infra – even more, with many small functions ➢Automate it with Java ➔e.g. AWS CDK, Pulumi

Ondro Mihályi @OndroMih Infrastructure as a Java code (CDK) Function myFunction = new Function(this, "MyFunctionID", FunctionProps.builder() .functionName("MyFunction") .handler("com.MyFunction::handleRequest") .runtime(Runtime.JAVA_17) .code(Code.fromAsset("my-function-package.jar")) .memorySize(3_000) .logRetention(RetentionDays.ONE_DAY) .build());

What about cold starts?

Ondro Mihályi @OndroMih Techniques to reduce cold starts ➢ Optimize JVM for fast startup (- XX:TieredStopAtLevel=1) ➢ Restore from a checkpoint ➢ Build-time optimizations ➢ Native compilation

Ondro Mihályi @OndroMih Restore from a checkpoint ➢ AWS Snapstart, OpenJDK CRaC, OpenJ9 CRIU ➔ Start app ➔ Store checkpoint of the JVM before serving requests ➔ Restore from checkpoint during cold start

Ondro Mihályi @OndroMih Init in a static block before checkpoint public APIGatewayProxyResponseEvent handleRequest( APIGatewayProxyRequestEvent req, Context context) { MyApp app = new MyApp(); app.initialize(config); return app.process(req); }

Ondro Mihályi @OndroMih Init in a static block before checkpoint static MyApp app; static { app = new MyApp(); app.initialize(config); } public APIGatewayProxyResponseEvent handleRequest(… return app.process( request );

Ondro Mihályi @OndroMih Build-time optimizations ➢ Generate config or bytecode during build ➢ Compile to native code ➔ Quarkus framework supports both

Ondro Mihályi @OmniFish Questions

Ondro Mihályi @OmniFishEE OmniFish - Jakarta EE Consulting & Support Jakarta EE Consultancy (migrations, tuning) Jakarta EE Application Development GlassFish Server Support omnifish.ee Jakarta EE Training