The Call of C-Tooling The Secrets Behind Native Image Building

About Me • Edoardo Vacchi @evacchi • Research @ UniMi / MaTe • Research @ UniCredit R&D • Kogito / Drools / jBPM @ Red Hat

Kogito Cloud-Native Business Automation for building intelligent applications, backed by battle-tested capabilities http:/ /kogito.kie.org

Native Java

Java Applications Build Time Run Time 3 Classloaders ~500 Classes ~160 Static Init 100+ Classloaders 1000+ Classes 1000+ Static Init 100++ Classloaders 1000++ Classes 1000++ Static Init static void Main Framework Initialization Application Initialization Source: Dan Heidinga - “Starting Fast” (QCon Plus 2021)

A Bit of History

Native Java Compilers • Compilation into machine code is not innovative per se • Prior art: native java compilers early 2000s. • GCJ (source code) • ExcelsiorJET (bytecode) • ... • More Recently: RoboVM (~2013)

Pros • Native code, possibly faster to start-up • Smaller memory footprint • by avoiding JIT+scratch memory in address space • possibly aggressive dead code elimination • Self-contained • avoid full JDK class library bundle

Cons Limitations • Not a JDK: different runtime environment, not cross-platform • May get out-of-sync with the spec • Trade-offs with dynamicity • Difference in run-time behavior (dynamic vs static) • Possibly need compromises with peak-performance (PGO ?) Moreover • The benefits of a native compilation are not compelling enough • Startup time is negligible • "You boot up your application once, you keep it running for a long time" • "Disk is cheap" • Dynamic Linking vs Static Linking • You can still achieve faster startup time through laziness

Laziness • Defer initialization to a later stage of execution, • Benefits: Shorter Startup Time • Downsides: Less predictable performance profile. Build Time Run Time static void Main Framework Initialization Application Initialization Delayed Inits...

Today

Getting Closer to Today • Shared Managed Infrastructure • Serverless • More interest in “Stateless” Apps • Suddenly attractive: • Fast Startup • Smaller Disk Footprint • Smaller Memory Footprint • Time to revisit?

Run-Time vs Build-Time • Generate code at build-time • Pre-initialize for boot time • e.g. Read config files, turn them into configuration commands • e.g. Read annotations, produce code for dependency injection • At startup, just execute that code • Benefits: faster startup time • Downsides • you have to write the code that generates code • possibly non-trivial, certainly time-consuming Build Time Run Time static void Main Framework Initialization Application Initialization Codegen

SmallTalk VMs

Smalltalk Environment • Concept of image • At run-time you do not just write code, you manipulate the state of such machine • contributing to the environment itself • possibly altering it or even turning it upside-down • When it is shut down, you do not just save the code you wrote you persist the state of machine to the image • When you start it you do not only run a program the state is restored, and execution resumes from the last saved state Run Time Load State Shutdown Save State

Checkpointing

CRIU + Java Build Time Run Time static void Main Framework Initialization Application Initialization Checkpoint • CRIU: Checkpoint and Restore in Userspace • https:/ /www.criu.org • CRaC: Coordinated Restore at Checkpoint • https:/ /github.com/CRaC/docs#crac • Jigawatts: • https:/ /github.com/chflood/jigawatts • OpenJ9 Snapshot+Restore • https:/ /danheidinga.github.io/Everyone_wants_fast_startup

GraalVM

GraalVM • GraalVM is an umbrella of technologies • A just-in-time compiler • The Truffle framework to implement dynamic languages • they can be seamlessly JITted across language boundaries. • SubstrateVM: the native image builder • reuses the compilation backend for Ahead-of-Time compilation • static init • image heap

Native Image Restrictions • Native binary compilation • Restriction: “closed-world assumption” • Limitations on reflection • No dynamic code loading: forbidden ClassLoader#defineClass(...byte[]...) • Allows more aggressive optimization (e.g, dead code elimination) • Static initializers are not lazy* ! • Evaluated at build time ! * originally opt-out, now opt-in. In some cases default on (e.g. Quarkus) Build Time static void Main Framework Initialization Application Initialization Run Time

Image Heap Generation

Image-Gen Heap That is another embarrassing pun

• We run parts of an application at build time and snapshot the objects allocated by this initialization code, using an iterative approach that is intertwined with points-to analysis. • We use points-to analysis results to only AOT-compile the parts of an application that are reachable at run time. “ Source: Initialize Once, Start Fast: Application Initialization at Build Time (Wimmer et al. OOPSLA 2019)

Static initializers

Static initializers public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } }

Static initializers $ java Example hello world

Static initializers $ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB

Static initializers $ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB

Static initializers $ ./example world

Static initializers public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } }

Static initializers public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } } A string constant

Static initializers public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } } A string constant A method invocation Over a PrintStream

Static initializers public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } } A string constant A method invocation A field resolution Over a subtype of OutputStream

Static initializers public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } } A string constant A method invocation A field resolution A static class initializer Over a subtype of OutputStream

Initialization Code First, class initializers are executed. • In Java, every class can have a class initializer ("static initializer") • represented as a method named in the class file. • It computes the initial value of static fields. • The developer decides which classes are initialized at image build time

Heap Snapshotting • Builds an object graph i.e., the transitive closure of reachable objects • starts with root pointers e.g. static fields. • This object graph is written into the native image as the image heap

Heap Snapshotting • Builds an object graph i.e., the transitive closure of reachable objects • starts with root pointers e.g. static fields. • This object graph is written into the native image as the image heap

Points-To Analysis • determine which classes, methods, and fields are reachable at run time. • starts with all entry points, e.g., the main method of the application, • iteratively processes all transitively reachable methods until a fixed point is reached

Points-To Analysis (Example) • System.out.println("hello") • java.lang.String • System.out • java.io.PrintStream • java.io.FilterOutputStream • java.io.OutputStream • System

Ahead-of-Time Compilation • methods marked as reachable by the points-to analysis • placed in the text section of the executable.

Image Heap at Run-Time • Execution at run-time starts with an already pre-populated Java heap • Relocatable: references relative to the start of the image heap • Objects of the image heap and objects allocated at run-time • i.e., also objects allocated at run time use relative references • (use of a fixed register r14 on x64 architectures). Build Time static void Main Framework Initialization Application Initialization Run Time

https:/ /twitter.com/reibitto/status/1384795560436113415 Perils of Static Initialization

No content

Project Leyden

Project Leyden • Goals • Address Java’s slow startup time • Reduce time to peak performance • Reduce memory footprint • Introduce static images at spec level (TCK) • stand-alone • closed-world

Qbicc • Experimental sandbox project for Leyden • Intended for compiler developers and experts • Goal: prototype approaches to native Java • New self-contained codebase • Allows to experiment with different trade-offs • GraalVM’s choices are known, • possible to explore different trade-offs of the solution space • Currently: Java-based compiler to LLVM IR • Future: different backend? (e.g. C2) https:/ /github.com/qbicc/qbicc https:/ /github.com/qbicc/qbicc/discussions https:/ /qbicc.zulipchat.com

Qbicc: Architecture • Points-to analysis (static entry points) • Flow graph copied between phases, dropping unreachable nodes • Approaches to static init being investigated ADD ANALYZE LOWER GENERATE ● TRANSFORM ● CORRECT ● OPTIMIZE ● INTEGRITY ● TRANSFORM ● CORRECT ● OPTIMIZE ● INTEGRITY ● TRANSFORM ● CORRECT ● OPTIMIZE ● INTEGRITY ● TRANSFORM ● CORRECT ● OPTIMIZE ● INTEGRITY

Qbicc: Static Initialization • Ongoing discussion topic • Defining feature • Different approaches being evaluated • Avoid pitfalls

mmap + offset Qbicc Build-time serialization + Fast deserialization routines initially static first + opt-out now runtime first + opt-in Qbicc currently all run-time eventually as close to “all build-time” as possible investigating explicit opt-in (code hints? annotations? language changes?) Qbicc: Static Initialization Trade-Offs

Further Reading

References Andrew Dinn (2021) Leyden: Lessons from Graal Native C. Wimmer et al. (OOPSLA 2019) Initialize Once, Start Fast: Application Initialization at Build Time Dan Heidinga (QCon Plus 2021) Starting Fast Duke Art at OpenJDK Wiki

developers.redhat.com/register