JUGNsk Meetup #13. Владимир Иванов: Project Panama: как сделать Java "ближе к железу"?

Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
| Project Panama Making Java “closer to hardware” Vladimir Ivanov Senior Principal Software Engineer HotSpot JVM Compilers, JPG Oracle February, 28, 2020

| Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 2

| How it all started? 3 https://www.youtube.com/watch?v=P10nuG2meR0 https://www.oracle.com/technetwork/java/jvmls2013nutter-2013526.pdf

| How it all started? 4 https://www.youtube.com/watch?v=P10nuG2meR0 https://www.oracle.com/technetwork/java/jvmls2013nutter-2013526.pdf

| How it all started? 5 JVMLS 2013: “Java Native Runtime” by Charles Nutter

| How it all started? 6 JVMLS 2013: “Java Native Runtime” by Charles Nutter

| 7 https://mail.openjdk.java.net/pipermail/discuss/2014-March/003306.html

| 8 https://openjdk.java.net/projects/panama/

| Project Panama 9 “Bridging the gap”

| Project Panama: What’s in there? (ca. February, 2020) $ hg branches foreign-allocator 59768:abf7713ba43e foreign-jextract 59690:a58f76f7ddc8 foreign-abi 59688:cd5834a0a884 (inactive) foreign-memaccess 59685:7f22f2342067 (inactive) vectorIntrinsics 59939:0785295027bc vector-unstable 59502:876e468f46d9 foreign 59760:63dc2eb214ef … 10 http://hg.openjdk.java.net/panama/dev

| Project Panama: What’s in there? (ca. February, 2020) $ hg branches foreign-allocator 59768:abf7713ba43e foreign-jextract 59690:a58f76f7ddc8 foreign-abi 59688:cd5834a0a884 (inactive) foreign-memaccess 59685:7f22f2342067 (inactive) vectorIntrinsics 59939:0785295027bc vector-unstable 59502:876e468f46d9 foreign 59760:63dc2eb214ef … 11 http://hg.openjdk.java.net/panama/dev FFI (JNI 2.0) (branch merges) Vector API

| Project Panama: What’s in there? (February, 2020) 12 https://github.com/openjdk/panama-foreign/branches/active

| 13 $ hg branches … foreign-jextract 59690:a58f76f7ddc8 foreign-memaccess 59685:7f22f2342067 foreign-abi 59688:cd5834a0a884 foreign-allocator 59768:abf7713ba43e … FFI (JNI 2.0) Better JNI Easier, Safer, Faster

| Java Frame Java Heap Native Memory Native Frame GC roots

| Java Heap Native Memory Java Frame Native Frame GC roots

| jobject raw ptr address Java Heap Native Memory ptr

| Java Heap Native Memory Handles Java Frame Native Frame GC roots

| JNI @since 1.1

| getpid in Java 20 JNI workflow

| JNI Usage scenario class LibC { static native long getpid(); } jlong JNICALL Java_LibC_getpid( JNIEnv* env, jclass c) { return getpid(); }

| JNI: Upcall jlong JNICALL Java_...(JNIEnv* env, jclass cls, jobject obj) { jmethodID mid = env->GetMethodID(cls, “m”, “(I)J”); jlong result = env->CallLongMethod(obj, mid, 10);

| JNI: Data Access jlong JNICALL Java_...(JNIEnv* env, jclass cls, jobject obj) { jfieldID fid = env->GetFieldID(cls, “f”, “J”); jlong result = env->GetLongField(obj, fid); jlong result = env->SetLongField(obj, fid, 10);

| Java Native Java Heap Native Memory VM Thread State Anatomy of JNI call

| Java Native Java Heap Native Memory VM Thread State Safepoints Anatomy of JNI call

| JNI • Pros – seamless integration • looks like a Java method – rich native API to interact with Java • Cons – manual binding – invocation overhead

| JNI Victim of its own success?

| JNI Sum array elements jint JNICALL Java_...(JNIEnv *env, jclass c, jobject arr) { jint len = (*env)->GetArrayLength(env, arr); jbyte* a = (*env)->GetPrimitiveArrayCritical(env, arr, 0); … return sum; } empty sum 1 sum 103 sum 106 JNI 11.4±0.3 ns 178.0±7.1 ns 798±32 ns 641±51 μs

| Critical JNI Sum array elements jint JNICALL JavaCritical_...(jint length, jbyte* first) { ... return sum; } empty sum 1 sum 103 sum 106 JNI 11.4±0.3 ns 178.0±7.1 ns 798±32 ns 641±51 μs Critical JNI 11.4±0.3 ns 17.2±0.8 ns 680±22 ns 636±12 μs

| Critical JNI • only static, non-synchronized methods supported • no JNIEnv* • arguments: primitives or primitive arrays – [I => (length, I*) – null => (0, NULL) • no object arguments

| int printf(const char *format, ...)

| void qsort( void* base, size_t nel, size_t width, int (*cmp)(const void*, const void*));

| JNR JNR Java Native bindings Interfaces libffi Target User-defined generated on-the-fly public interface LibC { @pid_t long getpid(); } LibC lib = LibraryLoader .create(LibC.class) .load("c"); libc.getpid()

| JNR • Pros – automatic binding of native methods • Cons – manual interface extraction • doesn’t scale – still uses JNI to perform native calls

| getpid in Java 35 JNI workflow

| Can we do better? 36 ?

| Native Interface • Real-world libraries usually have rich interfaces – many entry points (100s), data types definitions • Manual extraction – Laborous – Error prone • lots of details to care about – Hard to reuse the work when migrating to a newer version 37 Automatic Extraction

| c_api.h • 161 functions • 23 structs • 50 constants • 2 callbacks tensorflow/java/src/main • C header/impl: 3809 LoC • Java: 7034 LoC 38 TensorFlow

| 39

| 2015 - 2018 40

| 41

| 42

| unistd.class pid_t getpid(void); jextract: .h -> .class package mypackage; @Header(path=”unistd.h”) public interface unistd { @C(file=”unistd.h”, line=3, column=1, USR=”c:@F@getpid”) @NativeType(ctype=”pid_t (void)”, size=1) @CallingConvention(1) public int getpid(); } 43 unistd.h

| Example: getpid() // Load native library Library lib = NativeLibrary.load(“c”); // Weave a class for the unistd interface, // and return an instance unistd unistd = NativeLibrary.create(lib, unistd.class) // Call the system getpid() function int pid = unistd.getpid(); 44

| Off-heap Access • Native pointers are modelled with generic class Pointer<X> – Pointer<X> = address + layout pointee + carrier • Basic operations – Offset, cast, dereference (get/set), iteration • Pointers lifecycle managed by Scope – Cannot dereference a pointer whose owning scope has been closed! • Native arrays are modelled with generic class Array<X> – Array<X> = Pointer<X> + size 45 Pointers and Arrays

| Off-heap Access 46 Scope

| Rethinking Panama • Panama FFI scored pretty well in terms of ease of use, but fell short when it came to providing reusable components – No primitives to see here, move along! • In hindsight, it had been about “jacking up the house and laying new pipes & plumbing while decorating the living room furniture” (J. Rose) 47 2018 foreign jextract, binder, API, …

| 48 The Quest for Low-level API Foreign ABI foreign-abi Memory Access API foreign-memaccess foreign jextract

| 2018 - now 49

| Workflow 50 foreign $ jextract … --source unistd.h $ javac … unistd_h.java Getpid.java $ cat Getpid.java … unistd_h.getpid() …

| Foreign 51 http://hg.openjdk.java.net/panama/dev/raw-file/63dc2eb214ef/doc/panama_foreign.html

| pid_t getpid(void); public final class unistd_h { […] public static final MethodHandle getpid = RuntimeHelper.downcallHandle( LIBRARIES, "getpid", "()I", FunctionDescriptor.of(C_UINT), false); public static final int getpid() { try { return (int)getpid.invokeExact(); } catch (Throwable ex) { throw new AssertionError(ex); } } […] } 52

| System ABI 1. SystemABI abi = SystemABI.getInstance(); 2. MemoryAddress entry = LibraryLookup.ofDefault().lookup("getpid"); 3. MethodType type = MethodType.methodType(int.class); 4. FunctionDescriptor desc = FunctionDescriptor.of(MemoryLayouts.C_INT); 5. MethodHandle getpid = abi.downcallHandle(entry, type, desc); 53 Example

| Native Method Handles MethodHandle.linkToNative, Universal Adapter, Programmable Invoker 54

| Java Native Construction Lookup.findVirtual() et al Lookup.findNative() Reference (typed) DirectMethodHandle NativeMethodHandle Reference (direct) MemberName NativeEntryPoint Linker MH.linkToVirtual() et al MH.linkToNative() Invocation indy, MH.invoke(), MH.invokeExact() “Making native calls from the JVM” by John Rose http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html 55 Native Method Handles

| // int pid = mh.invokeExact(); @ 8 LambdaForm$MH::invokeExact_MT (29 bytes) force inline by annotation @ 11 Invokers::checkExactType (17 bytes) force inline by annotation @ 1 MethodHandle::type (5 bytes) accessor @ 15 Invokers::checkCustomized (23 bytes) force inline by annotation @ 1 MethodHandleImpl::isCompileConstant (2 bytes) (intrinsic) @ 25 LambdaForm$NMH::invokeNative_I (27 bytes) force inline by … @ 7 NativeMethodHandle::internalNativeEntryPoint (8 bytes) force inline … @ 23 MethodHandle::linkToNative(L)I (0 bytes) direct native call 56

| Native Method Handles getpid JNI 13.7 ± 0.5 ns Direct call (unsafe) 3.4 ± 0.2 ns callq 0x1057b2eb0 ; native method entry 57

| “Universal Adapter” • jdk.internal.nicl.NativeInvoker – alternative to MH.linkToNative() – static native void invokeNative(long[] args, long[] rets, long[] recipe, long addr); • Very powerful mechanism – fully programmable through the “recipe” • Rich functionality support – varargs – struct-by-value argument + return – callbacks (function pointers) – multiple values return • Hard to optimize by JITs 58

| 59 The Quest for Low-level API foreign-abi System ABI, Programmable Invoker foreign-memaccess Memory Access API foreign Jextract, Universal invoker linkToNative

| Programmable Invoker • MH.linkToNative and Universal Invoker are complementary approaches – l2N • JIT-friendly, but only when the call site ABI can be represented – Universal Invoker is generic • covers many intricate corner ABI cases at the cost of performance • Unify interface between – MethodHandle.linkToNative(…)… (signature polymorphic) – static native void invokeNative(long[] args, long[] rets, long[] recipe, long addr) • Choose the implementation at runtime – intrinsify during JIT-compilation; fallback to interpretering ABI recipe 60 foreign-abi

| Memory Access API Low-level, safe and efficient memory access across a variety of memory sources (on-heap, off-heap) • Key abstractions – MemorySegment → memory region with spatial and temporal bounds – MemoryAddress → offset within a segment – MemoryLayout → description of a segment’s contents – MemoryHandles → factory for memory access handles (VarHandles) 61

| Safety • Principle: safe API, hard VM crashes should not be possible! – Non-critical conditions (e.g. reading an int value as a float) treated as user errors • Memory access - what can go wrong? – Out-of-bounds access – Access to already freed memory • To achieve safety, the memory access API provides strong spatial and temporal safety guarantees – Bounds and liveness state checks enforced on every access 62 Spatial and temporal safety

| Working with segments 63 An example // struct { int x; int y; } points[5]; // for (int i = 0; i < 5; i++) { points[i].x = i; } VarHandle intHandle = MemoryHandles.varHandle(int.class, ByteOrder.nativeOrder()); VarHandle xHandle = MemoryHandles.withStride(intHandle, 8); try (MemorySegment points = MemorySegment.allocateNative(4 * 2 * 5)) { MemoryAddress base = points.baseAddress(); for (long i = 0 ; i < 5 ; i++) { xHandle.set(base, i, (int)i); // safety checks } } // points.close() frees memory

| Working with layouts 64 An example struct Point { int x; int y; } pts[5]; MemoryLayout.ofSequence(5, MemoryLayout.ofStruct( MemoryLayout.ofValueBits(32) .withName(“x”), MemoryLayout.ofValueBits(32) .withName(“y”) ) );

| 65 // struct { int x; int y; } points[5]; SequenceLayout seq = MemoryLayout.ofSequence(5, MemoryLayout.ofStruct( MemoryLayout.ofValueBits(32, ByteOrder.nativeOrder()).withName("x"), MemoryLayout.ofValueBits(32, ByteOrder.nativeOrder()).withName("y"))); // i -> points[i].x VarHandle seqXHandle = seq.varHandle(int.class, MemoryLayout.PathElement.sequenceElement(), MemoryLayout.PathElement.groupElement("x")); // for (int i = 0; i < 5; i++) { points[i].x = i; } try (MemorySegment points = MemorySegment.allocateNative(seq)) { MemoryAddress base = points.baseAddress(); long size = seq.elementCount().getAsLong(); for (long i = 0; i < size; i++) { seqXHandle.set(base, i, (int) i); } }

| Upcalls Native => Java 66

| qsort // void qsort(void* base, size_t nel, size_t width, // int (*compar)(const void*, const void*)); // public interface stdlib_h.qsort$__compar { // int apply(MemoryAddress x0, MemoryAddress x1); // } stdlib_h.qsort$__compar comparator = (addr1, addr2) -> { int e1 = (int)INT_VH.get(addr1.rebase(segment)); int e2 = (int)INT_VH.get(addr2.rebase(segment)); return e1 - e2; }; MemoryAddress comparatorAddr = stdlib_h.qsort$__compar$make(comparator); stdlib_h.qsort(arrayBase, elementCount, elementSize, comparatorAddr); 67

| qsort MethodHandle qsort = abi.downcallHandle(lookup.lookup("qsort"), MethodType.methodType(void.class, MemoryAddress.class, long.class, long.class, MemoryAddress.class), FunctionDescriptor.ofVoid(false, C_POINTER, C_ULONG, C_ULONG, C_POINTER)); // Upcall handler static int qsortCompare(MemoryAddress addr1, MemoryAddress addr2) { return (int)intHandle.get(addr1) - (int)intHandle.get(addr2); } MethodHandle compar = MethodHandles.lookup().findStatic(StdLibTest.class, "qsortCompare", MethodType.methodType(int.class, MemoryAddress.class, MemoryAddress.class)); MemoryAddress qsortUpcallAddr = abi.upcallStub(compar, qsortFunction); // qsort call qsort.invokeExact(nativeArr.baseAddress(), seq.elementsCount().getAsLong(), C_INT.byteSize(), qsortUpcallAddr); 68

| 69 Preparation: 1. interface QsortComparator { int compare(MemoryAddress addr1, MemoryAddress addr2); } 2. MethodHandle qsortCompare = MethodHandles.lookup(). findVirtual(QsortComparator.class, "compare", methodType(int.class, MemoryAddress.class, MemoryAddress.class)); 3. MethodHandle qsort = abi.downcallHandle(library.lookup("qsort"), methodType(void.class, MemoryAddress.class, long.class, long.class, MemoryAddress.class), FunctionDescriptor.ofVoid(C_POINTER, C_ULONG, C_ULONG, C_POINTER)); Invocation: 4. QsortComparator comparator = (addr1, addr2) -> ... // user-defined comparator 5. MemoryAddress comparatorAddr = abi.upcallStub( qsortCompare.bindTo(comparator), FunctionDescriptor.of(C_INT, C_POINTER, C_POINTER)); 6. qsort.invokeExact(arrayAddr, elemCount, elemSize, comparatorAddr);

| Upcalls 70 Call Java code from native code lower is better

| qsort 71 higher is better

| Downcalls getpid JNI 13.7 ± 0.5 ns Direct call (unsafe) 3.4 ± 0.2 ns Direct call (safe) 13.6 ± 0.6 ns callq 0x1057b2eb0 ; native method entry 72

| Calling Convention hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp: “The Java calling convention is a "shifted" version of the C ABI. By skipping the first C ABI register we can call non-static jni methods with small numbers of arguments without having to shuffle the arguments at all. Since we control the java ABI we ought to at least get some advantage out of it.“ 73 Java vs C arg 1st 2nd 3rd 4th 5th 6th … C RDI RSI RDX RCX R8 R9 stack Java RSI RDX RCX R8 R9 RDI stack System V AMD64 ABI

| Optimize Java <=> native transitions void run() { nativeFunc1(); // checks & state trans. nativeFunc2(); // checks & state trans. nativeFunc3(); // checks & state trans. } 74

| 75 Java Java Heap Native Memory VM Native Thread State

| Optimize Java <=> native transitions void run() { transitionJavaToNative(); call nativeFunc1(); call nativeFunc2(); call nativeFunc3(); transitionNativeToJava(); } 76

| Java Java Heap Native Memory VM Native Thread State 77

| foreign + foreign-abi + foreign-memaccess • Ease of use: from header files to native bundles with jextract • Rich API provides seamless integration with native code – much of the JNI boilerplate can now be expressed in Java! • A safer alternative to JNI – Scope API manages resource lifecycles (pointers, structs, callbacks, ...) • Room for performance improvement is huge – Reduce latency of native calls, hoist native transitions out of loops, ... • Not just for C! – Solid low-level foundation 78 Scorecard

| 79 https://openjdk.java.net/jeps/370

| JDK 14 80

| Safe Harbor Statement The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. 81

JUGNsk Meetup #13. Владимир Иванов: Project Panama: как сделать Java "ближе к железу"?

JUGNsk Meetup #13. Владимир Иванов: Project Panama: как сделать Java "ближе к железу"?

More Decks by jugnsk

Other Decks in Programming

Featured

Transcript