Foreign Function and Memory API - jextract

Slide 1

Slide 1 text

Bridge Java to the native world FFM API and jextract Brice Dutheil

Slide 2

Slide 2 text

Bio Brice DUTHEIL est un ingénieur logiciel chez Datadog. Il a construit son expérience avec du backend type EJB/WAR/EAR, du Spring Boot, puis des pods kubernetes – d’abord en tant que développeur puis ingénieur de production – dans le secteur du transport et de la téléphonie avant de rejoindre Datadog. Il a également été un des gros contributeurs à Mockito. Aujourd'hui il travaille sur le développement de plugin pour les produits JetBrains. https://blog.arkey.fr https://twitter.com/BriceDutheil

Slide 3

Slide 3 text

Native access history Jogl (Java OpenGL) Gluegen generate bindings (java & C code) Opensourced in 2010 FFM (Foreign Function & Memory API) to bind to native libraries JDK 1.0 1996 Native interface ? JDK 1.1 1997 JNI (Java Native Interface) JDK 1.5 2004 JNA (Java Native Access) 2006 JDK 8 2014 JDK 22 2023-03 JNR (Java Native Runtime) 2010 JDK 7 2011 Method handles JExtract utility (Oracle) to Java generate bindings (source and/or bytecode) SWIG (Simpliﬁed Wrapper and Interface Generator) Covers more that C or Java JavaCPP generate bindings (java & C code) JEP: 472: Prepare to Restrict the Use of JNI

Slide 4

Slide 4 text

2024-03-19 22 https://www.oracle.com/a/ocom/img/obic-java-cup.svg https://upload.wikimedia.org/wikipedia/commons/1/18/OpenJDK_logo.svg The people The Java spec version

Slide 5

Slide 5 text

JEP-454 : Foreign Function & Memory API Introduce an API by which Java programs can interoperate with code and data outside of the Java runtime. By efﬁciently invoking foreign functions (i.e., code outside the JVM), and by safely accessing foreign memory (i.e., memory not managed by the JVM), the API enables Java programs to call native libraries and process native data without the brittleness and danger of JNI. “

Slide 6

Slide 6 text

Slide 7

Slide 7 text

9 years in the making JEP-191 Foreign Function Interface JEP-338 Vector API (Incubator) JEP-370 Foreign Memory Access API (Incubator) JEP-383 Foreign Memory Access API (2nd Incubator) JEP-389 Foreign Linker API (Incubator) JEP-393 Foreign Memory Access API (3rd Incubator) JEP-414 Vector API (2nd Incubator) JEP-412 Foreign Function & Memory API JEP-417 Vector API (3nd Incubator) JEP-419 Foreign Function & Memory API (Second Incubator) JEP-424 Foreign Function & Memory API (Preview) JEP-426 Vector API (Fourth Incubator) JEP-434 Foreign Function & Memory API (Second Preview) JEP-438 Vector API (Fifth Incubator) JEP-442 Foreign Function & Memory API (Third Preview) JEP-448 Vector API (Sixth Incubator) JEP 454 Foreign Function & Memory API JEP 460 Vector API (Seventh Incubator) JDK 22 2014 2023

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Why native ? Native functions are needed for various reasons ● OS: interacting with devices, ﬁle systems, network interfaces, … ● Third party libraries ○ Unique feature, unique implementation, performance ■ e.g. Harfbuzz, Libsodium, blake3, vectorscan, fswatch, tcmalloc, libVLC, CEF, clang, etc.

Slide 11

Slide 11 text

Slide 12

Slide 12 text

JNI package q.r.s; class NativeBinding { static { System.load("path/to/libNative.so"); } public static native boolean isatty( int fileDescriptor ); } #include "q_r_s_Native.h" #include JNIEXPORT jboolean JNICALL Java_q_r_s_NativeBinding_isatty( JNIEnv *env, jclass cls, jint fileDescriptor ) { return isatty(fileDescriptor)? JNI_TRUE: JNI_FALSE; } $ javac -d classes -cp src \ -h jni q/r/s/Native.java $ gcc \ -I $JAVA_HOME/include \ -I $JAVA_HOME/include/linux \ -fpic \ -shared \ -o libNative.so \ Native.c 🥇 Available since Java 1.1 🏎 Eﬃcient 🏗 Complex and painful build ☣ Can terminate the JVM 💡 Before JDK10 use javah. See JEP-313

Slide 13

Slide 13 text

JNI The base of everything To call native code, the JVM must perform some tricks ● Initialize a valid C stack ● Initialize CPU register for the OS and CPU calling convention, and switch them back on return ● Use CPU memory barrier instructions (fencing) ● Prevent some JIT optimisations like inlining ● If critical sections are declared (Get*Critical) prevent the GC to run at the very least on the impacted région ● Data exchange (objects, arrays) likely require copy (unless in critical section) ○ And likely serialization

Slide 14

Slide 14 text

JNI JNA package q.r.s; class NativeBinding { static { System.load("path/to/libNative.so"); } public static native boolean isatty( int fileDescriptor ); } #include "q_r_s_Native.h" #include JNIEXPORT jboolean JNICALL Java_q_r_s_NativeBinding_isatty( JNIEnv *env, jclass cls, jint fileDescriptor ) { return isatty(fileDescriptor)? JNI_TRUE: JNI_FALSE; } $ javac -d classes -cp src \ -h jni q/r/s/Native.java $ gcc \ -I $JAVA_HOME/include \ -I $JAVA_HOME/include/linux \ -fpic \ -shared \ -o Native.so \ Native.c public interface JNA_Library extends Library { JNA_Library INSTANCE = (JNA_Library) Native.loadLibrary( "c", JNA_Library.class ); boolean isatty(int fileDescriptor); } 💪 Concept 1996, released 2006 📦 Third party 🎚 Easy to use, no C build 🚜 Versatile, not fast

Slide 15

Slide 15 text

Featureful JNA ● Supports native types (wstring, structure, union, etc.) ● Lots of reflection used for ease-of-use ● Handles serialization, memory segments ● Based upon JNI (invisible) to invoke its own native library ● JNA Native lib will perform dispatch via libffi libffi is an assembly handwritten library that sets up the C callsite (c stack, registers)

Slide 16

Slide 16 text

JNI JNA JNR-FFI package q.r.s; class NativeBinding { static { System.load("path/to/libNative.so"); } public static native boolean isatty( int fileDescriptor ); } #include "q_r_s_Native.h" #include JNIEXPORT jboolean JNICALL Java_q_r_s_NativeBinding_isatty( JNIEnv *env, jclass cls, jint fileDescriptor ) { return isatty(fileDescriptor)? JNI_TRUE: JNI_FALSE; } $ javac -d classes -cp src \ -h jni q/r/s/Native.java $ gcc \ -I $JAVA_HOME/include \ -I $JAVA_HOME/include/linux \ -fpic \ -shared \ -o Native.so \ Native.c public interface JNA_Library extends Library { JNA_Library INSTANCE = (JNA_Library) Native.loadLibrary( "c", JNA_Library.class ); boolean isatty(int fileDescriptor); } public interface IsATTY_JNRFFI { boolean isatty(int fileDescriptor); } import jnr.ffi.LibraryLoader; IsATTY_JNRFFI c = LibraryLoader .create(IsATTY_JNRFFI.class) .load("c"); 📦 Third party 🎚 Easy to use 🏎 Almost as fast as JNI

Slide 17

Slide 17 text

JNR-FFI Simple and faster than JNA ● Supports native types (wstring, structure, union, etc.) ● Handles classloader ● Generates bytecode instead of reflection, way more efficient ● Based upon JNI (invisible) to invoke its own native library ● Also rely on libffi ● Inspiration for JEP-191, and Project Panama

Slide 18

Slide 18 text

Foreign Function and Memory var linker = Linker.nativeLinker(); var lookup = linker.defaultLookup();

Slide 19

Slide 19 text

Foreign Function and Memory var linker = Linker.nativeLinker(); var lookup = linker.defaultLookup(); var isatty = lookup.find("isatty").orElseThrow();

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Foreign Function and Memory var linker = Linker.nativeLinker(); var lookup = linker.defaultLookup(); var isatty = lookup.find("isatty").orElseThrow(); var isatty_MH = linker.downcallHandle(isatty, FunctionDescriptor.of( ValueLayout.JAVA_INT, ValueLayout.JAVA_INT )); int result = isatty_MH.invokeExact(fileDescriptor); ☕ First party, made by Oracle 🎚 Easy to use 🏎 Faster than JNI Thanks to inlining and less conversions

Slide 22

Slide 22 text

Understand FFM API usage

Slide 23

Slide 23 text

JVM Restrictions JDK Team starting to enforce integrity and safety by default e.g. JDK 21, dynamic agent loading raise a warning, in later JDK disabled by default JNI might be restricted too Native memory must be explicitly enabled via cli or manifest (JDK 22) --enable-native-access=ALL-UNNAMED | {modules} Enable-Native-Access: ALL-UNNAMED ℹ JEP 472: Prepare to Restrict The Use of JNI: https://openjdk.org/jeps/472

Slide 24

Slide 24 text

Bad code can crash the VM # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x0000000104a0e9d8, pid=52896, tid=9987 # # JRE version: OpenJDK Runtime Environment (22.0+26) (build 22-ea+26-2112) # Java VM: OpenJDK 64-Bit Server VM (22-ea+26-2112, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64) # Problematic frame: # V [libjvm.dylib+0x9b69d8] Unsafe_GetByte(JNIEnv_*, _jobject*, _jobject*, long)+0x14c # # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # An error report file with more information is saved as: # /Users/brice.dutheil/opensource/panama-watch/hs_err_pid52896.log # # If you would like to submit a bug report, please visit: # https://bugreport.java.com/bugreport/crash.jsp #

Slide 25

Slide 25 text

Foreign Function and Memory Suppose the printf function

Slide 26

Slide 26 text

Foreign Function and Memory Suppose the printf function Standard libc function are documented in man pages man 3 printf

Slide 27

Slide 27 text

Foreign Function and Memory Suppose the printf function Standard libc function are documented in man pages man 3 printf

Slide 28

Slide 28 text

Foreign Function and Memory The printf is very similar to System.out.printf, 1. A format string 2. Variable arguments

Slide 29

Slide 29 text

Foreign Function and Memory The printf is very similar to System.out.printf System.out.printf("%-10.3f%n", pi); printf("%-10.3f\n", pi);

Slide 30

Slide 30 text

Simpliﬁed call without format arguments: printf("printed to stdout") Foreign Function and Memory var printf = LINKER.downcallHandle( SYMBOL_LOOKUP.find("printf").orElseThrow(), FunctionDescriptor.of( ValueLayout.JAVA_LONG, ValueLayout.ADDRESS ) );

Slide 31

Slide 31 text

The process And then …handling memory The JVM The Over Simpliﬁed “memory chart” ™ A native lib Java heap message

Slide 32

Slide 32 text

The process And then …handling memory The JVM A native lib Java heap message 1. Allocate new Off-heap segment of memory and copy content message The Over Simpliﬁed “memory chart” ™

Slide 33

Slide 33 text

The process And then …handling memory The JVM A native lib Java heap message 2. Pass the pointer of native segment to function function message The Over Simpliﬁed “memory chart” ™

Slide 34

Slide 34 text

The process And then …handling memory The JVM A native lib Java heap message Now there’s litter in the process memory ? function message The Over Simpliﬁed “memory chart” ™

Slide 35

Slide 35 text

Foreign Function and Memory 4 concepts to grasp ● Arenas : allow memory lifecycle management, form allocation to disposal ● Memory segments : a contiguous region of bytes ● Memory layouts : data model in a memory segment ● MethodHandle, VarHandle : polymorphic glue to access native function & memory

Slide 36

Slide 36 text

Foreign Function and Memory ● FFM API is designed in a way to prevent misuse for memory in particular. ● FFM comes with Arenas, they control the lifecycle of native memory segments. ○ Multiple characteristics are available ■ Global ⟹ Deallocation ❌ ■ Automatic ⟹ Deallocation❓ (when GC is passing by) ■ Conﬁned ⟹ Deallocate at close 🚧 Singled 🧵 ■ Shared ⟹ Deallocate at close 🚧, Shared by multiple 🧵🧵🧵🧵… ○ Implementing custom memory management is possible, ■ Interfaces: Arena ➝ SegmentAllocator

Slide 37

Slide 37 text

For the printf example, let’s create a conﬁned arena Foreign Function and Memory try (var arena = Arena.ofConfined()) { }

Slide 38

Slide 38 text

For the printf example, then let’s allocate and copy Java string content Foreign Function and Memory try (var arena = Arena.ofConfined()) { var memorySegment = arena.allocateFrom(str); }

Slide 39

Slide 39 text

For the printf example, then make the call Foreign Function and Memory try (var arena = Arena.ofConfined()) { var memorySegment = arena.allocateFrom(str); return (long) printf.invoke(memorySegment); }

Slide 40

Slide 40 text

Slide 41

Slide 41 text

MemorySegment Represents a contiguous “segment” of bytes, on-heap or off-heap All segments have a start address in memory, this value is used as the pointer Accessible via MemorySegment::address All segments have a size Accessible via MemorySegment::byteSize A segment with a zero-length is an address

Slide 42

Slide 42 text

MemorySegment Represents a contiguous “segment” of memory The process The JVM A native lib Java heap 2E502E2E A0FB992E function Segments of memory The Over Simpliﬁed “memory chart” ™

Slide 43

Slide 43 text

Pointers everywhere Pointers are links to everything Even native functions are actually memory address Actually an address, returns a zero-length MemorySegment var printf = LINKER.downcallHandle( SYMBOL_LOOKUP.find("printf").orElseThrow(), FunctionDescriptor.of( ValueLayout.JAVA_LONG, ValueLayout.ADDRESS ) );

Slide 44

Slide 44 text

MemorySegment Represents a contiguous “segment” of memory The process The JVM A native lib Java heap 2E502E2E A0FB992E function Segments of memory 0-length The Over Simpliﬁed “memory chart” ™

Slide 45

Slide 45 text

Exchanging structured data Passing strings and raw types is almost easy as usual factories are there, e.g. allocateFrom(elementLayout, elementsArray) Fun begin with complex layout, use MemoryLayout factories

Slide 46

Slide 46 text

Exchanging structured data typedef struct { uint8_t buf_len; uint8_t flags; } basic_struct; MemoryLayout.structLayout( JAVA_BYTE.withName("buf_len"), JAVA_BYTE.withName("flags"), ).withName("basic_struct"); uint8_t buf[8]; MemoryLayout.sequenceLayout( 64, JAVA_BYTE ).withName("buf") typedef struct { uint8_t buf[8]; uint8_t buf_len; uint8_t flags; } basic_struct; MemoryLayout.structLayout( bufLayout, JAVA_BYTE.withName("buf_len"), JAVA_BYTE.withName("flags"), ).withName("basic_struct");

Slide 47

Slide 47 text

Exchanging structured data Passing strings and raw types is almost easy as usual factories are there, e.g. allocateFrom(elementLayout, elementsArray) Fun begins with complex layout, use MemoryLayout factories ● structLayout(…), structure like ● sequenceLayout(…), array like ● unionLayout(…), allow different structs over the same storage ● paddingLayout(…), padding require understanding of what means alignment

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Exchanging structured data Structured access to elements basic_struct_LAYOUT = MemoryLayout.structLayout(bufLayout, JAVA_BYTE.withName("buf_len"), JAVA_BYTE.withName("flags")).withName("basic_struct"); VarHandle flags = basic_struct_LAYOUT.varHandle( MemoryLayout.PathElement.groupElement("flags")); flags.get(memorySegment, 0L); flags.set(memorySegment, 0L, (byte) 0x2); Exception in thread "main" java.lang.invoke.WrongMethodTypeException: cannot convert MethodHandle(VarHandle,MemorySegment,long,byte)void to (VarHandle,MemorySegment,int)void Notice the enforced type, WrongMethodTypeException if incorrect

Slide 53

Slide 53 text

Day to day tips

Slide 54

Slide 54 text

Loading libraries Standard or common libraries are already loaded by the JVM process ● Nothing to do ● Symbols are available via Linker.nativeLinker().defaultLookup() e.g. printf, isatty, etc. SYMBOL_LOOKUP = Linker.nativeLinker().defaultLookup() SYMBOL_LOOKUP.find("symbol") Tied to symbol lookup

Slide 55

Slide 55 text

Loading libraries JNI library loading mechanism via System::load or System::loadLibrary ● ⚠ Lifecycle tied to the classloader ● Library and symbols can only be “appearing” once SYMBOL_LOOKUP = SymbolLookup.loaderLookup() SYMBOL_LOOKUP.find("symbol")

Slide 56

Slide 56 text

Loading libraries FFM API has a ﬂexible loading mechanism ● Not tied to classloader, but to an Arena, that can be explicitly controlled SYMBOL_LOOKUP = SymbolLookup.libraryLookup("path/to/libblake3.so", LIBRARY_ARENA) SYMBOL_LOOKUP.find("symbol") Usually LIBRARY_ARENA = Arena.ofAuto();

Slide 57

Slide 57 text

Loading libraries FFM API has a ﬂexible loading mechanism ● Not tied to classloader, but to an Arena, that can be explicitly controlled ● Composable lookup SYMBOL_LOOKUP = SymbolLookup.libraryLookup("path/to/libblake3.so", LIBRARY_ARENA) .or(SymbolLookup.loaderLookup()) .or(Linker.nativeLinker().defaultLookup()); SYMBOL_LOOKUP.find("symbol")

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Handling error in C with errno errno is like a global error state in the C standard library, ● it’s an integer variable set by the function that was called https://man7.org/linux/man-pages/man3/errno.3.html ioctl = LINKER.downcallHandle( SYMBOL_LOOKUP.find("ioctl").orElseThrow(), ioctlFunctionDescriptor, Linker.Option.captureCallState("errno") ); var csl = Linker.Option.captureStateLayout(); var errnoHandle = csl.varHandle( PathElement.groupElement("errno") ); result = (int) ioctl.invokeExact(…)

Slide 61

Slide 61 text

Slide 62

Slide 62 text

MethodHandles As you noticed, this API relies a lot on invokedynamic features. ● VarHandle ● MethodHandle This means we beneﬁt from all the features ● binding arguments (similar to currying) ● Interception ● etc. and it’s errors ● WrongMethodTypeException in particular

Slide 63

Slide 63 text

Bad code example using printf Foreign Function and Memory try (var arena = Arena.ofConfined()) { var memorySegment = arena.allocateFrom(str); return (long) printf.invoke(memorySegment.address()); } Wrong

Slide 64

Slide 64 text

Understanding WrongMethodTypeException If WrongMethodTypeException shows up, read the message Exception in thread "main" java.lang.invoke.WrongMethodTypeException: handle's method type (MemorySegment)long but found (long)long at java.base/java.lang.invoke.Invokers.newWrongMethodTypeException(Invokers.java:521) at java.base/java.lang.invoke.Invokers.checkExactType(Invokers.java:530) at io.whatever.App.c_printf(App.java:41)

Slide 65

Slide 65 text

Understanding WrongMethodTypeException handle's method type (MemorySegment)long Indicates what was described in the descriptor but found (long)long Indicates what was seen on call site FunctionDescriptor.of( ValueLayout.JAVA_LONG, ValueLayout.ADDRESS ) ℹ ValueLayout.ADDRESS carrier type is MemorySegment (long) printf .invokeExact(segment.address());

Slide 66

Slide 66 text

Variadic function in the MethodHandle section E.g. int printf(const char * restrict format, ...) ● It’s not really supported in the JDK ● JDK team opted for specialized downcall ○ new downcall declaration for new sequence of argos ● JDK advocate use of jextract for this feature to work. ● Required to declare the downcallHandle with the index of the ﬁrst variadic param LINKER.downcallHandle(…, Linker.Option.firstVariadicArg(2));

Slide 67

Slide 67 text

Interacting with non-C language Linker and SymbolLookup works with C conventions Foreign code needs to visible as a C function …(usually declared with extern) Swift @_cdecl("authenticate_user_touchid") public func authenticateUserApi() { … } Rust #[no_mangle] pub extern "C" fn abort() { … } Objective-C extern void cameraCapture(); Also look at cbindgen to generate C binding ﬁrst https://jornvernee.github.io/java/panama/rust/panama-fﬁ/2021/09/03/rust-panama-helloworld.html

Slide 68

Slide 68 text

Graalvm native-image support Currently baking, e.g. in JDK 21 preview API ● Missing features: variadic functions, shared arenas, upcalls, etc. ● Requires some special conﬁguration at build time https://www.graalvm.org/latest/reference-manual/native-image/native-code-interoperability/foreign-interface/ class FFMRegistrationFeature implements Feature { public void duringSetup(DuringSetupAccess access) { RuntimeForeignAccess.registerForDowncall( FunctionDescriptor.ofVoid() ); RuntimeForeignAccess.registerForDowncall( FunctionDescriptor.ofVoid(), Linker.Option.isTrivial() ); … } } --features=com.example.FFMRegistrationFeature -H:+UnlockExperimentalVMOptions -H:+ForeignAPISupport --enable-native-access=ALL-UNNAMED native-image.properties

Slide 69

Slide 69 text

About sun.misc.Unsafe Used for low level concurrency library, and others ● Netty (used by quarkus), Akka, Hazelcast, Apache Spark, Java based implementation of some codecs JDK developers aim to remove it at some point …but alternative supported API are not yet performant enough JEP 471: Deprecate the Memory-Access Methods in sun.misc.Unsafe for Removal A collection of methods for performing low-level, unsafe operations.

Slide 70

Slide 70 text

Automating with jextract

Slide 71

Slide 71 text

Binding Automation: JavaCPP Supports rich C++ constructs Generation at build time Conﬁguration medium : Java class Existing binding (javacpp-presets)

Slide 72

Slide 72 text

Binding Automation: jextract https://twitter.com/devoxx/status/1715655857797550378 External CLI tool Supports C/C++ constructs ● Even complex declarations ● No C code generated Maintained by JDK team ⚠ Stable but still as mentions early access

Slide 73

Slide 73 text

Binding Automation: jextract Navigate to https://jdk.java.net/jextract/ ℹ Possibly add the executable permission : chmod +x jextract-22/jextract Or remove macOs quarantine ﬂags : sudo xattr -r -d com.apple.quarantine jextract-22/jextract/

Slide 74

Slide 74 text

Basic usage If you have a simple library $ jextract \ --target-package org.lib \ --header-class-name Lib \ lib.h org.lib.Lib

Slide 75

Slide 75 text

Generated code E.g. with BLAKE3/c/blake3.h (and --header-class-name blake3_h) Generated API usage var hasher = blake3_hasher.allocate(arena); blake3_h.blake3_hasher_init(hasher); var content = arena.allocateFrom("Hello Panama!"); blake3_h.blake3_hasher_update( hasher, content, content.byteSize() - 1 ); allocate returns MemorySegment with the blake3_hasher struct layout Functions and other symbols exposed as statics under blake3_h

Slide 76

Slide 76 text

Generated code E.g. with BLAKE3/c/blake3.h (and --header-class-name blake3_h) /** * {@snippet lang=c : * struct { * uint32_t key[8]; * blake3_chunk_state chunk; * uint8_t cv_stack_len; * uint8_t cv_stack[1760]; * } * } */ public class blake3_hasher { … /** * {@snippet lang=c : * void blake3_hasher_init(blake3_hasher *self) * } */ public static void blake3_hasher_init(MemorySegment self) { …

Slide 77

Slide 77 text

Conﬁguring library loading Is library on system library path? If yes Specify shared lib name $ jextract … \ --library GL \ --library GLU \ … Lookup will look for standard locations /usr/lib /usr/lib64 /lib /lib64 ├── libGL.so └── libGLU.so /usr/local/lib /usr/local/lib64 generates SymbolLookup.libraryLookup(System.mapLibraryName("GL"), …); SymbolLookup.libraryLookup(System.mapLibraryName("GLU"), …);

Slide 78

Slide 78 text

Or, prefix path with a colon : Configuring library loading Is library on system library path ? If no, either don’t pass --library flag, and load the library yourself before using bindings. Specify library path $ jextract … \ --library :path/to/libblake3.so \ … SymbolLookup.libraryLookup("path/to/libblake3.so", …) Explicit load SymbolLookup.libraryLookup( "path/to/libblake3.so", … ); // Then use generated bindings generates Note the colon

Slide 79

Slide 79 text

Dealing with includes Add include dirs $ jextract \ --target-package org.lib \ --include-dir path1/to/include \ --include-dir path2/to/include \ lib.h E.g. Speciﬁc library include path : /usr/local/Cellar/ffmpeg@4/4.4.4/include Or system includes macOs: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/ …or: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include Linux: /usr/include/ lib.h #include "relative/path1/a.h" #include "relative/path2/b.h" …

Slide 80

Slide 80 text

Multiple headers Juste create a synthetic single header ﬁle Declare every included headers Pass synthetic header $ jextract …\ my-multiple.h my-multiple.h #include "path/to/libOneThing.h" #include "path/to/libOtherThing.h" …

Slide 81

Slide 81 text

Choose generated binding explicitly Dumping symbol include conﬁguration Dump all symbols ﬁrst $ jextract … \ --dump-includes my-dump.txt \ blake3.h

Slide 82

Slide 82 text

Choose generated binding explicitly How to be speciﬁc about which symbol ? Dump all symbols ﬁrst $ jextract … \ --dump-includes my-dump.txt \ blake3.h my-dump.txt … #### Extracted from: …/blake3.h --include-struct blake3_chunk_state # header: …/c/blake3.h --include-struct blake3_hasher # header: …/blake3.h … --include-struct --include-function --include-constant --include-typedef --include-union --include-var

Slide 83

Slide 83 text

Choose generated binding explicitly How to be speciﬁc about which symbol ? Dump all symbols ﬁrst $ jextract … \ --dump-includes my-dump.txt \ blake3.h my-dump.txt … #### Extracted from: …/blake3.h --include-struct blake3_chunk_state # header: …/c/blake3.h --include-struct blake3_hasher # header: …/blake3.h … Selected “symbols” $ jextract … \ --include-struct blake3_chunk_state \ --include-struct blake3_hasher \ blake3.h Select which symbol should have a binding

Slide 84

Slide 84 text

Use a argument file Like the java command, jextract supports arguments file my-config.txt $ jextract … \ @my-config.txt --output path/to/gen-sources/jextract-blake3/java --target-package blake3 --header-class-name blake3_h --include-struct blake3_chunk_state # header: …/c/blake3.h --include-struct blake3_hasher # header: …/blake3.h … blake3.h

Slide 85

Slide 85 text

Caution Native headers may not be platform independent ● OS speciﬁc macros (preprocessed by the C preprocessor before the binding) ● Imprecise C types, e.g. long (different length depending on the platform), Precise types: int64_t, long long (instead of long) JExtract may produce bindings that may behave incorrectly on other platforms. ➡ Double check, jextract’s generated code on other platform, if unsure about library ☣ ℹ OpenGL is even more tricky, check out PanamaGL : https://gitlab.com/jzy3d/panama-gl (needs contributors)

Slide 86

Slide 86 text

JEP 454: Foreign Function & Memory API Javadoc JDK 22 java.base/java.lang.foreign Oracle Devguide for JDK 21: Foreign Function and Memory API JExtract : https://github.com/openjdk/jextract ● From C to Java Code using Panama by Johannes Bechberger ● The Panama Dojo: Black Belt Programming with Java 21 & the FFM API by Per Minborg ● JVMLS: Project Panama - Foreign Function & Memory API by Maurizio Cimadamore Some projects ● Vulkan on macOs : GitHub - chengenzhao/java-vulkan-mac Foreign Function and Memory API - jextract