Exploiter facilement des fonctions natives avec le projet panama

Courte introduction du tools-in-action au Devoxx France 2022 sur l'appel de fonctions natives sur le JDK18 et la JEP-419.
Code: https://github.com/bric3/panama-watch

1. Exploiter facilement des fonctions natives avec le Projet Panama Sans JNI et sans librairies tierces
2. Orateur Brice Dutheil @BriceDutheil Attribution: https://cr.openjdk.java.net/~mcimadamore/panama/panama-hat.png
3. Qui a déjà utilisé du code natif depuis la JVM ?
4. Agenda 1. Le projet panama, c’est quoi ? 2. Demo JEP-419 3. Demo de jextract 4. Question
5. Le projet Panama ?
6. Panama Interconnecting Java with the native world.
7. We had alternatives package q.r.s; class NativeBinding { static { System.load("{path/to/libNative.so} "); } public static native boolean isatty( int fileDescriptor ); } #include "q_r_s_Native.h" #include <unistd.h> JNIEXPORT jboolean JNICALL Java_q_r_s_NativeBinding_isatty( JNIEnv *env, jclass cls, jint fileDescriptor ) { return isatty(fileDescriptor)? JNI_TRUE: JNI_FALSE; } $ javac -d classes -cp src \ -h jni q/r/s/Native.java $ gcc \ -I $JAVA_HOME/include \ -I $JAVA_HOME/include/linux \ -fPIC \ -shared \ -o Native.so \ Native.c JNI 🏎 Efficient 🏗 Complex Build ☣ Can terminate the JVM
8. We had alternatives package q.r.s; class NativeBinding { static { System.load("{path/to/libNative.so} "); } public static native boolean isatty( int fileDescriptor ); } #include "q_r_s_Native.h" #include <unistd.h> JNIEXPORT jboolean JNICALL Java_q_r_s_NativeBinding_isatty( JNIEnv *env, jclass cls, jint fileDescriptor ) { return isatty(fileDescriptor)? JNI_TRUE: JNI_FALSE; } $ javac -d classes -cp src \ -h jni q/r/s/Native.java $ gcc \ -I $JAVA_HOME/include \ -I $JAVA_HOME/include/linux \ -fPIC \ -shared \ -o Native.so \ Native.c JNI JNA public interface JNA_Library extends Library { JNA_Library INSTANCE = (JNA_Library) Native.loadLibrary( "c", JNA_Library.class ); boolean isatty(int fileDescriptor); } 📦 Third party 🎚 Easy to use 🚜 Versatile, not fast
9. We had alternatives package q.r.s; class NativeBinding { static { System.load("{path/to/libNative.so} "); } public static native boolean isatty( int fileDescriptor ); } #include "q_r_s_Native.h" #include <unistd.h> JNIEXPORT jboolean JNICALL Java_q_r_s_NativeBinding_isatty( JNIEnv *env, jclass cls, jint fileDescriptor ) { return isatty(fileDescriptor)? JNI_TRUE: JNI_FALSE; } $ javac -d classes -cp src \ -h jni q/r/s/Native.java $ gcc \ -I $JAVA_HOME/include \ -I $JAVA_HOME/include/linux \ -fPIC \ -shared \ -o Native.so \ Native.c JNI JNA public interface JNA_Library extends Library { JNA_Library INSTANCE = (JNA_Library) Native.loadLibrary( "c", JNA_Library.class ); boolean isatty(int fileDescriptor); } JNR-FFI public interface IsATTY_JNRFFI { boolean isatty(int fileDescriptor); } import jnr.ffi.LibraryLoader; IsATTY_JNRFFI c = LibraryLoader.create(IsATTY_JNRFFI.class) .load("c"); 📦 Third party 🎚 Easy to use 🏎 Almost as fast as JNI
10. Support from the runtime Objective: • easy support of foreign function without 3rd party • efficient • flexible • improved security • rely on MethodHandles
11. • JEP-191 • 14 ⇒ JEP-370 • 15 ⇒ JEP-383 • 16 ⇒ JEP-338, JEP-389, JEP-393 • 17 ⇒ JEP-412, JEP-414 • 18 ⇒ JEP-417 Vector API (3nd Incubator) JEP-419 Foreign Function & Memory API (2nd Incubator) • 19 ⇒ JEP-424 Foreign Function & Memory API(Preview)
12. Invoke native code with JDK 18 Project is still incubating, so it requires to add modules • for compilation • and for execution $ javac --add-modules jdk.incubator.foreign ... $ java --enable-native-access=ALL-UNNAMED \ --add-modules jdk.incubator.foreign ...
13. Invoke native code with JDK 18 var lookup = CLinker.systemCLinker(); System.load(libPath.toAbsolutePath().toString()); var lookup = SymbolLookup.loaderLookup(); Choose the right lookup • systemLookup for system symbols like those in libc • loaderLookup for loaded libraries
14. Invoke native code with JDK 18 lookup.lookup("crypto_box_keypair").get() Get the address of the native function
15. Invoke native code with JDK 18 MethodHandle crypto_box_keypair = CLinker.systemCLinker().downcallHandle( lookup.lookup("crypto_box_keypair").get(), FunctionDescriptor.ofVoid(ADDRESS, ADDRESS) ); Create the method handle
16. Invoke native code with JDK 18 try (var scope = ResourceScope.newConfinedScope()) { var segmentAllocator = SegmentAllocator.nativeAllocator(scope); var recipientPublicKey = segmentAllocator.allocate(crypto_box_publickeybytes()); var recipientSecretKey = segmentAllocator.allocate(crypto_box_secretkeybytes()); crypto_box_keypair.invoke(recipientPublicKey.address(), recipientSecretKey.address()); } Open a scope (for memory safety) ⚠ newImplicitScope ⟹ closed by GC 👌 newConfinedScope ⟹ full control
17. Invoke native code with JDK 18 try (var scope = ResourceScope.newConfinedScope()) { var segmentAllocator = SegmentAllocator.nativeAllocator(scope); var recipientPublicKey = segmentAllocator.allocate(crypto_box_publickeybytes()); var recipientSecretKey = segmentAllocator.allocate(crypto_box_secretkeybytes()); crypto_box_keypair.invoke(recipientPublicKey.address(), recipientSecretKey.address()); } Allocate memory within that scope
18. Invoke native code with JDK 18 try (var scope = ResourceScope.newConfinedScope()) { var segmentAllocator = SegmentAllocator.nativeAllocator(scope); var recipientPublicKey = segmentAllocator.allocate(crypto_box_publickeybytes()); var recipientSecretKey = segmentAllocator.allocate(crypto_box_secretkeybytes()); crypto_box_keypair.invoke(recipientPublicKey.address(), recipientSecretKey.address()); } Make the call
19. Practice
20. Better memory mapped files Memory mapping files in memory is a OS feature Allows to put the region of a file in memory MappedByteBuffer implements this feature • ⚠ stays in memory until the buffer itself is garbage collected
21. Replacing MappedByteBuffer try (var fileChannel = FileChannel.open(src, StandardOpenOption.READ)) { var mappedByteBuffer = fileChannel.map( FileChannel.MapMode.READ_ONLY, 0, fileChannel.size()); } FileChannel.map create the memory mapped region
22. Replacing MappedByteBuffer try (var scope = ResourceScope.newConfinedScope()) { var mappedFile = MemorySegment.mapFile( src, 0, Files.size(src), FileChannel.MapMode.READ_ONLY, scope); } Map the file in memory
23. Practice
24. Le projet Panama ? • JEP 419 (Java 18) https://openjdk.java.net/jeps/419 • A practical look at JEP 412 (Java 17) with Libsodium https://blog.arkey.fr/2021/09/04/a-practical-look-at-jep-412-in-j dk17-with-libsodium/ • Java Project Panama au ParisJug Nov 2021 (deep dive de 1h30) https://youtu.be/hrqi-KJ_74I • https://inside.java • https://github.com/bric3/panama-watch
25. Big API changes after Java 17 (JEP-412) • https://bugs.openjdk.java.net/browse/JDK-8273905 https://github.com/openjdk/panama-foreign/pull/576 •