Exploiter facilement des fonctions natives avec le projet panama

Courte introduction du tools-in-action au Devoxx France 2022 sur l'appel de fonctions natives sur le JDK18 et la JEP-419.
Code: https://github.com/bric3/panama-watch

Transcript

1. Exploiter facilement
   des fonctions natives
   avec le Projet Panama
   Sans JNI et sans librairies tierces
   

   View Slide

2. Orateur
   Brice Dutheil
   @BriceDutheil
   Attribution: https://cr.openjdk.java.net/~mcimadamore/panama/panama-hat.png
   

   View Slide

3. Qui a déjà utilisé du code natif depuis la JVM ?
   

   View Slide

4. Agenda
   1. Le projet panama, c’est quoi ?
   2. Demo JEP-419
   3. Demo de jextract
   4. Question
   

   View Slide

5. Le projet Panama ?
   

   View Slide

6. Panama
   Interconnecting Java with the native world.
   

   View Slide

7. We had alternatives
   package q.r.s;
   class NativeBinding {
   static {
   System.load("{path/to/libNative.so} ");
   }
   public static native boolean isatty(
   int fileDescriptor
   );
   }
   #include "q_r_s_Native.h"
   #include
   JNIEXPORT jboolean JNICALL
   Java_q_r_s_NativeBinding_isatty(
   JNIEnv *env, jclass cls, jint fileDescriptor
   ) {
   return isatty(fileDescriptor)?
   JNI_TRUE:
   JNI_FALSE;
   }
   $ javac -d classes -cp src \
   -h jni q/r/s/Native.java
   $ gcc \
   -I $JAVA_HOME/include \
   -I $JAVA_HOME/include/linux \
   -fPIC \
   -shared \
   -o Native.so \
   Native.c
   JNI
   🏎 Efficient
   🏗 Complex Build
   ☣ Can terminate the JVM
   

   View Slide

8. We had alternatives
   package q.r.s;
   class NativeBinding {
   static {
   System.load("{path/to/libNative.so} ");
   }
   public static native boolean isatty(
   int fileDescriptor
   );
   }
   #include "q_r_s_Native.h"
   #include
   JNIEXPORT jboolean JNICALL
   Java_q_r_s_NativeBinding_isatty(
   JNIEnv *env, jclass cls, jint fileDescriptor
   ) {
   return isatty(fileDescriptor)?
   JNI_TRUE:
   JNI_FALSE;
   }
   $ javac -d classes -cp src \
   -h jni q/r/s/Native.java
   $ gcc \
   -I $JAVA_HOME/include \
   -I $JAVA_HOME/include/linux \
   -fPIC \
   -shared \
   -o Native.so \
   Native.c
   JNI JNA
   public interface JNA_Library extends Library
   {
   JNA_Library INSTANCE =
   (JNA_Library) Native.loadLibrary(
   "c",
   JNA_Library.class
   );
   boolean isatty(int fileDescriptor);
   }
   📦 Third party
   🎚 Easy to use
   🚜 Versatile, not fast
   

   View Slide

9. We had alternatives
   package q.r.s;
   class NativeBinding {
   static {
   System.load("{path/to/libNative.so} ");
   }
   public static native boolean isatty(
   int fileDescriptor
   );
   }
   #include "q_r_s_Native.h"
   #include
   JNIEXPORT jboolean JNICALL
   Java_q_r_s_NativeBinding_isatty(
   JNIEnv *env, jclass cls, jint fileDescriptor
   ) {
   return isatty(fileDescriptor)?
   JNI_TRUE:
   JNI_FALSE;
   }
   $ javac -d classes -cp src \
   -h jni q/r/s/Native.java
   $ gcc \
   -I $JAVA_HOME/include \
   -I $JAVA_HOME/include/linux \
   -fPIC \
   -shared \
   -o Native.so \
   Native.c
   JNI JNA
   public interface JNA_Library extends Library
   {
   JNA_Library INSTANCE =
   (JNA_Library) Native.loadLibrary(
   "c",
   JNA_Library.class
   );
   boolean isatty(int fileDescriptor);
   }
   JNR-FFI
   public interface IsATTY_JNRFFI {
   boolean isatty(int fileDescriptor);
   }
   import jnr.ffi.LibraryLoader;
   IsATTY_JNRFFI c =
   LibraryLoader.create(IsATTY_JNRFFI.class)
   .load("c");
   📦 Third party
   🎚 Easy to use
   🏎 Almost as fast as
   JNI
   

   View Slide

10. Support from the runtime
    Objective:
    ● easy support of foreign function without 3rd party
    ● efficient
    ● flexible
    ● improved security
    ● rely on MethodHandles
    

    View Slide

11. ● JEP-191
    ● 14 ⇒ JEP-370
    ● 15 ⇒ JEP-383
    ● 16 ⇒ JEP-338, JEP-389, JEP-393
    ● 17 ⇒ JEP-412, JEP-414
    ● 18 ⇒ JEP-417 Vector API (3nd Incubator)
    JEP-419 Foreign Function & Memory API
    (2nd Incubator)
    ● 19 ⇒ JEP-424 Foreign Function & Memory API(Preview)
    

    View Slide

12. Invoke native code with JDK 18
    Project is still incubating, so it requires to add modules
    ● for compilation
    ● and for execution
    $ javac --add-modules jdk.incubator.foreign ...
    $ java --enable-native-access=ALL-UNNAMED \
    --add-modules jdk.incubator.foreign ...
    

    View Slide

13. Invoke native code with JDK 18
    var lookup = CLinker.systemCLinker();
    System.load(libPath.toAbsolutePath().toString());
    var lookup = SymbolLookup.loaderLookup();
    Choose the right lookup
    ● systemLookup for system symbols
    like those in libc
    ● loaderLookup for loaded libraries
    

    View Slide

14. Invoke native code with JDK 18
    lookup.lookup("crypto_box_keypair").get()
    Get the address of the native function
    

    View Slide

15. Invoke native code with JDK 18
    MethodHandle crypto_box_keypair =
    CLinker.systemCLinker().downcallHandle(
    lookup.lookup("crypto_box_keypair").get(),
    FunctionDescriptor.ofVoid(ADDRESS, ADDRESS)
    );
    Create the method handle
    

    View Slide

16. Invoke native code with JDK 18
    try (var scope = ResourceScope.newConfinedScope()) {
    var segmentAllocator = SegmentAllocator.nativeAllocator(scope);
    var recipientPublicKey = segmentAllocator.allocate(crypto_box_publickeybytes());
    var recipientSecretKey = segmentAllocator.allocate(crypto_box_secretkeybytes());
    crypto_box_keypair.invoke(recipientPublicKey.address(),
    recipientSecretKey.address());
    }
    Open a scope (for memory safety)
    ⚠ newImplicitScope ⟹ closed by GC
    👌 newConfinedScope ⟹ full control
    

    View Slide

17. Invoke native code with JDK 18
    try (var scope = ResourceScope.newConfinedScope()) {
    var segmentAllocator = SegmentAllocator.nativeAllocator(scope);
    var recipientPublicKey = segmentAllocator.allocate(crypto_box_publickeybytes());
    var recipientSecretKey = segmentAllocator.allocate(crypto_box_secretkeybytes());
    crypto_box_keypair.invoke(recipientPublicKey.address(),
    recipientSecretKey.address());
    }
    Allocate memory within that scope
    

    View Slide

18. Invoke native code with JDK 18
    try (var scope = ResourceScope.newConfinedScope()) {
    var segmentAllocator = SegmentAllocator.nativeAllocator(scope);
    var recipientPublicKey = segmentAllocator.allocate(crypto_box_publickeybytes());
    var recipientSecretKey = segmentAllocator.allocate(crypto_box_secretkeybytes());
    crypto_box_keypair.invoke(recipientPublicKey.address(),
    recipientSecretKey.address());
    }
    Make the call
    

    View Slide

19. Practice
    

    View Slide

20. Better memory mapped files
    Memory mapping files in memory is a OS feature
    Allows to put the region of a file in memory
    MappedByteBuffer implements this feature
    ● ⚠ stays in memory until the buffer itself is garbage collected
    

    View Slide

21. Replacing MappedByteBuffer
    try (var fileChannel = FileChannel.open(src, StandardOpenOption.READ)) {
    var mappedByteBuffer = fileChannel.map(
    FileChannel.MapMode.READ_ONLY,
    0,
    fileChannel.size());
    }
    FileChannel.map create the memory
    mapped region
    

    View Slide

22. Replacing MappedByteBuffer
    try (var scope = ResourceScope.newConfinedScope()) {
    var mappedFile = MemorySegment.mapFile(
    src,
    0,
    Files.size(src),
    FileChannel.MapMode.READ_ONLY,
    scope);
    }
    Map the file in memory
    

    View Slide

23. Practice
    

    View Slide

24. Le projet Panama ?
    ● JEP 419 (Java 18)
    https://openjdk.java.net/jeps/419
    ● A practical look at JEP 412 (Java 17) with Libsodium
    https://blog.arkey.fr/2021/09/04/a-practical-look-at-jep-412-in-j
    dk17-with-libsodium/
    ● Java Project Panama au ParisJug Nov 2021
    (deep dive de 1h30)
    https://youtu.be/hrqi-KJ_74I
    ● https://inside.java
    ● https://github.com/bric3/panama-watch
    

    View Slide

25. Big API changes after Java 17 (JEP-412)
    ● https://bugs.openjdk.java.net/browse/JDK-8273905
    https://github.com/openjdk/panama-foreign/pull/576
    ●
    

    View Slide