$30 off During Our Annual Pro Sale. View Details »

Tu Modules ?

forax
October 05, 2017

Tu Modules ?

Presentation at Breizh JUG

forax

October 05, 2017
Tweet

More Decks by forax

Other Decks in Programming

Transcript

  1. Rémi Forax Tu Modules Breizh JUG Sept 2017

  2. Me, Myself and I Maitre de conférence University Paris Est

    Marne la vallée
  3. Me, Myself and I Maitre de conférence University Paris Est

    Marne la vallée Developer: OpenJDK, ASM, valuetypify, … Java Community Process Expert – Invokedynamic (JSR 292) – Lambda (JSR 335 + java.util/java.util.stream) – Jigsaw (JSR 376) – Amber (Java 18.x), Valhalla (Java 18.x + 19.x)
  4. Disclaimer ! No Duke will be harm during this presentation

  5. Dramatic Intro !

  6. None
  7. No reliable configuration ClassPath ? – Linear scan :( –

    If two versions of the same package exist ? => Jar Hell Example: CLASSPATH = …:asm-2.3.jar:…:asm-3.1.jar A newly introduced class of ASM 3.1 will use an old class of ASM 2.3 => compile time information is different from runtime info
  8. No way to scale Down the platform Some libraries tend

    to be too big – OpenJDK 8 rt.jar => 66 Mb – Guava 20 => 2.3 Mb – Spring Context => 1.1 Mb on Maven Central IoT anyone ?
  9. No Strong Encapsulation The JDK has only one line of

    defense :( And 66 megs of privileged classes
  10. Why it takes so long ?

  11. The Jigsaw Sandwich Compile time Runtime

  12. Draw me a module

  13. Module descriptor in mymodule/module-info.java module foo { requires java.base; requires

    bar; exports com.acme.my.package; } requires → module dependencies exports → packages visible by other modules (classpath == unamed module)
  14. java.se java.sql.rowset java.sql java.xml java.base java.logging java.scripting java.datatransfer java.compiler java.desktop

    Module graph of java.se (part of)
  15. A modular jar Modular jar == plain old jar +

    module-info.class Configuration fidelity – module-info.java is verified by the compiler – module-info.class is verified by the VM
  16. Module Graph & module path From a module, the compiler

    and VM computes the acyclic dependency graph the ClassPath is not needed anymore The ClassPath is dead* Long live to the Module-Path * if you don’t care about backward compat. and migration
  17. Java 9 compatibility issues (1/3) The Java 9 JDK disallow

    split packages, one package == one module disallow access to non exported packages deep reflection on modules setAccessible can now fail
  18. Java 9 compatibility issues (2/3) Allowing illegal access with a

    migration flag --illegal-access=<value> permit or deny access to members of types in modules <value> is one of "deny", "permit", "warn", or "debug" This option will be removed in a future release
  19. Java 9 compatibility issues (3/3) Downsizing the JDK – Some

    packages have been deprecated java.corba, java.transaction, java.xml.bind, java.xml.ws, java.xml.ws.annotations They are not root modules and will be removed in Java 10? – Some classes (sun.*, com.sun.*) of the JDK have been deleted are in an non exported packages (may be deleted in the future)
  20. Command line java/javac Override root modules configuration --add-modules modules to

    add as root modules --upgrade-module-path to replace some root modules Patch a module --patch-module module=files patch existing classes of a module Add a dependency (can be done by reflection too) --add-reads module=anotherModule Break encapsulation / allow setAccessible --add-exports module/package=anotherModule --add-opens modue/package=anotherModule
  21. Discovering usage of internal APIs $ jdeps --jdk-internals target deps/vertx-*.jar

    target -> JDK removed internal API target -> jdk.unsupported io.snowcamp.papaya.doc.Document -> sun.misc.BASE64Decoder (JDK removed internal) io.snowcamp.papaya.doc.Document -> sun.misc.BASE64Encoder (JDK removed internal) io.snowcamp.papaya.reflect.ReflectionSupport -> sun.misc.Unsafe internal (jdk.unsupported) vertx-internal-3.3.3.jar -> java.base vertx-internal-3.3.3.jar -> jdk.unsupported io.netty.handler.ssl.util.OpenJdkSelfSignedCertGenerator -> sun.security.util.ObjectIdentifier internal (java.base) io.netty.util.internal.PlatformDependent0 -> sun.misc.Unsafe internal (jdk.unsupported) io.netty.util.internal.UnsafeAtomicIntegerFieldUpdater -> sun.misc.Unsafe internal (jdk.unsupported) ... JDK Internal API Suggested Replacement ---------------- --------------------- sun.misc.BASE64Decoder Use java.util.Base64 @since 1.8 sun.misc.BASE64Encoder Use java.util.Base64 @since 1.8 sun.misc.Unsafe See http://openjdk.java.net/jeps/260 sun.security.x509.X500Name Use javax.security.auth.x500.X500Principal @since 1.4
  22. Modularizing

  23. Example - ModuleTools A simple application – Allow to read/transform/write

    a module-info 4 packages – fr.umlv.moduletools.main • command line – fr.umlv.moduletools.trasform • chain transformations – fr.umlv.moduletools.api • module input, module output – fr.umlv.moduletools.internal • Uses ASM to read/write module-info.class • Uses javac to read module-info.java
  24. Packages dependencies – Java 8 java.* fr.umlv.moduletools.api fr.umlv.moduletools.internal fr.umlv.moduletools.main fr.umlv.moduletools.transform

    org.objectweb.asm.* asm6.jar
  25. Modules - jigsaw java.base org.objectweb.asm.* asm6.jar module moduletools.main { requires

    java.base; requires moduletools.api: exports fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.api moduletools.api fr.umlv.moduletools.internal java.compiler
  26. Modules - jigsaw java.base org.objectweb.asm.* asm6.jar module moduletools.main { requires

    java.base; requires moduletools.api: exports fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.api moduletools.api fr.umlv.moduletools.internal java.compiler
  27. 3 kind of requires 3 kinds of requires – default

    • not transitive – transitive (1 hop) • If B requires transitive C, if A requires B, A get all exports of C – static • Requires at compile-time, maybe not present at runtime java --add-modules foo
  28. Layout of modules on disc Two layouts – Legacy layout,

    1 module in 1 project, use --source-path – Multi-module layout, several modules in 1 project, use --module-source-path javac -d target/main/exploded \ --module-source-path src/main/java \ $(find src/main/java -name "*.java")
  29. Multi-modules / module-source-path javac -d target/main/exploded \ --module-source-path src/main/java \

    $(find src/main/java -name "*.java") On disc src/main/java • moduletools.main module-info.java fr/umlv/moduletools/transform • … .java fr/umlv/moduletools/main • Main.java • moduletools.api module-info.java ... a folder with a qualified name
  30. Create module moduletools.main javac -d target/main/exploded \ --module-source-path src/main/java/ \

    $(find src/main/java/ -name "*.java") jar --create \ --file target/main/artifact/moduletools.main-1.0.jar \ --module-version=1.0 \ --main-class fr.umlv.moduletools.main.Main \ -C target/exploded/moduletools.main .
  31. Automatic module

  32. Modules java.base org.objectweb.asm.* asm-6.jar module moduletools.main { requires moduletools.api: exports

    fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.parser moduletools.api fr.umlv.moduletools.asm java.compiler module moduletools.api { requires java.compiler: requires asm; exports fr.umlv.moduletools.api; }
  33. Create module moduletools.api javac -d target/main/exploded \ --module-source-path src/main/java \

    --class-path deps/asm-6.jar \ $(find src/main/java -name "*.java") This doesn't compile !!
  34. Mixing modules and jars Modules in module-path do not read

    (see) classes in classpath modulepath classpath (unamed module) java.compiler moduletools.main moduletools.api asm-6.jar OK ! NO ! java.base
  35. Automatic modules (migration mode) Plain old jar in module-path –

    can be used in requires – can read modules – can read any jars from the classpath modulepath classpath (unamed module) another-jar.jar modulepath module moduletools.api { requires java.compiler: requires asm; exports …; } java.compiler moduletools.main moduletools.api asm-6.jar java.base
  36. Name of an automatic module How the runtime find the

    module name ? – Look the property Automatic-Module-Name in the Manifest – Otherwise, use the name of the jar If you are responsible of a Maven Central artifact • Add a AutomaticModuleName in the Manifest
  37. None
  38. Create module moduletools.api javac -d target/main/exploded \ --module-source-path src/main/java \

    --module-path deps \ $(find src/main/java -name "*.java") It works !
  39. Run moduletools As legacy jars (Java 9 is compatible with

    Java <9) $ /usr/jdk/jdk-9/bin/java --class-path \ target/main/artifact/moduletools.api-1.0.jar: \ target/main/artifact/moduletools.main-1.0.jar: \ deps/asm-6.jar fr.umlv.moduletools.main.Main \ ... As modular jars $ java --module-path target/main/artifact:deps \ --module moduletools.main \ ...
  40. Better encapsulation

  41. With 3 Modules java.base org.objectweb.asm.* asm-6.jar module moduletools.main { requires

    moduletools.api: exports fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.api moduletools.api java.compiler module moduletools.api { requires moduletools.impl; exports fr.umlv.moduletools.api; } moduletools.impl fr.umlv.moduletools.internal module moduletools.impl { requires java.compiler: requires asm; exports fr.umlv.moduletools.internal; }
  42. Restricted export An exported package can be restricted to a

    set of modules java.base org.objectweb.asm.* asm-6.jar fr.umlv.moduletools.api moduletools.api java.compiler module moduletools.api { requires moduletools.impl; exports fr.umlv.moduletools.api; } moduletools.impl fr.umlv.moduletools.internal module moduletools.impl { requires java.compiler: requires asm; exports fr.umlv.moduletools.internal to moduletools.api; }
  43. sun.misc.Unsafe in 9 ? $ javap -m java.base module-info module

    java.base { …, exports sun.misc; exports jdk.internal.misc to java.rmi, java.sql, jdk.charsets, … } $ javap -m jdk.unsupported module-info module jdk.unsupported { exports sun.misc; } Keep sun.misc in 9, remove features when a public replacement APIs exist New internal unsafe API (hidden)
  44. Deep reflection Changing final fields, accessing private members, etc is

    now invalid ! The VM doesn’t allow deep reflection by default setAccessible can now fail at runtime !
  45. Open Module / Open package Deep reflection must to be

    allowed explicitly with the directive opens module foo.bean { exports boo.bean.business; opens boo.bean.business; } Or you can open all the packages of a module open module foo.bean { exports boo.bean.business; }
  46. Module Service

  47. Module Service Want to decouple the interface ModuleInput (resp. ModuleOutput)

    from its implementations Moduletools.api – fr.umlv.moduletools.api • ModuleInput.java – static List<ModuleInput> availableInputs(); • ClassModuleInput.java • SourceModuleInput.java API Implementations
  48. Module Service / Code package fr.umlv.moduletools.api; public interface ModuleInput {

    Optionial<ModuleDescriptor> read(String name); } package fr.umlv.moduletools.api; … public interface ModuleInput { public static List<ModuleInput> availableInputs() { return ServiceLoader.load(ModuleInput.class).stream() ...; } } package fr.umlv.moduletools.service; public class ClassMdouleInput implements ModuleInput { … } interface an implementation usage
  49. Uses / Provides … with java.base org.objectweb.asm.* asm-6.jar module moduletools.main

    { … } moduletools.main fr.umlv.moduletools.api moduletools.api java.compiler module moduletools.api { exports fr.umlv.moduletools.api; uses ... .api.ModuleInput; } moduletools.impl fr.umlv.moduletools.internal module moduletools.impl { requires java.compiler: requires asm; exports fr.umlv.moduletools.internal to moduletools.service; } module moduletools.service { requires moduletools.api; requires moduletools.impl; provides … .api.ModuleInput with … .service.SouceModuleInput; } fr.umlv.moduletools.service moduletools.service
  50. Going Native !

  51. jlink Create a custom runtime image from a set of

    root modules Image is specific for a platform Intel 64bits Linux, ARM 32bits Windows, etc For small devices or in the cloud (BTW, jlink is not mandatory !)
  52. jlink and automatic module ! $ jlink --module-path $JAVA_HOME/jmods: \

    target/main/artifact: \ deps \ --add-modules moduletools.main, \ moduletools.service \ --output image Error: jdk.tools.jlink.plugin.PluginException : module-info.class not found for asm module Automatic modules can see classes from the classpath so the world in not closed :(
  53. jlink launcher In fact, ASM 6 is already a jigsaw

    module ! $ rm -fr image $ jlink --module-path $JAVA_HOME/jmods: \ target/main/artifact: \ deps \ --add-modules moduletools.main, \ moduletools.service \ --strip-debug \ --launcher moduletools=moduletools.main \ --output image
  54. jlink image - Run moduletools $ ls -R image image:

    bin conf lib release image/bin: moduletools java keytool image/lib: amd64 classlist jexec modules security tzdb.dat … image/lib/amd64/server: classes.jsa libjsig.so libjvm.so Xusage.txt Run the executable $ ./image/bin/moduletools Run with the 'java' wrapper $ ./image/bin/java -m moduletools.main
  55. Minimal VM On small device, use the minimal VM, $

    jlink --module-path $JAVA_HOME/jmods: \ target/main/artifact:deps \ --add-modules moduletools.main, \ moduletools.service \ --strip-debug \ --launcher moduletools=moduletools.main \ --vm=minimal \ --output image size of the classical VM: 20 megs size of the minimal VM: 3 megs
  56. Ahead Of Time compilation experimental feature ! Create a shared

    library (.so, .dll, .dynlib) from a module 2 modes – Non-tiered-mode: use AOT libs + interpreter – tiered-mode: use AOT libs + interpreter + JIT $ jaotc --module-path target/main/artifact --module moduletools.main
  57. Ahead Of Time compilation experimental feature ! On disk $

    ls lib/ libasm.all.so libjdk.compiler.so libmoduletools.impl.so libmoduletools.service.so libjava.base.so libmoduletools.api.so libmoduletools.main.so Run in AOT mode ./target/image/bin/java \ -XX:+UseAOT \ -XX:AOTLibrary=./lib/libjava.base.so \ -XX:AOTLibrary=./lib/libasm.all.so \ -XX:AOTLibrary=./lib/libmoduletools.impl.so \ -XX:AOTLibrary=./lib/libmoduletools.api.so \ -XX:AOTLibrary=./lib/libmoduletools.service.so \ -XX:AOTLibrary=./lib/libmoduletools.main.so \ -m moduletools.main/fr.umlv.moduletools.main.Main \ ...
  58. BenchMarks !

  59. Time to run moduletools

  60. Size on disc

  61. Jigsaw Goals Reliable configuration – No classpath anymore – compile

    time / runtime fidelity Scale down the platform – downsized unit of re-use Strong encapsulation – Separate public types from implementation – Enhanced security Closed World – Ahead of time optimizations
  62. Questions ? Code is available at https://github.com/forax/moduletools