Tu Modules ?

5ce0156bc2f1864459fd77229eff3fe3?s=47 forax
October 05, 2017

Tu Modules ?

Presentation at Breizh JUG

5ce0156bc2f1864459fd77229eff3fe3?s=128

forax

October 05, 2017
Tweet

Transcript

  1. Rémi Forax Tu Modules Breizh JUG Sept 2017

  2. Me, Myself and I Maitre de conférence University Paris Est

    Marne la vallée
  3. Me, Myself and I Maitre de conférence University Paris Est

    Marne la vallée Developer: OpenJDK, ASM, valuetypify, … Java Community Process Expert – Invokedynamic (JSR 292) – Lambda (JSR 335 + java.util/java.util.stream) – Jigsaw (JSR 376) – Amber (Java 18.x), Valhalla (Java 18.x + 19.x)
  4. Disclaimer ! No Duke will be harm during this presentation

  5. Dramatic Intro !

  6. None
  7. No reliable configuration ClassPath ? – Linear scan :( –

    If two versions of the same package exist ? => Jar Hell Example: CLASSPATH = …:asm-2.3.jar:…:asm-3.1.jar A newly introduced class of ASM 3.1 will use an old class of ASM 2.3 => compile time information is different from runtime info
  8. No way to scale Down the platform Some libraries tend

    to be too big – OpenJDK 8 rt.jar => 66 Mb – Guava 20 => 2.3 Mb – Spring Context => 1.1 Mb on Maven Central IoT anyone ?
  9. No Strong Encapsulation The JDK has only one line of

    defense :( And 66 megs of privileged classes
  10. Why it takes so long ?

  11. The Jigsaw Sandwich Compile time Runtime

  12. Draw me a module

  13. Module descriptor in mymodule/module-info.java module foo { requires java.base; requires

    bar; exports com.acme.my.package; } requires → module dependencies exports → packages visible by other modules (classpath == unamed module)
  14. java.se java.sql.rowset java.sql java.xml java.base java.logging java.scripting java.datatransfer java.compiler java.desktop

    Module graph of java.se (part of)
  15. A modular jar Modular jar == plain old jar +

    module-info.class Configuration fidelity – module-info.java is verified by the compiler – module-info.class is verified by the VM
  16. Module Graph & module path From a module, the compiler

    and VM computes the acyclic dependency graph the ClassPath is not needed anymore The ClassPath is dead* Long live to the Module-Path * if you don’t care about backward compat. and migration
  17. Java 9 compatibility issues (1/3) The Java 9 JDK disallow

    split packages, one package == one module disallow access to non exported packages deep reflection on modules setAccessible can now fail
  18. Java 9 compatibility issues (2/3) Allowing illegal access with a

    migration flag --illegal-access=<value> permit or deny access to members of types in modules <value> is one of "deny", "permit", "warn", or "debug" This option will be removed in a future release
  19. Java 9 compatibility issues (3/3) Downsizing the JDK – Some

    packages have been deprecated java.corba, java.transaction, java.xml.bind, java.xml.ws, java.xml.ws.annotations They are not root modules and will be removed in Java 10? – Some classes (sun.*, com.sun.*) of the JDK have been deleted are in an non exported packages (may be deleted in the future)
  20. Command line java/javac Override root modules configuration --add-modules modules to

    add as root modules --upgrade-module-path to replace some root modules Patch a module --patch-module module=files patch existing classes of a module Add a dependency (can be done by reflection too) --add-reads module=anotherModule Break encapsulation / allow setAccessible --add-exports module/package=anotherModule --add-opens modue/package=anotherModule
  21. Discovering usage of internal APIs $ jdeps --jdk-internals target deps/vertx-*.jar

    target -> JDK removed internal API target -> jdk.unsupported io.snowcamp.papaya.doc.Document -> sun.misc.BASE64Decoder (JDK removed internal) io.snowcamp.papaya.doc.Document -> sun.misc.BASE64Encoder (JDK removed internal) io.snowcamp.papaya.reflect.ReflectionSupport -> sun.misc.Unsafe internal (jdk.unsupported) vertx-internal-3.3.3.jar -> java.base vertx-internal-3.3.3.jar -> jdk.unsupported io.netty.handler.ssl.util.OpenJdkSelfSignedCertGenerator -> sun.security.util.ObjectIdentifier internal (java.base) io.netty.util.internal.PlatformDependent0 -> sun.misc.Unsafe internal (jdk.unsupported) io.netty.util.internal.UnsafeAtomicIntegerFieldUpdater -> sun.misc.Unsafe internal (jdk.unsupported) ... JDK Internal API Suggested Replacement ---------------- --------------------- sun.misc.BASE64Decoder Use java.util.Base64 @since 1.8 sun.misc.BASE64Encoder Use java.util.Base64 @since 1.8 sun.misc.Unsafe See http://openjdk.java.net/jeps/260 sun.security.x509.X500Name Use javax.security.auth.x500.X500Principal @since 1.4
  22. Modularizing

  23. Example - ModuleTools A simple application – Allow to read/transform/write

    a module-info 4 packages – fr.umlv.moduletools.main • command line – fr.umlv.moduletools.trasform • chain transformations – fr.umlv.moduletools.api • module input, module output – fr.umlv.moduletools.internal • Uses ASM to read/write module-info.class • Uses javac to read module-info.java
  24. Packages dependencies – Java 8 java.* fr.umlv.moduletools.api fr.umlv.moduletools.internal fr.umlv.moduletools.main fr.umlv.moduletools.transform

    org.objectweb.asm.* asm6.jar
  25. Modules - jigsaw java.base org.objectweb.asm.* asm6.jar module moduletools.main { requires

    java.base; requires moduletools.api: exports fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.api moduletools.api fr.umlv.moduletools.internal java.compiler
  26. Modules - jigsaw java.base org.objectweb.asm.* asm6.jar module moduletools.main { requires

    java.base; requires moduletools.api: exports fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.api moduletools.api fr.umlv.moduletools.internal java.compiler
  27. 3 kind of requires 3 kinds of requires – default

    • not transitive – transitive (1 hop) • If B requires transitive C, if A requires B, A get all exports of C – static • Requires at compile-time, maybe not present at runtime java --add-modules foo
  28. Layout of modules on disc Two layouts – Legacy layout,

    1 module in 1 project, use --source-path – Multi-module layout, several modules in 1 project, use --module-source-path javac -d target/main/exploded \ --module-source-path src/main/java \ $(find src/main/java -name "*.java")
  29. Multi-modules / module-source-path javac -d target/main/exploded \ --module-source-path src/main/java \

    $(find src/main/java -name "*.java") On disc src/main/java • moduletools.main module-info.java fr/umlv/moduletools/transform • … .java fr/umlv/moduletools/main • Main.java • moduletools.api module-info.java ... a folder with a qualified name
  30. Create module moduletools.main javac -d target/main/exploded \ --module-source-path src/main/java/ \

    $(find src/main/java/ -name "*.java") jar --create \ --file target/main/artifact/moduletools.main-1.0.jar \ --module-version=1.0 \ --main-class fr.umlv.moduletools.main.Main \ -C target/exploded/moduletools.main .
  31. Automatic module

  32. Modules java.base org.objectweb.asm.* asm-6.jar module moduletools.main { requires moduletools.api: exports

    fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.parser moduletools.api fr.umlv.moduletools.asm java.compiler module moduletools.api { requires java.compiler: requires asm; exports fr.umlv.moduletools.api; }
  33. Create module moduletools.api javac -d target/main/exploded \ --module-source-path src/main/java \

    --class-path deps/asm-6.jar \ $(find src/main/java -name "*.java") This doesn't compile !!
  34. Mixing modules and jars Modules in module-path do not read

    (see) classes in classpath modulepath classpath (unamed module) java.compiler moduletools.main moduletools.api asm-6.jar OK ! NO ! java.base
  35. Automatic modules (migration mode) Plain old jar in module-path –

    can be used in requires – can read modules – can read any jars from the classpath modulepath classpath (unamed module) another-jar.jar modulepath module moduletools.api { requires java.compiler: requires asm; exports …; } java.compiler moduletools.main moduletools.api asm-6.jar java.base
  36. Name of an automatic module How the runtime find the

    module name ? – Look the property Automatic-Module-Name in the Manifest – Otherwise, use the name of the jar If you are responsible of a Maven Central artifact • Add a AutomaticModuleName in the Manifest
  37. None
  38. Create module moduletools.api javac -d target/main/exploded \ --module-source-path src/main/java \

    --module-path deps \ $(find src/main/java -name "*.java") It works !
  39. Run moduletools As legacy jars (Java 9 is compatible with

    Java <9) $ /usr/jdk/jdk-9/bin/java --class-path \ target/main/artifact/moduletools.api-1.0.jar: \ target/main/artifact/moduletools.main-1.0.jar: \ deps/asm-6.jar fr.umlv.moduletools.main.Main \ ... As modular jars $ java --module-path target/main/artifact:deps \ --module moduletools.main \ ...
  40. Better encapsulation

  41. With 3 Modules java.base org.objectweb.asm.* asm-6.jar module moduletools.main { requires

    moduletools.api: exports fr.umlv.moduletools.main; } fr.umlv.moduletools.main moduletools.main fr.umlv.moduletools.transform fr.umlv.moduletools.api moduletools.api java.compiler module moduletools.api { requires moduletools.impl; exports fr.umlv.moduletools.api; } moduletools.impl fr.umlv.moduletools.internal module moduletools.impl { requires java.compiler: requires asm; exports fr.umlv.moduletools.internal; }
  42. Restricted export An exported package can be restricted to a

    set of modules java.base org.objectweb.asm.* asm-6.jar fr.umlv.moduletools.api moduletools.api java.compiler module moduletools.api { requires moduletools.impl; exports fr.umlv.moduletools.api; } moduletools.impl fr.umlv.moduletools.internal module moduletools.impl { requires java.compiler: requires asm; exports fr.umlv.moduletools.internal to moduletools.api; }
  43. sun.misc.Unsafe in 9 ? $ javap -m java.base module-info module

    java.base { …, exports sun.misc; exports jdk.internal.misc to java.rmi, java.sql, jdk.charsets, … } $ javap -m jdk.unsupported module-info module jdk.unsupported { exports sun.misc; } Keep sun.misc in 9, remove features when a public replacement APIs exist New internal unsafe API (hidden)
  44. Deep reflection Changing final fields, accessing private members, etc is

    now invalid ! The VM doesn’t allow deep reflection by default setAccessible can now fail at runtime !
  45. Open Module / Open package Deep reflection must to be

    allowed explicitly with the directive opens module foo.bean { exports boo.bean.business; opens boo.bean.business; } Or you can open all the packages of a module open module foo.bean { exports boo.bean.business; }
  46. Module Service

  47. Module Service Want to decouple the interface ModuleInput (resp. ModuleOutput)

    from its implementations Moduletools.api – fr.umlv.moduletools.api • ModuleInput.java – static List<ModuleInput> availableInputs(); • ClassModuleInput.java • SourceModuleInput.java API Implementations
  48. Module Service / Code package fr.umlv.moduletools.api; public interface ModuleInput {

    Optionial<ModuleDescriptor> read(String name); } package fr.umlv.moduletools.api; … public interface ModuleInput { public static List<ModuleInput> availableInputs() { return ServiceLoader.load(ModuleInput.class).stream() ...; } } package fr.umlv.moduletools.service; public class ClassMdouleInput implements ModuleInput { … } interface an implementation usage
  49. Uses / Provides … with java.base org.objectweb.asm.* asm-6.jar module moduletools.main

    { … } moduletools.main fr.umlv.moduletools.api moduletools.api java.compiler module moduletools.api { exports fr.umlv.moduletools.api; uses ... .api.ModuleInput; } moduletools.impl fr.umlv.moduletools.internal module moduletools.impl { requires java.compiler: requires asm; exports fr.umlv.moduletools.internal to moduletools.service; } module moduletools.service { requires moduletools.api; requires moduletools.impl; provides … .api.ModuleInput with … .service.SouceModuleInput; } fr.umlv.moduletools.service moduletools.service
  50. Going Native !

  51. jlink Create a custom runtime image from a set of

    root modules Image is specific for a platform Intel 64bits Linux, ARM 32bits Windows, etc For small devices or in the cloud (BTW, jlink is not mandatory !)
  52. jlink and automatic module ! $ jlink --module-path $JAVA_HOME/jmods: \

    target/main/artifact: \ deps \ --add-modules moduletools.main, \ moduletools.service \ --output image Error: jdk.tools.jlink.plugin.PluginException : module-info.class not found for asm module Automatic modules can see classes from the classpath so the world in not closed :(
  53. jlink launcher In fact, ASM 6 is already a jigsaw

    module ! $ rm -fr image $ jlink --module-path $JAVA_HOME/jmods: \ target/main/artifact: \ deps \ --add-modules moduletools.main, \ moduletools.service \ --strip-debug \ --launcher moduletools=moduletools.main \ --output image
  54. jlink image - Run moduletools $ ls -R image image:

    bin conf lib release image/bin: moduletools java keytool image/lib: amd64 classlist jexec modules security tzdb.dat … image/lib/amd64/server: classes.jsa libjsig.so libjvm.so Xusage.txt Run the executable $ ./image/bin/moduletools Run with the 'java' wrapper $ ./image/bin/java -m moduletools.main
  55. Minimal VM On small device, use the minimal VM, $

    jlink --module-path $JAVA_HOME/jmods: \ target/main/artifact:deps \ --add-modules moduletools.main, \ moduletools.service \ --strip-debug \ --launcher moduletools=moduletools.main \ --vm=minimal \ --output image size of the classical VM: 20 megs size of the minimal VM: 3 megs
  56. Ahead Of Time compilation experimental feature ! Create a shared

    library (.so, .dll, .dynlib) from a module 2 modes – Non-tiered-mode: use AOT libs + interpreter – tiered-mode: use AOT libs + interpreter + JIT $ jaotc --module-path target/main/artifact --module moduletools.main
  57. Ahead Of Time compilation experimental feature ! On disk $

    ls lib/ libasm.all.so libjdk.compiler.so libmoduletools.impl.so libmoduletools.service.so libjava.base.so libmoduletools.api.so libmoduletools.main.so Run in AOT mode ./target/image/bin/java \ -XX:+UseAOT \ -XX:AOTLibrary=./lib/libjava.base.so \ -XX:AOTLibrary=./lib/libasm.all.so \ -XX:AOTLibrary=./lib/libmoduletools.impl.so \ -XX:AOTLibrary=./lib/libmoduletools.api.so \ -XX:AOTLibrary=./lib/libmoduletools.service.so \ -XX:AOTLibrary=./lib/libmoduletools.main.so \ -m moduletools.main/fr.umlv.moduletools.main.Main \ ...
  58. BenchMarks !

  59. Time to run moduletools

  60. Size on disc

  61. Jigsaw Goals Reliable configuration – No classpath anymore – compile

    time / runtime fidelity Scale down the platform – downsized unit of re-use Strong encapsulation – Separate public types from implementation – Enhanced security Closed World – Ahead of time optimizations
  62. Questions ? Code is available at https://github.com/forax/moduletools