Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Jigsaw will save us, ToulouseJUG Edition

Jigsaw will save us, ToulouseJUG Edition

What are the goals of jigsaw, what jigsaw is, what can we do with jigsaw, etc

forax

May 17, 2016
Tweet

More Decks by forax

Other Decks in Programming

Transcript

  1. Me, Myself and I Maitre de conférence University Paris Est

    Marne la vallée Developer: OpenJDK, ASM, Tatoo, Dragon, … Java Community Process Expert – Invokedynamic (JSR 292) – Lambda (JSR 335 + java.util/java.util.stream) – Jigsaw (JSR 376)
  2. No reliable configuration ClassPath ? – Linear scan :( –

    If two versions of the same package exist ? => Jar Hell Example: CLASSPATH = …:asm-2.3.jar:…:asm-3.1.jar A newly introduced class of ASM 3.1 will use an old class of ASM 2.3 => compile time information is different from runtime info
  3. No way to scale Down the platform Some libraries tend

    to be too big – OpenJDK 8 rt.jar => 66 Mb – Guava 1.9 => 2.3 Mb – Spring Context => 1.1 Mb on Maven Central IoT anyone ? and JDK classes are privileged :(
  4. No Strong Encapsulation 2013 is a very bad year for

    Java security The JDK has only one line of defense :(
  5. No Closed World Java Startup is slow – VM startup

    + JIT ramp up – Application initialization is slow • Load + verify classes dynamically • static final object graphs Java runtime is big – No 'tree shaking' Android does AOT !
  6. Jigsaw Goals Reliable configuration – No classpath anymore – compile

    time / runtime fidelity Scale down the platform – downsized unit of re-use Strong encapsulation – Separate public types from implementation – Enhanced security Closed World – Ahead of time optimization
  7. Meet the Dragon An interpreter of a toy language def(fib:

    \ n "returns the fibonacci number of n" if(n < 2 1 fib(n - 1) + fib(n – 2))) A small application delivered as a 'fat' jar (dragon.jar) – 244 classes (dependencies included) – 20 packages • 3 auto-generated from the grammar • 14 from tatoo-runtime.jar – 1 external dependency (tatoo-runtime.jar) agon
  8. jdeps – find package dependencies $ /usr/jdk/jdk-9/bin/jdeps dragon.jar dragon.jar ->

    java.base fr.umlv.dragon.ast (dragon.jar) -> fr.umlv.dragon.grammar.tools -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.grammar.lexer (dragon.jar) -> fr.umlv.tatoo.runtime. ... (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.parser (dragon.jar) -> fr.umlv.tatoo.runtime.parser (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.tools (dragon.jar) -> fr.umlv.dragon.grammar.lexer -> fr.umlv.dragon.grammar.parser -> fr.umlv.dragon.rt -> fr.umlv.tatoo.runtime. … (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.main (dragon.jar) -> fr.umlv.dragon.ast -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.rt (dragon.jar) -> java ...
  9. dragon.rt module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser

    dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { requires java.base; exports fr.umlv.dragon.rt; }
  10. Create module fr.umlv.dragon.rt javac -d build/modules \ -modulesourcepath src \

    $(find src/fr.umlv.dragon.rt -name "*.java") jar --create \ --file mlib/fr.umlv.dragon.rt-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.rt .
  11. dragon.rt module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser

    dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { // requires java.base; exports fr.umlv.dragon.rt; }
  12. dragon.grammar module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser

    dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; }
  13. Create module dragon.grammar javac -d build/modules \ -modulesourcepath grammar/gen-src \

    -modulepath mlib \ -classpath lib/tatoo-runtime.jar \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") This doesn't compile !!
  14. Mixing modules and jar Modules in modulepath do not read

    (see) classes in classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo-runtime.jar OK ! NO !
  15. Automatic modules (migration mode) Plain old jar in modulepath –

    can be used in requires – can read modules – can read any jars from the classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo.runtime-1.0.jar another-jar.jar modulepath module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; ...; } the jar also needs to be renamed !
  16. Create module dragon.grammar mv lib/tatoo-runtime.jar \ mlib/fr.umlv.tatoo.runtime-1.0.jar javac -d build/modules

    \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") It works !
  17. dragon.ast module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser

    dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }
  18. dragon.main module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser

    dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
  19. requires is not transitive by default ! dragon.rt java.base fr.umlv.tatoo.runtime.*

    dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; }
  20. requires public to the rescue ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast

    dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { // requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
  21. require public The public API of the module fr.umlv.dragon.ast refers

    to classes from a required module package fr.umlv.dragon.ast; import ... import fr.umlv.dragon.rt.Block; public class ASTBuilder { ... public static Block buildAST(Reader reader) { TerminalEvaluator<CharSequence> terminalEvaluator = new TokenEvaluator(); ASTEvaluator grammarEvaluator = new ASTEvaluator(); Analyzers.run(reader, terminalEvaluator, grammarEvaluator, null, null); return grammarEvaluator.script; } ... } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }
  22. Same thing with dragon.grammar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools

    dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast; }
  23. Create dragon.main module The main class is specified when creating

    the jar as usual $ javac -d build/modules \ -modulesourcepath src \ -modulepath mlib \ $(find src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.main-1.0.jar \ --module-version=1.0 \ --main-class fr.umlv.dragon.main.Main \ -C build/modules/fr.umlv.dragon.main
  24. Run dragon As jars (Java 9 is compatible with Java

    <9) $ /usr/jdk/jdk-9/bin/java -classpath \ mlib/fr.umlv.tatoo.runtime-1.0.jar: \ mlib/fr.umlv.dragon.grammar-1.0.jar: \ mlib/fr.umlv.dragon.ast-1.0.jar: \ mlib/fr.umlv.dragon.rt-1.0.jar: \ mlib/fr.umlv.dragon.main-1.0.jar \ fr.umlv.dragon.main.Main As modular jars $ java --modulepath mlib \ --m fr.umlv.dragon.main
  25. Restricted export module fr.umlv.dragon.grammar exports methods that are specific to

    the Tatoo parser compiler what if I want to change the parser in the future ? => restrict the visibility of a module !
  26. Now, only dragon.ast can 'requires' dragon.grammar ! dragon.rt java.base fr.umlv.tatoo.runtime.*

    dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools to fr.umlv.dragon.ast; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast; }
  27. sun.misc.Unsafe in 9 ? $ javap -m java.base module-info module

    java.base { …, exports sun.misc; exports jdk.internal.misc to java.rmi, java.sql, jdk.charsets, … } $ javap -m jdk.unsupported module-info module jdk.unsupported { exports sun.misc; } Keep sun.misc in 9, remove features when a public replacement APIs exist New internal unsafe API (hidden)
  28. Discovering usage of internal APIs $ jdeps -jdkinternals jruby-9.0.5.0/lib/jruby.jar jruby.jar

    -> java.base jruby.jar -> java.management org.jruby.management.BeanManagerImpl (jruby.jar) -> sun.management.Agent JDK internal API (java.management) org.jruby.util.encoding.ISO_8859_16 (jruby.jar) -> sun.nio.cs.US_ASCII JDK internal API (java.base) $ jdeps -jdkinternals gradle/lib/*.jar gradle/lib/plugins/*.jar gradle-core-2.12.jar -> java.base org.gradle.internal.filewatch.jdk7.WatchServiceRegistrar (gradle-core-2.12.jar) -> com.sun.nio.file.ExtendedWatchEventModifier JDK internal API (java.base) -> com.sun.nio.file.SensitivityWatchEventModifier JDK internal API (java.base) bsh-2.0b4.jar -> java.desktop bsh.util.AWTConsole (bsh-2.0b4.jar) -> java.awt.peer.ComponentPeer JDK internal API (java.desktop) -> java.awt.peer.TextComponentPeer JDK internal API (java.desktop) ...
  29. Command line java/javac Overidde root modules configuration -addmods module to

    add root modules -upgrademodulepath to replace some root modules Patch a module -Xpatch:module=files patch existing classes of a module Add a dependency (can be done by reflection too) -XaddReads:module=anotherModule Break encapsulation -XaddExports:module/package=anotherModule
  30. jlink Create a custom runtime image from a set of

    root modules Image is specific for a platform – Intel 64bits linux, ARM 32bits Windows, etc For small devices or in the cloud (jlink is not mandatory !)
  31. jlink and automatic module ! $ jlink --modulepath /usr/jdk/jdk-9-jigsaw/jmods:mlib --addmods

    fr.umlv.dragon.main --output image Error: jdk.tools.jlink.plugin.PluginException : module-info.class not found for fr.umlv.tatoo.runtime module Automatic modules can see classes from the classpath so the world in not closed :(
  32. Jigsawification by hand $ jdeps -genmoduleinfo tmp lib/tatoo-runtime.jar writing to

    tmp/tatoo.runtime/module-info.java $ cat tmp/tatoo.runtime/module-info.java module tatoo.runtime { exports fr.umlv.tatoo.runtime.ast; exports fr.umlv.tatoo.runtime.buffer; exports fr.umlv.tatoo.runtime.lexer; exports fr.umlv.tatoo.runtime.lexer.rules; exports fr.umlv.tatoo.runtime.log; exports fr.umlv.tatoo.runtime.parser; exports fr.umlv.tatoo.runtime.regex; exports fr.umlv.tatoo.runtime.tools; ... }
  33. Jigsawification by hand (2) $ javac -d build/modules \ -modulesourcepath

    tmp \ fr.umlv.tatoo.runtime/module-info.java $ jar --update \ --file mlib/fr.umlv.tatoo.runtime-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.tatoo.runtime \ module-info.class
  34. jlink my dragon Generating a custom image for dragon $

    rm -fr image $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --compress=2 \ --strip-debug \ --output image
  35. jlink image - Run dragon $ ls -R image image:

    bin conf lib release image/bin: fr.umlv.dragon.main java keytool image/lib: amd64 classlist jexec modules security tzdb.dat … image/lib/amd64/server: classes.jsa libjsig.so libjvm.so Xusage.txt Run the executable $ image/bin/fr.umlv.dragon.main Run with the 'java' wrapper $ image/bin/java -m fr.umlv.dragon.main
  36. Compact VM / AOT On small device, use the compact

    VM, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --compress=2 \ --strip-debug \ --vm=compact \ --output image and Oracle is working on a way to pre-JIT the code https://www.youtube.com/watch?v=Xybzyv8qbOc
  37. jlink plugin Jlink has an experimental plugin API By example,

    to generate a CDS file, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --plugin-module-path mlib \ --genclasslist=image/lib/classlist2 \ --compress=2 \ --strip-debug \ --output image
  38. jlink plugin - code import jdk.tools.jlink.plugin.TransformerPlugin; public class ClassListGeneratorPlugin implements

    TransformerPlugin { public String getName() { return "genclasslist"; } public boolean hasArguments() { return true; } ... public void configure(Map<String, String> config) { … } public void visit(Pool in, Pool out) { ... } }
  39. jlink plugin – Service Use the mechanism of Service integrated

    with jigsaw modules module fr.umlv.dragon.jlinkplugin { requires jdk.jlink; provides jdk.tools.jlink.plugin.TransformerPlugin with fr.umlv.dragon.jlinkplugin.ClassListGeneratorPlugin; } $ javap -m jdk.jlink module-info module jdk.jlink { … exports jdk.tools.jlink.builder; exports jdk.tools.jlink.plugin; exports jdk.tools.jlink; uses jdk.tools.jlink.plugin.TransformerPlugin; }
  40. jlink plugin – create module A module that defines a

    service is compiled like any other modules $ javac -d build/modules \ -modulesourcepath jlink-plugin/src \ -modulepath mlib \ $(find jlink-plugin/src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.jlinkplugin-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.jlinkplugin .