Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Jigsaw our savior

forax
April 22, 2016

Jigsaw our savior

Learn how jigsaw will save us from our sins

forax

April 22, 2016
Tweet

More Decks by forax

Other Decks in Programming

Transcript

  1. #DevoxxF R 1 Rémi Forax Université Paris Est Marne la

    vallée Jigsaw will save us Jigsaw will save us
  2. #DevoxxFR Me, Myself and I Maitre de conférence University Paris

    Est Marne la vallée Developer: OpenJDK, ASM, Tatoo, Dragon, … Java Community Process Expert – Invokedynamic (JSR 292) – Lambda (JSR 335 + java.util/java.util.stream) – Jigsaw (JSR 376)
  3. #DevoxxFR No reliable configuration ClassPath ? – Linear scan :(

    – If two versions of the same package exist ? => Jar Hell Example: CLASSPATH = …:asm-2.3.jar:…:asm-3.1.jar A newly introduced class of ASM 3.1 will use an old class of ASM 2.3 => compile time information is different from runtime info
  4. #DevoxxFR No way to scale Down the platform Some libraries

    tend to be too big – OpenJDK 8 rt.jar => 66 Mb – Guava 1.9 => 2.3 Mb – Spring Context => 1.1 Mb on Maven Central IoT anyone ? and JDK classes are privileged :(
  5. #DevoxxFR No Strong Encapsulation 2013 is a very bad year

    for Java security The JDK has only one line of defense :(
  6. #DevoxxFR Meet the Dragon An interpreter of a toy language

    def(fib: \ n "returns the fibonacci number of n" if(n < 2 1 fib(n - 1) + fib(n – 2))) A small application delivered as a 'fat' jar (dragon.jar) – 244 classes (dependencies included) – 20 packages • 3 auto-generated from the grammar • 14 from tatoo-runtime.jar – 1 external dependency (tatoo-runtime.jar)
  7. #DevoxxFR jdeps – find package dependencies $ /usr/jdk/jdk-9/bin/jdeps dragon.jar dragon.jar

    -> java.base fr.umlv.dragon.ast (dragon.jar) -> fr.umlv.dragon.grammar.tools -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.grammar.lexer (dragon.jar) -> fr.umlv.tatoo.runtime. ... (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.parser (dragon.jar) -> fr.umlv.tatoo.runtime.parser (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.tools (dragon.jar) -> fr.umlv.dragon.grammar.lexer -> fr.umlv.dragon.grammar.parser -> fr.umlv.dragon.rt -> fr.umlv.tatoo.runtime. … (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.main (dragon.jar) -> fr.umlv.dragon.ast -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.rt (dragon.jar) -> java ...
  8. #DevoxxFR dragon.rt module-info.java module fr.umlv.dragon.rt { requires java.base; exports fr.umlv.dragon.rt;

    } dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
  9. #DevoxxFR Create module fr.umlv.dragon.rt javac -d build/modules \ -modulesourcepath src

    \ $(find src/fr.umlv.dragon.rt -name "*.java") jar --create \ --file mlib/fr.umlv.dragon.rt-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.rt .
  10. #DevoxxFR dragon.rt module-info.java module fr.umlv.dragon.rt { // requires java.base; exports

    fr.umlv.dragon.rt; } dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
  11. #DevoxxFR dragon.grammar module-info.java module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt;

    } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
  12. #DevoxxFR Create module dragon.grammar javac -d build/modules \ -modulesourcepath grammar/gen-src

    \ -modulepath mlib \ -classpath lib/tatoo-runtime.jar \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") This doesn't compile !!
  13. #DevoxxFR Mixing modules and jar Modules in modulepath do not

    read (see) classes in classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo-runtime.jar OK ! NO !
  14. #DevoxxFR Automatic modules (migration mode) Plain old jar in modulepath

    – can be used in requires – can read modules – can read any jars from the classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo.runtime-1.0.jar another-jar.jar modulepath the jar also needs to be renamed ! module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; }
  15. #DevoxxFR Create module dragon.grammar mv lib/tatoo-runtime.jar \ mlib/fr.umlv.tatoo.runtime-1.0.jar javac -d

    build/modules \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") It works !
  16. #DevoxxFR dragon.ast module-info.java dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer

    dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; }
  17. #DevoxxFR dragon.main module-info.java dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer

    dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
  18. #DevoxxFR Create dragon.main module The main class is specified when

    creating the jar as usual $ javac -d build/modules \ -modulesourcepath src \ -modulepath mlib \ $(find src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.main-1.0.jar \ --module-version=1.0 \ --main-class fr.umlv.dragon.main.Main \ -C build/modules/fr.umlv.dragon.main
  19. #DevoxxFR requires is not transitive by default ! dragon.rt java.base

    fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
  20. #DevoxxFR requires public to the rescue ! dragon.rt java.base fr.umlv.tatoo.runtime.*

    dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
  21. #DevoxxFR require public The public API of the module fr.umlv.dragon.ast

    refers to classes from a required module package fr.umlv.dragon.ast; import ... import fr.umlv.dragon.rt.Block; public class ASTBuilder { • public static Block buildAST(Reader reader) { TerminalEvaluator<CharSequence> terminalEvaluator = new TokenEvaluator(); ASTEvaluator grammarEvaluator = new ASTEvaluator(); Analyzers.run(reader, terminalEvaluator, grammarEvaluator, null, null); return grammarEvaluator.script; } ... } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }
  22. #DevoxxFR require public only works with one hop dragon.rt java.base

    fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
  23. #DevoxxFR Restricted export module fr.umlv.dragon.grammar exports methods that are specific

    to the Tatoo parser compiler what if I want to change the parser in the future ? => restrict the visibility of a module !
  24. #DevoxxFR Only dragon.ast can 'requires' dragon.grammar ! dragon.rt java.base fr.umlv.tatoo.runtime.*

    dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools to fr.umlv.dragon.ast; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
  25. #DevoxxFR sun.misc.Unsafe in 9 ? $ javap -m java.base module-info

    module java.base { …, exports sun.misc; exports jdk.internal.misc to java.rmi, java.sql, jdk.charsets, … } $ javap -m jdk.unsupported module-info module jdk.unsupported { exports sun.misc; } Keep sun.misc in 9, remove features when a public replacement APIs exist New internal unsafe API (hidden)
  26. #DevoxxFR Discovering usage of internal APIs $ jdeps -jdkinternals jruby-9.0.5.0/lib/jruby.jar

    jruby.jar -> java.base jruby.jar -> java.management org.jruby.management.BeanManagerImpl (jruby.jar) -> sun.management.Agent JDK internal API (java.management) org.jruby.util.encoding.ISO_8859_16 (jruby.jar) -> sun.nio.cs.US_ASCII JDK internal API (java.base) $ jdeps -jdkinternals gradle/lib/*.jar gradle/lib/plugins/*.jar gradle-core-2.12.jar -> java.base org.gradle.internal.filewatch.jdk7.WatchServiceRegistrar (gradle-core-2.12.jar) -> com.sun.nio.file.ExtendedWatchEventModifier JDK internal API (java.base) -> com.sun.nio.file.SensitivityWatchEventModifier JDK internal API (java.base) bsh-2.0b4.jar -> java.desktop bsh.util.AWTConsole (bsh-2.0b4.jar) -> java.awt.peer.ComponentPeer JDK internal API (java.desktop) -> java.awt.peer.TextComponentPeer JDK internal API (java.desktop) ...
  27. #DevoxxFR Module Service I want to be able to plug

    different kinds of interpreter for dragon – AST interpreter, – Bytecode interpreter, etc. Need to decouple the interpreter interface from its implementation At least one interpreter should be present at runtime
  28. #DevoxxFR Module Service / Code package fr.umlv.dragon.ast; public interface Interpreter

    { Object interpret(Array<Parameter> parameters, Block body, ...); } package fr.umlv.dragon.main; … public static void main(String[] args) { Interpreter interpreter = ServiceLoader.load(Interpreter.class).iterator().next(); } package fr.umlv.dragon.eval; public class InterpreterImpl implements Interpreter { … } interface an implementation usage
  29. #DevoxxFR dragon.rt java.base dragon.ast dragon.main module fr.umlv.dragon.eval { requires fr.umlv.dragon.ast

    // export fr.umlv.tatoo.eval; provides fr.umlv.dragon.ast.Interpreter to fr.umlv.dragon.eval.InterpreterImpl; } module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast uses fr.umlv.dragon.ast.Interpreter;; } Uses / Provides … with dragon.eval
  30. #DevoxxFR Run dragon As jars (Java 9 is compatible with

    Java <9) $ /usr/jdk/jdk-9/bin/java -classpath \ mlib/fr.umlv.tatoo.runtime-1.0.jar: \ mlib/fr.umlv.dragon.grammar-1.0.jar: \ mlib/fr.umlv.dragon.ast-1.0.jar: \ mlib/fr.umlv.dragon.rt-1.0.jar: \ mlib/fr.umlv.dragon.eval-1.0.jar \ mlib/fr.umlv.dragon.main-1.0.jar \ fr.umlv.dragon.main.Main As modular jars $ java --modulepath mlib \ --m fr.umlv.dragon.main
  31. #DevoxxFR jlink Create a custom runtime image from a set

    of root modules Image is specific for a platform – Intel 64bits linux, ARM 32bits Windows, etc For small devices or in the cloud (jlink is not mandatory !)
  32. #DevoxxFR jlink and automatic module ! $ jlink --modulepath /usr/jdk/jdk-9-jigsaw/jmods:mlib

    --addmods "fr.umlv.dragon.main,fr.umlv.dragon.eval" \ --output image Error: jdk.tools.jlink.plugin.PluginException : module-info.class not found for fr.umlv.tatoo.runtime module Automatic modules can see classes from the classpath so the world in not closed :(
  33. #DevoxxFR Jigsawification by hand $ jdeps -genmoduleinfo tmp lib/tatoo-runtime.jar writing

    to tmp/tatoo.runtime/module-info.java $ cat tmp/tatoo.runtime/module-info.java module tatoo.runtime { exports fr.umlv.tatoo.runtime.ast; exports fr.umlv.tatoo.runtime.buffer; exports fr.umlv.tatoo.runtime.lexer; exports fr.umlv.tatoo.runtime.lexer.rules; exports fr.umlv.tatoo.runtime.log; exports fr.umlv.tatoo.runtime.parser; exports fr.umlv.tatoo.runtime.regex; exports fr.umlv.tatoo.runtime.tools; ... }
  34. #DevoxxFR Jigsawification by hand (2) $ javac -d build/modules \

    -modulesourcepath tmp \ fr.umlv.tatoo.runtime/module-info.java $ jar --update \ --file mlib/fr.umlv.tatoo.runtime-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.tatoo.runtime \ module-info.class
  35. #DevoxxFR jlink my dragon Generating a custom image for dragon

    $ rm -fr image $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --strip-debug \ --output image
  36. #DevoxxFR jlink image - Run dragon $ ls -R image

    image: bin conf lib release image/bin: fr.umlv.dragon.main java keytool image/lib: amd64 classlist jexec modules security tzdb.dat … image/lib/amd64/server: classes.jsa libjsig.so libjvm.so Xusage.txt Run the executable $ ./image/bin/fr.umlv.dragon.main Run with the 'java' wrapper $ ./image/bin/java -m fr.umlv.dragon.main
  37. #DevoxxFR Compact VM / AOT On small device, use the

    compact VM, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --strip-debug \ --vm=compact \ --output image and Oracle is working on a way to pre-JIT the code https://www.youtube.com/watch?v=Xybzyv8qbOc
  38. #DevoxxFR Jigsaw Goals Reliable configuration • No classpath anymore •

    compile time / runtime fidelity Scale down the platform • downsized unit of re-use Strong encapsulation • Separate public types from implementation • Enhanced security Closed World • Ahead of time optimization
  39. #DevoxxFR Command line java/javac Override root modules configuration -addmods module

    to add root modules -upgrademodulepath to replace some root modules Patch a module -Xpatch:module=files patch existing classes of a module Add a dependency (can be done by reflection too) -XaddReads:module=anotherModule Break encapsulation -XaddExports:module/package=anotherModule
  40. #DevoxxFR jlink plugin Jlink has an experimental plugin API By

    example, to generate a CDS file, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --plugin-module-path mlib \ --genclasslist=image/lib/classlist2 \ --strip-debug \ --output image
  41. #DevoxxFR jlink plugin - code import jdk.tools.jlink.plugin.TransformerPlugin; public class ClassListGeneratorPlugin

    implements TransformerPlugin { public String getName() { return "genclasslist"; } public boolean hasArguments() { return true; } ... public void configure(Map<String, String> config) { … } public void visit(Pool in, Pool out) { ... } }
  42. #DevoxxFR jlink plugin – Service Use the mechanism of Service

    integrated with jigsaw modules module fr.umlv.dragon.jlinkplugin { requires jdk.jlink; provides jdk.tools.jlink.plugin.TransformerPlugin with fr.umlv.dragon.jlinkplugin.ClassListGeneratorPlugin; } $ javap -m jdk.jlink module-info module jdk.jlink { … exports jdk.tools.jlink.builder; exports jdk.tools.jlink.plugin; exports jdk.tools.jlink; uses jdk.tools.jlink.plugin.TransformerPlugin; }
  43. #DevoxxFR jlink plugin – create module A module that defines

    a service is compiled like any other modules $ javac -d build/modules \ -modulesourcepath jlink-plugin/src \ -modulepath mlib \ $(find jlink-plugin/src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.jlinkplugin-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.jlinkplugin .