Jigsaw will save us Jigsaw will save us Rémi Forax (Bordeaux Edition, April 2016)

Me, Myself and I Maitre de conférence University Paris Est Marne la vallée Developer: OpenJDK, ASM, Tatoo, Dragon, … Java Community Process Expert – Invokedynamic (JSR 292) – Lambda (JSR 335 + java.util/java.util.stream) – Jigsaw (JSR 376)

Dramatic Introduction

No reliable configuration ClassPath ? – Linear scan :( – If two versions of the same package exist ? => Jar Hell Example: CLASSPATH = …:asm-2.3.jar:…:asm-3.1.jar A newly introduced class of ASM 3.1 will use an old class of ASM 2.3 => compile time information is different from runtime info

No way to scale Down the platform Some libraries tend to be too big – OpenJDK 8 rt.jar => 66 Mb – Guava 1.9 => 2.3 Mb – Spring Context => 1.1 Mb on Maven Central IoT anyone ? and JDK classes are privileged :(

No Strong Encapsulation 2013 is a very bad year for Java security The JDK has only one line of defense :(

No Closed World Java Startup is slow – VM startup + JIT ramp up – Application initialization is slow ● Load + verify classes dynamically ● static final object graphs Java runtime is big – No 'tree shaking' Android does AOT !

No content

Jigsaw Goals Reliable configuration – No classpath anymore – compile time / runtime fidelity Scale down the platform – downsized unit of re-use Strong encapsulation – Separate public types from implementation – Enhanced security Closed World – Ahead of time optimization

The Jigsaw Sandwich Compile time Runtime

Meet the Dragon An interpreter of a toy language def(fib: \ n "returns the fibonacci number of n" if(n < 2 1 fib(n - 1) + fib(n – 2))) A small application delivered as a 'fat' jar (dragon.jar) – 244 classes (dependencies included) – 20 packages ● 3 auto-generated from the grammar ● 14 from tatoo-runtime.jar – 1 external dependency (tatoo-runtime.jar) agon

Jigsawification !

jdeps – find package dependencies $ /usr/jdk/jdk-9/bin/jdeps dragon.jar dragon.jar -> java.base fr.umlv.dragon.ast (dragon.jar) -> fr.umlv.dragon.grammar.tools -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.grammar.lexer (dragon.jar) -> fr.umlv.tatoo.runtime. ... (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.parser (dragon.jar) -> fr.umlv.tatoo.runtime.parser (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.tools (dragon.jar) -> fr.umlv.dragon.grammar.lexer -> fr.umlv.dragon.grammar.parser -> fr.umlv.dragon.rt -> fr.umlv.tatoo.runtime. … (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.main (dragon.jar) -> fr.umlv.dragon.ast -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.rt (dragon.jar) -> java ...

Packages dependencies dragon.rt dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser fr.umlv.tatoo.runtime.* dragon.ast dragon.main java.*

Modules dependencies dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar

dragon.rt module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { requires java.base; exports fr.umlv.dragon.rt; }

Create module fr.umlv.dragon.rt javac -d build/modules \ -modulesourcepath src \ $(find src/fr.umlv.dragon.rt -name "*.java") jar --create \ --file mlib/fr.umlv.dragon.rt-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.rt .

dragon.rt module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { // requires java.base; exports fr.umlv.dragon.rt; }

dragon.grammar module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; }

Create module dragon.grammar javac -d build/modules \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ -classpath lib/tatoo-runtime.jar \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") This doesn't compile !!

Mixing modules and jar Modules in modulepath do not read (see) classes in classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo-runtime.jar OK ! NO !

Automatic modules (migration mode) Plain old jar in modulepath – can be used in requires – can read modules – can read any jars from the classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo.runtime-1.0.jar another-jar.jar modulepath module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; ...; } the jar also needs to be renamed !

Create module dragon.grammar mv lib/tatoo-runtime.jar \ mlib/fr.umlv.tatoo.runtime-1.0.jar javac -d build/modules \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") It works !

dragon.ast module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }

dragon.main module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }

requires is not transitive by default ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; }

requires public to the rescue ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { // requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }

require public The public API of the module fr.umlv.dragon.ast refers to classes from a required module package fr.umlv.dragon.ast; import ... import fr.umlv.dragon.rt.Block; public class ASTBuilder { ... public static Block buildAST(Reader reader) { TerminalEvaluator terminalEvaluator = new TokenEvaluator(); ASTEvaluator grammarEvaluator = new ASTEvaluator(); Analyzers.run(reader, terminalEvaluator, grammarEvaluator, null, null); return grammarEvaluator.script; } ... } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }

Same thing with dragon.grammar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast; }

Create dragon.main module The main class is specified when creating the jar as usual $ javac -d build/modules \ -modulesourcepath src \ -modulepath mlib \ $(find src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.main-1.0.jar \ --module-version=1.0 \ --main-class fr.umlv.dragon.main.Main \ -C build/modules/fr.umlv.dragon.main

Run dragon As jars (Java 9 is compatible with Java <9) $ /usr/jdk/jdk-9/bin/java -classpath \ mlib/fr.umlv.tatoo.runtime-1.0.jar: \ mlib/fr.umlv.dragon.grammar-1.0.jar: \ mlib/fr.umlv.dragon.ast-1.0.jar: \ mlib/fr.umlv.dragon.rt-1.0.jar: \ mlib/fr.umlv.dragon.main-1.0.jar \ fr.umlv.dragon.main.Main As modular jars $ java --modulepath mlib \ --m fr.umlv.dragon.main

No content

Restricted export module fr.umlv.dragon.grammar exports methods that is specific to the Tatoo parser compiler what if I want to change the parser in the future ? => restrict the visibility of a module !

Now, only dragon.ast can 'requires' dragon.grammar ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools to fr.umlv.dragon.ast; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast; }

sun.misc.Unsafe in 9 ? $ javap -m java.base module-info module java.base { …, exports sun.misc; exports jdk.internal.misc to java.rmi, java.sql, jdk.charsets, … } $ javap -m jdk.unsupported module-info module jdk.unsupported { exports sun.misc; } Keep sun.misc in 9, remove features when a public replacement APIs exist New internal unsafe API (hidden)

Discovering usage of internal APIs $ jdeps -jdkinternals jruby-9.0.5.0/lib/jruby.jar jruby.jar -> java.base jruby.jar -> java.management org.jruby.management.BeanManagerImpl (jruby.jar) -> sun.management.Agent JDK internal API (java.management) org.jruby.util.encoding.ISO_8859_16 (jruby.jar) -> sun.nio.cs.US_ASCII JDK internal API (java.base) $ jdeps -jdkinternals gradle/lib/*.jar gradle/lib/plugins/*.jar gradle-core-2.12.jar -> java.base org.gradle.internal.filewatch.jdk7.WatchServiceRegistrar (gradle-core-2.12.jar) -> com.sun.nio.file.ExtendedWatchEventModifier JDK internal API (java.base) -> com.sun.nio.file.SensitivityWatchEventModifier JDK internal API (java.base) bsh-2.0b4.jar -> java.desktop bsh.util.AWTConsole (bsh-2.0b4.jar) -> java.awt.peer.ComponentPeer JDK internal API (java.desktop) -> java.awt.peer.TextComponentPeer JDK internal API (java.desktop) ...

Command line java/javac Overidde root modules configuration -addmods module to add root modules -upgrademodulepath to replace some root modules Patch a module -Xpatch:module=files patch existing classes of a module Add a dependency (can be done by reflection too) -XaddReads:module=anotherModule Break encapsulation -XaddExports:module/package=anotherModule

No content

jlink Create a custom runtime image from a set of root modules Image is specific for a platform – Intel 64bits linux, ARM 32bits Windows, etc For small devices or in the cloud (jlink is not mandatory !)

jlink and automatic module ! $ jlink --modulepath /usr/jdk/jdk-9-jigsaw/jmods:mlib --addmods fr.umlv.dragon.main --output image Error: jdk.tools.jlink.plugin.PluginException : module-info.class not found for fr.umlv.tatoo.runtime module Automatic modules can see classes from the classpath so the world in not closed :(

Jigsawification by hand $ jdeps -genmoduleinfo tmp lib/tatoo-runtime.jar writing to tmp/tatoo.runtime/module-info.java $ cat tmp/tatoo.runtime/module-info.java module tatoo.runtime { exports fr.umlv.tatoo.runtime.ast; exports fr.umlv.tatoo.runtime.buffer; exports fr.umlv.tatoo.runtime.lexer; exports fr.umlv.tatoo.runtime.lexer.rules; exports fr.umlv.tatoo.runtime.log; exports fr.umlv.tatoo.runtime.parser; exports fr.umlv.tatoo.runtime.regex; exports fr.umlv.tatoo.runtime.tools; ... }

Jigsawification by hand (2) $ javac -d build/modules \ -modulesourcepath tmp \ fr.umlv.tatoo.runtime/module-info.java $ jar --update \ --file mlib/fr.umlv.tatoo.runtime-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.tatoo.runtime \ module-info.class

jlink my dragon Generating a custom image for dragon $ rm -fr image $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --compress=2 \ --strip-debug \ --output image

jlink image - Run dragon $ ls -R image image: bin conf lib release image/bin: fr.umlv.dragon.main java keytool image/lib: amd64 classlist jexec modules security tzdb.dat … image/lib/amd64/server: classes.jsa libjsig.so libjvm.so Xusage.txt Run the executable $ image/bin/fr.umlv.dragon.main Run with the 'java' wrapper $ image/bin/java -m fr.umlv.dragon.main

Compact VM / AOT On small device, use the compact VM, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --compress=2 \ --strip-debug \ --vm=compact \ --output image and Oracle is working on a way to pre-JIT the code https://www.youtube.com/watch?v=Xybzyv8qbOc

jlink plugin Jlink has an experimental plugin API By example, to generate a CDS file, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --plugin-module-path mlib \ --genclasslist=image/lib/classlist2 \ --compress=2 \ --strip-debug \ --output image

jlink plugin – Service Use the mechanism of Service integrated with jigsaw modules module fr.umlv.dragon.jlinkplugin { requires jdk.jlink; provides jdk.tools.jlink.plugin.TransformerPlugin with fr.umlv.dragon.jlinkplugin.ClassListGeneratorPlugin; } $ javap -m jdk.jlink module-info module jdk.jlink { … exports jdk.tools.jlink.builder; exports jdk.tools.jlink.plugin; exports jdk.tools.jlink; uses jdk.tools.jlink.plugin.TransformerPlugin; }

jlink plugin - code import jdk.tools.jlink.plugin.TransformerPlugin; public class ClassListGeneratorPlugin implements TransformerPlugin { public String getName() { return "genclasslist"; } public boolean hasArguments() { return true; } ... public void configure(Map config) { … } public void visit(Pool in, Pool out) { ... } }

jlink plugin – create module A module that defines a service is compiled like any other modules $ javac -d build/modules \ -modulesourcepath jlink-plugin/src \ -modulepath mlib \ $(find jlink-plugin/src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.jlinkplugin-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.jlinkplugin .

No content

The harsh reality (as today) Startup time :(

The harsh reality (as today) Size on disc

Questions ? Code is available at https://github.com/forax/dragon/tree/jigsaw