Jigsaw will save us

Jigsaw is the major feature of Java 9, introduction with an example on how to jigsawify an existing code and play with the new jlink command

Transcript

1. Jigsaw will save us Jigsaw will save us Rémi Forax (Bordeaux Edition, April 2016)
2. Me, Myself and I Maitre de conférence University Paris Est Marne la vallée Developer: OpenJDK, ASM, Tatoo, Dragon, … Java Community Process Expert – Invokedynamic (JSR 292) – Lambda (JSR 335 + java.util/java.util.stream) – Jigsaw (JSR 376)
3. Dramatic Introduction
4. No reliable configuration ClassPath ? – Linear scan :( – If two versions of the same package exist ? => Jar Hell Example: CLASSPATH = …:asm-2.3.jar:…:asm-3.1.jar A newly introduced class of ASM 3.1 will use an old class of ASM 2.3 => compile time information is different from runtime info
5. No way to scale Down the platform Some libraries tend to be too big – OpenJDK 8 rt.jar => 66 Mb – Guava 1.9 => 2.3 Mb – Spring Context => 1.1 Mb on Maven Central IoT anyone ? and JDK classes are privileged :(
6. No Strong Encapsulation 2013 is a very bad year for Java security The JDK has only one line of defense :(
7. No Closed World Java Startup is slow – VM startup + JIT ramp up – Application initialization is slow • Load + verify classes dynamically • static final object graphs Java runtime is big – No 'tree shaking' Android does AOT !
9. Jigsaw Goals Reliable configuration – No classpath anymore – compile time / runtime fidelity Scale down the platform – downsized unit of re-use Strong encapsulation – Separate public types from implementation – Enhanced security Closed World – Ahead of time optimization
10. The Jigsaw Sandwich Compile time Runtime
11. Meet the Dragon An interpreter of a toy language def(fib: \ n "returns the fibonacci number of n" if(n < 2 1 fib(n - 1) + fib(n – 2))) A small application delivered as a 'fat' jar (dragon.jar) – 244 classes (dependencies included) – 20 packages • 3 auto-generated from the grammar • 14 from tatoo-runtime.jar – 1 external dependency (tatoo-runtime.jar) agon
12. Jigsawification !
13. jdeps – find package dependencies $ /usr/jdk/jdk-9/bin/jdeps dragon.jar dragon.jar -> java.base fr.umlv.dragon.ast (dragon.jar) -> fr.umlv.dragon.grammar.tools -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.grammar.lexer (dragon.jar) -> fr.umlv.tatoo.runtime. ... (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.parser (dragon.jar) -> fr.umlv.tatoo.runtime.parser (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.grammar.tools (dragon.jar) -> fr.umlv.dragon.grammar.lexer -> fr.umlv.dragon.grammar.parser -> fr.umlv.dragon.rt -> fr.umlv.tatoo.runtime. … (tatoo-runtime.jar) -> java. ... fr.umlv.dragon.main (dragon.jar) -> fr.umlv.dragon.ast -> fr.umlv.dragon.rt -> java. ... fr.umlv.dragon.rt (dragon.jar) -> java ...
14. Packages dependencies dragon.rt dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser fr.umlv.tatoo.runtime.* dragon.ast dragon.main java.*
15. Modules dependencies dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar
16. dragon.rt module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { requires java.base; exports fr.umlv.dragon.rt; }
17. Create module fr.umlv.dragon.rt javac -d build/modules \ -modulesourcepath src \ $(find src/fr.umlv.dragon.rt -name "*.java") jar --create \ --file mlib/fr.umlv.dragon.rt-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.rt .
18. dragon.rt module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { // requires java.base; exports fr.umlv.dragon.rt; }
19. dragon.grammar module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo-runtime.jar module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; }
20. Create module dragon.grammar javac -d build/modules \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ -classpath lib/tatoo-runtime.jar \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") This doesn't compile !!
21. Mixing modules and jar Modules in modulepath do not read (see) classes in classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo-runtime.jar OK ! NO !
22. Automatic modules (migration mode) Plain old jar in modulepath – can be used in requires – can read modules – can read any jars from the classpath modulepath classpath (unamed module) dragon.main dragon.ast dragon.grammar tatoo.runtime-1.0.jar another-jar.jar modulepath module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; ...; } the jar also needs to be renamed !
23. Create module dragon.grammar mv lib/tatoo-runtime.jar \ mlib/fr.umlv.tatoo.runtime-1.0.jar javac -d build/modules \ -modulesourcepath grammar/gen-src \ -modulepath mlib \ $(find grammar/gen-src/fr.umlv.dragon.grammar -name "*.java") It works !
24. dragon.ast module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }
25. dragon.main module-info.jar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
26. requires is not transitive by default ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.ast { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; }
27. requires public to the rescue ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { // requires fr.umlv.dragon.rt; requires fr.umlv.dragon.ast; }
28. require public The public API of the module fr.umlv.dragon.ast refers to classes from a required module package fr.umlv.dragon.ast; import ... import fr.umlv.dragon.rt.Block; public class ASTBuilder { ... public static Block buildAST(Reader reader) { TerminalEvaluator<CharSequence> terminalEvaluator = new TokenEvaluator(); ASTEvaluator grammarEvaluator = new ASTEvaluator(); Analyzers.run(reader, terminalEvaluator, grammarEvaluator, null, null); return grammarEvaluator.script; } ... } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; }
29. Same thing with dragon.grammar dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast; }
30. Create dragon.main module The main class is specified when creating the jar as usual $ javac -d build/modules \ -modulesourcepath src \ -modulepath mlib \ $(find src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.main-1.0.jar \ --module-version=1.0 \ --main-class fr.umlv.dragon.main.Main \ -C build/modules/fr.umlv.dragon.main
31. Run dragon As jars (Java 9 is compatible with Java <9) $ /usr/jdk/jdk-9/bin/java -classpath \ mlib/fr.umlv.tatoo.runtime-1.0.jar: \ mlib/fr.umlv.dragon.grammar-1.0.jar: \ mlib/fr.umlv.dragon.ast-1.0.jar: \ mlib/fr.umlv.dragon.rt-1.0.jar: \ mlib/fr.umlv.dragon.main-1.0.jar \ fr.umlv.dragon.main.Main As modular jars $ java --modulepath mlib \ --m fr.umlv.dragon.main
33. Restricted export module fr.umlv.dragon.grammar exports methods that is specific to the Tatoo parser compiler what if I want to change the parser in the future ? => restrict the visibility of a module !
34. Now, only dragon.ast can 'requires' dragon.grammar ! dragon.rt java.base fr.umlv.tatoo.runtime.* dragon.ast dragon.main dragon.grammar.tools dragon.grammar.lexer dragon.grammar.parser dragon.grammar tatoo.runtime module fr.umlv.dragon.rt { //requires java.base; exports fr.umlv.dragon.rt; } module fr.umlv.dragon.grammar { requires public fr.umlv.dragon.rt; requires fr.umlv.tatoo.runtime; exports fr.umlv.dragon.grammar.tools to fr.umlv.dragon.ast; } module fr.umlv.dragon.ast { requires public fr.umlv.dragon.rt; requires fr.umlv.dragon.grammar; exports fr.umlv.dragon.ast; } module fr.umlv.dragon.main { requires fr.umlv.dragon.ast; }
35. sun.misc.Unsafe in 9 ? $ javap -m java.base module-info module java.base { …, exports sun.misc; exports jdk.internal.misc to java.rmi, java.sql, jdk.charsets, … } $ javap -m jdk.unsupported module-info module jdk.unsupported { exports sun.misc; } Keep sun.misc in 9, remove features when a public replacement APIs exist New internal unsafe API (hidden)
36. Discovering usage of internal APIs $ jdeps -jdkinternals jruby-9.0.5.0/lib/jruby.jar jruby.jar -> java.base jruby.jar -> java.management org.jruby.management.BeanManagerImpl (jruby.jar) -> sun.management.Agent JDK internal API (java.management) org.jruby.util.encoding.ISO_8859_16 (jruby.jar) -> sun.nio.cs.US_ASCII JDK internal API (java.base) $ jdeps -jdkinternals gradle/lib/*.jar gradle/lib/plugins/*.jar gradle-core-2.12.jar -> java.base org.gradle.internal.filewatch.jdk7.WatchServiceRegistrar (gradle-core-2.12.jar) -> com.sun.nio.file.ExtendedWatchEventModifier JDK internal API (java.base) -> com.sun.nio.file.SensitivityWatchEventModifier JDK internal API (java.base) bsh-2.0b4.jar -> java.desktop bsh.util.AWTConsole (bsh-2.0b4.jar) -> java.awt.peer.ComponentPeer JDK internal API (java.desktop) -> java.awt.peer.TextComponentPeer JDK internal API (java.desktop) ...
37. Command line java/javac Overidde root modules configuration -addmods module to add root modules -upgrademodulepath to replace some root modules Patch a module -Xpatch:module=files patch existing classes of a module Add a dependency (can be done by reflection too) -XaddReads:module=anotherModule Break encapsulation -XaddExports:module/package=anotherModule
39. jlink Create a custom runtime image from a set of root modules Image is specific for a platform – Intel 64bits linux, ARM 32bits Windows, etc For small devices or in the cloud (jlink is not mandatory !)
40. jlink and automatic module ! $ jlink --modulepath /usr/jdk/jdk-9-jigsaw/jmods:mlib --addmods fr.umlv.dragon.main --output image Error: jdk.tools.jlink.plugin.PluginException : module-info.class not found for fr.umlv.tatoo.runtime module Automatic modules can see classes from the classpath so the world in not closed :(
41. Jigsawification by hand $ jdeps -genmoduleinfo tmp lib/tatoo-runtime.jar writing to tmp/tatoo.runtime/module-info.java $ cat tmp/tatoo.runtime/module-info.java module tatoo.runtime { exports fr.umlv.tatoo.runtime.ast; exports fr.umlv.tatoo.runtime.buffer; exports fr.umlv.tatoo.runtime.lexer; exports fr.umlv.tatoo.runtime.lexer.rules; exports fr.umlv.tatoo.runtime.log; exports fr.umlv.tatoo.runtime.parser; exports fr.umlv.tatoo.runtime.regex; exports fr.umlv.tatoo.runtime.tools; ... }
42. Jigsawification by hand (2) $ javac -d build/modules \ -modulesourcepath tmp \ fr.umlv.tatoo.runtime/module-info.java $ jar --update \ --file mlib/fr.umlv.tatoo.runtime-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.tatoo.runtime \ module-info.class
43. jlink my dragon Generating a custom image for dragon $ rm -fr image $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --compress=2 \ --strip-debug \ --output image
44. jlink image - Run dragon $ ls -R image image: bin conf lib release image/bin: fr.umlv.dragon.main java keytool image/lib: amd64 classlist jexec modules security tzdb.dat … image/lib/amd64/server: classes.jsa libjsig.so libjvm.so Xusage.txt Run the executable $ image/bin/fr.umlv.dragon.main Run with the 'java' wrapper $ image/bin/java -m fr.umlv.dragon.main
45. Compact VM / AOT On small device, use the compact VM, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --compress=2 \ --strip-debug \ --vm=compact \ --output image and Oracle is working on a way to pre-JIT the code https://www.youtube.com/watch?v=Xybzyv8qbOc
46. jlink plugin Jlink has an experimental plugin API By example, to generate a CDS file, $ jlink --modulepath $JAVA_HOME/jmods:mlib \ --addmods fr.umlv.dragon.main \ --plugin-module-path mlib \ --genclasslist=image/lib/classlist2 \ --compress=2 \ --strip-debug \ --output image
47. jlink plugin – Service Use the mechanism of Service integrated with jigsaw modules module fr.umlv.dragon.jlinkplugin { requires jdk.jlink; provides jdk.tools.jlink.plugin.TransformerPlugin with fr.umlv.dragon.jlinkplugin.ClassListGeneratorPlugin; } $ javap -m jdk.jlink module-info module jdk.jlink { … exports jdk.tools.jlink.builder; exports jdk.tools.jlink.plugin; exports jdk.tools.jlink; uses jdk.tools.jlink.plugin.TransformerPlugin; }
48. jlink plugin - code import jdk.tools.jlink.plugin.TransformerPlugin; public class ClassListGeneratorPlugin implements TransformerPlugin { public String getName() { return "genclasslist"; } public boolean hasArguments() { return true; } ... public void configure(Map<String, String> config) { … } public void visit(Pool in, Pool out) { ... } }
49. jlink plugin – create module A module that defines a service is compiled like any other modules $ javac -d build/modules \ -modulesourcepath jlink-plugin/src \ -modulepath mlib \ $(find jlink-plugin/src -name "*.java") $ jar --create \ --file mlib/fr.umlv.dragon.jlinkplugin-1.0.jar \ --module-version=1.0 \ -C build/modules/fr.umlv.dragon.jlinkplugin .
51. The harsh reality (as today) Startup time :(
52. The harsh reality (as today) Size on disc
53. Questions ? Code is available at https://github.com/forax/dragon/tree/jigsaw