The Marriage Made on a Cloud

• Edoardo Vacchi @evacchi • Research @ UniMi / MaTe • Research @ UniCredit R&D • Kogito / Drools / jBPM @ Red Hat

• You are a team mate ( hey! ) • You are a wannabe contributor ( wow! ) • You are just curious ( welcome! )

Step 1 • Head over to • github.com/kiegroup/kogito-runtimes OK, now what?

• Kogito • Compile-Time vs Run-Time • Quarkus Extensions • Bytecode vs Source-Code Generation • GraalVM Native image • Scaffolding and Troubleshooting

Cloud-Native Business Automation for building intelligent applications, backed by battle-tested capabilities

No content

• There is always a pre-processing phase where you prepare your program for execution • Then, there's actual process execution phase

• Will that configuration change across runs? • Do you have to repackage the application to bundle the new configuration?

• You are building an Immutable Dockerized Microservice in the Cloud • You are not going to load new classes at run-time • You don’t need Runtime Dependency Injection

Build-time • Pre-process configuration • Generate code with configuration options pre-loaded • Resolve dependencies • Generate “wiring” code -- e.g. factories, startup methods etc.

Dagger 2

Dagger 2

No content

No content

Assets Wiring Pre-Built Assets REST/Event Handlers ∙ “Drools Canonical Model” ∙ BPMN Fluent API ∙ ... ∙ DRL ∙ BPMN ∙ DMN ∙ ... ∙ JAX-RS Endpoints ∙ Cloud Event Reactive Consumers ∙ ... ∙ CDI Producers ∙ Bean Factories ∙ ...

REST Resources class MyModelProcessResource { @Inject Process p; } class MyRuleUnitResource { @Inject RuleUnit r; } Wiring Code (CDI etc.) Application.java ApplicationConfig.java Processes.java Decisions.java ... (Inferred) Data Model (process variables, DMN ItemDefinitions, DMN context, Rule Unit descriptors...) e.g. MyModel, MyUnitData Typed Knowledge Asset API (and related wiring) exposes the engine e.g. Process, RuleUnit, DMNDecision

REST Resources class MyModelProcessResource { @Inject Process p; } class MyRuleUnitResource { @Inject RuleUnit r; } Wiring Code (CDI etc.) Application.java ApplicationConfig.java Processes.java Decisions.java ... (Inferred) Data Model (process variables, DMN ItemDefinitions, DMN context, Rule Unit descriptors...) e.g. MyModel, MyUnitData Typed Knowledge Asset API (and related wiring) exposes the engine e.g. Process, RuleUnit, DMNDecision Implementation Detail ! Heavy Rework in Progress ! Do Not Rely on The Details !

• Maven Plug-In • Quarkus Extension

No content

• Codegen-based framework • Plug into the codegen mechanism • Produce more code

… show path here ...

No content

• deployment • runtime

No content

• @BuildStep • annotates a method • method parameters: required items • return type: provided result • You can think of it as of • a CDI @Produces method or • a Spring @Factory • With a twist: • You can return more than one element at once (with a “trick”) @BuildStep public SomeBuildItem myBuildStep( SomeOtherBuildItem items ) { ... }

all: foo.o bar.o cc -o out foo.o bar.o foo.o: foo.c $(CC) -c foo.c -o foo.o bar.o: bar.c $(CC) -c bar.c -o bar.o

foo.h foo.o a.out bar.o foo.c bar.h bar.c

• Think of them as CDI/Spring beans • A special interface that represents things that can be • produced (provided) • consumed (required) @BuildStep public SomeBuildItem generate( SomeOtherBuildItem items) { ... }

• Think of them as CDI/Spring beans • A special interface that represents things that can be • produced (provided) • consumed (required) @BuildStep public SomeBuildItem generate( SomeOtherBuildItem items) { ... } @BuildStep public SomeBuildItem generate( Collection items) { ... } You can depend on a collection !

@BuildStep public void generate( SomeBuildItem items, BuildProducer bp) { ... bp.produce(t); } BuildProducer generatedBeans, BuildProducer resource, BuildProducer reflectiveClass, BuildProducer genResBI

@BuildStep public void generate( SomeBuildItem items, BuildProducer bp) { ... bp.produce(t); } BuildProducer generatedBeans, BuildProducer resource, BuildProducer reflectiveClass, BuildProducer genResBI You can have as many as you want ! Simulates multiple return values !

In the body of the methods // real work occurs here: invoke the code-generation procedure Collection generatedFiles = appGen.generate(); // Write files to disk dumpFilesToDisk(appPaths, generatedFiles);

private void registerResources(Collection generatedFiles, BuildProducer resource, BuildProducer genResBI) { for (GeneratedFile f : generatedFiles) { if (f.getType() == GeneratedFile.Type.GENERATED_CP_RESOURCE) { genResBI.produce(new GeneratedResourceBuildItem(f.relativePath(), f.contents())); resource.produce(new NativeImageResourceBuildItem(f.relativePath())); } } }

result.add(new ReflectiveHierarchyIgnoreWarningBuildItem( DotName.createSimple("org.kie.dmn.api.core.DMNContext"))); ... toReturn.add(new ReflectiveClassBuildItem(true, true, PMML4Result.class)); ...

• Avoid scanning class-path • Instead use a serialized index • Faster lookup: no scanning! IndexView index = combinedIndexBuildItem.getIndex(); DotName classDotName = DotName.createSimple(className); return !index.getAnnotations(classDotName).isEmpty() || index.getClassByName(classDotName) != null;

META-INF/services/io.quarkus.deployment.dev.CompilationProvider: org.kie.kogito.quarkus.deployment.ProcessCompilationProvider org.kie.kogito.quarkus.deployment.RulesCompilationProvider org.kie.kogito.quarkus.deployment.DecisionTablesCompilationProvider org.kie.kogito.quarkus.deployment.DMNCompilationProvider

if (liveReload.isLiveReload()) { return; }

No content

Developer Mode / Hot Reload e.g io.quarkus.it.kogito.drools.HotReloadTest

• integration-tests • integration-tests-kogito-plugin • integration-tests-quarkus • integration-tests-quarkus-decisions • integration-tests-quarkus-predictions • integration-tests-quarkus-processes • integration-tests-quarkus-rules • integration-tests-springboot

No content

• Quarkus general approach is to generate byte-code • In Kogito we generate Java source code • In some cases we compile it on-the-fly

• Templated Approach • Useful for Scaffolding • Learn more about this in a Future Meeting

No content

Here Be Dragons Charmander Vector by Pokinee (DeviantArt) deviantart.com/pokinee/art/Charmander-Vector-427462295

$ mvn kogito:scaffold ... [info] Generating...

• fiddling with generated code • target/generated-code/kogito/

No content

• Native binary compilation • Restriction: “closed-world assumption” • Limitations on reflection • No dynamic code loading: forbidden ClassLoader#defineClass(...byte[]...) • Allows more aggressive optimization (pruning) • Static initializers are not lazy* ! • Evaluated at build time ! * in Quarkus

• Native does not prevent dynamic code loading! • e.g. dlopen() for dynamic .so/.dll/.dylib loading • JVM mode does not prevent pruning! • e.g. ProGuard • These are just choices of the native image compiler

META-INF/ └── native-image └── groupID └── artifactID └── native-image.properties └── reflection-config.json └── resource-config.json └── jni-config.json └── proxy-config.json ReflectiveClassBuildItem

@TargetClass(ASMAccessorOptimizer.class) final class ASMAccessorOptimizer_Target { @Substitute public Accessor optimizeAccessor( ParserContext pCtx, char[] property, int start, int offset, Object ctx, Object thisRef, VariableResolverFactory factory, boolean rootThisRef, Class ingressType) { throw new UnsupportedOperationException(); } ... }

@Recorder class HelloRecorder { public void sayHello(String name) { System.out.println("Hello" + name); } } @Record(RUNTIME_INIT) @BuildStep public void helloBuildStep(HelloRecorder recorder) { recorder.sayHello("World"); }

public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } }

$ java Example hello world

$ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB

$ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB

$ ./example world

the check on a static method may not be equivalent from the perspective of the native image compiler, which does aggressive code pruning at build time. If the field is static, then the entire non-true branch is basically treated as unreachable; otherwise the native image compiler will try to visit the other branch too, which in some cases may mark as reachable code things that would not really be reached at runtime. this may not be an issue here, but it may be if that code eventually leads to some "banned" method call (e.g. ClassLoader#defineClass()), causing the native image build to fail.

private static final Collection readers = List.of( readResource("some/path..."), ... ); private static InputStreamReader readResource(InputStream stream) { return new java.io.InputStreamReader(stream); }

private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

private boolean isNativeImage() { return System.getProperty("org.graalvm.nativeimage.imagecode") != null; } private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!isNativeImage()) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } } Not Equivalent!

No content

I don’t want to contribute to the extension. What else can I do? - contribute documentation - contribute examples - hang out on the Zulip channel / mailing list and help other users out!

No content