Slide 1

Slide 1 text

The Marriage Made on a Cloud

Slide 2

Slide 2 text

• Edoardo Vacchi @evacchi • Research @ UniMi / MaTe • Research @ UniCredit R&D • Kogito / Drools / jBPM @ Red Hat

Slide 3

Slide 3 text

• You are a team mate ( hey! ) • You are a wannabe contributor ( wow! ) • You are just curious ( welcome! )

Slide 4

Slide 4 text

Step 1 • Head over to • github.com/kiegroup/kogito-runtimes OK, now what?

Slide 5

Slide 5 text

• Kogito • Compile-Time vs Run-Time • Quarkus Extensions • Bytecode vs Source-Code Generation • GraalVM Native image • Scaffolding and Troubleshooting

Slide 6

Slide 6 text

Cloud-Native Business Automation for building intelligent applications, backed by battle-tested capabilities

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

• There is always a pre-processing phase where you prepare your program for execution • Then, there's actual process execution phase

Slide 9

Slide 9 text

• Will that configuration change across runs? • Do you have to repackage the application to bundle the new configuration?

Slide 10

Slide 10 text

• You are building an Immutable Dockerized Microservice in the Cloud • You are not going to load new classes at run-time • You don’t need Runtime Dependency Injection

Slide 11

Slide 11 text

Build-time • Pre-process configuration • Generate code with configuration options pre-loaded • Resolve dependencies • Generate “wiring” code -- e.g. factories, startup methods etc.

Slide 12

Slide 12 text

Dagger 2

Slide 13

Slide 13 text

Dagger 2

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Assets Wiring Pre-Built Assets REST/Event Handlers ∙ “Drools Canonical Model” ∙ BPMN Fluent API ∙ ... ∙ DRL ∙ BPMN ∙ DMN ∙ ... ∙ JAX-RS Endpoints ∙ Cloud Event Reactive Consumers ∙ ... ∙ CDI Producers ∙ Bean Factories ∙ ...

Slide 17

Slide 17 text

REST Resources class MyModelProcessResource { @Inject Process p; } class MyRuleUnitResource { @Inject RuleUnit r; } Wiring Code (CDI etc.) Application.java ApplicationConfig.java Processes.java Decisions.java ... (Inferred) Data Model (process variables, DMN ItemDefinitions, DMN context, Rule Unit descriptors...) e.g. MyModel, MyUnitData Typed Knowledge Asset API (and related wiring) exposes the engine e.g. Process, RuleUnit, DMNDecision

Slide 18

Slide 18 text

REST Resources class MyModelProcessResource { @Inject Process p; } class MyRuleUnitResource { @Inject RuleUnit r; } Wiring Code (CDI etc.) Application.java ApplicationConfig.java Processes.java Decisions.java ... (Inferred) Data Model (process variables, DMN ItemDefinitions, DMN context, Rule Unit descriptors...) e.g. MyModel, MyUnitData Typed Knowledge Asset API (and related wiring) exposes the engine e.g. Process, RuleUnit, DMNDecision Implementation Detail ! Heavy Rework in Progress ! Do Not Rely on The Details !

Slide 19

Slide 19 text

• Maven Plug-In • Quarkus Extension

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

• Codegen-based framework • Plug into the codegen mechanism • Produce more code

Slide 22

Slide 22 text

… show path here ...

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

• deployment • runtime

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

• @BuildStep • annotates a method • method parameters: required items • return type: provided result • You can think of it as of • a CDI @Produces method or • a Spring @Factory • With a twist: • You can return more than one element at once (with a “trick”) @BuildStep public SomeBuildItem myBuildStep( SomeOtherBuildItem items ) { ... }

Slide 27

Slide 27 text

all: foo.o bar.o cc -o out foo.o bar.o foo.o: foo.c $(CC) -c foo.c -o foo.o bar.o: bar.c $(CC) -c bar.c -o bar.o

Slide 28

Slide 28 text

foo.h foo.o a.out bar.o foo.c bar.h bar.c

Slide 29

Slide 29 text

• Think of them as CDI/Spring beans • A special interface that represents things that can be • produced (provided) • consumed (required) @BuildStep public SomeBuildItem generate( SomeOtherBuildItem items) { ... }

Slide 30

Slide 30 text

• Think of them as CDI/Spring beans • A special interface that represents things that can be • produced (provided) • consumed (required) @BuildStep public SomeBuildItem generate( SomeOtherBuildItem items) { ... } @BuildStep public SomeBuildItem generate( Collection items) { ... } You can depend on a collection !

Slide 31

Slide 31 text

@BuildStep public void generate( SomeBuildItem items, BuildProducer bp) { ... bp.produce(t); } BuildProducer generatedBeans, BuildProducer resource, BuildProducer reflectiveClass, BuildProducer genResBI

Slide 32

Slide 32 text

@BuildStep public void generate( SomeBuildItem items, BuildProducer bp) { ... bp.produce(t); } BuildProducer generatedBeans, BuildProducer resource, BuildProducer reflectiveClass, BuildProducer genResBI You can have as many as you want ! Simulates multiple return values !

Slide 33

Slide 33 text

In the body of the methods // real work occurs here: invoke the code-generation procedure Collection generatedFiles = appGen.generate(); // Write files to disk dumpFilesToDisk(appPaths, generatedFiles);

Slide 34

Slide 34 text

private void registerResources(Collection generatedFiles, BuildProducer resource, BuildProducer genResBI) { for (GeneratedFile f : generatedFiles) { if (f.getType() == GeneratedFile.Type.GENERATED_CP_RESOURCE) { genResBI.produce(new GeneratedResourceBuildItem(f.relativePath(), f.contents())); resource.produce(new NativeImageResourceBuildItem(f.relativePath())); } } }

Slide 35

Slide 35 text

result.add(new ReflectiveHierarchyIgnoreWarningBuildItem( DotName.createSimple("org.kie.dmn.api.core.DMNContext"))); ... toReturn.add(new ReflectiveClassBuildItem(true, true, PMML4Result.class)); ...

Slide 36

Slide 36 text

• Avoid scanning class-path • Instead use a serialized index • Faster lookup: no scanning! IndexView index = combinedIndexBuildItem.getIndex(); DotName classDotName = DotName.createSimple(className); return !index.getAnnotations(classDotName).isEmpty() || index.getClassByName(classDotName) != null;

Slide 37

Slide 37 text

META-INF/services/io.quarkus.deployment.dev.CompilationProvider: org.kie.kogito.quarkus.deployment.ProcessCompilationProvider org.kie.kogito.quarkus.deployment.RulesCompilationProvider org.kie.kogito.quarkus.deployment.DecisionTablesCompilationProvider org.kie.kogito.quarkus.deployment.DMNCompilationProvider

Slide 38

Slide 38 text

if (liveReload.isLiveReload()) { return; }

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

Developer Mode / Hot Reload e.g io.quarkus.it.kogito.drools.HotReloadTest

Slide 41

Slide 41 text

• integration-tests • integration-tests-kogito-plugin • integration-tests-quarkus • integration-tests-quarkus-decisions • integration-tests-quarkus-predictions • integration-tests-quarkus-processes • integration-tests-quarkus-rules • integration-tests-springboot

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

• Quarkus general approach is to generate byte-code • In Kogito we generate Java source code • In some cases we compile it on-the-fly

Slide 44

Slide 44 text

• Templated Approach • Useful for Scaffolding • Learn more about this in a Future Meeting

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

Here Be Dragons Charmander Vector by Pokinee (DeviantArt) deviantart.com/pokinee/art/Charmander-Vector-427462295

Slide 47

Slide 47 text

$ mvn kogito:scaffold ... [info] Generating...

Slide 48

Slide 48 text

• fiddling with generated code • target/generated-code/kogito/

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

• Native binary compilation • Restriction: “closed-world assumption” • Limitations on reflection • No dynamic code loading: forbidden ClassLoader#defineClass(...byte[]...) • Allows more aggressive optimization (pruning) • Static initializers are not lazy* ! • Evaluated at build time ! * in Quarkus

Slide 51

Slide 51 text

• Native does not prevent dynamic code loading! • e.g. dlopen() for dynamic .so/.dll/.dylib loading • JVM mode does not prevent pruning! • e.g. ProGuard • These are just choices of the native image compiler

Slide 52

Slide 52 text

META-INF/ └── native-image └── groupID └── artifactID └── native-image.properties └── reflection-config.json └── resource-config.json └── jni-config.json └── proxy-config.json ReflectiveClassBuildItem

Slide 53

Slide 53 text

@TargetClass(ASMAccessorOptimizer.class) final class ASMAccessorOptimizer_Target { @Substitute public Accessor optimizeAccessor( ParserContext pCtx, char[] property, int start, int offset, Object ctx, Object thisRef, VariableResolverFactory factory, boolean rootThisRef, Class ingressType) { throw new UnsupportedOperationException(); } ... }

Slide 54

Slide 54 text

@Recorder class HelloRecorder { public void sayHello(String name) { System.out.println("Hello" + name); } } @Record(RUNTIME_INIT) @BuildStep public void helloBuildStep(HelloRecorder recorder) { recorder.sayHello("World"); }

Slide 55

Slide 55 text

public class Example { static { System.out.println("hello"); } public static void main(String... args) { System.out.println("world"); } }

Slide 56

Slide 56 text

$ java Example hello world

Slide 57

Slide 57 text

$ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB

Slide 58

Slide 58 text

$ native-image --initialize-at-build-time Example [example:23074] classlist: 1,032.11 ms, 1.18 GB [example:23074] (cap): 2,301.26 ms, 1.18 GB [example:23074] setup: 3,609.57 ms, 1.69 GB hello [example:23074] (clinit): 82.45 ms, 1.73 GB [example:23074] (typeflow): 3,032.00 ms, 1.73 GB [example:23074] (objects): 2,923.76 ms, 1.73 GB [example:23074] (features): 129.59 ms, 1.73 GB [example:23074] analysis: 6,307.81 ms, 1.73 GB [example:23074] universe: 277.17 ms, 1.73 GB [example:23074] (parse): 525.88 ms, 1.73 GB [example:23074] (inline): 877.57 ms, 1.78 GB [example:23074] (compile): 3,842.94 ms, 1.87 GB [example:23074] compile: 5,504.45 ms, 1.87 GB [example:23074] image: 463.22 ms, 1.87 GB [example:23074] write: 176.80 ms, 1.87 GB [example:23074] [total]: 17,528.27 ms, 1.87 GB

Slide 59

Slide 59 text

$ ./example world

Slide 60

Slide 60 text

the check on a static method may not be equivalent from the perspective of the native image compiler, which does aggressive code pruning at build time. If the field is static, then the entire non-true branch is basically treated as unreachable; otherwise the native image compiler will try to visit the other branch too, which in some cases may mark as reachable code things that would not really be reached at runtime. this may not be an issue here, but it may be if that code eventually leads to some "banned" method call (e.g. ClassLoader#defineClass()), causing the native image build to fail.

Slide 61

Slide 61 text

private static final Collection readers = List.of( readResource("some/path..."), ... ); private static InputStreamReader readResource(InputStream stream) { return new java.io.InputStreamReader(stream); }

Slide 62

Slide 62 text

private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

Slide 63

Slide 63 text

private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

Slide 64

Slide 64 text

private final static boolean IS_NATIVE_IMAGE = System.getProperty("org.graalvm.nativeimage.imagecode") != null; private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!IS_NATIVE_IMAGE) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

Slide 65

Slide 65 text

private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } }

Slide 66

Slide 66 text

private boolean isNativeImage() { return System.getProperty("org.graalvm.nativeimage.imagecode") != null; } private static final Collection readers = List.of(...); private static InputStreamReader readResource(java.io.InputStream stream) { if (!isNativeImage()) { return new java.io.InputStreamReader(stream); } try { byte[] bytes = stream.readAllBytes(); ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes); return new InputStreamReader(byteArrayInputStream); } catch (java.io.IOException e) { throw new java.io.UncheckedIOException(e); } } Not Equivalent!

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

I don’t want to contribute to the extension. What else can I do? - contribute documentation - contribute examples - hang out on the Zulip channel / mailing list and help other users out!

Slide 69

Slide 69 text

No content