[KIELive#39] Migrating Drools to the cloud with Kogito: a step by step path

Slide 1

Slide 1 text

1 Migrating to the cloud with : a step by step path Mario Fusco @mariofusco Principal Software Engineer Drools Project Lead July, 2021

Slide 2

Slide 2 text

From Drools to Kogito - Agenda Drools Application

Slide 3

Slide 3 text

Quarkus - REST endpoint From Drools to Kogito - Agenda Drools Application Rules evaluation in microservice

Slide 4

Slide 4 text

Quarkus - REST endpoint From Drools to Kogito - Agenda Drools Application Kogito Quarkus extension + legacy Drools API Rules evaluation in microservice Hot reload + Native image

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Quarkus - REST endpoint From Drools to Kogito - Agenda Drools Application Kogito Quarkus extension + legacy Drools API Kogito Quarkus extension + Rule units API Rules evaluation in microservice Hot reload + Native image Automatic REST endpoint generation https://github.com/kiegroup/kogito-examples/ tree/master/rules-legacy-quarkus-example https://github.com/kiegroup/kogito-examples/ tree/master/ruleunit-quarkus-example

Slide 7

Slide 7 text

A simple Drools project (domain model) public class Applicant { private String name; private int age; public Applicant(String name, int age) { this.name = name; this.age = age; } } public class LoanApplication { private String id; private Applicant applicant; private int amount; private int deposit; private boolean approved = false; public LoanApplication(String id, Applicant applicant, int amount, int deposit) { this.id = id; this.applicant = applicant; this.amount = amount; this.deposit = deposit; } }

Slide 8

Slide 8 text

A simple Drools project (rules) global Integer maxAmount; global java.util.List approvedApplications; rule LargeDepositApprove when $l: LoanApplication( applicant.age >= 20, deposit >= 1000, amount <= maxAmount ) then modify($l) { setApproved(true) }; // loan is approved end rule LargeDepositReject when $l: LoanApplication( applicant.age >= 20, deposit >= 1000, amount > maxAmount ) then modify($l) { setApproved(false) }; // loan is rejected end // ... more loans approval/rejections business rules ... rule CollectApprovedApplication when $l: LoanApplication( approved ) then approvedApplications.add($l); // collect all approved loan applications end

Slide 9

Slide 9 text

Rules evaluation as a Function as a Service ● Stateless rules evaluation can be seen as a pure function invocation ○ You invoke the function with a few arguments and it produces a result ○ The arguments are actually the facts inserted into the sessions and the result is the outcome of your rules set applied on those facts ○ But for the users of your function this is an implementation detail ● This function invocation can be exposed through a REST endpoint, making it a perfect microservice ● It can be eventually deployed in a Function as a Service environment ○ Maybe for this purpose a Java application is not a good fit and it will be necessary to create a native image (more on this later…) Stateless rules evaluation ... … as a pure function invocation ... … exposed as a REST endpoint ... … packaged into a native image

Slide 10

Slide 10 text

10 Step 1: Exposing rules evaluation with a REST endpoint through Quarkus

Slide 11

Slide 11 text

Integrating with Quarkus io.quarkus quarkus-resteasy io.quarkus quarkus-resteasy-jackson org.example drools-project 1.0-SNAPSHOT maven-compiler-plugin 3.1 11 11 io.quarkus quarkus-maven-plugin build The kie project containing the business rules

Slide 12

Slide 12 text

Exposing rules evaluation with a REST endpoint @Path("/find-approved") public class FindApprovedLoansEndpoint { private static final KieContainer kContainer = KieServices.Factory.get().newKieClasspathContainer(); @POST() @Produces(MediaType.APPLICATION_JSON) @Consumes(MediaType.APPLICATION_JSON) public List executeQuery(LoanAppDto loanAppDto) { KieSession session = kContainer.newKieSession(); List approvedApplications = new ArrayList<>(); session.setGlobal("approvedApplications", approvedApplications); session.setGlobal("maxAmount", loanAppDto.getMaxAmount()); loanAppDto.getLoanApplications().forEach(session::insert); session.fireAllRules(); session.dispose(); return approvedApplications; } }

Slide 13

Slide 13 text

The DTO public class LoanAppDto { private int maxAmount; private List loanApplications; public int getMaxAmount() { return maxAmount; } public void setMaxAmount(int maxAmount) { this.maxAmount = maxAmount; } public List getLoanApplications() { return loanApplications; } public void setLoanApplications(List loanApplications) { this.loanApplications = loanApplications; } }

Slide 14

Slide 14 text

Giving it a try mario@mario-pc:~/workspace/drools/drools-quarkus$ java -jar quarkus-app/target/quarkus-app/quarkus-run.jar __ ____ __ _____ ___ __ ____ ______ --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \ --\___\_\____/_/ |_/_/|_/_/|_|\____/___/ 2021-07-05 18:24:48,269 INFO [org.dro.com.kie.bui.imp.ClasspathKieProject] (main) Found kmodule: jar:file:/home/mario/workspace/drools/drools-quarkus/quarkus-app/t arget/quarkus-app/lib/main/org.example.drools-project-1.0-SNAPSHOT .jar!/META-INF/kmodule.xml 2021-07-05 18:24:49,391 INFO [org.dro.com.kie.bui.imp.InternalKieModuleProvider] (main) Creating KieModule for artifact org.example:drools-project:1.0-SNAPSHOT 2021-07-05 18:24:49,439 INFO [org.dro.mod.CanonicalKieModuleProvider] (main) Artifact org.example:drools-project:1.0-SNAPSHOT has executable model 2021-07-05 18:24:50,995 INFO [io.quarkus] (main) quarkus-app 1.0-SNAPSHOT on JVM (powered by Quarkus 1.13.6.Final) started in 4.728s. Listening on: http://0.0.0.0:8080 2021-07-05 18:24:50,998 INFO [io.quarkus] (main) Profile prod activated. 2021-07-05 18:24:50,998 INFO [io.quarkus] (main) Installed features: [cdi, resteasy, resteasy-jackson]

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Drools on Quarkus is cool, we are all done right? NOPE! We cannot leverage a big part of Quarkus features at the moment, and in particular we are missing ● Hot reload ● Native image compilation

Slide 17

Slide 17 text

17 Step 2: From Drools to Kogito without changing (almost) anything - Legacy API

Slide 18

Slide 18 text

From Drools to Kogito (with the legacy API) org.kie.kogito kogito-quarkus-rules org.kie.kogito kogito-legacy-api io.quarkus quarkus-maven-plugin build The Quarkus Kogito extension for rules This module allows you to use the legacy API (since we want to make minimal changes to our project) The use of the Kogito extension also makes possible to consolidate the former 2 modules into a single one

Slide 19

Slide 19 text

Exposing rules evaluation with a REST endpoint @Path("/find-approved") public class FindApprovedLoansEndpoint { @Inject KieRuntimeBuilder kieRuntimeBuilder; @POST() @Produces(MediaType.APPLICATION_JSON) @Consumes(MediaType.APPLICATION_JSON) public List executeQuery(LoanAppDto loanAppDto) { KieSession session = kieRuntimeBuilder.newKieSession(); List approvedApplications = new ArrayList<>(); session.setGlobal("approvedApplications", approvedApplications); session.setGlobal("maxAmount", loanAppDto.getMaxAmount()); loanAppDto.getLoanApplications().forEach(session::insert); session.fireAllRules(); session.dispose(); return approvedApplications; } } The KieRuntimeBuilder is generated by the Kogito extension and automatically injected by Quarkus You create KieSessions out of it exactly as you did for the KieContainer No other change is necessary

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Quarkus hot reload mario@mario-pc:~/workspace/kogito/kogito-examples/rules-legacy-quarkus-example$ mvn quarkus:dev [INFO] --- quarkus-maven-plugin:2.0.0.Final:dev (default-cli) @ rules-legacy-quarkus-example --- Listening for transport dt_socket at address: 5005 __ ____ __ _____ ___ __ ____ ______ --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \ --\___\_\____/_/ |_/_/|_/_/|_|\____/___/ 2021-07-06 16:44:38,165 INFO [io.quarkus] (Quarkus Main Thread) rules-legacy-quarkus-example 2.0.0-SNAPSHOT on JVM (powered by Quarkus 2.0.0.Final) started in 2.510s. Listening on: http://localhost:8080 2021-07-06 16:44:38,167 INFO [io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated. 2021-07-06 16:45:44,974 INFO [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Live reload total time: 0.627s 2021-07-06 16:46:01,489 WARN [io.qua.dep.dev.JavaCompilationProvider] (Quarkus Test Watcher - 0) bootstrap class path not set in conjunction with -source 8, line -1 in [unknown source] 2021-07-06 16:46:04,255 INFO [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-0) Restarting quarkus due to changes in HotReloadSupportClass.class. 2021-07-06 16:46:04,267 INFO [io.quarkus] (Quarkus Main Thread) rules-legacy-quarkus-example stopped in 0.010s

Slide 22

Slide 22 text

Generating a native image mario@mario-pc:~/workspace/kogito/kogito-examples/rules-legacy-quarkus-example$ mvn clean package -Pnative [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM 21.1.0 Java 11 CE (Java Version 11.0.11+8-jvmci-21.1-b05) [rules-legacy-quarkus-example-runner:663694] (clinit): 790.13 ms, 2.30 GB [rules-legacy-quarkus-example-runner:663694] (typeflow): 26,623.51 ms, 2.30 GB [rules-legacy-quarkus-example-runner:663694] (objects): 35,075.43 ms, 2.30 GB [rules-legacy-quarkus-example-runner:663694] (features): 1,498.52 ms, 2.30 GB [rules-legacy-quarkus-example-runner:663694] analysis: 68,379.29 ms, 2.30 GB [rules-legacy-quarkus-example-runner:663694] universe: 2,678.90 ms, 2.30 GB [rules-legacy-quarkus-example-runner:663694] (parse): 12,574.37 ms, 2.80 GB [rules-legacy-quarkus-example-runner:663694] (inline): 11,008.55 ms, 3.55 GB [rules-legacy-quarkus-example-runner:663694] (compile): 55,101.27 ms, 3.78 GB [rules-legacy-quarkus-example-runner:663694] compile: 81,843.69 ms, 3.78 GB [rules-legacy-quarkus-example-runner:663694] image: 8,130.90 ms, 3.67 GB [rules-legacy-quarkus-example-runner:663694] write: 899.46 ms, 3.67 GB # Printing build artifacts to: rules-legacy-quarkus-example-runner.build_artifacts.txt [rules-legacy-quarkus-example-runner:663694] [total]: 170,321.21 ms, 3.67 GB [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] objcopy --strip-debug rules-legacy-quarkus-example-runner [INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 176013ms [INFO] BUILD SUCCESS

Slide 23

Slide 23 text

Running the native image mario@mario-pc:~/workspace/kogito/kogito-examples/rules-legacy-quarkus-example$ ./target/rules-legacy-quarkus-example-runner __ ____ __ _____ ___ __ ____ ______ --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \ --\___\_\____/_/ |_/_/|_/_/|_|\____/___/ 2021-07-06 17:51:49,502 INFO [io.quarkus] (main) rules-legacy-quarkus-example 2.0.0-SNAPSHOT native (powered by Quarkus 2.0.0.Final) started in 0.014s. Listening on: http://0.0.0.0:8080 2021-07-06 17:51:49,503 INFO [io.quarkus] (main) Profile prod activated. 2021-07-06 17:51:49,503 INFO [io.quarkus] (main) Installed features: [cdi, kogito-rules, resteasy, resteasy-jackson, servlet, smallrye-context-propagation, smallrye-health, smallrye-openapi, swagger-ui] Negligible start time of the native image makes it a perfect fit for a function as a service deployment

Slide 24

Slide 24 text

Much better than before right? SURE! But what if I could better define the boundaries of my business domain so that Kogito could generate the biggest part of the code for me?

Slide 25

Slide 25 text

25 Step 3: Embracing Kogito rule units + automatic REST endpoint generation

Slide 26

Slide 26 text

Introducing rule units ● Rule Unit: a unit of computation for a rule evaluation including fact to be matched and rules

Slide 27

Slide 27 text

Introducing rule units ● DataStream: append-only ○ subscribers only receive new (and possibly past) messages ○ cannot update/remove ○ stream may also be hot/cold in “reactive streams” terminology ● DataStore: add/update/remove ○ subscribers may act upon the data store, by acting upon the fact handle ● Rule Unit: a unit of computation for a rule evaluation including fact to be matched and rules

Slide 28

Slide 28 text

Why rule units Data Defs Rule Defs Multiple cross-cutting features: ● supersede kiebase definition (modularity/grouping of rules) ● supersede agenda groups / rule-flow group (modularity of unit of execution+rules, on a per-rule basis) ● supersede entry point definition (modularity of the working memory) Rule Units: ● introduce a unified, top-down module concept ● introduce an abstraction over unit of execution ● introduce a data source abstraction

Slide 29

Slide 29 text

Defining a rule unit modeling the Loan application public class LoanUnit implements RuleUnitData { private int maxAmount; private DataStore loanApplications; public LoanUnit() { this(DataSource.createStore(), 0); } public LoanUnit(DataStore loanApplications, int maxAmount) { this.loanApplications = loanApplications; this.maxAmount = maxAmount; } public DataStore getLoanApplications() { return loanApplications; } public void setLoanApplications(DataStore loanApplications) { this.loanApplications = loanApplications; } public int getMaxAmount() { return maxAmount; } public void setMaxAmount(int maxAmount) { this.maxAmount = maxAmount; } } Just a marker interface to flag this class as a container of rule unit datas

Slide 30

Slide 30 text

Introducing oopath ▸ Similar to XPath / JSONPath $street: /persons/address/street [ number == 18 ] ▸ roughly equivalent to: Person( address.street.number == 18 ) from entry-point persons ▸ or: $person: Person() from entry-point persons $address: Address() from $person.address $street: Street ( number == 18 ) from $address.street private final DataStream persons = ....

Slide 31

Slide 31 text

Rewriting rules in the rule unit context with oopath package org.kie.kogito.queries; unit LoanUnit; // no need to using globals, all variables and facts are stored in the rule unit rule LargeDepositApprove when $l: /loanApplications[ applicant.age >= 20, deposit >= 1000, amount <= maxAmount ] // oopath style then modify($l) { setApproved(true) }; end rule LargeDepositReject when $l: /loanApplications[ applicant.age >= 20, deposit >= 1000, amount > maxAmount ] then modify($l) { setApproved(false) }; end // ... more loans approval/rejections business rules ... // approved loan applications are now retrieved through a query query FindApproved $l: /loanApplications[ approved ] end

Slide 32

Slide 32 text

Rule Unit ● Automated serialization ○ inferred from the RuleUnitData definition ● native JSON ○ not a custom serialization format ● Clear definition of input and output ○ RuleUnitData for the shape of the input ○ query for the shape of the output clear definition of computation boundaries automatic REST endpoint generation

Slide 33

Slide 33 text

Automatically generated query executor public class LoanUnitQueryFindApproved implements org.kie.kogito.rules.RuleUnitQuery> { private final RuleUnitInstance instance; public LoanUnitQueryFindApproved(RuleUnitInstance instance) { this.instance = instance; } @Override public List execute() { return instance.executeQuery("FindApproved").stream().map(this::toResult).collect(toList()); } private org.kie.kogito.queries.LoanApplication toResult(Map tuple) { return (org.kie.kogito.queries.LoanApplication) tuple.get("$l"); } }

Slide 34

Slide 34 text

Automatically generated REST endpoint @Path("/find-approved") public class LoanUnitQueryFindApprovedEndpoint { @javax.inject.Inject RuleUnit ruleUnit; public LoanUnitQueryFindApprovedEndpoint() { } public LoanUnitQueryFindApprovedEndpoint(RuleUnit ruleUnit) { this.ruleUnit = ruleUnit; } @POST() @Produces(MediaType.APPLICATION_JSON) @Consumes(MediaType.APPLICATION_JSON) public List executeQuery(org.kie.kogito.queries.LoanUnit unitDTO) { RuleUnitInstance instance = ruleUnit.createInstance(unitDTO); return instance.executeQuery(LoanUnitQueryFindApproved.class); } }