Tweet
Tweet

Slide 1

Slide 1 text

Online Conformance Checking Using Behavioural Patterns A. Burattin1, S. van Zelst 2, A. Armas-Cervantes3, B. van Dongen 2, J. Carmona 4 1 Technical University of Denmark, Kgs. Lyngby, Denmark 2 Eindhoven University of Technology, Eindhoven, The Netherlands 3 The University of Melbourne, Melbourne, Australia 4 Universitat Politècnica de Catalunya, Barcelona, Spain

Slide 2

Slide 2 text

What is conformance checking? • Conformance checking aims at comparing prescriptive process models with actual executions, pinpointing deviations 2

Slide 3

Slide 3 text

What is conformance checking? • Conformance checking aims at comparing prescriptive process models with actual executions, pinpointing deviations • State of the art uses alignments between a given execution trace and the most similar (i.e., cheapest) prescriptive trace • An alignment cost of 0 indicates no deviation • An alignment cost > 0 indicated some sort of deviation 2

Slide 4

Slide 4 text

What is conformance checking? • Conformance checking aims at comparing prescriptive process models with actual executions, pinpointing deviations • State of the art uses alignments between a given execution trace and the most similar (i.e., cheapest) prescriptive trace • An alignment cost of 0 indicates no deviation • An alignment cost > 0 indicated some sort of deviation • Alignments have been introduces as offline (or post-mortem) techniques, but online (or in vivo) approaches are desirable 2

Slide 5

Slide 5 text

Motivation for online conformance Not a real xkcd comic. 3

Slide 6

Slide 6 text

What is online conformance checking? • Given a sequence of observations and prescriptive process models pinpoint deviations • Weak notion of trace 4

Slide 7

Slide 7 text

What is online conformance checking? • Given a sequence of observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) 4

Slide 8

Slide 8 text

What is online conformance checking? • Given a sequence of observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) 4

Slide 9

Slide 9 text

What is online conformance checking? • Given a sequence of observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) • No head and no tail is allowed (partial trace with warm start) 4

Slide 10

Slide 10 text

What is online conformance checking? • Given a sequence of observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) • No head and no tail is allowed (partial trace with warm start) • Real-time processing of event streams 4

Slide 11

Slide 11 text

What is online conformance checking? • Given a sequence of observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) • No head and no tail is allowed (partial trace with warm start) • Real-time processing of event streams • Technical challenges with event stream real-time processing • Impossible to store the entire log, i.e., approximations needed • Variable system conditions and fluctuating stream rates • Unbounded backtracking is not feasible • One pass on data: scale linearly with respect to number of observations 4

Slide 12

Slide 12 text

Related work • Mainly two lines of research in online conformance checking 5

Slide 13

Slide 13 text

Related work • Mainly two lines of research in online conformance checking 1. Approaches based on prefix-alignment computation • Explanation of prefixes of complete behaviour • High complexity, cannot cope with warm start 5

Slide 14

Slide 14 text

Related work • Mainly two lines of research in online conformance checking 1. Approaches based on prefix-alignment computation • Explanation of prefixes of complete behaviour • High complexity, cannot cope with warm start 2. Approaches where all possible deviations are pre-computed • High offline cost • Important bias (no contextual information) 5

Slide 15

Slide 15 text

Related work • Mainly two lines of research in online conformance checking 1. Approaches based on prefix-alignment computation • Explanation of prefixes of complete behaviour • High complexity, cannot cope with warm start 2. Approaches where all possible deviations are pre-computed • High offline cost • Important bias (no contextual information) • No approach can handle partial traces in warm start scenarios in an effective way 5

Slide 16

Slide 16 text

Table of contents • Introduction and motivation • Online conformance checking • Intuition • General approach for behavioural patterns • Instance to direct following relations • Experimental evaluation • Conclusion 6

Slide 17

Slide 17 text

Running example • Let’s consider this process model • Offline conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7

Slide 18

Slide 18 text

Running example • Let’s consider this process model • Offline conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7

Slide 19

Slide 19 text

Running example • Let’s consider this process model • Offline conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7

Slide 20

Slide 20 text

Running example • Let’s consider this process model • Offline conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7

Slide 21

Slide 21 text

Online conformance checking intuition • Key idea: split the metric to capture each concept independently 8

Slide 22

Slide 22 text

Online conformance checking intuition • Key idea: split the metric to capture each concept independently 8

Slide 23

Slide 23 text

Online conformance checking intuition • Key idea: split the metric to capture each concept independently 8

Slide 24

Slide 24 text

Online conformance checking intuition • Key idea: split the metric to capture each concept independently 8

Slide 25

Slide 25 text

Online conformance checking intuition • Key idea: split the metric to capture each concept independently 8

Slide 26

Slide 26 text

Online conformance checking intuition • Key idea: split the metric to capture each concept independently 8

Slide 27

Slide 27 text

Online conformance checking intuition • Key idea: split the metric to capture each concept independently 8

Slide 28

Slide 28 text

Running example Observation Offline conformance Online measure Conformance Completeness Confidence , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9

Slide 29

Slide 29 text

Running example Observation Offline conformance Online measure Conformance Completeness Confidence , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9

Slide 30

Slide 30 text

Running example Observation Offline conformance Online measure Conformance Completeness Confidence , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9

Slide 31

Slide 31 text

Running example Observation Offline conformance Online measure Conformance Completeness Confidence , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9

Slide 32

Slide 32 text

Running example Observation Offline conformance Online measure Conformance Completeness Confidence , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9

Slide 33

Slide 33 text

Formal definitions • Behavioural pattern • Given set of activities and set of control-flow relations ℛ, a behavioural pattern is 1 , 2 where ∈ ℛ and 1 , 2 ∈ • Observation (or observable unit) • Given set of case ids , an observable unit is = , where ∈ and ∈ ℛ × × • Stream of observations • An [infinite] sequence of observable units: 1 , 2 , 3 , … 10

Slide 34

Slide 34 text

Formal definitions • Behavioural pattern • Given set of activities and set of control-flow relations ℛ, a behavioural pattern is 1 , 2 where ∈ ℛ and 1 , 2 ∈ • Observation (or observable unit) • Given set of case ids , an observable unit is = , where ∈ and ∈ ℛ × × • Stream of observations • An [infinite] sequence of observable units: 1 , 2 , 3 , … 10

Slide 35

Slide 35 text

Pattern instances as observations • A stream of behavioural patterns represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) 11

Slide 36

Slide 36 text

Pattern instances as observations • A stream of behavioural patterns represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) • Example with direct following relation 11

Slide 37

Slide 37 text

Pattern instances as observations • A stream of behavioural patterns represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) • Example with direct following relation 11

Slide 38

Slide 38 text

Pattern instances as observations • A stream of behavioural patterns represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) • Example with direct following relation 11

Slide 39

Slide 39 text

Intermediate structure • To calculate the online conformance we convert the process model into corresponding PMOC (Process Model for Online Conformance) • PMOC is a tiplet = , , • ⊆ ℛ × × is the set of possible behavioural patterns • : → ℕ × ℕ maps each pattern ∈ to the minimum and maximum number of distinct patters expected before • : → ℕ maps each pattern ∈ to the minimum number of distinct patters expected after 12

Slide 40

Slide 40 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 13

Slide 41

Slide 41 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 13

Slide 42

Slide 42 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13

Slide 43

Slide 43 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13

Slide 44

Slide 44 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 13

Slide 45

Slide 45 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 13

Slide 46

Slide 46 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13

Slide 47

Slide 47 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 3 13

Slide 48

Slide 48 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 3 4 13

Slide 49

Slide 49 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 3 4 13

Slide 50

Slide 50 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 13

Slide 51

Slide 51 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 13

Slide 52

Slide 52 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13

Slide 53

Slide 53 text

Example with ≺ (direct following relation) • Consider the model • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13

Slide 54

Slide 54 text

PMOC for ≺ and Petri nets • Give a safe Petri net of our process 14

Slide 55

Slide 55 text

PMOC for ≺ and Petri nets • Give a safe Petri net of our process • Construct finite representation with complete prefix unfolding • Unfolding with 2 markings equivalent if: (i) same places; (ii) same relations between activities to reach them; (iii) same set of lastly activities executed 14

Slide 56

Slide 56 text

PMOC for ≺ and Petri nets • Give a safe Petri net of our process • Construct finite representation with complete prefix unfolding • Unfolding with 2 markings equivalent if: (i) same places; (ii) same relations between activities to reach them; (iii) same set of lastly activities executed 14

Slide 57

Slide 57 text

PMOC for ≺ and Petri nets • Give a safe Petri net of our process • Construct finite representation with complete prefix unfolding • Unfolding with 2 markings equivalent if: (i) same places; (ii) same relations between activities to reach them; (iii) same set of lastly activities executed 14

Slide 58

Slide 58 text

PMOC for ≺ and Petri nets (cont.) To construct = , , With the reachability graph of the unfolded net we can extract 15

Slide 59

Slide 59 text

PMOC for ≺ and Petri nets (cont.) To construct = , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets

Slide 60

Slide 60 text

PMOC for ≺ and Petri nets (cont.) To construct = , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets

Slide 61

Slide 61 text

PMOC for ≺ and Petri nets (cont.) To construct = , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets

Slide 62

Slide 62 text

PMOC for ≺ and Petri nets (cont.) To construct = , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets

Slide 63

Slide 63 text

Online conformance checking algorithm 16

Slide 64

Slide 64 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16

Slide 65

Slide 65 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16

Slide 66

Slide 66 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16

Slide 67

Slide 67 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16

Slide 68

Slide 68 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16

Slide 69

Slide 69 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16

Slide 70

Slide 70 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16

Slide 71

Slide 71 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end Approximations in the computation 16

Slide 72

Slide 72 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end Constant time required per event Approximations in the computation 16

Slide 73

Slide 73 text

Online conformance checking algorithm Input: PMOC = , , and the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end Constant time required per event General to any behavioural pattern Approximations in the computation 16

Slide 74

Slide 74 text

Evaluation / Time performance • System implemented in ProM (StreamConformance package, no optimization, plain Java structures) 17

Slide 75

Slide 75 text

Evaluation / Time performance • System implemented in ProM (StreamConformance package, no optimization, plain Java structures) • Stress test • Random BPMN model with 64 activities and 26 gateways • 2 million events, standard desktop machine 17

Slide 76

Slide 76 text

Evaluation / Time performance • System implemented in ProM (StreamConformance package, no optimization, plain Java structures) • Stress test • Random BPMN model with 64 activities and 26 gateways • 2 million events, standard desktop machine 0 50 100 150 200 250 0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 Space (MB) Time (ms) Events Average processing time per event Total space used (MB) 17

Slide 77

Slide 77 text

Evaluation / Time performance • System implemented in ProM (StreamConformance package, no optimization, plain Java structures) • Stress test • Random BPMN model with 64 activities and 26 gateways • 2 million events, standard desktop machine 0 50 100 150 200 250 0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 Space (MB) Time (ms) Events Average processing time per event Total space used (MB) 17

Slide 78

Slide 78 text

Evaluation / Comparison with other metric • Comparison of proposed metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18

Slide 79

Slide 79 text

Evaluation / Comparison with other metric • Comparison of proposed metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18

Slide 80

Slide 80 text

Evaluation / Comparison with other metric • Comparison of proposed metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18

Slide 81

Slide 81 text

Evaluation / Comparison with other metric • Comparison of proposed metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18

Slide 82

Slide 82 text

Evaluation / Comparison with other metric • Combinations where at least one of the techniques identifies a deviation 19

Slide 83

Slide 83 text

Evaluation / Comparison with other metric • Combinations where at least one of the techniques identifies a deviation 19

Slide 84

Slide 84 text

Evaluation / Comparison with other metric • Combinations where at least one of the techniques identifies a deviation • Conclusions • When no deviations, the techniques agree • Deviations, though, are captured in different ways due to different representation biases 19

Slide 85

Slide 85 text

Evaluation / Real event log • We tested our approach on the “road traffic fines log” • 316868 events / 83614 cases, DOI: 10.1007/s00607-015-0441-1 • Total time: 44967 ms; 0.141 ms/event 20

Slide 86

Slide 86 text

Evaluation / Real event log • We tested our approach on the “road traffic fines log” • 316868 events / 83614 cases, DOI: 10.1007/s00607-015-0441-1 • Total time: 44967 ms; 0.141 ms/event 0 0.25 0.5 0.75 1 Conformance 0 0.25 0.5 0.75 1 Completeness 0 0.25 0.5 0.75 1 0 50000 100000 150000 200000 250000 300000 Confidence Events 20

Slide 87

Slide 87 text

Evaluation / Real event log • We tested our approach on the “road traffic fines log” • 316868 events / 83614 cases, DOI: 10.1007/s00607-015-0441-1 • Total time: 44967 ms; 0.141 ms/event 0 0.25 0.5 0.75 1 Conformance 0 0.25 0.5 0.75 1 Completeness 0 0.25 0.5 0.75 1 0 50000 100000 150000 200000 250000 300000 Confidence Events Observations • 93.5% events conformance 1 99.6% events conformance ≥ 0.5 • 99.8% events completeness 1: most executions from beginning • 99.4% events confidence 1: the model which allows immediate termination after first activity 20

Slide 88

Slide 88 text

Conclusion and future work • Approach to compute online conformance starting from stream of behavioural patterns 21

Slide 89

Slide 89 text

Conclusion and future work • Approach to compute online conformance starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after 21

Slide 90

Slide 90 text

Conclusion and future work • Approach to compute online conformance starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after • Online measure split into • Actual conformance: ratio of correct relations being observed over all obs • Completeness: is the instance starting from the beginning (warm start)? • Confidence: can the current conformance change in the future (partial traces)? 21

Slide 91

Slide 91 text

Conclusion and future work • Approach to compute online conformance starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after • Online measure split into • Actual conformance: ratio of correct relations being observed over all obs • Completeness: is the instance starting from the beginning (warm start)? • Confidence: can the current conformance change in the future (partial traces)? • An approach based on unfolding of Petri nets has been presented with direct following relation 21

Slide 92

Slide 92 text

Conclusion and future work • Approach to compute online conformance starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after • Online measure split into • Actual conformance: ratio of correct relations being observed over all obs • Completeness: is the instance starting from the beginning (warm start)? • Confidence: can the current conformance change in the future (partial traces)? • An approach based on unfolding of Petri nets has been presented with direct following relation • Future work includes the extension to new implementation (i.e., patterns/perspectives) and optimization towards optimal costs 21