Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Online Conformance Checking Using Behavioural Patterns

0b6203b08e1c063c97bb25abfc3842ec?s=47 Andrea Burattin
September 11, 2018

Online Conformance Checking Using Behavioural Patterns

New and compelling regulations (e.g., the GDPR in Europe) impose tremendous pressure on organizations, in order to adhere to standard procedures, processes, and practices. The field of conformance checking aims to quantify the extent to which the execution of a process, captured within recorded corresponding event data, conforms to a given reference process model. Existing techniques assume a post-mortem scenario, i.e. they detect deviations based on complete executions of the process. This limits their applicability in an online setting. In such context, we aim to to detect deviations online (i.e., in-vivo), in order to provide recovery possibilities before the execution of a process instance is completed. Also, current techniques assume cases to start from the initial stage of the process, whereas this assumption is not feasible in online settings. In this paper, we present a generic framework for online conformance checking, in which the underlying process is represented in terms of behavioural patterns and no assumption on the starting point of cases is needed. We instantiate the framework on the basis of Petri nets, with an accompanying new unfolding technique. The approach is implemented in the process mining tool ProM, and evaluated by means of several experiments including a stress-test and a comparison with a similar technique.

More info: https://andrea.burattin.net/publications/2018-bpm

0b6203b08e1c063c97bb25abfc3842ec?s=128

Andrea Burattin

September 11, 2018
Tweet

More Decks by Andrea Burattin

Other Decks in Science

Transcript

  1. Online Conformance Checking Using Behavioural Patterns A. Burattin1, S. van

    Zelst 2, A. Armas-Cervantes3, B. van Dongen 2, J. Carmona 4 1 Technical University of Denmark, Kgs. Lyngby, Denmark 2 Eindhoven University of Technology, Eindhoven, The Netherlands 3 The University of Melbourne, Melbourne, Australia 4 Universitat Politècnica de Catalunya, Barcelona, Spain
  2. What is conformance checking? • Conformance checking aims at comparing

    prescriptive process models with actual executions, pinpointing deviations 2
  3. What is conformance checking? • Conformance checking aims at comparing

    prescriptive process models with actual executions, pinpointing deviations • State of the art uses alignments between a given execution trace and the most similar (i.e., cheapest) prescriptive trace • An alignment cost of 0 indicates no deviation • An alignment cost > 0 indicated some sort of deviation 2
  4. What is conformance checking? • Conformance checking aims at comparing

    prescriptive process models with actual executions, pinpointing deviations • State of the art uses alignments between a given execution trace and the most similar (i.e., cheapest) prescriptive trace • An alignment cost of 0 indicates no deviation • An alignment cost > 0 indicated some sort of deviation • Alignments have been introduces as offline (or post-mortem) techniques, but online (or in vivo) approaches are desirable 2
  5. Motivation for online conformance Not a real xkcd comic. 3

  6. What is online conformance checking? • Given a sequence of

    observations and prescriptive process models pinpoint deviations • Weak notion of trace 4
  7. What is online conformance checking? • Given a sequence of

    observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) 4
  8. What is online conformance checking? • Given a sequence of

    observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) 4
  9. What is online conformance checking? • Given a sequence of

    observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) • No head and no tail is allowed (partial trace with warm start) 4
  10. What is online conformance checking? • Given a sequence of

    observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) • No head and no tail is allowed (partial trace with warm start) • Real-time processing of event streams 4
  11. What is online conformance checking? • Given a sequence of

    observations and prescriptive process models pinpoint deviations • Weak notion of trace • No trace tail is allowed (sequence prefix/partial trace) • No trace head is allowed (sequence suffix/warm start) • No head and no tail is allowed (partial trace with warm start) • Real-time processing of event streams • Technical challenges with event stream real-time processing • Impossible to store the entire log, i.e., approximations needed • Variable system conditions and fluctuating stream rates • Unbounded backtracking is not feasible • One pass on data: scale linearly with respect to number of observations 4
  12. Related work • Mainly two lines of research in online

    conformance checking 5
  13. Related work • Mainly two lines of research in online

    conformance checking 1. Approaches based on prefix-alignment computation • Explanation of prefixes of complete behaviour • High complexity, cannot cope with warm start 5
  14. Related work • Mainly two lines of research in online

    conformance checking 1. Approaches based on prefix-alignment computation • Explanation of prefixes of complete behaviour • High complexity, cannot cope with warm start 2. Approaches where all possible deviations are pre-computed • High offline cost • Important bias (no contextual information) 5
  15. Related work • Mainly two lines of research in online

    conformance checking 1. Approaches based on prefix-alignment computation • Explanation of prefixes of complete behaviour • High complexity, cannot cope with warm start 2. Approaches where all possible deviations are pre-computed • High offline cost • Important bias (no contextual information) • No approach can handle partial traces in warm start scenarios in an effective way 5
  16. Table of contents • Introduction and motivation • Online conformance

    checking • Intuition • General approach for behavioural patterns • Instance to direct following relations • Experimental evaluation • Conclusion 6
  17. Running example • Let’s consider this process model • Offline

    conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7
  18. Running example • Let’s consider this process model • Offline

    conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7
  19. Running example • Let’s consider this process model • Offline

    conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7
  20. Running example • Let’s consider this process model • Offline

    conformance • , 1, , , conformance = 1 • , 1, 2, 1, conformance = 0.8 • , , , conformance = 0.78 • , , conformance = 0.62 Adriansyah, A.: Aligning Observed and Modeled Behavior. Ph.D. thesis, Technische Universiteit Eindhoven (2014) 7
  21. Online conformance checking intuition • Key idea: split the metric

    to capture each concept independently 8
  22. Online conformance checking intuition • Key idea: split the metric

    to capture each concept independently 8
  23. Online conformance checking intuition • Key idea: split the metric

    to capture each concept independently 8
  24. Online conformance checking intuition • Key idea: split the metric

    to capture each concept independently 8
  25. Online conformance checking intuition • Key idea: split the metric

    to capture each concept independently 8
  26. Online conformance checking intuition • Key idea: split the metric

    to capture each concept independently 8
  27. Online conformance checking intuition • Key idea: split the metric

    to capture each concept independently 8
  28. Running example Observation Offline conformance Online measure Conformance Completeness Confidence

    , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9
  29. Running example Observation Offline conformance Online measure Conformance Completeness Confidence

    , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9
  30. Running example Observation Offline conformance Online measure Conformance Completeness Confidence

    , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9
  31. Running example Observation Offline conformance Online measure Conformance Completeness Confidence

    , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9
  32. Running example Observation Offline conformance Online measure Conformance Completeness Confidence

    , 1, , , 1.00 ✔ ✔ High , 1, 2, 1, 0.80 ✔ ✔ Low , , , 0.78 ✔ ✘ High , , 0.62 ✔ ✘ Low 9
  33. Formal definitions • Behavioural pattern • Given set of activities

    and set of control-flow relations ℛ, a behavioural pattern is 1 , 2 where ∈ ℛ and 1 , 2 ∈ • Observation (or observable unit) • Given set of case ids , an observable unit is = , where ∈ and ∈ ℛ × × • Stream of observations • An [infinite] sequence of observable units: 1 , 2 , 3 , … 10
  34. Formal definitions • Behavioural pattern • Given set of activities

    and set of control-flow relations ℛ, a behavioural pattern is 1 , 2 where ∈ ℛ and 1 , 2 ∈ • Observation (or observable unit) • Given set of case ids , an observable unit is = , where ∈ and ∈ ℛ × × • Stream of observations • An [infinite] sequence of observable units: 1 , 2 , 3 , … 10
  35. Pattern instances as observations • A stream of behavioural patterns

    represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) 11
  36. Pattern instances as observations • A stream of behavioural patterns

    represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) • Example with direct following relation 11
  37. Pattern instances as observations • A stream of behavioural patterns

    represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) • Example with direct following relation 11
  38. Pattern instances as observations • A stream of behavioural patterns

    represents high level information • To obtain such stream, pre-processing might be required • For many control-flow relations, approaches already available (e.g., online control-flow mining approaches) • Gives room to extensions (e.g., to non control-flow relations) • Example with direct following relation 11
  39. Intermediate structure • To calculate the online conformance we convert

    the process model into corresponding PMOC (Process Model for Online Conformance) • PMOC is a tiplet = , , • ⊆ ℛ × × is the set of possible behavioural patterns • : → ℕ × ℕ maps each pattern ∈ to the minimum and maximum number of distinct patters expected before • : → ℕ maps each pattern ∈ to the minimum number of distinct patters expected after 12
  40. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 13
  41. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 13
  42. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13
  43. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13
  44. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 13
  45. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 13
  46. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13
  47. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 3 13
  48. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 3 4 13
  49. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 3 4 13
  50. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 13
  51. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 13
  52. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13
  53. Example with ≺ (direct following relation) • Consider the model

    • Relation ≺ • ≺ = (2, 4) • ≺ = 2 1 2 13
  54. PMOC for ≺ and Petri nets • Give a safe

    Petri net of our process 14
  55. PMOC for ≺ and Petri nets • Give a safe

    Petri net of our process • Construct finite representation with complete prefix unfolding • Unfolding with 2 markings equivalent if: (i) same places; (ii) same relations between activities to reach them; (iii) same set of lastly activities executed 14
  56. PMOC for ≺ and Petri nets • Give a safe

    Petri net of our process • Construct finite representation with complete prefix unfolding • Unfolding with 2 markings equivalent if: (i) same places; (ii) same relations between activities to reach them; (iii) same set of lastly activities executed 14
  57. PMOC for ≺ and Petri nets • Give a safe

    Petri net of our process • Construct finite representation with complete prefix unfolding • Unfolding with 2 markings equivalent if: (i) same places; (ii) same relations between activities to reach them; (iii) same set of lastly activities executed 14
  58. PMOC for ≺ and Petri nets (cont.) To construct =

    , , With the reachability graph of the unfolded net we can extract 15
  59. PMOC for ≺ and Petri nets (cont.) To construct =

    , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets
  60. PMOC for ≺ and Petri nets (cont.) To construct =

    , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets
  61. PMOC for ≺ and Petri nets (cont.) To construct =

    , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets
  62. PMOC for ≺ and Petri nets (cont.) To construct =

    , , With the reachability graph of the unfolded net we can extract • By reversing the original net*, and computing the same unfolding we can extract component 15 * We need to restrict to sound workflow nets
  63. Online conformance checking algorithm 16

  64. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16
  65. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16
  66. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16
  67. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16
  68. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16
  69. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16
  70. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end 16
  71. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end Approximations in the computation 16
  72. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end Constant time required per event Approximations in the computation 16
  73. Online conformance checking algorithm Input: PMOC = , , and

    the stream of observations 1: define obs ⊳ map from case ids to sets of allowed patterns 2: define inc ⊳ map from case ids to integers (wrong observations) 3: repeat forever: read (, ) from 4: if ∈ then add to obs else increment inc 5: conformance of ← obs obs + inc ⊳ obs and inc behaves as expected when no key 6: if ∈ then 7: completeness of ← ቐ 1 if ≤ obs ≤ min 1, obs + 1 otherwise 8: confidence of ← 1 − () max ′∈B ′ 9: end 10: notify new values and clean-up operations on obs and inc 11: end Constant time required per event General to any behavioural pattern Approximations in the computation 16
  74. Evaluation / Time performance • System implemented in ProM (StreamConformance

    package, no optimization, plain Java structures) 17
  75. Evaluation / Time performance • System implemented in ProM (StreamConformance

    package, no optimization, plain Java structures) • Stress test • Random BPMN model with 64 activities and 26 gateways • 2 million events, standard desktop machine 17
  76. Evaluation / Time performance • System implemented in ProM (StreamConformance

    package, no optimization, plain Java structures) • Stress test • Random BPMN model with 64 activities and 26 gateways • 2 million events, standard desktop machine 0 50 100 150 200 250 0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 Space (MB) Time (ms) Events Average processing time per event Total space used (MB) 17
  77. Evaluation / Time performance • System implemented in ProM (StreamConformance

    package, no optimization, plain Java structures) • Stress test • Random BPMN model with 64 activities and 26 gateways • 2 million events, standard desktop machine 0 50 100 150 200 250 0 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 2,000,000 Space (MB) Time (ms) Events Average processing time per event Total space used (MB) 17
  78. Evaluation / Comparison with other metric • Comparison of proposed

    metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18
  79. Evaluation / Comparison with other metric • Comparison of proposed

    metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18
  80. Evaluation / Comparison with other metric • Comparison of proposed

    metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18
  81. Evaluation / Comparison with other metric • Comparison of proposed

    metric and [2] (optimal cost) • 12 random processes, incremental noise levels (from 0.1 to 0.5) for each log containing 1000 traces (in total 2.9M events), DOI: 10.5281/zenodo.1194057 [2] van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.P.: Online Conformance Checking: Relating Event Streams to Process Models using Prefix-Alignments. Int J Data Sci Anal (Oct 2017) 18
  82. Evaluation / Comparison with other metric • Combinations where at

    least one of the techniques identifies a deviation 19
  83. Evaluation / Comparison with other metric • Combinations where at

    least one of the techniques identifies a deviation 19
  84. Evaluation / Comparison with other metric • Combinations where at

    least one of the techniques identifies a deviation • Conclusions • When no deviations, the techniques agree • Deviations, though, are captured in different ways due to different representation biases 19
  85. Evaluation / Real event log • We tested our approach

    on the “road traffic fines log” • 316868 events / 83614 cases, DOI: 10.1007/s00607-015-0441-1 • Total time: 44967 ms; 0.141 ms/event 20
  86. Evaluation / Real event log • We tested our approach

    on the “road traffic fines log” • 316868 events / 83614 cases, DOI: 10.1007/s00607-015-0441-1 • Total time: 44967 ms; 0.141 ms/event 0 0.25 0.5 0.75 1 Conformance 0 0.25 0.5 0.75 1 Completeness 0 0.25 0.5 0.75 1 0 50000 100000 150000 200000 250000 300000 Confidence Events 20
  87. Evaluation / Real event log • We tested our approach

    on the “road traffic fines log” • 316868 events / 83614 cases, DOI: 10.1007/s00607-015-0441-1 • Total time: 44967 ms; 0.141 ms/event 0 0.25 0.5 0.75 1 Conformance 0 0.25 0.5 0.75 1 Completeness 0 0.25 0.5 0.75 1 0 50000 100000 150000 200000 250000 300000 Confidence Events Observations • 93.5% events conformance 1 99.6% events conformance ≥ 0.5 • 99.8% events completeness 1: most executions from beginning • 99.4% events confidence 1: the model which allows immediate termination after first activity 20
  88. Conclusion and future work • Approach to compute online conformance

    starting from stream of behavioural patterns 21
  89. Conclusion and future work • Approach to compute online conformance

    starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after 21
  90. Conclusion and future work • Approach to compute online conformance

    starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after • Online measure split into • Actual conformance: ratio of correct relations being observed over all obs • Completeness: is the instance starting from the beginning (warm start)? • Confidence: can the current conformance change in the future (partial traces)? 21
  91. Conclusion and future work • Approach to compute online conformance

    starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after • Online measure split into • Actual conformance: ratio of correct relations being observed over all obs • Completeness: is the instance starting from the beginning (warm start)? • Confidence: can the current conformance change in the future (partial traces)? • An approach based on unfolding of Petri nets has been presented with direct following relation 21
  92. Conclusion and future work • Approach to compute online conformance

    starting from stream of behavioural patterns • The approach represents a model as • Set of expected relations • For each relation, how many of them (min/max) expected before/after • Online measure split into • Actual conformance: ratio of correct relations being observed over all obs • Completeness: is the instance starting from the beginning (warm start)? • Confidence: can the current conformance change in the future (partial traces)? • An approach based on unfolding of Petri nets has been presented with direct following relation • Future work includes the extension to new implementation (i.e., patterns/perspectives) and optimization towards optimal costs 21