Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach

Research on the process of process modeling (PPM) studies how process models are created. It typically uses the logs of the interactions with the modeling tool to assess the modeler's behavior. In this paper we suggest to introduce an additional stream of data (i.e., eye tracking) to improve the analysis of the PPM.

We show that, by exploiting this additional source of information, we can refine the detection of comprehension phases (introducing activities such as "semantic validation" or "problem understanding") as well as provide more exploratory visualizations (e.g., combined modeling phase diagram, heat maps, fixations distributions) both static and dynamic (i.e., movies with the evolution of the model and eye tracking data on top).

More info: https://andrea.burattin.net/publications/2016-taproviz

Transcript

1. Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach Andrea Burattin 1 Michael Kaiser 1 Manuel Neurauter 1 Barbara Weber 1,2 1 University of Innsbruck, Austria 2 Technical University of Denmark This research is supported by Austrian Science Fund (FWF): P26609.
2. Table of Contents • Background • Wrap up on process of process modeling • Eye tracking technology • Visualization of the eye tracking + process modeling • Fixation distribution • Fixation heat maps • Combined Modeling Phase Diagram • Animations • Demonstration • Conclusion and future work Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 2
3. Process of Process Modeling • Our focus is in analyzing the process that leads to process models • A typical modeling session is characterized by the repetition of three phases • Comprehension • Modeling • Reconciliation • Cheetah Experimental Platform • Tool to model a process model • Records all model interactions • Allows replaying the process modeling step-by-step Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 3 J. Pinggera, S. Zugal and B. Weber: Investigating the Process of Process Modeling with Cheetah Experimental Platform. In: Proc. ER-POIS ’10, pp. 13–18, 2010 0 13 26 39 52 65 0 400 800 1200 Number of Elements Time [s] COMPREHENSION MODELING RECONCILIATION … Cheetah Experimental Platform (CEP) Traditional Research: Resulting Model
4. Modeling Phases – Comprehension • Comprehension phases are characterized by no interaction with the modeling tool • Comprehension = not (Modeling or Reconciliation) • Very coarse-grained classification • Is there any activity going on? • Can we detect sub-activities? • That’s all we can do with the currently available data Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 4
5. Modeling Phases – Comprehension • Comprehension phases are characterized by no interaction with the modeling tool • Comprehension = not (Modeling or Reconciliation) • Very coarse-grained classification • Is there any activity going on? • Can we detect sub-activities? • That’s all we can do with the currently available data Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 4 Let’s focus on more than just interactions with the modeling tool!
6. Eye Tracking • Idea: where a person is looking at • Useful to know which information is processed by a subject • Eye trackers collect data regarding • Fixations: ≥ 200 ms on a stimulus • Saccades: 30 − 80 ms shifting attention from one point to another • Typically, Areas Of Interest (AOI) are identified within the target • Abstraction: fixations on AOIs and saccades from one AOI to another Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 5
7. Eye Tracking • Idea: where a person is looking at • Useful to know which information is processed by a subject • Eye trackers collect data regarding • Fixations: ≥ 200 ms on a stimulus • Saccades: 30 − 80 ms shifting attention from one point to another • Typically, Areas Of Interest (AOI) are identified within the target • Abstraction: fixations on AOIs and saccades from one AOI to another Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 5
8. Areas Of Interest of CEP Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 6
9. Areas Of Interest of CEP Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 6 Textual description
10. Areas Of Interest of CEP Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 6 Textual description Modeling area
11. Areas Of Interest of CEP Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 6 Textual description Modeling area Toolbox
12. Analysis with Additional Data Streams • Experimental environment • Steps of our analysis • Collect all CEP interactions • Compute the Modeling Phase Diagram • Collect all eye tracking data • Refer all eye tracking data to corresponding AOIs • As a result, we build different representations Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 7 Eye tracker Stimulus video stream CEP Database
13. Visualization 1: Fixation Distributions Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 8
14. Visualization 1: Fixation Distributions Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 8
15. Visualization 2: Heat Map Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 9
16. Visualization 3: Combined Modeling Phase Diagram Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 10 No. Elements Time Modeling phase Comprehension phases Reconciliation phases Fixation Points Switches Create node Create edge Move node Delete edge Model Text Toolbox Time fragment 1 Time fragment 2
17. Time Dimension • Time dimension not always considered • Fixation distribution: all fixations are reported • Heat map: total time spent • Animations allow us to incorporate the time dimension • Importance of time dimension: plot the evolution of the model as well! • Two approaches possible • Sliding window • Incremental Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 11
18. Time Dimension • Time dimension not always considered • Fixation distribution: all fixations are reported • Heat map: total time spent • Animations allow us to incorporate the time dimension • Importance of time dimension: plot the evolution of the model as well! • Two approaches possible • Sliding window • Incremental Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 11 Time Frame 1 Frame 2 Frame 3 Time Frame 1 Frame 2 Frame 3 Sliding window Incremental approach
19. Video Representations of Textual description Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 12
20. Video Representations of Modeling Area Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 13
21. Demonstration • Data collected during a modeling experiment performed in 2015 • 116 subjects, all novices • We analyzed, in particular, the Combined Modeling Phase Diagram • We identified possible patterns occurring during reconciliation phases • Problem understanding • Method finding • Semantic validation • Syntactic validation • But, we can improve the accuracy of the MDP as well! Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 14
22. Problem Understanding and Method Finding Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 15
23. Problem Understanding and Method Finding Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 15 Problem understanding
24. Problem Understanding and Method Finding Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 15 Problem understanding Method finding
25. Semantic and Syntactic Validation Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 16
26. Semantic and Syntactic Validation Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 16 Semantic validation
27. Semantic and Syntactic Validation Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 16 Semantic validation Syntactic validation
28. Example of improvement of the MPD Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 17
29. Example of improvement of the MPD Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 17 Semantic validation
30. Heat Maps per Modeling Phase Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 18
31. Heat Maps per Modeling Phase Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 18 Long back loop Exit point of the long back loop Entry point of the long back loop Short loop
32. Heat Maps per Modeling Phase Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 18 Long back loop Exit point of the long back loop Entry point of the long back loop Short loop Short loop Short loop
33. Heat Maps per Modeling Phase Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 18 Long back loop Exit point of the long back loop Entry point of the long back loop Short loop Short loop Short loop
34. Conclusions and Future work • We introduced the eye tracking into the process of process modeling • We suggested several visualizations useful to • Improve the quality of data • Detect additional activities • Syntactic validation • Semantic validation • Problem understanding • Method finding • Refine existing phases of the MPD • Future work • Eye tracking data improvement (e.g., offset compensation) • Validation of the refined activities (with post hoc interviews) • Automatic identification of refined activities • Identification of additional activities for other phase types as well Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 19