Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach

0b6203b08e1c063c97bb25abfc3842ec?s=47 Andrea Burattin
September 19, 2016

Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach

Research on the process of process modeling (PPM) studies how process models are created. It typically uses the logs of the interactions with the modeling tool to assess the modeler's behavior. In this paper we suggest to introduce an additional stream of data (i.e., eye tracking) to improve the analysis of the PPM.

We show that, by exploiting this additional source of information, we can refine the detection of comprehension phases (introducing activities such as "semantic validation" or "problem understanding") as well as provide more exploratory visualizations (e.g., combined modeling phase diagram, heat maps, fixations distributions) both static and dynamic (i.e., movies with the evolution of the model and eye tracking data on top).

More info: https://andrea.burattin.net/publications/2016-taproviz

0b6203b08e1c063c97bb25abfc3842ec?s=128

Andrea Burattin

September 19, 2016
Tweet

More Decks by Andrea Burattin

Other Decks in Research

Transcript

  1. Eye Tracking Meets the Process of Process Modeling: a Visual

    Analytic Approach Andrea Burattin 1 Michael Kaiser 1 Manuel Neurauter 1 Barbara Weber 1,2 1 University of Innsbruck, Austria 2 Technical University of Denmark This research is supported by Austrian Science Fund (FWF): P26609.
  2. Table of Contents • Background • Wrap up on process

    of process modeling • Eye tracking technology • Visualization of the eye tracking + process modeling • Fixation distribution • Fixation heat maps • Combined Modeling Phase Diagram • Animations • Demonstration • Conclusion and future work Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 2
  3. Process of Process Modeling • Our focus is in analyzing

    the process that leads to process models • A typical modeling session is characterized by the repetition of three phases • Comprehension • Modeling • Reconciliation • Cheetah Experimental Platform • Tool to model a process model • Records all model interactions • Allows replaying the process modeling step-by-step Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 3 J. Pinggera, S. Zugal and B. Weber: Investigating the Process of Process Modeling with Cheetah Experimental Platform. In: Proc. ER-POIS ’10, pp. 13–18, 2010 0 13 26 39 52 65 0 400 800 1200 Number of Elements Time [s] COMPREHENSION MODELING RECONCILIATION … Cheetah Experimental Platform (CEP) Traditional Research: Resulting Model
  4. Modeling Phases – Comprehension • Comprehension phases are characterized by

    no interaction with the modeling tool • Comprehension = not (Modeling or Reconciliation) • Very coarse-grained classification • Is there any activity going on? • Can we detect sub-activities? • That’s all we can do with the currently available data Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 4
  5. Modeling Phases – Comprehension • Comprehension phases are characterized by

    no interaction with the modeling tool • Comprehension = not (Modeling or Reconciliation) • Very coarse-grained classification • Is there any activity going on? • Can we detect sub-activities? • That’s all we can do with the currently available data Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 4 Let’s focus on more than just interactions with the modeling tool!
  6. Eye Tracking • Idea: where a person is looking at

    • Useful to know which information is processed by a subject • Eye trackers collect data regarding • Fixations: ≥ 200 ms on a stimulus • Saccades: 30 − 80 ms shifting attention from one point to another • Typically, Areas Of Interest (AOI) are identified within the target • Abstraction: fixations on AOIs and saccades from one AOI to another Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 5
  7. Eye Tracking • Idea: where a person is looking at

    • Useful to know which information is processed by a subject • Eye trackers collect data regarding • Fixations: ≥ 200 ms on a stimulus • Saccades: 30 − 80 ms shifting attention from one point to another • Typically, Areas Of Interest (AOI) are identified within the target • Abstraction: fixations on AOIs and saccades from one AOI to another Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 5
  8. Areas Of Interest of CEP Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 6
  9. Areas Of Interest of CEP Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 6 Textual description
  10. Areas Of Interest of CEP Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 6 Textual description Modeling area
  11. Areas Of Interest of CEP Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 6 Textual description Modeling area Toolbox
  12. Analysis with Additional Data Streams • Experimental environment • Steps

    of our analysis • Collect all CEP interactions • Compute the Modeling Phase Diagram • Collect all eye tracking data • Refer all eye tracking data to corresponding AOIs • As a result, we build different representations Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 7 Eye tracker Stimulus video stream CEP Database
  13. Visualization 1: Fixation Distributions Eye Tracking Meets the Process of

    Process Modeling: a Visual Analytic Approach 8
  14. Visualization 1: Fixation Distributions Eye Tracking Meets the Process of

    Process Modeling: a Visual Analytic Approach 8
  15. Visualization 2: Heat Map Eye Tracking Meets the Process of

    Process Modeling: a Visual Analytic Approach 9
  16. Visualization 3: Combined Modeling Phase Diagram Eye Tracking Meets the

    Process of Process Modeling: a Visual Analytic Approach 10 No. Elements Time Modeling phase Comprehension phases Reconciliation phases Fixation Points Switches Create node Create edge Move node Delete edge Model Text Toolbox Time fragment 1 Time fragment 2
  17. Time Dimension • Time dimension not always considered • Fixation

    distribution: all fixations are reported • Heat map: total time spent • Animations allow us to incorporate the time dimension • Importance of time dimension: plot the evolution of the model as well! • Two approaches possible • Sliding window • Incremental Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 11
  18. Time Dimension • Time dimension not always considered • Fixation

    distribution: all fixations are reported • Heat map: total time spent • Animations allow us to incorporate the time dimension • Importance of time dimension: plot the evolution of the model as well! • Two approaches possible • Sliding window • Incremental Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 11 Time Frame 1 Frame 2 Frame 3 Time Frame 1 Frame 2 Frame 3 Sliding window Incremental approach
  19. Video Representations of Textual description Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 12
  20. Video Representations of Modeling Area Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 13
  21. Demonstration • Data collected during a modeling experiment performed in

    2015 • 116 subjects, all novices • We analyzed, in particular, the Combined Modeling Phase Diagram • We identified possible patterns occurring during reconciliation phases • Problem understanding • Method finding • Semantic validation • Syntactic validation • But, we can improve the accuracy of the MDP as well! Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 14
  22. Problem Understanding and Method Finding Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 15
  23. Problem Understanding and Method Finding Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 15 Problem understanding
  24. Problem Understanding and Method Finding Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 15 Problem understanding Method finding
  25. Semantic and Syntactic Validation Eye Tracking Meets the Process of

    Process Modeling: a Visual Analytic Approach 16
  26. Semantic and Syntactic Validation Eye Tracking Meets the Process of

    Process Modeling: a Visual Analytic Approach 16 Semantic validation
  27. Semantic and Syntactic Validation Eye Tracking Meets the Process of

    Process Modeling: a Visual Analytic Approach 16 Semantic validation Syntactic validation
  28. Example of improvement of the MPD Eye Tracking Meets the

    Process of Process Modeling: a Visual Analytic Approach 17
  29. Example of improvement of the MPD Eye Tracking Meets the

    Process of Process Modeling: a Visual Analytic Approach 17 Semantic validation
  30. Heat Maps per Modeling Phase Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 18
  31. Heat Maps per Modeling Phase Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 18 Long back loop Exit point of the long back loop Entry point of the long back loop Short loop
  32. Heat Maps per Modeling Phase Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 18 Long back loop Exit point of the long back loop Entry point of the long back loop Short loop Short loop Short loop
  33. Heat Maps per Modeling Phase Eye Tracking Meets the Process

    of Process Modeling: a Visual Analytic Approach 18 Long back loop Exit point of the long back loop Entry point of the long back loop Short loop Short loop Short loop
  34. Conclusions and Future work • We introduced the eye tracking

    into the process of process modeling • We suggested several visualizations useful to • Improve the quality of data • Detect additional activities • Syntactic validation • Semantic validation • Problem understanding • Method finding • Refine existing phases of the MPD • Future work • Eye tracking data improvement (e.g., offset compensation) • Validation of the refined activities (with post hoc interviews) • Automatic identification of refined activities • Identification of additional activities for other phase types as well Eye Tracking Meets the Process of Process Modeling: a Visual Analytic Approach 19