Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Prototyping Haptic Data Visualizations

Prototyping Haptic Data Visualizations

Presentation of the HITPROTO toolkit (Visualization and Medical Graphics Group Seminar, Bangor)

Panagiotis D. Ritsos

October 02, 2013
Tweet

More Decks by Panagiotis D. Ritsos

Other Decks in Research

Transcript

  1. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
    Panagiotis D. Ritsos1  Sabrina A. Panëels2  Peter J. Rodgers3  Jonathan C. Roberts1
    [1] School of Computer Science, Bangor University, UK - {p.ritsos, j.c.roberts}@bangor.ac.uk)
    [2] CEA, LIST, France – [email protected]
    [3] School of Computer Science, University of Kent, UK – [email protected]
    VMG OCTOBER 2013
    Prototyping Haptic Data Visualizations

    View Slide

  2. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     Introduction to Haptic Data Visualization (HDV).
     Related work using Tactile Representations
     Overview of the concept of prototyping interactions
     In-depth description of the HITPROTO toolkit
     Examples of HDVs produced using the HITPROTO toolkit
     User evaluation of the HITPROTO toolkit
     Preliminary investigation with users with vision impairments
     Lessons learned
    Presentation Outline

    View Slide

  3. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     S.A. Paneels, P.D. Ritsos, P.J.Rodgers, and J.C.Roberts, Prototyping
    3D haptic data visualizations, Computers & Graphics, Volume 37, Issue
    3, May 2013, pp. 179-192
     P.D. Ritsos, S.A. Paneels, P.J.Rodgers, and J.C. Roberts, Towards a
    Formalized Process for Creating Haptic Data Visualization, Visweek
    2013, Atlanta, Georgia, USA
    Related Publications

    View Slide

  4. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     The use of haptic devices is becoming widespread, particularly with their
    growth in the home-games market.
     Likewise, the number of haptic devices and APIs is increasing.
     However, it is still difficult to program and develop haptic applications.
     One way to tackle this is by providing prototyping tools or frameworks.
     This paper presents a visual prototyping tool for interactions for…
     … haptic data visualization (HDV), also called haptification - the use of
    haptic devices to represent and realize information and data.
    Introduction

    View Slide

  5. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     In software engineering (SE) the term prototyping refers to the rapid
    development of a software solution and has, typically, two forms:
     evolutionary prototyping, where the user refines the prototype in an
    ongoing process before it turns into the final system
     throw-away prototyping where subsequent systems are created and
    then discarded in turn before the final development and delivery of a
    software produce.
     Our approach fuses both forms through the rapid creation of haptic
    interactions…
     …and the generation of reusable code that can be built-upon by
    developers.
    Prototyping

    View Slide

  6. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     The haptic visualization display can be considered to consist of three parts:
     First a model is created that contains the haptic information that is going to
    be perceived.
     The generation of the model closely follows the visualization process, where
    the data is enhanced and mapped into an abstract/haptic model.
     Second, haptic rendering is used to compute the forces required to realize
    the model.
    The Haptic Data Visualization
     Third, an electro-mechanical
    device is used to exert the forces
    that the user perceives.

    View Slide

  7. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     Braille tactile writing system (paper
    embossing)
     microcapsule paper, thermo-form
    and vacuum-form
     Wikkistix, which are strings doped
    with wax to make the string keep
    its form
    Tactile Graphics and Static Representations

    View Slide

  8. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     HITPROTO attempts to ameliorate the technical complexities and
    provide an interface that is closer to a natural language (e.g., “Wait for a
    button press, then add and start guidance”).
     We use a modular approach in HITPROTO, where users drag-and-
    drop components onto a canvas and connect them together to provide
    the logic for the haptic visualization.
     Parameters of the blocks can be set to describe specific behaviours.
     The arrangement of the blocks describes the semantic structure of the
    haptic interaction
    The HITPROTO Toolkit - Design

    View Slide

  9. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     There are four main parts to the process:
    1. The user makes the HDV scenario by connecting modular blocks on the canvas,
    2. this information is saved in the .hit XML file,
    3. HITPROTO generates a H3D Python file,
    4. H3DAPI is used to execute the haptic scene
    The HITPROTO Toolkit – The process

    View Slide

  10. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     HITPROTO uses H3DAPI (h3dapi.org), is implemented in C++ with
    WxWidgets and combines X3D, C++ and Python
     Current version tested with the Geomagic Touch and Touch X (formerly
    Sensable Phantom Omni and Desktop, respectively), but in theory can be
    used with devices supported by H3DAPI
    The HITPROTO Toolkit – The application
     GUI is like Lego NXT
    Mindstorms visual
    programming language

    View Slide

  11. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
    HITPROTO Blocks
    • Action blocks - describe the addition, removal, creation and modification of
    haptic and guidance effects
    Stop – compulsory block that delimits the end of the ‘interaction scenario’
    Guidance Add – creates a guidance instance. Includes a spring to attach to the device and an
    anchor to visualize the spring and parameters such as path and speed/duration.
    Haptic Effect – creates a chosen haptic effect. The available haptic effects are: SpringEffect,
    Magnetic Line(s) and PositionFunctionEffect (model-based).
    • Flow blocks that control the flow of the data by listening to events.
    Wait For – enables the interruption of a sequence of actions until a chosen event happens
    such as a haptic device/mouse button or a keyboard key being pressed/released, an
    elapsed time or the activation of a spring
    Switch and Switch End – Checks if a condition is satisfied or not before executing a set of
    actions contained between the two blocks

    View Slide

  12. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
    1
    2
    3
    4
    5
    6
    9
    10
    11
    12
    13
    14
    Block Diagram Intermediate Form

    View Slide

  13. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
    • We parse the .hit file sequentially, and the parameters stored as XML
    attributes are passed to the respective Python code components.
    • However, code generation is not a linear process. This is because the
    labels and values may be defined at a lower position in the .hit file
    • The Python code generated provides a runnable implementation and
    can be executed directly from HITPROTO’s interface, or the Python file
    run separately.
    • For the mentor/blind-user situation this may be enough, and it affords
    quick development and deployment.
    • However the code, being standard H3DAPI Python, can be reused and
    extended for the needs of another application
    Block Diagram Parsing and Python Output

    View Slide

  14. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
    HITPROTO Examples
    Irises data set – force model
    UK Public Spending since 1963 – magnetic lines + X3D visual elements

    View Slide

  15. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     HITPROTO was evaluated by postgraduates that fitted the profile of
    potential support workers for blind users
     We used a ‘convenience sample’ of nine participants: ages 18-65, 6
    female and male, field study anything but Computer Science
     Participants completed four tasks of gradually increasing difficulty and
    success rates, time for task completion was recorded.
     A post-experiment questionnaire was completed, consisting of the
    System Usability Scale (SUS) and open ended questions.
    Usability Evaluation - I

    View Slide

  16. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     The average SUS score, rating overall usability, was 67%. Individual
    scores from participants ranged between 50% and 92.5%, except for one
    at 17.5%.
     The results indicate that earlier tasks were completed successfully
    with no or little help, whereas help was required for latter ones.
     This behaviour was expected because the latter tasks were designed to
    be more challenging than earlier ones.
     Overall 88.9% of the attempts at the tasks resulted in a working
    interaction, with or without help, while only 8.3% of them resulted in
    failures, despite the help given.
    Usability Evaluation - II

    View Slide

  17. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     We performed a talk-aloud session of the Guided
    Tour and the magnetic line-graphs
     They were very positive over haptic visualization
    techniques and saw much potential in the tool
     They discussed how they explored information in
    general, and explained that they had used static
    examples such as thermoformed images.
    Using HITPROTO with Blind Students
     Emphasised the fact that many tasks take them longer to perform in comparison to
    a sighted user.
     Saw the benefit of the Guided Tour example that led them to points of interest.
     We discussed how audio annotation could be incorporated into the
    representations

    View Slide

  18. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     Visual dataflow programming assists novice users
     Medium granularity required
     Block Appearance improves User Experience
     Alternative notation (block diagram and Python code) assists expert users
     Reuse of models/patterns towards Learning by Example
     Reliance on standards (e.g., X3D) and popular APIs (e.g., H3DAPI) allows
    reuse and extensibility
     Usage patterns
     Error Checking
     Default parameter values
    Lessons Learnt – Prototyping tools

    View Slide

  19. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     Objective mapping - clear mapping from the data to the haptic
    representation.
     Value - enable the user to understand the underlying data quantity.
    Values may be represented through other modalities (such as sound).
     Reproducible - identical if the same data is loaded in another session
     Different data can be loaded to be represented by HDV.
     Systematic set of interactions that allow exploring in 3D
     Provide feature guidance or touring mechanisms
     Exploration is encouraged
    Lessons Learnt – HDVs

    View Slide

  20. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
     There are many areas that can be improved in HITPROTO.
     new block diagram editor with an error-checking parser,
     new custom block creator,
     ‘hinting’ mechanisms to the dataflow block diagram editor,
     use of ‘Patterns’ for users to build typical configurations.
    Future Work & Conclusions

    View Slide

  21. School of Computer Science
    Ysgol Gwyddorau Cyfrifiadurol
    Thank You!

    View Slide