Prototyping Haptic Data Visualizations

School of Computer Science
Ysgol Gwyddorau Cyfrifiadurol
Panagiotis D. Ritsos1  Sabrina A. Panëels2  Peter J. Rodgers3  Jonathan C. Roberts1
[1] School of Computer Science, Bangor University, UK - {p.ritsos, j.c.roberts}@bangor.ac.uk)
[2] CEA, LIST, France – [email protected]
[3] School of Computer Science, University of Kent, UK – [email protected]
VMG OCTOBER 2013
Prototyping Haptic Data Visualizations

View Slide

 Introduction to Haptic Data Visualization (HDV).
 Related work using Tactile Representations
 Overview of the concept of prototyping interactions
 In-depth description of the HITPROTO toolkit
 Examples of HDVs produced using the HITPROTO toolkit
 User evaluation of the HITPROTO toolkit
 Preliminary investigation with users with vision impairments
 Lessons learned
Presentation Outline

View Slide

 S.A. Paneels, P.D. Ritsos, P.J.Rodgers, and J.C.Roberts, Prototyping
3D haptic data visualizations, Computers & Graphics, Volume 37, Issue
3, May 2013, pp. 179-192
 P.D. Ritsos, S.A. Paneels, P.J.Rodgers, and J.C. Roberts, Towards a
Formalized Process for Creating Haptic Data Visualization, Visweek
2013, Atlanta, Georgia, USA
Related Publications

View Slide

 The use of haptic devices is becoming widespread, particularly with their
growth in the home-games market.
 Likewise, the number of haptic devices and APIs is increasing.
 However, it is still difficult to program and develop haptic applications.
 One way to tackle this is by providing prototyping tools or frameworks.
 This paper presents a visual prototyping tool for interactions for…
 … haptic data visualization (HDV), also called haptification - the use of
haptic devices to represent and realize information and data.
Introduction

View Slide

 In software engineering (SE) the term prototyping refers to the rapid
development of a software solution and has, typically, two forms:
 evolutionary prototyping, where the user refines the prototype in an
ongoing process before it turns into the final system
 throw-away prototyping where subsequent systems are created and
then discarded in turn before the final development and delivery of a
software produce.
 Our approach fuses both forms through the rapid creation of haptic
interactions…
 …and the generation of reusable code that can be built-upon by
developers.
Prototyping

View Slide

 The haptic visualization display can be considered to consist of three parts:
 First a model is created that contains the haptic information that is going to
be perceived.
 The generation of the model closely follows the visualization process, where
the data is enhanced and mapped into an abstract/haptic model.
 Second, haptic rendering is used to compute the forces required to realize
the model.
The Haptic Data Visualization
 Third, an electro-mechanical
device is used to exert the forces
that the user perceives.

View Slide

 Braille tactile writing system (paper
embossing)
 microcapsule paper, thermo-form
and vacuum-form
 Wikkistix, which are strings doped
with wax to make the string keep
its form
Tactile Graphics and Static Representations

View Slide

 HITPROTO attempts to ameliorate the technical complexities and
provide an interface that is closer to a natural language (e.g., “Wait for a
button press, then add and start guidance”).
 We use a modular approach in HITPROTO, where users drag-and-
drop components onto a canvas and connect them together to provide
the logic for the haptic visualization.
 Parameters of the blocks can be set to describe speciﬁc behaviours.
 The arrangement of the blocks describes the semantic structure of the
haptic interaction
The HITPROTO Toolkit - Design

View Slide

 There are four main parts to the process:
1. The user makes the HDV scenario by connecting modular blocks on the canvas,
2. this information is saved in the .hit XML ﬁle,
3. HITPROTO generates a H3D Python ﬁle,
4. H3DAPI is used to execute the haptic scene
The HITPROTO Toolkit – The process

View Slide

 HITPROTO uses H3DAPI (h3dapi.org), is implemented in C++ with
WxWidgets and combines X3D, C++ and Python
 Current version tested with the Geomagic Touch and Touch X (formerly
Sensable Phantom Omni and Desktop, respectively), but in theory can be
used with devices supported by H3DAPI
The HITPROTO Toolkit – The application
 GUI is like Lego NXT
Mindstorms visual
programming language

View Slide

HITPROTO Blocks
• Action blocks - describe the addition, removal, creation and modification of
haptic and guidance effects
Stop – compulsory block that delimits the end of the ‘interaction scenario’
Guidance Add – creates a guidance instance. Includes a spring to attach to the device and an
anchor to visualize the spring and parameters such as path and speed/duration.
Haptic Effect – creates a chosen haptic effect. The available haptic effects are: SpringEffect,
Magnetic Line(s) and PositionFunctionEffect (model-based).
• Flow blocks that control the flow of the data by listening to events.
Wait For – enables the interruption of a sequence of actions until a chosen event happens
such as a haptic device/mouse button or a keyboard key being pressed/released, an
elapsed time or the activation of a spring
Switch and Switch End – Checks if a condition is satisfied or not before executing a set of
actions contained between the two blocks

View Slide

1
2
3
4
5
6
9
10
11
12
13
14
Block Diagram Intermediate Form

View Slide

• We parse the .hit file sequentially, and the parameters stored as XML
attributes are passed to the respective Python code components.
• However, code generation is not a linear process. This is because the
labels and values may be defined at a lower position in the .hit file
• The Python code generated provides a runnable implementation and
can be executed directly from HITPROTO’s interface, or the Python file
run separately.
• For the mentor/blind-user situation this may be enough, and it affords
quick development and deployment.
• However the code, being standard H3DAPI Python, can be reused and
extended for the needs of another application
Block Diagram Parsing and Python Output

View Slide

HITPROTO Examples
Irises data set – force model
UK Public Spending since 1963 – magnetic lines + X3D visual elements

View Slide

 HITPROTO was evaluated by postgraduates that ﬁtted the proﬁle of
potential support workers for blind users
 We used a ‘convenience sample’ of nine participants: ages 18-65, 6
female and male, field study anything but Computer Science
 Participants completed four tasks of gradually increasing difficulty and
success rates, time for task completion was recorded.
 A post-experiment questionnaire was completed, consisting of the
System Usability Scale (SUS) and open ended questions.
Usability Evaluation - I

View Slide

 The average SUS score, rating overall usability, was 67%. Individual
scores from participants ranged between 50% and 92.5%, except for one
at 17.5%.
 The results indicate that earlier tasks were completed successfully
with no or little help, whereas help was required for latter ones.
 This behaviour was expected because the latter tasks were designed to
be more challenging than earlier ones.
 Overall 88.9% of the attempts at the tasks resulted in a working
interaction, with or without help, while only 8.3% of them resulted in
failures, despite the help given.
Usability Evaluation - II

View Slide

 We performed a talk-aloud session of the Guided
Tour and the magnetic line-graphs
 They were very positive over haptic visualization
techniques and saw much potential in the tool
 They discussed how they explored information in
general, and explained that they had used static
examples such as thermoformed images.
Using HITPROTO with Blind Students
 Emphasised the fact that many tasks take them longer to perform in comparison to
a sighted user.
 Saw the beneﬁt of the Guided Tour example that led them to points of interest.
 We discussed how audio annotation could be incorporated into the
representations

View Slide

 Visual dataﬂow programming assists novice users
 Medium granularity required
 Block Appearance improves User Experience
 Alternative notation (block diagram and Python code) assists expert users
 Reuse of models/patterns towards Learning by Example
 Reliance on standards (e.g., X3D) and popular APIs (e.g., H3DAPI) allows
reuse and extensibility
 Usage patterns
 Error Checking
 Default parameter values
Lessons Learnt – Prototyping tools

View Slide

 Objective mapping - clear mapping from the data to the haptic
representation.
 Value - enable the user to understand the underlying data quantity.
Values may be represented through other modalities (such as sound).
 Reproducible - identical if the same data is loaded in another session
 Different data can be loaded to be represented by HDV.
 Systematic set of interactions that allow exploring in 3D
 Provide feature guidance or touring mechanisms
 Exploration is encouraged
Lessons Learnt – HDVs

View Slide

 There are many areas that can be improved in HITPROTO.
 new block diagram editor with an error-checking parser,
 new custom block creator,
 ‘hinting’ mechanisms to the dataﬂow block diagram editor,
 use of ‘Patterns’ for users to build typical configurations.
Future Work & Conclusions

View Slide

Thank You!

View Slide

Prototyping Haptic Data Visualizations

Prototyping Haptic Data Visualizations

Panagiotis D. Ritsos

More Decks by Panagiotis D. Ritsos

Other Decks in Research

Featured

Transcript