Towards User-defined Cross-Device Interaction

Presentation given at DUI 2016, 5th Workshop on Distributed User Interfaces, Lugano, Switzerland.

ABSTRACT: Over the last decade we have seen various research on distributed user interfaces (DUIs). We provide an overview of existing DUI approaches and classify the different solutions based on the granularity of the distributed UI components, location constraints as well as their support for the distribution of state. We propose an approach for user-defined cross-device interaction where users can author their customised user interfaces based on a hypermedia metamodel and the concept of
active components. Furthermore, we discuss the configuration and sharing of customised distributed user interfaces by end users where the focus is on an authoring rather than programming approach .

Research paper: https://beatsigner.com/publications/towards-user-defined-cross-device-interaction.pdf

Transcript

1. Towards User-defined Cross-Device Interaction AUDREY SANCTORUM & BEAT SIGNER VRIJE UNIVERSITEIT BRUSSEL
3. 0% 5% 10% 15% 20% 25% 30% 0 1 2 3 4 5 or more Percentage of users Number of devices How many connected devices do people use? Belgium Switzerland USA Spain Japan The Connected Consumer Survey 2014/2015 Google Inc. 65%
4. Conductor (2014) HuddleLamp (2014) Connichiwa (2015) Panelrama (2014) MultiSpace (2006) ReticularSpaces (2012)
5. Existing Classifications "The 4C Reference Model for Distributed User Interfaces" by Demeure et al. • computation, configuration, communication and coordination "Distributed User Interfaces: State of the Art" by Niklas Elmqvist • input, output, platform, space and time
6. UI + data + UI elements UI + data Table / Camera Room Network connection to the server Anywhere HuddleLamp MultiSpace ReticularSpaces Panelrama Conductor Connichiwa Granularity of distribution Location constraint
7. UI + data + UI elements UI + data Table / Camera Room Network connection to the server Anywhere HuddleLamp Airlift MultiSpace ReticularSpaces ARIS GroupTogether iLand iRoom Panelrama Conductor IMPROMPTU THAW Deep Shot Weave Connichiwa XDStudio WebSplitter Melchior et al. (2009) CAMELEON-RT Granularity of distribution Location constraint Robertson et al. (1996) Frosini et al. (2013) MultiMasher Pick-and-Drop End Users
8. How to allow end users to define customised cross-device interactions? How much control will end users have in terms of the granularity of the UI components to be distributed? Will end users be limited by a specific location, space or office setting? Will end users be able to share their configuration of customised DUIs? Can end users reuse parts of other configurations? What will end users be able to modify? …
9. <?xml version="1.0"encoding="UTF-8"?> <activeComponents> <activeComponent> <identifier>Sound< /identifier> <logic>…</logic> </activeComponent> </activeComponents> Active Component Architecture Active Components Resource/Selector Plug-ins Developer Registry Core RSL User Profile End-User Registry Configuration Pool User Developer Button 1 Signer and Norrie (2007)
10. Scenarios
11. Swipe AC Double Swipe AC Triple Swipe AC Direction AC Swipe Area TABLET Sound AC TV Runtime environment Runtime environment ... Gesture AC Play AC Data Transfer AC Data Transfer AC Proposed Approach
12. Authoring Rather Than Programming Swipe Gesture Device: Tablet Area Device: Tablet Data Transfer SDevice: Tablet TDevice: TV Play Device: TV Sound Device: TV Volume: 80% _ X   Components Data Transfer Play Sound Swipe Area Double Swipe Triple Swipe Direction ...
13. Conclusion • Classification of DUI systems • User-defined cross-device interactions • Linking UI components and application logic • hypermedia metamodel • at any level of granularity • Sharing of user-defined interactions
14. References • A. Sanctorum and B. Signer, Towards User-defined Cross-Device Interaction, Proceedings of DUI 2016, 5th Workshop on Distributed User Interfaces, Lugano, Switzerland, June 2016 • http://beatsigner.com/publications/sanctorum_DUI2016.pdf