Towards User-defined Cross-Device Interaction

Presentation given at DUI 2016, 5th Workshop on Distributed User Interfaces, Lugano, Switzerland.

ABSTRACT: Over the last decade we have seen various research on distributed user interfaces (DUIs). We provide an overview of existing DUI approaches and classify the different solutions based on the granularity of the distributed UI components, location constraints as well as their support for the distribution of state. We propose an approach for user-defined cross-device interaction where users can author their customised user interfaces based on a hypermedia metamodel and the concept of
active components. Furthermore, we discuss the configuration and sharing of customised distributed user interfaces by end users where the focus is on an authoring rather than programming approach .

Research paper: https://beatsigner.com/publications/towards-user-defined-cross-device-interaction.pdf

Transcript

1. Towards User-defined
   Cross-Device Interaction
   AUDREY SANCTORUM & BEAT SIGNER
   VRIJE UNIVERSITEIT BRUSSEL
   

   View Slide

2. View Slide

3. 0% 5% 10% 15% 20% 25% 30%
   0
   1
   2
   3
   4
   5 or more
   Percentage of users
   Number of devices
   How many connected devices do people use?
   Belgium Switzerland USA Spain Japan
   The Connected Consumer
   Survey 2014/2015
   Google Inc.
   65%
   

   View Slide

4. Conductor (2014)
   HuddleLamp (2014)
   Connichiwa (2015)
   Panelrama (2014)
   MultiSpace (2006)
   ReticularSpaces (2012)
   

   View Slide

5. Existing Classifications
   "The 4C Reference Model for Distributed User Interfaces"
   by Demeure et al.
   • computation, configuration, communication and coordination
   "Distributed User Interfaces: State of the Art"
   by Niklas Elmqvist
   • input, output, platform, space and time
   

   View Slide

6. UI + data +
   UI elements
   UI + data
   Table /
   Camera
   Room
   Network connection
   to the server
   Anywhere
   HuddleLamp
   MultiSpace
   ReticularSpaces
   Panelrama
   Conductor
   Connichiwa
   Granularity of
   distribution
   Location
   constraint
   

   View Slide

7. UI + data +
   UI elements
   UI + data
   Table /
   Camera
   Room
   Network connection
   to the server
   Anywhere
   HuddleLamp
   Airlift
   MultiSpace
   ReticularSpaces
   ARIS
   GroupTogether
   iLand
   iRoom
   Panelrama
   Conductor
   IMPROMPTU
   THAW
   Deep Shot
   Weave
   Connichiwa
   XDStudio
   WebSplitter
   Melchior et al. (2009)
   CAMELEON-RT
   Granularity of
   distribution
   Location
   constraint
   Robertson et al. (1996)
   Frosini et al. (2013)
   MultiMasher
   Pick-and-Drop
   End Users
   

   View Slide

8. How to allow end users to define customised
   cross-device interactions?
   How much control will end users have in terms of the
   granularity of the UI components to be distributed?
   Will end users be limited by a specific
   location, space or office setting?
   Will end users be able to share their
   configuration of customised DUIs?
   Can end users reuse parts of other
   configurations?
   What will end users be able to modify?
   …
   

   View Slide

9. version="1.0"encoding="UTF-8"?>
   
   
   Sound< /identifier>
   …
   
   
   Active Component
   Architecture
   Active
   Components
   Resource/Selector
   Plug-ins
   Developer Registry
   Core
   RSL
   User Profile
   End-User Registry
   Configuration
   Pool
   User Developer
   Button 1
   Signer and Norrie (2007)
   

   View Slide

10. Scenarios
    

    View Slide

11. Swipe AC
    Double
    Swipe AC
    Triple
    Swipe AC
    Direction
    AC
    Swipe Area
    TABLET
    Sound
    AC
    TV Runtime
    environment
    Runtime environment
    ...
    Gesture AC
    Play
    AC
    Data
    Transfer AC
    Data
    Transfer AC
    Proposed Approach
    

    View Slide

12. Authoring Rather Than Programming
    Swipe Gesture
    Device: Tablet
    Area
    Device: Tablet
    Data Transfer
    SDevice: Tablet
    TDevice: TV
    Play
    Device: TV
    Sound
    Device: TV
    Volume: 80%
    _ X
    
     Components
    Data Transfer
    Play
    Sound
    Swipe
    Area
    Double Swipe
    Triple Swipe
    Direction
    ...
    

    View Slide

13. Conclusion
    • Classification of DUI systems
    • User-defined cross-device interactions
    • Linking UI components and application logic
    • hypermedia metamodel
    • at any level of granularity
    • Sharing of user-defined interactions
    

    View Slide

14. References
    • A. Sanctorum and B. Signer, Towards User-defined Cross-Device
    Interaction, Proceedings of DUI 2016, 5th Workshop on Distributed
    User Interfaces, Lugano, Switzerland, June 2016
    • http://beatsigner.com/publications/sanctorum_DUI2016.pdf
    

    View Slide