Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Towards Cross-Media Information Spaces and Architectures

Towards Cross-Media Information Spaces and Architectures

Keeynote given at RCIS 2019, Brussels, Belgium.

ABSTRACT: The efficient management and retrieval of information via dedicated devices and data structures has been investigated since the early days of Vannevar Bush’s seminal article 'As We May Think' introducing the Memex. However, nowadays information is usually fragmented across different media types, devices as well as digital and physical environments, and we are often struggling to retrieve specific information. We discuss three main issues to be addressed when developing solutions for managing information in these co-called cross-media information spaces. First, we have a look at an extensible cross-media linking solution based on the resource-selector-link (RSL) hypermedia metamodel where information can be integrated across applications, devices as well as digital and physical information environments. We then outline some of the limitations of existing digital document formats which are often just a simulation of paper documents and their affordances on desktop computers, and discuss more flexible document representations for cross-media information spaces. Further, new forms of human-information interaction and cross-media user interfaces—including some recent work on dynamic data physicalisation—are discussed. A number of research artefacts are used to illustrate different aspects of the presented data-centric approach for crossmedia information spaces and architectures. Last but not least, we provide an outlook on how the embedding of the presented concepts at the level of an operating system might ultimately lead to new possibilities for cross-media information management and innovative forms of human-information interaction

Research paper: https://beatsigner.com/publications/towards-cross-media-information-spaces-and-architectures.pdf

Beat Signer
PRO

May 31, 2019
Tweet

More Decks by Beat Signer

Other Decks in Science

Transcript

  1. 2 December 2005
    Towards Cross-Media Information Spaces
    and Architectures
    Prof. Beat Signer
    beatsigner.com
    Web & Information Systems Engineering Lab
    Department of Computer Science
    Vrije Universiteit Brussel

    View Slide

  2. "As We May Think" (1945)
    ▪ Memex
    ▪ memory extender
    ▪ information retrieval
    ▪ associative trails
    ▪ trail blazers
    information overload

    View Slide

  3. 01001000100
    0101001100010010100010010
    0100100010010001000100001001000
    100010001100001001000110010101010110
    1100010010100010010010010100010101011001
    101001100001100 101010100
    0010100 100 110101010101
    00100 01010101000
    1001 1111111111
    111 0000000010
    0 11111111
    0101001011
    010101010
    111010101
    001010010110
    Cross-Media Information Spaces

    View Slide

  4. Fragmentation of information
    across different applications,
    devices and digital & physical
    information environments

    View Slide

  5. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  6. Data-centric approach for
    cross-media information
    solutions based on a
    hypermedia metamodel

    View Slide

  7. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  8. RSL Metamodel and Link Service
    Beat Signer and Moira C. Norrie,
    As We May Link: A General Metamodel
    for Hypermedia Systems, Proceedings of
    ER 2007, November 2007

    View Slide

  9. EdFest: Paper-Digital Guide
    Global Information Systems Group
    ETH Zurich

    View Slide

  10. PimVis: Personal Information Management
    Sandra Trullemans and Audrey Sanctorum

    View Slide

  11. PimVis Document View
    Audrey Sanctorum

    View Slide

  12. PimVis Focus View
    Audrey Sanctorum

    View Slide

  13. Open Cross-Media Linking
    local visual plug-ins
    external visual plug-ins
    communicate
    Cross-Media Link Browser & Editor
    Ahmed A.O. Tayeh

    View Slide

  14. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  15. What is Wrong with Digital Documents?
    Most people don't understand the logic of the concept:
    "What You See Is What You Get" is based on printing
    the document out ("get" means "get WHEN YOU
    PRINT IT OUT"). And that means a metaphysical shift:
    a document can only consist of what can be printed!
    [...] No overlays [...] – PAPER UNDER GLASS.
    Ted Nelson, Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009
    Existing document formats often based on
    the simulation of paper affordances
    on desktop computers

    View Slide

  16. What is Wrong with Slideware?
    ▪ Simulation of physical slides
    ▪ limited space due to slide concept
    ▪ linear navigation
    ▪ monolithic documents
    ▪ difficult to reuse content and embed rich media types

    View Slide

  17. MindXpres Presentation Platform
    ▪ Flexible representation
    of presentations
    ▪ use of structural RSL links
    ▪ separation of content and
    structure
    ▪ Extensible platform
    ▪ content-based approach
    ▪ cross-media content reuse
    ▪ non-linear navigation and
    zoomable user interface
    ▪ associative linking
    ▪ rich media types
    Reinout Roels

    View Slide

  18. MindXpres Document Representation

    View Slide

  19. video

    View Slide

  20. Different context-dependent
    adaptive overlay structures on top
    of (parts of) content
    Clear separation between content
    and structure enables cross-media
    content reuse (transclusion)

    View Slide

  21. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  22. PimVis: Paper-Digital User Interface
    Sandra Trullemans and Audrey Sanctorum

    View Slide

  23. ArtVis: Tangible User Interface
    Bram Moerman

    View Slide

  24. Real-Time Cross-Media Data Exploration
    dynamic data
    physicalisation

    View Slide

  25. Adaptive Distributed Hybrid User Interfaces
    Audrey Sanctorum and Beat Signer,
    A Unifying Reference Framework and
    Model for Adaptive Distributed Hybrid
    User Interfaces, Proceedings of
    RCIS 2019, May 2019
    Audrey Sanctorum
    eSPACE reference
    framework and model

    View Slide

  26. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  27. Unified Cross-Media Information Spaces

    View Slide

  28. Common underlying
    representation of cross-media
    links and structures via
    RSL hypermedia metamodel

    View Slide

  29. "Operating System" of the Future
    RSL-based Hypermedia Engine

    Legacy Applications
    File System Driver
    Hypermedia-enabled Applications
    RCIS2019.docx
    RCIS2019.docx

    View Slide

  30. References
    ▪ B. Signer, Fundamental Concepts for
    Interactive Paper and Cross-Media Information
    Spaces, Second Edition, ISBN 978-3-8370-2713-6,
    August 2017
    ▪ B. Signer and M.C. Norrie, As We May Link: A General
    Metamodel for Hypermedia Systems, Proceedings of ER
    2007, 26th International Conference on Conceptual
    Modeling, Auckland, New Zealand, November 2007
    ▪ http://beatsigner.com/publications/signer_ER2007.pdf

    View Slide

  31. References ...
    ▪ B. Signer, What is Wrong with Digital
    Documents? A Conceptual Model for Structural Cross-
    Media Content Composition and Reuse, Proceedings of
    the 29th International Conference on Conceptual Mod-
    eling (ER 2010), Vancouver, Canada, November 2010
    ▪ http://beatsigner.com/publications/signer_ER2010.pdf
    ▪ R. Roels and B. Signer, A Conceptual Framework and
    Content Model for Next Generation Presentation
    Solutions, Proceedings of the ACM on Human-Computer
    Interaction (PACMHCI), 2019
    ▪ http://beatsigner.com/publications/sanctorum_RCIS2019.pdf

    View Slide

  32. References ...
    ▪ S. Trullemans, A. Sanctorum and B. Signer,
    PimVis: Exploring and Re-finding Documents in Cross-
    Media Information Spaces, Proceedings of AVI 2016,
    International Working Conference on Advanced Visual
    Interfaces, Bari, Italy, June 2016
    ▪ http://beatsigner.com/publications/trullemans_AVI2016.pdf
    ▪ B. Dumas, B. Moerman, S. Trullemans and B. Signer,
    ArtVis: Combining Advanced Visualisation and Tangible
    Interaction for the Exploration, Analysis and Browsing of
    Digital Artwork Collections, Proceedings of AVI 2014,
    Como, Italy, May 2014
    ▪ http://beatsigner.com/publications/dumas_AVI2014.pdf

    View Slide

  33. References ...
    ▪ B. Signer and T.J. Curtin, Tangible Holograms:
    Towards Mobile Physical Augmentation of Virtual
    Objects, Technical Report WISE Lab, WISE-2017-01,
    March 2017
    ▪ http://beatsigner.com/publications/signer_arXiv2017.pdf
    ▪ A.A.O Tayeh and B. Signer, A Dynamically Extensible
    Open Cross-Document Link Service, Proceedings of
    WISE 2015, 16th International Conference on Web
    Information Systems Engineering, Miami, USA,
    November, 2015
    ▪ http://beatsigner.com/publications/tayeh_WISE2015.pdf

    View Slide

  34. References ...
    ▪ A. Sanctorum and B. Signer, A Unifying Reference
    Framework and Model for Adaptive Distributed Hybrid
    User Interfaces, Proceedings of RCIS 2019, 13th
    International Conference on Research Challenges in
    Information Science, Brussels, Belgium, May 2019
    ▪ https://beatsigner.com/publications/sanctorum_RCIS2019.pdf

    View Slide