Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The RSL Hypermedia Metamodel and Its Application in Cross-Media Solutions

The RSL Hypermedia Metamodel and Its Application in Cross-Media Solutions

Guest lecture given in the 'Hypertext and Hypernarratives' seminar organised by Prof. Claus Atzenbeck at Hof University, Germany.

Beat Signer
PRO

June 06, 2023
Tweet

More Decks by Beat Signer

Other Decks in Research

Transcript

  1. 2 December 2005
    The RSL Hypermedia Metamodel and Its
    Application in Cross-Media Solutions
    Prof. Beat Signer
    beatsigner.com
    Web & Information Systems Engineering Lab
    Department of Computer Science
    Vrije Universiteit Brussel

    View Slide

  2. 01001000100
    0101001100010010100010010
    0100100010010001000100001001000
    100010001100001001000110010101010110
    1100010010100010010010010100010101011001
    101001100001100 101010100
    0010100 100 110101010101
    00100 01010101000
    1001 1111111111
    111 0000000010
    0 11111111
    0101001011
    010101010
    111010101
    001010010110
    Cross-Media Information Spaces

    View Slide

  3. Fragmentation of information
    across different applications,
    devices and digital & physical
    information environments

    View Slide

  4. Information
    Systems &
    Management
    Information
    Visualisation
    & Navigation
    Information
    Visualisation
    nd Navigation
    Human-Machine &
    Human-Information
    Interaction
    CISA
    RSL
    MindXpres
    OC2
    associative file system
    data-driven storytelling
    XIMA
    cross-device interaction
    iServer/iPaper
    ViDaX
    data physicalisation
    OpenHPS
    non-linear storytelling
    PaperProof
    iGesture
    PaperSketch
    TangHo
    Context Modelling Toolkit
    Midas
    SpeeG2
    PaperPoint
    Print-n-Link
    digital libraries
    Mudra mixed reality
    ArtVis
    open cross-media linking
    PimVis
    DocTr
    EdFest
    source code visualisation
    INFEX
    technology-enhanced learning
    Cross-Media Technologies
    WEB & INFORMATION
    SYSTEMS ENGINEERING

    View Slide

  5. Data-centric approach for
    cross-media information
    solutions based on a
    hypermedia metamodel

    View Slide

  6. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  7. RSL Hypermedia Metamodel & Link Server
    B. Signer and M.C. Norrie, As We May Link: A General Meta-
    model for Hypermedia Systems, Proceedings of ER 2007x

    View Slide

  8. RSL Hypermedia Metamodel
    B. Signer and M.C. Norrie, As We May Link: A General Meta-
    model for Hypermedia Systems, Proceedings of ER 2007x

    View Slide

  9. Interactive Paper (EdFest)
    B. Signer, M. Grossniklaus and M.C. Norrie, Interactive Paper as a Mobile
    Client for a Multi-Channel Web Information System WWWJ 10(4), 2007 x

    View Slide

  10. PimVis Personal Information Management
    S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
    Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

    View Slide

  11. PimVis Setup
    S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
    Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

    View Slide

  12. PimVis Document View
    S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
    Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

    View Slide

  13. PimVis Focus View
    S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
    Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

    View Slide

  14. Object-Concept-Context (OC2) Model
    object layer
    structural link
    concept layer
    context layer
    navigational link
    associative link
    weighted link
    Object 1 Object n
    Object 2
    Context 1 Context n
    Concept 1
    Concept 2 Concept n
    extent link
    OC2 conceptual model
    RSL-based metamodel
    S. Trullemans and B. Signer, Towards a Conceptual Framework and Metamodel for Context-
    aware Personal Cross-Media Information Management Systems, Proceedings of ER 2014

    View Slide

  15. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  16. What is Wrong with Digital Documents?
    Most people don't understand the logic of the concept:
    "What You See Is What You Get" is based on printing
    the document out ("get" means "get WHEN YOU
    PRINT IT OUT"). And that means a metaphysical shift:
    a document can only consist of what can be printed!
    [...] No overlays [...] – PAPER UNDER GLASS.
    Ted Nelson, Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009
    Existing document formats often based on
    the simulation of paper affordances on
    desktop computers
    B. Signer, What is Wrong with Digital Documents? A Conceptual Model for Struc-
    tural Cross-Media Content Composition and Reuse, Proceedings of ER 2010

    View Slide

  17. What is Wrong with Slideware?
    ▪ Simulation of physical slides
    ▪ limited space due to slide concept
    ▪ linear navigation
    ▪ monolithic documents
    ▪ difficult to reuse content and embed rich media types

    View Slide

  18. MindXpres Presentation Platform
    ▪ Flexible representation
    of presentations
    ▪ use of structural RSL links
    ▪ separation of content and
    structure
    ▪ Extensible platform
    ▪ content-based approach
    ▪ cross-media content reuse
    via transclusion
    ▪ non-linear navigation and
    zoomable user interface
    ▪ associative linking
    ▪ rich media types
    R. Roels and B. Signer, MindXpres: An Extensible Content-driven
    Cross-Media Presentation Platform, Proceedings of WISE 2014

    View Slide

  19. MindXpres Document Representation
    R. Roels and B. Signer, A Conceptual Framework and Content Model for
    Next Generation Presentation Solutions, PACMHCI 3(Issue EICS), 2019

    View Slide

  20. MindXpres (Video)

    View Slide

  21. Different context-dependent
    adaptive overlay structures on top
    of (parts of) content
    Clear separation between content
    and structure enables cross-media
    content reuse (transclusion)

    View Slide

  22. Cross-Media Linking
    Flexible Document Representation
    Cross-Media User Interfaces

    View Slide

  23. PimVis Personal Information Management
    S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding
    Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

    View Slide

  24. eSPACE Framework for Hybrid UIs
    A. Sanctorum and B. Signer, A Unifying Reference Framework and Model
    for Adaptive Distributed Hybrid User Interfaces, Proceedings of RCIS 2019
    x

    View Slide

  25. XR-based Interaction
    ▪ Controlling individual IoT
    devices
    ▪ Define relationships and
    rules between IoT devices
    ▪ Linking any physical
    artefact to other physical
    artefacts or digital
    information and services
    ▪ use of natural (pointing)
    gestures and interactions
    ▪ “drag and drop” between
    IoT devices
    Elena Zambon
    Tangible Holograms (TangHo)

    View Slide

  26. The Lost Cosmonaut
    ▪ Writing as a collaborative
    act of memory and story-
    telling
    ▪ Reading, writing and
    narrative as an act of
    making sense
    ▪ Handwriting as a tool
    for human-computer
    interaction
    In collaboration with Axel Vogelsang,
    Artists in Labs Programme
    A. Vogelsang and B. Signer, The Lost Cosmonaut: An Interactive Narrative En-
    vironment on Basis of Digitally Enhanced Paper, Proceedings of ICVS 2005

    View Slide

  27. Open Cross-Media Linking
    local visual plug-ins
    external visual plug-ins
    communicate
    Cross-Media Link Browser & Editor

    View Slide

  28. Unified Cross-Media Information Spaces

    View Slide

  29. Common underlying
    representation of data, cross-
    media links and structures via
    RSL-based hypermedia engine
    supporting legacy applications

    View Slide

  30. RSL-based Hypermedia Engine
    RSL-based Hypermedia Engine
    Legacy Applications
    File System Driver
    Hypermedia-enabled Applications
    HT2021.docx
    HT2021.docx

    B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper-
    media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

    View Slide

  31. RSL-based Hypermedia Engine …
    signer
    My Pictures
    Vancouver ER 2010
    Granville
    Island
    Vancouver
    Harbour
    Public
    Market
    Keynote
    J. Thorp
    Olga and
    Lamia
    Dinner
    ER 2010
    multiple classification
    structural link
    navigational link
    My Videos
    Vancouver
    Aquarium
    ER 2010
    Keynote
    cross-media association
    My Presentations
    ER 2010 WISE 2009
    cross-media
    transclusion
    single slide
    transclusion
    (via OOXML
    RSL selector)

    View Slide

  32. Next Generation File Manager
    B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper-
    media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

    View Slide

  33. System-wide hypermedia engine
    at operating system level, enabling
    cross-media information manage-
    ment across application boundaries

    View Slide

  34. Beat Signer - Department of Computer Science - [email protected] 34
    June 6, 2023
    References
    ▪ B. Signer, Fundamental Concepts for
    Interactive Paper and Cross-Media Information
    Spaces, Second Edition, ISBN 978-3-8370-2713-6,
    August 2017
    ▪ B. Signer and M.C. Norrie, As We May Link: A General
    Metamodel for Hypermedia Systems, Proceedings of
    ER 2007, 26th International Conference on Conceptual
    Modeling, Auckland, New Zealand, November 2007
    ▪ https://beatsigner.com/publications/signer_ER2007.pdf

    View Slide

  35. Beat Signer - Department of Computer Science - [email protected] 35
    June 6, 2023
    References ...
    ▪ B. Signer, Towards Cross-Media Information
    Spaces and Architectures, Proceedings of RCIS 2019,
    13th International Conference on Research Challenges
    in Information Science, Brussels, Belgium, May 2019
    ▪ https://beatsigner.com/publications/signer_RCIS2019.pdf
    ▪ B. Signer, M. Grossniklaus and M.C. Norrie,
    Interactive Paper as a Mobile Client for a Multi-Channel
    Web Information System, World Wide Web Journal
    (WWW), Vol. 10, No. 4, Springer, December 2007
    ▪ https://beatsigner.com/publications/signer_WWWJ2007.pdf

    View Slide

  36. Beat Signer - Department of Computer Science - [email protected] 36
    June 6, 2023
    References ...
    ▪ S. Trullemans, A. Sanctorum and B. Signer,
    PimVis: Exploring and Re-finding Documents in Cross-
    Media Information Spaces, Proceedings of AVI 2016,
    International Working Conference on Advanced Visual
    Interfaces, Bari, Italy, June 2016
    ▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf
    ▪ S. Trullemans and B. Signer, Towards a Conceptual
    Framework and Metamodel for Context-aware Personal
    Cross-Media Information Management Systems, Pro-
    ceedings of ER 2014, 33rd International Conference on
    Conceptual Modelling, Atlanta, USA, October, 2014
    ▪ https://beatsigner.com/publications/trullemans_ER2014.pdf

    View Slide

  37. Beat Signer - Department of Computer Science - [email protected] 37
    June 6, 2023
    References ...
    ▪ B. Signer, What is Wrong with Digital Documents?
    A Conceptual Model for Structural Cross-Media Content
    Composition and Reuse, Proceedings of ER 2010,
    29th International Conference on Conceptual Modeling,
    Vancouver, Canada, November 2010
    ▪ https://beatsigner.com/publications/signer_ER2010.pdf
    ▪ R. Roels and B. Signer, MindXpres: An
    Extensible Content-driven Cross-Media Presentation
    Platform, Proceedings of WISE 2014, 15th International
    Conference on Web Information System Engineering,
    Thessaloniki, Greece, October 2014
    ▪ https://beatsigner.com/publications/roels_WISE2014.pdf

    View Slide

  38. Beat Signer - Department of Computer Science - [email protected] 38
    June 6, 2023
    References ...
    ▪ R. Roels and B. Signer, A Conceptual Frame-
    work and Content Model for Next Generation
    Presentation Solutions, Proceedings of the ACM on
    Human-Computer Interaction (PACMHCI), 2019
    ▪ https://beatsigner.com/publications/roels_PACMHCI2019.pdf
    ▪ A. Sanctorum and B. Signer, A Unifying Reference
    Framework and Model for Adaptive Distributed Hybrid
    User Interfaces, Proceedings of RCIS 2019,
    13th International Conference on Research Challenges
    in Information Science, Brussels, Belgium, May 2019
    ▪ https://beatsigner.com/publications/sanctorum_RCIS2019.pdf

    View Slide

  39. Beat Signer - Department of Computer Science - [email protected] 39
    June 6, 2023
    References ...
    ▪ A. Vogelsang and B. Signer, The Lost Cosmonaut:
    An Interactive Narrative Environment on Basis of
    Digitally Enhanced Paper, Proceedings of the Inter-
    national Conference on Virtual Storytelling 2005,
    Strasbourg, France, December 2005
    ▪ https://beatsigner.com/publications/vogelsang_ICVS2005.pdf
    ▪ A.A.O. Tayeh, Payam Ebrahimi and Beat Signer, Cross-
    Media Document Linking and Navigation, Proceedings
    of DocEng 2018, 18th ACM Symposium on Document
    Engineering, Halifax, Canada, August 2018
    ▪ https://beatsigner.com/publications/tayeh_DocEng2018.pdf

    View Slide

  40. Beat Signer - Department of Computer Science - [email protected] 40
    June 6, 2023
    References ...
    ▪ B. Signer, R. Roels, R. van Barlingen and
    B. Willems, Back to the Future: Bringing Original
    Hypermedia and Cross-Media Concepts to Modern
    Desktop Environments, Proceedings of Hypertext 2021,
    Virtual Event, August 2021
    ▪ https://beatsigner.com/publications/signer_Hypertext2021.pdf

    View Slide