Slide 1

Slide 1 text

2 December 2005 From Hypertext to Cross-Media Information Spaces Prof.Beat Signer beatsigner.com Web & Information Systems Engineering Lab Department of Computer Science Vrije Universiteit Brussel

Slide 2

Slide 2 text

"As We May Think" (1945) ▪ Vannevar Bush's article As We May Think (1945) is often seen as the "origin" of hypertext ▪ The article introduces the Memex ▪ memory extender ▪ store and access information (on microfilms) ▪ follow cross-references in the form of associative trails between pieces of information ▪ prototypical hypertext machine ▪ trail blazers are those who find delight in the task of establishing useful trails Memex

Slide 3

Slide 3 text

Hypertext (1965) ▪ Ted Nelson coined the term hypertext ▪ Nelson started Project Xanadu in 1960 ▪ first hypertext project ▪ non-sequential writing ▪ referencing / embedding parts of a document in another document (transclusion) ▪ bidirectional (bivisible) links ▪ version and rights management ▪ OpenXanadu demo / deliverable released in 2014 Ted Nelson

Slide 4

Slide 4 text

OpenXanadu (2014) [https://xanadu.com/xanademos/MoeJusteOrigins.html]

Slide 5

Slide 5 text

Hypertext Editing System, HES (1967) ▪ Early hypertext system ▪ developed at Brown University (1967) by Andries van Dam and his team ▪ Ted Nelson was a visitor at Brown University at that time ▪ Limitations ▪ unidirectional links ▪ non-overlapping links ▪ only embedded links

Slide 6

Slide 6 text

World Wide Web (WWW) ▪ WWW is a network-enabled version of the HES hypertext model ▪ unidirectional links between heterogeneous resources ▪ What about all the richer functionality researched by the hypertext community? ▪ bidirectional links ▪ transclusion ▪ external (non-embedded) links ▪ version management ▪ … Tim Berners-Lee Robert Cailliau

Slide 7

Slide 7 text

01001000100 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 101001100001100 101010100 0010100 100 110101010101 00100 01010101000 1001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101 001010010110 Cross-Media Information Spaces

Slide 8

Slide 8 text

Fragmentation of information across different applications, devices and digital & physical information environments

Slide 9

Slide 9 text

Information Systems & Management Information Visualisation & Navigation Information Visualisation nd Navigation Human-Machine & Human-Information Interaction CISA RSL MindXpres OC2 associative file system data-driven storytelling XIMA cross-device interaction iServer/iPaper ViDaX data physicalisation OpenHPS non-linear storytelling PaperProof iGesture PaperSketch TangHo Context Modelling Toolkit Midas SpeeG2 PaperPoint Print-n-Link digital libraries Mudra mixed reality ArtVis open cross-media linking PimVis DocTr EdFest source code visualisation INFEX technology-enhanced learning Cross-Media Technologies WEB & INFORMATION SYSTEMS ENGINEERING

Slide 10

Slide 10 text

Data-centric approach for cross-media information solutions based on a hypermedia metamodel

Slide 11

Slide 11 text

Cross-Media Linking Flexible Document Representation Cross-Media User Interfaces

Slide 12

Slide 12 text

RSL Hypermedia Metamodel & Link Server B. Signer and M.C. Norrie, As We May Link: A General Meta- model for Hypermedia Systems, Proceedings of ER 2007x

Slide 13

Slide 13 text

RSL Hypermedia Metamodel B. Signer and M.C. Norrie, As We May Link: A General Meta- model for Hypermedia Systems, Proceedings of ER 2007x

Slide 14

Slide 14 text

Interactive Paper (EdFest) B. Signer, M. Grossniklaus and M.C. Norrie, Interactive Paper as a Mobile Client for a Multi-Channel Web Information System WWWJ 10(4), 2007 x

Slide 15

Slide 15 text

Cross-Media Link Server (iServer) Global Information Systems Group ETH Zurich

Slide 16

Slide 16 text

PimVis Personal Information Management S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

Slide 17

Slide 17 text

PimVis Setup S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

Slide 18

Slide 18 text

PimVis Document View S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

Slide 19

Slide 19 text

PimVis Focus View S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

Slide 20

Slide 20 text

Cross-Media Linking Flexible Document Representation Cross-Media User Interfaces

Slide 21

Slide 21 text

What is Wrong with Digital Documents? Most people don't understand the logic of the concept: "What You See Is What You Get" is based on printing the document out ("get" means "get WHEN YOU PRINT IT OUT"). And that means a metaphysical shift: a document can only consist of what can be printed! [...] No overlays [...] – PAPER UNDER GLASS. Ted Nelson, Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009 Existing document formats often based on the simulation of paper affordances on desktop computers B. Signer, What is Wrong with Digital Documents? A Conceptual Model for Struc- tural Cross-Media Content Composition and Reuse, Proceedings of ER 2010

Slide 22

Slide 22 text

What is Wrong with Slideware? ▪ Simulation of physical slides ▪ limited space due to slide concept ▪ linear navigation ▪ monolithic documents ▪ difficult to reuse content and embed rich media types

Slide 23

Slide 23 text

MindXpres Presentation Platform ▪ Flexible representation of presentations ▪ use of structural RSL links ▪ separation of content and structure ▪ Extensible platform ▪ content-based approach ▪ cross-media content reuse via transclusion ▪ non-linear navigation and zoomable user interface ▪ associative linking ▪ rich media types R. Roels and B. Signer, MindXpres: An Extensible Content-driven Cross-Media Presentation Platform, Proceedings of WISE 2014

Slide 24

Slide 24 text

MindXpres Document Representation R. Roels and B. Signer, A Conceptual Framework and Content Model for Next Generation Presentation Solutions, PACMHCI 3(Issue EICS), 2019

Slide 25

Slide 25 text

MindXpres (Video)

Slide 26

Slide 26 text

Different context-dependent adaptive overlay structures on top of (parts of) content Clear separation between content and structure enables cross-media content reuse (transclusion)

Slide 27

Slide 27 text

Cross-Media Linking Flexible Document Representation Cross-Media User Interfaces

Slide 28

Slide 28 text

PimVis Personal Information Management S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross-Media Information Spaces, Proceedings of AVI 2016

Slide 29

Slide 29 text

eSPACE Framework for Hybrid UIs A. Sanctorum and B. Signer, A Unifying Reference Framework and Model for Adaptive Distributed Hybrid User Interfaces, Proceedings of RCIS 2019 x

Slide 30

Slide 30 text

XR-based Interaction ▪ Controlling individual IoT devices ▪ Define relationships and rules between IoT devices ▪ Linking any physical artefact to other physical artefacts or digital information and services ▪ use of natural (pointing) gestures and interactions ▪ “drag and drop” between IoT devices Elena Zambon Tangible Holograms (TangHo)

Slide 31

Slide 31 text

Unified Cross-Media Information Spaces

Slide 32

Slide 32 text

Common underlying representation of data, cross- media links and structures via RSL-based hypermedia engine supporting legacy applications

Slide 33

Slide 33 text

RSL-based Hypermedia Engine RSL-based Hypermedia Engine Legacy Applications File System Driver Hypermedia-enabled Applications HT2021.docx HT2021.docx … B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper- media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

Slide 34

Slide 34 text

RSL-based Hypermedia Engine … signer My Pictures Vancouver ER 2010 Granville Island Vancouver Harbour Public Market Keynote J. Thorp Olga and Lamia Dinner ER 2010 multiple classification structural link navigational link My Videos Vancouver Aquarium ER 2010 Keynote cross-media association My Presentations ER 2010 WISE 2009 cross-media transclusion single slide transclusion (via OOXML RSL selector)

Slide 35

Slide 35 text

Next Generation File Manager B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hyper- media and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021

Slide 36

Slide 36 text

System-wide hypermedia engine at operating system level, enabling cross-media information manage- ment across application boundaries

Slide 37

Slide 37 text

Beat Signer - Department of Computer Science - [email protected] 37 October 25, 2023 References ▪ B. Signer, Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, Second Edition, ISBN 978-3-8370-2713-6, August 2017 ▪ B. Signer, Towards Cross-Media Information Spaces and Architectures, Proceedings of RCIS 2019, 13th International Conference on Research Challenges in Information Science, Brussels, Belgium, May 2019 ▪ https://beatsigner.com/publications/signer_RCIS2019.pdf ▪ Interview with Beat Signer, Claus Atzenbeck, ACM SIGWEB Newsletter 2021(Winter), February 2021 ▪ https://beatsigner.com/publications/interview-with-beat-signer.pdf

Slide 38

Slide 38 text

Beat Signer - Department of Computer Science - [email protected] 38 October 25, 2023 References ... ▪ B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems, Proceedings of ER 2007, 26th International Conference on Conceptual Modeling, Auckland, New Zealand, November 2007 ▪ https://beatsigner.com/publications/signer_ER2007.pdf ▪ B. Signer, R. Roels, R. van Barlingen and B. Willems, Back to the Future: Bringing Original Hypermedia and Cross-Media Concepts to Modern Desktop Environments, Proceedings of Hypertext 2021, Virtual Event, August 2021 ▪ https://beatsigner.com/publications/signer_Hypertext2021.pdf

Slide 39

Slide 39 text

Beat Signer - Department of Computer Science - [email protected] 39 October 25, 2023 References ... ▪ B. Signer, M. Grossniklaus and M.C. Norrie, Interactive Paper as a Mobile Client for a Multi-Channel Web Information System, World Wide Web Journal (WWW), Vol. 10, No. 4, Springer, December 2007 ▪ https://beatsigner.com/publications/signer_WWWJ2007.pdf ▪ S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross- Media Information Spaces, Proceedings of AVI 2016, International Working Conference on Advanced Visual Interfaces, Bari, Italy, June 2016 ▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf

Slide 40

Slide 40 text

Beat Signer - Department of Computer Science - [email protected] 40 October 25, 2023 References ... ▪ B. Signer, What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content Composition and Reuse, Proceedings of ER 2010, 29th International Conference on Conceptual Modeling, Vancouver, Canada, November 2010 ▪ https://beatsigner.com/publications/signer_ER2010.pdf ▪ R. Roels and B. Signer, MindXpres: An Extensible Content-driven Cross-Media Presentation Platform, Proceedings of WISE 2014, 15th International Conference on Web Information System Engineering, Thessaloniki, Greece, October 2014 ▪ https://beatsigner.com/publications/roels_WISE2014.pdf

Slide 41

Slide 41 text

Beat Signer - Department of Computer Science - [email protected] 41 October 25, 2023 References ... ▪ R. Roels and B. Signer, A Conceptual Frame- work and Content Model for Next Generation Presentation Solutions, Proceedings of the ACM on Human-Computer Interaction (PACMHCI), 2019 ▪ https://beatsigner.com/publications/roels_PACMHCI2019.pdf ▪ A. Sanctorum and B. Signer, A Unifying Reference Framework and Model for Adaptive Distributed Hybrid User Interfaces, Proceedings of RCIS 2019, 13th International Conference on Research Challenges in Information Science, Brussels, Belgium, May 2019 ▪ https://beatsigner.com/publications/sanctorum_RCIS2019.pdf

Slide 42

Slide 42 text

Beat Signer - Department of Computer Science - [email protected] 42 October 25, 2023 References ... ▪ A. Vogelsang and B. Signer, The Lost Cosmonaut: An Interactive Narrative Environment on Basis of Digitally Enhanced Paper, Proceedings of the Inter- national Conference on Virtual Storytelling 2005, Strasbourg, France, December 2005 ▪ https://beatsigner.com/publications/vogelsang_ICVS2005.pdf ▪ A.A.O. Tayeh, Payam Ebrahimi and Beat Signer, Cross- Media Document Linking and Navigation, Proceedings of DocEng 2018, 18th ACM Symposium on Document Engineering, Halifax, Canada, August 2018 ▪ https://beatsigner.com/publications/tayeh_DocEng2018.pdf

Slide 43

Slide 43 text

Beat Signer - Department of Computer Science - [email protected] 43 October 25, 2023 References ... ▪ Vannevar Bush, As We May Think, Atlantic Monthly, July 1945 ▪ https://doi.org/10.1145/227181.227186 ▪ https://www.youtube.com/watch?v=c539cK58ees ▪ Ted Nelson demonstrates Xanadu Space ▪ https://www.youtube.com/watch?v=En_2T7KH6RA