2 December 2005 Cross-Media Technologies, Information Science and Human-Information Interaction Prof. Dr. Beat Signer beatsigner.com Web & Information Systems Engineering Lab Department of Computer Science Vrije Universiteit Brussel WEB & INFORMATION SYSTEMS ENGINEERING

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 2 March 27, 2024

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 3 March 27, 2024 01001000100 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 101001100001100 101010100 0010100 100 110101010101 00100 01010101000 1001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101 001010010110 Fluid Cross-Media Information Spaces

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 4 March 27, 2024 Information Systems & Management Information Visualisation & Navigation Information Visualisation nd Navigation Human-Machine & Human-Information Interaction CISA RSL [4] MindXpres [8] OC2 [5] associative file system [2] data-driven storytelling [11] XIMA [13] cross-device interaction [9] iServer/iPaper [3] ViDaX [16] data physicalisation [32] OpenHPS [18] non-linear storytelling [27] PaperProof [29] iGesture [23] PaperSketch [30] TangHo [31] Context Modelling Toolkit [10] Midas [21] SpeeG2 [24] PaperPoint [26] Print-n-Link [17] digital libraries [12] Mudra [22] mixed reality [28] ArtVis [15] open cross-media linking [1] PimVis [14] DocTr [6] EdFest [25] source code visualisation [19] INFEX [7] technology-enhanced learning [20] Cross-Media Information Spaces and Architectures

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 5 March 27, 2024 Prof. Dr. Beat Signer Cross-Media Technology, Interac- tive Paper, Data Physicalisation Dr. Audrey Sanctorum User-defined XDI and IoT Inter- action, Human-AI Interaction CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation WEB & INFORMATION SYSTEMS ENGINEERING CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Maxim Van de Wynckel Hybrid Positioning, Implicit Human-Computer Interaction Yoshi Malaise Technology-enhanced Learning, Content-driven Presentations

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 6 March 27, 2024 WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Ekene Attoh IoT Middleware, Context-aware Computing, Implicit HCI Isaac Valadez Knowledge Physicalisation and Augmentation, Tangible UIs Xuyao Zhang Extensible Platform for Dynamic Data Physicalisation Ingela Rossing Dynamic Data Physicalisation Framework and Guidelines

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 7 March 27, 2024 WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Arun Sojan Physicalisation for Digital Twins Piet Van Der Paelt Julia-based Framework for Simulation and Optimisation Evan Cole Technology-enhanced Learning, Study Lenses

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 8 March 27, 2024 RSL Hypermedia Metamodel Global Information Systems Group, ETH Zurich

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 9 March 27, 2024 Personal Information Management (PIM) ▪ Keeping, organising and re-finding information ▪ digital and physical ▪ Study of human- information interaction ▪ files, piles, mixtures, … ▪ OC2 PIM model ▪ based on RSL hypermedia metamodel ▪ Cross-Media PIM system ▪ explicit as well as implicit associations between entities

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 10 March 27, 2024 PimVis Setup

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 11 March 27, 2024 Mixed Reality in IoT Environments ▪ eSPACE IoT platform ▪ end-user authoring ▪ mashup tool for digital and physical (IoT) components ▪ Context Modelling Toolkit (CMT) ▪ OpenHPS hybrid positioning system ▪ stream-based processing ▪ combination of existing and new positioning techniques ▪ decentralised sensor fusion Elena Zambon

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 12 March 27, 2024 Dynamic Data Physicalisation ▪ Physical objects used for input as well as output ▪ Dynamic data physicali- sation ▪ experimental tangible holo- grams (TangHo) platform ▪ use physical variables such as temperature or texture ▪ exploration of big data sets ▪ dynamic data physicalisation framework - data physicalisation grammar

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 13 March 27, 2024 Real-Time Cross-Media Data Exploration

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 14 March 27, 2024 MindXpres Presentation Platform ▪ Extensible prototyping platform for novel presen- tation concepts ▪ content-based approach - separation of content and presen- tation (automatic visualisation) ▪ cross-media content reuse ▪ non-linear navigation via zoomable user interface ▪ connectivity and interactivity ▪ Rich-media plug-ins ▪ e.g.source code or interactive data visualisation

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 15 March 27, 2024 Interactive Source Code Plug-in Paul Mestereaga

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 16 March 27, 2024 Personalised Learning Environments ▪ Diagnostic assessments to detect knowledge gaps ▪ Suggestion of learning content ▪ domain knowledge graph ▪ user knowledge graph ▪ learning paths ▪ Automatic content adap- tation ▪ based on user preferences and constraints Enhanced Curriculum Assessments Tagged Learning Content Knowledge Graphs Adaptive Learning Content Learning Paths Knowledge Representation Framework Assessment Module Assessment Analyser Knowledge Gap Analyser Adaptive Learning Environment Suggestion Module User Knowledge Graph Personalised Presentation Module Learner Constraints Groups Module

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 17 March 27, 2024 Thesis Topics ▪ Possibilities for BA, MA and PhD theses ▪ Mixed Reality Solutions - e.g. for museums, photography etc. ▪ Tangible User Interfaces ▪ Dynamic Data Physicalisation ▪ MindXpres Presentation Platform ▪ Cross-Device and Cross-Media Interfaces ▪ Smart Environments and Cross-Domain Internet of Things (IoT) ▪ Hybrid Tracking and Implicit Human-Computer Interaction ▪ Personal Information Management (PIM) ▪ Technology-enhanced Learning ▪ Interactive Paper Solutions ▪ … ▪ Do you have your own ideas? Come along to discuss them...

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 18 March 27, 2024 References ▪ B. Signer, Towards Cross-Media Information Spaces and Architectures, Proceedings of RCIS 2019, Brussels, Belgium, May 2019 ▪ https://beatsigner.com/publications/signer_RCIS2019.pdf ▪ B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems, Proceedings of ER 2007, 26th International Conference on Conceptual Modeling, Auckland, New Zealand, November 2007 ▪ https://beatsigner.com/publications/signer_ER2007.pdf

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 19 March 27, 2024 References … ▪ B. Signer, What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content Composition and Reuse, Proceedings of the 29th International Conference on Conceptual Modeling (ER 2010), Vancouver, Canada, November 2010 ▪ https://beatsigner.com/publications/signer_ER2010.pdf ▪ R. Roels and B. Signer, A Conceptual Framework and Content Model for Next Generation Presentation Solutions, Proceedings of the ACM on Human-Computer Interaction (PACMHCI), 3(Issue EICS), June 2019 ▪ https://beatsigner.com/publications/roels_PACMHCI2019.pdf

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 20 March 27, 2024 References … ▪ B. Signer, P. Ebrahimi, T.J. Curtin and A. K.A. Abdullah, Towards a Framework for Dynamic Data Physicalisation, International Workshop Toward a Design Language for Data Physicalization, Berlin, Germany, October 2018 ▪ https://beatsigner.com/publications/signer_DataPhys2018.pdf ▪ B. Signer and T.J. Curtin, Tangible Holograms Towards Mobile Physical Augmentation of Virtual Objects, Technical Report WISE Lab, WISE-2017-01, March 2017 ▪ https://beatsigner.com/publications/signer_arXiv2017.pdf

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 21 March 27, 2024 References … ▪ A. Sanctorum and B. Signer, A Unifying Reference Framework and Model for Adaptive Distributed Hybrid User Interfaces, Proceedings of RCIS 2019, 13th International Conference on Research Challenges in Information Science, Brussels, Belgium, May 2019 ▪ https://beatsigner.com/publications/sanctorum_RCIS2019.pdf ▪ M. Van de Wynckel and B. Signer, OpenHPS: An Open Source Hybrid Positioning System, Technical Report WISE Lab, WISE-2020-01, December 2020 ▪ https://beatsigner.com//publications/vanDeWynckel_CoRR2020.pdf

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 22 March 27, 2024 References … ▪ R. Roels and B. Signer, MindXpres: An Extensible Content-driven Cross-Media Presentation Platform, Proceedings of WISE 2014, 15th International Conference on Web Information Systems Engineering, Thessaloniki, Greece, October, 2014 ▪ https://beatsigner.com/publications/roels_WISE2014.pdf ▪ R. Roels, P. Mestereaga and B. Signer, An Interactive Source Code Visualisation Plug-in for the MindXpres Presentation Platform, Communications in Computer and Information Science (CCIS), 583, 2016 ▪ https://beatsigner.com/publications/roels_CCIS2016.pdf

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 23 March 27, 2024 References … ▪ Y. Malaise and B. Signer, Personalised Learning Environments Based on Knowledge Graphs and the Zone of Proximal Development, Proceedings of CSEDU 2022, 14th International Conference on Computer Supported Education, April 2022 ▪ https://beatsigner.com/publications/malaise_CSEDU2022.pdf ▪ S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross- Media Information Spaces, Proceedings of AVI 2016, International Working Conference on Advanced Visual Interfaces, Bari, Italy, June 2016 ▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf

Beat Signer - Department of Computer Science - bsigner@vub.ac.be 24 March 27, 2024 References … ▪ MindXpres ▪ https://mindxpres.com ▪ OpenHPS ▪ https://openhps.org