2 December 2005 Cross-Media Technologies and Applications Future Directions for Personal Information Management Prof.Dr.Beat Signer beatsigner.com Web & Information Systems Engineering Lab Department of Computer Science Vrije Universiteit Brussel WEB & INFORMATION SYSTEMS ENGINEERING speakerdeck.com/signer/icity

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 01001000100 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 101001100001100 101010100 0010100 100 110101010101 00100 01010101000 1001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101 001010010110 Cross-Media Information Spaces and Architectures

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Information Systems & Management Information Visualisation & Navigation Information Visualisation nd Navigation Human-Machine & Human-Information Interaction CISA RSL [4] MindXpres [8] OC2 [5] associative file system [2] data-driven storytelling [11] XIMA [13] cross-device interaction [9] iServer/iPaper [3] ViDaX [16] data physicalisation [32] OpenHPS [18] non-linear storytelling [27] PaperProof [29] iGesture [23] PaperSketch [30] TangHo [31] Context Modelling Toolkit [10] Midas [21] SpeeG2 [24] PaperPoint [26] Print-n-Link [17] digital libraries [12] Mudra [22] mixed reality [28] ArtVis [15] open cross-media linking [1] PimVis [14] DocTr [6] EdFest [25] source code visualisation [19] INFEX [7] technology-enhanced learning [20] Cross-Media Information Spaces and Architectures

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 RSL Hypermedia Metamodel Global Information Systems Group, ETH Zurich

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Personal Information Management (PIM) ▪ Keeping, organising and re-finding information ▪ digital and physical ▪ Study of human- information interaction ▪ files, piles, mixtures, … ▪ Cross-Media PIM system ▪ explicit as well as implicit associations between entities

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 PimVis Setup

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Mixed Reality in IoT Environments ▪ eSPACE IoT platform ▪ end-user authoring ▪ mashup tool for digital and physical (IoT) components ▪ Context Modelling Toolkit (CMT) ▪ OpenHPS hybrid positioning system ▪ stream-based processing ▪ combination of existing and new positioning techniques ▪ decentralised sensor fusion Elena Zambon

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Interactive Paper and Tangible Interfaces ▪ Bridging the paper-digital divide ▪ seamless transition ▪ multimodal interaction ▪ new interaction and design patterns ▪ Advanced visualisation techniques in combination with tangible user interfaces (TUI) ▪ e.g. ArtVis Bram Moerman Global Information Systems Group, ETH Zurich

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Tangible Hologram (TangHo) Prototype

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Collaborative TangHo Interaction

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Dynamic Data Physicalisation ▪ Physical objects used for input as well as output ▪ Dynamic data physicali- sation ▪ use physical variables such as temperature or texture ▪ exploration of big data sets ▪ dynamic data physicalisation framework - data physicalisation grammar

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 What is Wrong with Slideware? ▪ Simulation of physical slides ▪ limited space due to the slide concept ▪ linear navigation from slide to slide ▪ difficult to reuse content and embed rich media types ▪ MindXpres addresses these issues ▪ unlimited canvas with zoomable user interface ▪ non-linear navigation and associative linking ▪ content-based approach with automatic visualisation

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 MindXpres Presentation Platform ▪ Extensible prototyping platform for novel presen- tation concepts ▪ content-based approach - separation of content and presen- tation (automatic visualisation) ▪ cross-media content reuse ▪ non-linear navigation via zoomable user interface ▪ connectivity and interactivity ▪ Rich-media plug-ins ▪ e.g. source code or interactive data visualisation

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Interactive Source Code Plug-in Paul Mestereaga

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Personalised Learning Environments ▪ Suggestion of learning content ▪ knowledge graphs ▪ user knowledge graph ▪ learning paths ▪ Diagnostic assessments to detect knowledge gaps ▪ Automatic content adap- tation ▪ based on user preferences and constraints Enhanced Curriculum Assessments Tagged Learning Content Knowledge Graphs Adaptive Learning Content Learning Paths Knowledge Representation Framework Assessment Module Assessment Analyser Knowledge Gap Analyser Adaptive Learning Environment Suggestion Module User Knowledge Graph Personalised Presentation Module Learner Constraints Groups Module

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 RSL-based Hypermedia Engine RSL-based Hypermedia Engine Legacy Applications File System Driver Hypermedia-enabled Applications HT2021.docx HT2021.docx …

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 Models, Frameworks and Platforms dynamic data physicalisation framework RSL hypermedia metamodel eSPACE IoT platform iGesture framework OpenHPS hybrid positioning system MindXpres presentation platform

No content

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References [4] B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems, Proceedings of ER 2007, 26th International Conference on Conceptual Modeling, Auckland, New Zealand, November 2007 ▪ https://beatsigner.com/publications/signer_ER2007.pdf [8] R. Roels and B. Signer, MindXpres: An Exten- sible Content-driven Cross-Media Presentation Platform, Proceedings of WISE 2014, 15th Interna- tional Conference on Web Information System Engineering, Thessaloniki, Greece, October, 2014 ▪ https://beatsigner.com/publications/roels_WISE2014.pdf

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References ... [10] S. Trullemans and B. Signer, A Multi-layered Context Modelling Approach for End Users, Expert Users and Programmers, Proceedings of SERVE 2016, International Workshop on Smart Ecosystems cReation by Visual dEsign, Bari, Italy, June 2016 ▪ https://beatsigner.com/publications/trullemans_SERVE2016.pdf [14] S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross- Media Information Spaces, Proceedings of AVI 2016, International Working Conference on Advanced Visual Interfaces, Bari, Italy, June 2016 ▪ https://beatsigner.com/publications/trullemans_AVI2016.pdf

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References ... [15] B. Dumas, B. Moerman, S. Trullemans and B. Signer, ArtVis: Combining Advanced Visualisation and Tangible Interaction for the Exploration, Analysis and Browsing of Digital Artwork Collections, Proceed- ings of AVI 2014, Como, Italy, May 2014 ▪ https://beatsigner.com/publications/dumas_AVI2014.pdf [18] M. Van de Wynckel and B. Signer, OpenHPS: An Open Source Hybrid Positioning System, Technical Report WISE Lab, WISE-2020-01, December 2020 ▪ https://beatsigner.com/publications/vanDeWynckel_CoRR2020.pdf

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References ... [19] R. Roels, P. Mestereaga and B. Signer, An Interactive Source Code Visualisation Plug-in for the MindXpres Presentation Platform, Communications in Computer and Information Science (CCIS), 583, 2016 ▪ https://beatsigner.com/publications/roels_CCIS2016.pdf [20] Y. Malaise and B. Signer, Personalised Learning Environments Based on Knowledge Graphs and the Zone of Proximal Development, Proceedings of CSEDU 2022, International Conference on Computer Supported Education, Online Event, April 2022 ▪ https://beatsigner.com/publications/malaise_CSEDU2022.pdf

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References ... [15] B. Dumas, B. Moerman, S. Trullemans and B. Signer, ArtVis: Combining Advanced Visualisation and Tangible Interaction for the Exploration, Analysis and Browsing of Digital Artwork Collections, Proceed- ings of AVI 2014, Como, Italy, May 2014 ▪ https://beatsigner.com/publications/dumas_AVI2014.pdf [18] M. Van de Wynckel and B. Signer, OpenHPS: An Open Source Hybrid Positioning System, Technical Report WISE Lab, WISE-2020-01, December 2020 ▪ https://beatsigner.com/publications/vanDeWynckel_CoRR2020.pdf

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References ... [25] B. Signer, M. Grossniklaus and M.C. Norrie, Interactive Paper as a Mobile Client for a Multi- Channel Web Information System, World Wide Web Journal (WWW), Vol. 10, No. 4, Springer, December 2007 ▪ https://beatsigner.com/publications/signer_WWWJ2007.pdf [28] L. Hoste and B. Signer, Expressive Control of Indirect Augmented Reality During Live Music Performances, Proceedings of NIME 2013, 13th International Confe- rence on New Interfaces for Musical Expression, Daejeon, Korea Republic, May 2013 ▪ https://beatsigner.com/publications/hoste_NIME2013.pdf

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References ... [31] B. Signer and T.J. Curtin, Tangible Holograms: Towards Mobile Physical Augmentation of Virtual Objects, Technical Report WISE Lab, WISE-2017-01, March 2017 ▪ https://beatsigner.com/publications/signer_arXiv2017.pdf [32] B. Signer, P. Ebrahimi, T.J. Curtin and Ahmed K.A. Abdullah, Towards a Framework for Dynamic Data Physicalisation, International Workshop Toward a Design Language for Data Physicalisation, Berlin, Germany, October 2018 ▪ https://beatsigner.com/publications/signer_DataPhys2018.pdf

Beat Signer - Department of Computer Science - bsigner@vub.be June 9, 2022 References ... ▪ B. Signer, Towards Cross-Media Information Spaces and Architectures, Proceedings of RCIS 2019, Brussels, Belgium, May 2019 ▪ https://beatsigner.com/publications/signer_RCIS2019.pdf ▪ MindXpres Project ▪ https://mindxpres.com ▪ OpenHPS ▪ https://openhps.org ▪ iGesture ▪ http://igesture.org