The Remediation of Paper - Creating Affinities between Paper and Cross-Media Information Spaces

Inaugural lecture given at the Vrije Universiteit Brussel, Brussels, Belgium, April 2009

ABSTRACT: While there have been dramatic increases in the use of digital technologies for information storage, processing and delivery over the last few decades, the affordances of paper have ensured its retention as a key information medium. Despite predictions of the paperless office, paper is ever more present in our daily work. However, there is a gap between the paper and digital worlds: information present in paper documents cannot be transferred seamlessly to digital media and digital services are not easily accessible from the paper world.

In this seminar talk I will present an information-centric approach for integrating paper with digital as well as physical media based on a general cross-media information platform (iServer). Some details about the architecture and implementation of the iServer platform as well as the underlying RSL metamodel for cross-media linking will be highlighted. A selection of interactive paper applications that have been developed based on this platform over the past few years will be presented, including the EdFest interactive paper guide for the Edinburgh festivals and the PaperPoint tool for paper-driven PowerPoint presentations. Challenges and solutions for novel forms of interactive paper and cross-media publishing are discussed based on the presented applications.

The second part of my talk will address future directions for research on interactive paper and cross-media information management. This includes specific extensions of the iServer platform and RSL model as well as the application of our solution in new domains such as digital libraries, cross-media annotation and retrieval or personal cross-media information management that goes beyond the hierarchical information management imposed by the desktop metaphor.

Transcript

1. Cross-Media Information Spaces, April 22, 2009 The Remediation of Paper Beat Signer Department of Computer Science Vrije Universiteit Brussel Creating Affinities between Paper and Cross-Media Information Spaces
2. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be The Paperless Office
3. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be The Myth of the Paperless Office
4. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Bridging the Paper-Digital Divide
5. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be EdFest Project
6. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Digital Pen and Paper ▪ Anoto offers ▪ camera technology ▪ pattern license (virtual paper space of 60 million km2) ▪ Pen manufacturers ▪ Nokia ▪ Maxell ▪ Adapx ▪ Livescribe
7. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be EdFest Demo
8. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be PaperPoint Presentation Tool ▪ Mobile presentation tool ▪ Non-linear presentations ▪ Real-time annotations ▪ Digital whiteboard ▪ Multi-pen support ▪ brainstorming
9. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Applications ▪ Enhanced Reading ▪ interactive paper maps ▪ interactive festival brochures ▪ educational materials ▪ auction brochures ▪ scientific publications ▪ … ▪ Enhanced Writing ▪ photo album ▪ researcher's notebook ▪ mammography ▪ query by sketching ▪ … ▪ Paper-based Interfaces ▪ PaperPoint ▪ document proof-editing ▪ digital libraries ▪ video analysis tool ▪ educational games ▪ Artists' Installations ▪ Lost Cosmonaut ▪ Generosa Enterprise ▪ Interactive Tabletops ▪ photo browser ▪ collaborative sketch tool
10. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Research Link Metamodel (RSL) Tools Cross-Media Server / Publishing Applications
11. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Cross-Media Server (iServer)
12. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Active Page Areas and Layers image video clip
13. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be RSL Metamodel and iServer
14. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Links Users Layers Plug-ins Structure Active Content
15. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be iServer Resource Plug-ins Resource document page XHTML document mpeg file, avi file etc. mpeg file, avi file etc. mp3 file, wav file etc. gif file, jpeg file etc. RFID space ... Medium paper web page movie movie sound image physical object ... Selector shape XPointer time span shape time span shape RFID tag ...
16. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Active Components ▪ Execute code (services) on the client and server side Active Component A Active Component B Active Component C iServer AC Manager Server Request Sender Active Component Stub B AC Names ACS Manager Client Device AC Names Client HTTP
17. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Active Components … Button Slider Slider2D DateSelector Browser MapSelector Rating Capture CaptureAndSpeak Image Sound Movie SQLClient XCMRequest ClientCtrlRequest PaperPointControl PaperPointAnnot ...
18. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Cross-Media Authoring and Publishing ▪ Manual link authoring ▪ Content-driven publishing ▪ e.g. EdFest brochure ▪ Automatic link authoring ▪ e.g. Print-n-Link ▪ Collaborative authoring
19. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Manual Authoring Tool
20. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Cross-Media Publishing Framework XCM content publishing XML data of festival venues and events iServer cross-media link server • paper link data • active areas on paper • layers • digital resources • user model import publish PDF XML link definition data iPublish publishing framework Structure Style XSL CSS Information-centric approach
21. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be EdFest Architecture Overview iPaper Client XCM content publishing platform Metadata DB Appln DB iServer/iPaper cross-media link server Link DB Context Engine Context DB Client Controller Active Components Text-to-Speech Engine ICR Software handwriting recognition
22. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Print-n-Link ▪ Citations build the basis for a scientific web of publications ▪ Use digital services to search / retrieve cited publications ▪ Automatic authoring
23. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Collaborative Authoring consumers / publishers books publisher web pages videos presentation slides
24. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Paper-Digital Proof-Editing (PaperProof)
25. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be iGesture Framework ▪ Gesture recognition tool ▪ create/test gesture sets ▪ create/test algorithms ▪ Device independent ▪ digital pen, tablet PC, mouse, Wii remote, … ▪ Open Source ▪ initial release in Feb 2007 ▪ >4500 downloads http://www.igesture.org
26. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Rough Guide
27. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Interactive Table (iTable) Pen-Based Google Earth Browser Pen-Based Drawing Tool Photo Browser Photo Browser (Slider)
28. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Interactive Table (iTable) …
29. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Pulse Smartpen ▪ Record and replay synchronised ink & audio ▪ 1 GB memory ▪ 16'000 pages ▪ 200 hours voice recording ▪ 96x18 OLED Display ▪ Open application platform
30. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Ongoing and Future Work ▪ Cross-Media Information Spaces and Architectures (CISA) research team ▪ cross-media lab space ("meeting room of the future") ▪ General and extensible representations for cross-media information spaces ▪ interlinked structural components rather than document-centric representation of information ▪ data no longer "owned" by applications ▪ replacement of traditional hierarchical file systems
31. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be CISA Research ▪ New forms of cross-media publishing methodologies and tools ▪ global publishing infrastructures and architectures ▪ personal cross-media information management ▪ information sharing and collaborative authoring ▪ cross-media annotations ▪ Innovative multimodal user interfaces (digital and physical) to cross-media information spaces ▪ includes further research on interactive paper
32. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Conclusions ▪ Interactive paper framework (iPaper) ▪ information-centric approach supports different technologies, functionalities and applications ▪ rapid prototyping (reusable active components) ▪ separation of authoring and programming ▪ Extensible platform for cross-media information management (iServer) based on RSL metamodel ▪ Cross-media authoring and publishing tools ▪ manual, content-driven, automatic and collaborative
33. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Acknowledgements ▪ GlobIS group @ ETH Zurich ▪ in particular Prof. Moira C. Norrie, Nadir Weibel and Adriana Ispas ▪ Paper++ and PaperWorks members ▪ ... and many other collaborators in individual projects
34. Cross-Media Information Spaces, April 22, 2009 Beat Signer, bsigner@vub.ac.be Further Information ▪ http://vub.academia.edu/BeatSigner ▪ Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, Beat Signer, ISBN 978-3-8370-2713-6, May 2008