Y-Toolbox, April 17, 2008 Interactive Paper Introduction and Basic Technology Beat Signer Global Information Systems Research Group Department of Computer Science ETH Zurich 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] The Paperless Office (1975) 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] The "Paperless Office" (2003) 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] The Myth of the Paperless Office ▪ For decades, people have predicted the office of the future as a paperless office ▪ documents generated, published and distributed electronically ▪ documents read electronically ▪ What has happened to this imminent revolution? 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Affordances of Paper ▪ The physical properties of an object determine how people use that object ▪ Properties of paper ▪ light, flexible, robust, porous, opaque, transparent, …. ▪ Human actions ▪ grasping, folding, tearing, carrying, writing, on …. ▪ Paper supports forms of collaboration and interaction difficult to mimic in the digital world 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Paper in Everyday Settings 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Electronic Paper (e-paper) ▪ E-Ink Cooperation ▪ paper-like high contrast appearance ▪ low power consumption ▪ thin, light form 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Augmenting the Physical Space ▪ Embedding computing functionality in everyday objects instead of digitising the physical environment At breakfast Sal reads the news. She still prefers the paper form, as do most people. She spots an interesting quote from a columnist in the business section. She wipes her pen over the newspaper’s name, date, section, and page number and then circles the quote. The pen sends a message to the paper, which transmits the quote to her office. The Computer for the 21st Century, Weiser, 1991 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Paper-Digital Integration Technologies ▪ Document (object) identification / tracking ▪ barcodes ▪ RFID tags ▪ optical tracking ▪ … ▪ Within-document position tracking ▪ relative/absolute positions ▪ support for mobility ▪ … 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Listen Reader (Maribeth Back et al.) ▪ Combines the look and feel of a real book with an interactive soundtrack ▪ RFID tags embedded in each page ▪ Electric field sensors in the book binding 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Digital Pen and Paper ▪ Developed by Anoto ▪ Different pen manufacturers ▪ Logitech, Maxel, Nokia, Adapx, Livescribe ... ▪ Pattern space ▪ 60 million km2 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Forms Automation Systems ▪ Capture form data ▪ data is processed and stored in a database ▪ Applications ▪ policies ▪ hospitals ▪ transporation ▪ ... ▪ No real-time interaction! 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Active Areas and Virtual Page Layers image video clip 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Cross-Media Link Server (iServer) consumers / publishers books publisher web pages videos presentation slides 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] iPaper @ ETH Zurich Cross-Media Link Infrastructure Tools Publishing Applications 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Edinburgh Fringe Festival ▪ World's largest international arts festival ▪ 4 weeks ▪ ~1700 events ▪ ~27000 performances 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] EdFest Project 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] EdFest User Trials 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] EdFest Demo 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Design of Interactive Paper Documents 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] References ▪ The Myth of the Paperless Office, A.J. Sellen and R. Harper, MIT Press, November 2001 ▪ Scrolling Forward: Making Sense of Documents in the Digital Age, D.M. Levy, October 2001 ▪ Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, B. Signer, Diss ETH Zurich Nr. 16218, 2006 ▪ https://beatsigner.com/interactive-paper.html 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Interactive Paper Applications ▪ Enhanced Reading ▪ interactive paper maps ▪ interactive festival brochures ▪ educational materials ▪ auction brochures ▪ scientific publications ▪ … ▪ Enhanced Writing ▪ photo album ▪ researcher's notebook ▪ mammography ▪ search system ▪ … ▪ Paper-Based Interfaces ▪ PaperPoint ▪ digital libraries ▪ video analysis tool ▪ educational games ▪ Artists' Installations ▪ Lost Cosmonaut ▪ Generosa Enterprise ▪ Interactive TableTops http://www.globis.ethz.ch/research/paper/applications/ 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] PaperPoint Presentation Tool ▪ Mobile presentation tool ▪ Non-linear presentations ▪ Paper-based real-time annotations ▪ Digital whiteboard ▪ Multi-pen support ▪ brainstorming 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] PaperPoint Stickers ▪ Empty Stickers linked to specific PowerPoint slides ▪ Digital information (text, image, movie, ...) embedded in slides ▪ Mock-ups for interactive paper applications 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Rough Guide 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Natural History Museum, London 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Natural History Museum, London ... 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Natural History Museum, London ... 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Natural History Museum, London ... 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] DiDA (Diploma in Digital Applications) 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Mammography Annotations 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Print-n-Link ▪ Citations build the basis for a scientific web of publications ▪ Use digital services to search / retrieve cited publications ▪ Automatic authoring 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Print-n-Link … 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] PaperWeb ▪ Print arbitrary web pages with iDoc printer driver ▪ All hyperlinks become interactive on the printout version ▪ Automatic authoring of links 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Document Editing ▪ Digital and paper-based user interface ▪ Support for collaborative editing ▪ Multiple digital/physical editing iteration cycles Digital Document Printed Document 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] PaperProof 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Paper-Digital Structural Mapping ▪ Dual document representation in digital and physical document spaces 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] PaperProof 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] The Lost Cosmonaut ▪ Writing as a collaborative act of memory and story- telling ▪ Reading, writing and narrative as an act of making sense ▪ Handwriting as a tool for human computer interaction In collaboration with Axel Vogelsang, Artists in Labs Programme 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] The Lost Cosmonaut … 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] The Lost Cosmonaut … 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Generosa Enterprise ▪ Interactive drawing installation ▪ Exhibited at the 150 Years ETH Zurich Jubilee (more than 200'000 visitors) 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Generosa Enterprise … ▪ Generation of a new artwork 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Generosa Enterprise … 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Interactive Table (iTable) Pen-Based Google Earth Browser Pen-Based Drawing Tool Photo Browser Photo Browser (Slider) 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Interactive Table (iTable) … 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Remote Sketching on Paper ▪ Integration of Skype and paper-based remote sketching, Hermann Hofstetter ▪ HCI group University of Konstanz, H. Reiterer ▪ GlobIS group 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Query by Sketching (QbS) ▪ Pen-based sketching interface ▪ University of Basel ▪ GlobIS group 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] FLY Fusion™ Pentop Computer ▪ Everything integrated into the pen ▪ Voice feedback ▪ Applications on cartridge ▪ english to spanish translator ▪ calculator ▪ ... 

Y-Toolbox, April 17, 2008 Beat Signer, [email protected] Pulse Smartpen ▪ Record and replay synchronised ink & audio ▪ 1 GB memory ▪ 16'000 pages ▪ 200 hours voice recording ▪ 96x18 OLED Display ▪ Open application platform