iPaper@GlobIS - Interactive Paper Research

Presentation given at CSCW Seminar, ETH Zurich.

Transcript

1. CSCW Seminar 2007, June 12, 2007 iPaper@GlobIS – Interactive Paper Research CSCW Seminar Beat Signer, Nadir Weibel, Adriana Ispas and Moira C. Norrie Global Information Systems Research Group Institute for Information Systems Department of Computer Science, ETH Zurich
2. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Problems of Existing Applications ▪ Many projects focus on the input device, paper, printing and other hardware technologies rather than on the data integration and information management aspects → isolated solutions ▪ The linking of paper tends to be based on physical rather than information-centric concepts → difficult to integrate new input / output devices
3. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Our Approach ▪ General interactive framework ▪ different forms of paper-based interactions (enhanced reading, enhanced writing, …) ▪ rapid prototyping as well as large scale publishing ▪ integration of new types of resources and services ▪ integration of new input devices ▪ The key to a highly integrated interactive paper solution lies in the introduction of a platform for general cross-media information management
4. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Active Areas and Virtual Page Layers image video clip
5. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iPaper Architecture
6. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iServer Metamodel and Resource Plug-ins iPaper Shape Page Application Database DB Object Data/Metadata Query Movie Clips Time Span / Spatial Info Movie Clip HTML Documents XPointer XHTML Document Selectors Entities Resources link
7. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iServer layer |Layers| layer Active Layers resource Containers activeComponent Active Com- ponents medium Media medium Movies medium Images medium Webpages OnLayer Contains (1,1) (1,1) (0,*) (0,*) document Documents (0,*) shape Shapes partition (2,*) (0,*) rectangle Rectangles ellipse Ellipses polygon Polygons circle Circles complexShape Complex Shapes HasShapes page Pages OnPage Contains Pages (0,*) (1,1) (1,1) (0,*) Resources parameter Preferences link Links user Users group Groups individual Individuals HasMembers HasTarget (0,*) (0,*) (0,*) (0,*) (0,*) (0,*) (0,*) (1,*) (0,*) (0,*) (1,1) (0,*) partition partition (1,*) CreatedBy RefersTo Has Preferences parameter Properties selector Selectors resource Resources HasSource (0,*) AccessibleTo InaccessibleTo (0,*) (0,*) entity Entities HasProperties Users Plug-ins Links (0,*) Layers iPAPER iSERVER
8. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iServer Interfaces ▪ Java API ▪ XML import and export ▪ Interface based on XML and XSLT ▪ XHTML ▪ WAP, i-mode ▪ VoiceXML ▪ Web Service ▪ Peer-to-peer (P2P)
9. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iWeb Firefox Extension
10. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iWeb Plug-in Firefox Web Client SOAP Request SOAP Response World Wide Web HTTP Request HTTP Response Link DB iWeb Plug-in iServExt
11. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Active Content ▪ Execute code (services) on the client and server side Active Component A Active Component B Active Component C iServer AC Manager Server Request Sender Active Component Stub B AC Names ACS Manager Client Device AC Names Client HTTP
12. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Active Components Button Slider Slider2D DateSelector Browser MapSelector Rating Capture CaptureAndSpeak Image Sound Movie SQLClient XCMRequest ClientCtrlRequest PaperPointControl PaperPointAnnot ...
13. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Active Components … <?xml version="1.0" encoding="UTF-8" ?> <iserver> <activeComponent id="s1a" creator="beat"> <name>Show slide 1</name> <properties> <parameter> <key>org.ximtec.iserver.ac:command</key> <value>showSlide</value> </parameter> <parameter> <key>org.ximtec.iserver.ac:slide</key> <value>1</value> </parameter> </properties> <identifier>PAPERPOINT_CONTROL</identifier> </activeComponent> ... </iserver> PAPERPOINT_CONTROL Show slide 1 command showSlide slide 1
14. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch PaperPoint Presentation Tool
15. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch XML-Based Authoring ... <document id="handout" creator="beat"> <name>Slide Handout</name> <id>h1</id> <size> <width>210</width> <height>297</height> </size> <content>handout.pdf</content> </document> <page id="p1" creator="beat" document="h1"> <name>Page 1, Slide Controller</name> <number>1</number> </page> ... <rectangle id="fs" creator="beat" layer="default" resource="p1"> <name>First Slide</name> <upperLeft> <point><x>24</x><y>12</y></point> </upperLeft> <size> <width>25</width><height>8</height> </size> </rectangle> ... ... <activeComponent id="fsa" creator="beat"> <name>First slide action</name> <properties> <parameter> <key>org.ximtec.iserver.ac:command</key> <value>firstSlide</value> </parameter> </properties> <identifier>PAPERPOINT_CONTROL</identifier> </activeComponent> ... <link id="fsl" creator="beat" sources="fs" targets="fsa"> <name>Show the first slide</name> </link> ...
16. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Digital Link Authoring Tool
17. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch PaperPoint Architecture
18. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch EdFest Components
19. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch EdFest Architecture Overview iPaper Client XCM content publishing platform Metadata DB Appln DB iServer/iPaper cross-media link server Link DB Context Engine Context DB Client Controller Active Components Text-to-Speech Engine ICR Software handwriting recognition
20. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Content Publishing Framework XCM content publishing XML data of festival venues and events iServer cross-media link server • paper link data • active areas on paper • layers • digital resources • user model import publish PDF XML link definition data iPublish publishing framework Structure Style XSL CSS
21. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Print-n-Link ▪ Citations build the basis for a scientific web of publications ▪ Use digital services to search / retrieve cited publications ▪ Automatic authoring
22. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Print-n-Link Architecture Display Printer Driver Document DB iPAPER iDOC Publish Document Pattern Semantic Mapper Citation Analyser Citation Retrieval Document Lookup Citation DB Print-n-Link Engine iSERVER Active Content Link DB iPaper Plug-in Publish Positions and Active Content Definition Publish Citations Desktop Print (x,y) Coordinates Text-to- Speech Web Search Services iPaper Client Print-n-Link: Weaving the Paper Web Moira C. Norrie Dept. of Computer Science ETH Zurich 8092 Zurich, Switzerland norrie@inf.ethz.ch Beat Signer Dept. of Computer Science ETH Zurich 8092 Zurich, Switzerland signer@inf.ethz.ch Nadir Weibel Dept. of Computer Science ETH Zurich 8092 Zurich, Switzerland weibel@inf.ethz.ch ABSTRACT Citations form the basis for a web of scientific publications. Search engines, embedded hyperlinks and digital libraries all simplify the task of finding publications of interest on the web and navigating to cited publications or web sites. However the actual reading of publications often takes place on paper and frequently on the move. We present a system Print-n-Link that uses technologies for interactive paper to enhance the reading process by enabling users to access dig- ital information and/or searches for cited documents from a printed version of a publication using a digital pen for inter- action. A special virtual printer driver automatically gener- ates links from paper to digital services during the printing process based on an analysis of PDF documents. Depend- ing on the user setting and interaction gesture, the system may retrieve metadata about the citation and inform the user through an audio channel or directly display the cited document on the user’s screen. Categories and Subject Descriptors D.2.11 [ Software Engineering ]: Software Architectures; H.4.m [ Information Systems Applications ]: Miscella- neous; H.5.m [ Information Interfaces and Presenta- tion ]: Miscellaneous General Terms Design, Algorithms, Experimentation Keywords Interactive paper, document integration, digital library, ci- tation management 1. INTRODUCTION Nowadays we are all familiar with the notion of hyperlinks as a means of creating interconnected document spaces that can be easily searched and navigated with the help of search Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. DocEng’06, October 10–13, 2006, Amsterdam, The Netherlands. Copyright 2006 ACM 1-59593-515-0/06/0010 ...$5.00. engines and web browsers. The ease with which we can not only find documents on the web, but also follow links to related documents is considered to be one of the main advantages of working with digital documents as opposed to paper equivalents. Y et many studies have shown that paper is still the preferred med ium when it comes to reading documents for reasons of comfort, mobility and also ease of annotation. In this paper, we show how emerging technologies for in- teractive paper enable a bridge to be built between paper and digital document spaces that allows users to travel back and forth between the two using the medium most appropri- ate to the current task and sit uation. Users can print doc- uments for reading and, by activating hyperlinks on paper with a digital pen, they can not only access digital informa- tion about related documents, but also initiate web searches to retrieve copies of those documents. The retrieved docu- mentsmay in turn be printed for reading with automatically generated embedded hyperlinks on the paper. Mobility is supported by providing disconnected userson the move with audio information about related documents and the option of adding them to a request list to be processed when recon- nected. As an application, we take the everyday academic task of reading scientific publications. Whether the reading activ- ity is for the purpose of knowledge discovery or reviewing, readers regularly look up citations to other publications by flicking to the reference list at the back of the paper. In some cases, a search for the cited publication may follow and, if found, often it is printed and added to a pile of documentsto be read. Citationsare the established meansof providing the links that form the basis of the scientific web of publications that spans the digital and paper worlds. Just as citations within digital documents can be used to automatically gen- erate hyperlinks to cited documents, we can automatically generate links within paper documents through an analysis of the PDF file when the document is printed. Section 2 presents a more detailed look at the activity of reading scientific publications and a discussion of related work. Section 3 describes the digital pen and paper tech- nologies and how these can be used to support interaction between paper and digital services. We then present the general functionality of our enhanced reading system in Sec- tion 4 and the overall architecture in Section 5. Details of the virtual printer component used to analyse PDF docu- ments for automatic link generation as well as printing are given in Section 6. Concluding remarks follow in Section 7. Interactive Paper Architecture Printing Semantic Mapper Document Retrieval
23. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Printed-Digital Mapping static physical rendering dynamic logical document
24. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Printed-Digital Mapping Source Digital Document Paginated (Print-Ready) Document Enhanced Virtual Printer Interactive Document (e.g. Anoto) Augmenting Technology (ex. Anoto) Geometrical Shapes, Active Components Logical Representation Physical Representation iServer (ID, page, x,y) iDoc Digital Elements
25. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Mixed Physical-Digital Model
26. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Interactive Paper Applications ▪ Enhanced Reading ▪ interactive paper maps ▪ interactive festival brochures ▪ educational materials ▪ auction brochures ▪ scientific publications ▪ … ▪ Enhanced Writing ▪ photo album ▪ researcher's notebook ▪ mammography ▪ search system ▪ … ▪ Paper-Based Interfaces ▪ PaperPoint ▪ digital libraries ▪ video analysis tool ▪ educational games ▪ Artists' Installations ▪ Lost Cosmonaut ▪ Generosa Enterprise ▪ Interactive TableTops http://www.globis.ethz.ch/research/paper/applications/
27. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Mammography Annotations
28. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch The Lost Cosmonaut ▪ Writing as a collaborative act of memory and story- telling ▪ Reading, writing and narrative as an act of making sense ▪ Handwriting as a tool for human computer interaction In collaboration with Axel Vogelsang, Artists in Labs Programme
29. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Generosa Enterprise ▪ Interactive drawing installation ▪ Exhibited at the 150 Years ETH Zurich Jubilee (more than 200'000 visitors)
30. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iGesture Framework ▪ Common interface for different gesture recognition algorithms ▪ Tools for implementing and evaluating new algorithms ▪ Gesture set management
31. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch iGesture Admin Tab
32. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Capture of Samples
33. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Conclusions ▪ Interactive paper framework (iPaper) ▪ information-centric approach supports different technologies, functionalities and applications ▪ rapid prototyping (reusable active components) ▪ Extensible platform for cross-media information management (iServer) ▪ Cross-media authoring / publishing tools ▪ manual, content-driven and automatic link authoring
34. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch Student Projects ▪ Basic technologies and infrastructure ▪ iServer resource plug-ins, … ▪ Integration of new tracking technologies ▪ Interactive paper authoring tools ▪ iGesture recognition framework (www.igesture.org) ▪ Paper-based applications ▪ PaperPoint, paper-based interface to digital agenda/todo list (e.g. Outlook), paper-digital notebook, paper-digital photo album, … 34 We offer Master and Semester projects and are also looking for Hilfsassistenten
35. CSCW Seminar 2007, June 12, 2007 Beat Signer, signer@inf.ethz.ch References ▪ http://www.ipaper.inf.ethz.ch ▪ http://www.igesture.org ▪ Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, B. Signer, Diss ETH Zurich Nr. 16218, 2006 35