2 December 2005 Open Cross-Document Linking and Browsing based on a Visual Plug-in Architecture Ahmed A.O.Tayeh and Beat Signer Web & Information Systems Engineering Lab (WISE) Department of Computer Science Vrije Universiteit Brussel WEB & INFORMATION SYSTEMS ENGINEERING 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Isolated Digital Documents × 1 HTML document PDF document 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Isolated Digital Documents … 2 × × × Word document PDF document 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Isolated Digital Documents …  Limited possibilities to create hyperlinks between snippets of different document formats  document formats mainly address links to web resources (HTML)  simple embedded unidirectional links  linked documents are not aware of any links that have been defined from other source documents 3 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Existing Support for Hyperlinks 4 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Cross-Document Linking Anchor (page index + ec(x,y,w,h)) Anchor (XPointer-like expression) Anchor (start and end indices) Anchor(XPointer expression) 5 PDF (linear-like model) HTML (tree model) Text (linear model) XML (tree model) 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Open Hypermedia  No support for cross-document linking  Limited extensibility  on the data as well as on the visual level 6 MADCOW: A Multimedia Digital Annotation System, Paolo Bottoni et al., Proceedings of AVI 2004 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Requirements for a Linking Service  No changes required to existing document formats and standards  No assumptions about the document models  e.g. WYSIWYG, tree models, constrained tree models, …  Support existing as well as emerging document formats  Support advanced linking features 7 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Open Cross-Media Architecture 9 Annotation/Link Service Client Application Annotation/Link Browser & Editor Visual Plug-ins Resource Plug-in Repository Data Plug-ins Visual Plug-ins Data Plug-ins Visual Plug-ins An Architecture for Open Cross-Media Annotation Services, Beat Signer and Moria C. Norrie, Proceeding of WISE 2009 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Data Level Extensibility (RSL Approach) As We May Link: A General Metamodel for Hypermedia Systems, Beat Signer and Moria C. Norrie, Proceeding of ER 2007 … … 11 file:// URL file:// URL … shape XPointer (s,e) XPointer … 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Visualisation Layer Extensibility 12 local visual plug-ins external visual plug-ins communicate Link Browser 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Visualisation Layer Extensibility …  Local visual plug-ins vs. external visual plug-ins Link Browser Link visual plug-in Link visual plug-in 13 Acrobat Reader Google Chrome 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Visualisation Layer Extensibility …  DefaultDocument class  necessary methods to visualise any document format - getSelector() - openDocument() - …  event listeners and handlers  Local visual plug-ins  extend DefaultDocument class  might use existing visualisation libraries  Link browser  instantiates DefaultDocument 14 DocFormat1 DocFormat3 DocFormat2 Visual Plug-ins Visualisation Gateway DocFormat3 DocFormat3 Application Third-Party Applications Visualisation DocFormat1 Visualisation DocFormat2 Browser 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Visualisation Layer Extensibility …  External visual plug-ins  provide methods to handle selections in third-party applications  communicate with the link browser via a specific gateway plug-in  Gateway plug-ins  launch third-party application  offer different communication protocols - TCP sockets - WebSockets - REST API as a fallback DocFormat1 DocFormat3 DocFormat2 Visual Plug-ins Visualisation Gateway DocFormat3 DocFormat3 Application Third-Party Applications Visualisation DocFormat1 Visualisation DocFormat2 Browser 15 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Open Cross-Document Link Service Architecture DocFormat3 Database DocFormat2 DocFormat1 DocFormat1 DocFormat3 DocFormat2 Visual Plug-ins Visualisation Gateway DocFormat3 RSL Database Manager DocFormat3 Application Third-Party Applications Data Plug-ins Core Data Layer Visualisation DocFormat1 Visualisation DocFormat2 Browser 16 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Open Service Gateway initiative (OSGi)  Enhances modularisation of our linking service  Use OSGi dynamic extensibility for dynamic extensibility of the linking service  OSGi versioning and dependency resolution reduces problems in dealing with multiple libraries (“JAR hell”)  linking service might provide different visual plug-ins (versions) for a single document format 17 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Open Service Gateway initiative (OSGi) … User Interface Manifest Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: UserInterface Bundle-SymbolicName: com.rsl.userInterface Bundle-Version: 1.0.0.qualifier Bundle-RequiredExecutionEnvironment: JavaSE-1.7 Export-Package: org.rsl.userInterface Import-Package: org.rsl.core, org.rsl.service, org.rsl.databasemanager, org.associations.collections, org.osgi.framework;version="1.7.0" PDF Local Visual Plug-in Manifest Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Visual Bundle-SymbolicName: org.rsl.pdf.visual Bundle-Version: 1.0.0.qualifier Bundle-RequiredExecutionEnvironment: JavaSE-1.7 Import-Package: org.rsl.core, org.rsl.userInterface, org.rsl.pdf.data Extension-Class: org.rsl.pdf.visual.Pdf Extension-Type: visual Extension-Format: application/pdf 18 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Link Navigation Scenario Browser RSL Registry DocFormat1 Gateway DocFormat3 DocFormat2 DocFormat3 getSelection() selector alt getPluginType(formatName) instantiateClass() openDocument(resource, selectors, selector) instantiateClass() openDocument(resource, selectors, selector) openDocument(resource, selectors, selector) launchApp() [visualPlugin==local] [else] getLinkTarget(selector) getDocument(resource) 19 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Evaluation: PDF, XML, Text and HTML Plug-ins 20 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 Conclusion and Future Work  Cross-document link service prototype  Extensibility on the data as well as the visualisation layer  OSGi framework for extensible architecture  Evaluation with PDF, XML, Text and HTML plug-ins  Dynamic plug-in extensibility  Investigate extensibility in a study with developers  Usability evaluation 21 

Ahmed Tayeh - Department of Computer Science - [email protected] October 14, 2014 References  A.A.O Tayeh and Beat Signer, Open Cross- Document Linking and Browsing based on a Visual Plug-in Architecture, Proceedings of WISE 2014,15th International Conference on Web Information System Engineering, Thessaloniki, Greece, October, 2014  http://wise.vub.ac.be/sites/default/files/publications/WISE2014.pdf