Open Cross-Document Linking and Browsing Based on a Visual Plug-in Architecture

Presentation given at WISE 2014, 15th International Conference on Web Information System Engineering, Thessaloniki, Greece.

ABSTRACT: Digital documents often do not exist in isolation but are implicitly or explicitly linked to parts of other documents. Nevertheless,most existing document formats only support links to web resources but not to parts of third-party documents. An open cross-document link service should address the multitude of existing document formats and be extensible to support emerging document formats and models. We present an architecture and prototype of an open cross-document link service and browser that is based on the RSL hypermedia metamodel. A main contribution is the specification and development of a visual plug-in solution that enables the integration of new document formats without requiring changes to the cross-document browser’s main user interface component. The presented visual plug-in mechanism makes use of the Open Service Gateway initiative (OSGi) specification for modularisation and plug-in extensibility and has been validated by developing data as well as visual plug-ins for a number of existing document formats.

Research paper: https://beatsigner.com/publications/open-cross-document-linking-and-browsing-based-on-a-visual-plug-in-architecture.pdf

Transcript

1. 2 December 2005 Open Cross-Document Linking and Browsing based on a Visual Plug-in Architecture Ahmed A.O.Tayeh and Beat Signer Web & Information Systems Engineering Lab (WISE) Department of Computer Science Vrije Universiteit Brussel WEB & INFORMATION SYSTEMS ENGINEERING
2. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Isolated Digital Documents × 1 HTML document PDF document
3. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Isolated Digital Documents … 2 × × × Word document PDF document
4. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Isolated Digital Documents …  Limited possibilities to create hyperlinks between snippets of different document formats  document formats mainly address links to web resources (HTML)  simple embedded unidirectional links  linked documents are not aware of any links that have been defined from other source documents 3
5. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Existing Support for Hyperlinks 4
6. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Cross-Document Linking Anchor (page index + ec(x,y,w,h)) Anchor (XPointer-like expression) Anchor (start and end indices) Anchor(XPointer expression) 5 PDF (linear-like model) HTML (tree model) Text (linear model) XML (tree model)
7. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Open Hypermedia  No support for cross-document linking  Limited extensibility  on the data as well as on the visual level 6 MADCOW: A Multimedia Digital Annotation System, Paolo Bottoni et al., Proceedings of AVI 2004
8. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Requirements for a Linking Service  No changes required to existing document formats and standards  No assumptions about the document models  e.g. WYSIWYG, tree models, constrained tree models, …  Support existing as well as emerging document formats  Support advanced linking features 7
9. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Open Cross-Media Architecture 9 Annotation/Link Service Client Application Annotation/Link Browser & Editor Visual Plug-ins Resource Plug-in Repository Data Plug-ins Visual Plug-ins Data Plug-ins Visual Plug-ins An Architecture for Open Cross-Media Annotation Services, Beat Signer and Moria C. Norrie, Proceeding of WISE 2009
10. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Data Level Extensibility (RSL Approach) As We May Link: A General Metamodel for Hypermedia Systems, Beat Signer and Moria C. Norrie, Proceeding of ER 2007 … … 11 file:// URL file:// URL … shape XPointer (s,e) XPointer …
11. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Visualisation Layer Extensibility 12 local visual plug-ins external visual plug-ins communicate Link Browser
12. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Visualisation Layer Extensibility …  Local visual plug-ins vs. external visual plug-ins Link Browser Link visual plug-in Link visual plug-in 13 Acrobat Reader Google Chrome
13. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Visualisation Layer Extensibility …  DefaultDocument class  necessary methods to visualise any document format - getSelector() - openDocument() - …  event listeners and handlers  Local visual plug-ins  extend DefaultDocument class  might use existing visualisation libraries  Link browser  instantiates DefaultDocument 14 DocFormat1 DocFormat3 DocFormat2 Visual Plug-ins Visualisation Gateway DocFormat3 DocFormat3 Application Third-Party Applications Visualisation DocFormat1 Visualisation DocFormat2 Browser
14. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Visualisation Layer Extensibility …  External visual plug-ins  provide methods to handle selections in third-party applications  communicate with the link browser via a specific gateway plug-in  Gateway plug-ins  launch third-party application  offer different communication protocols - TCP sockets - WebSockets - REST API as a fallback DocFormat1 DocFormat3 DocFormat2 Visual Plug-ins Visualisation Gateway DocFormat3 DocFormat3 Application Third-Party Applications Visualisation DocFormat1 Visualisation DocFormat2 Browser 15
15. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Open Cross-Document Link Service Architecture DocFormat3 Database DocFormat2 DocFormat1 DocFormat1 DocFormat3 DocFormat2 Visual Plug-ins Visualisation Gateway DocFormat3 RSL Database Manager DocFormat3 Application Third-Party Applications Data Plug-ins Core Data Layer Visualisation DocFormat1 Visualisation DocFormat2 Browser 16
16. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Open Service Gateway initiative (OSGi)  Enhances modularisation of our linking service  Use OSGi dynamic extensibility for dynamic extensibility of the linking service  OSGi versioning and dependency resolution reduces problems in dealing with multiple libraries (“JAR hell”)  linking service might provide different visual plug-ins (versions) for a single document format 17
17. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Open Service Gateway initiative (OSGi) … User Interface Manifest Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: UserInterface Bundle-SymbolicName: com.rsl.userInterface Bundle-Version: 1.0.0.qualifier Bundle-RequiredExecutionEnvironment: JavaSE-1.7 Export-Package: org.rsl.userInterface Import-Package: org.rsl.core, org.rsl.service, org.rsl.databasemanager, org.associations.collections, org.osgi.framework;version="1.7.0" PDF Local Visual Plug-in Manifest Manifest-Version: 1.0 Bundle-ManifestVersion: 2 Bundle-Name: Visual Bundle-SymbolicName: org.rsl.pdf.visual Bundle-Version: 1.0.0.qualifier Bundle-RequiredExecutionEnvironment: JavaSE-1.7 Import-Package: org.rsl.core, org.rsl.userInterface, org.rsl.pdf.data Extension-Class: org.rsl.pdf.visual.Pdf Extension-Type: visual Extension-Format: application/pdf 18
18. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Link Navigation Scenario Browser RSL Registry DocFormat1 Gateway DocFormat3 DocFormat2 DocFormat3 getSelection() selector alt getPluginType(formatName) instantiateClass() openDocument(resource, selectors, selector) instantiateClass() openDocument(resource, selectors, selector) openDocument(resource, selectors, selector) launchApp() [visualPlugin==local] [else] getLinkTarget(selector) getDocument(resource) 19
19. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Evaluation: PDF, XML, Text and HTML Plug-ins 20
20. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 Conclusion and Future Work  Cross-document link service prototype  Extensibility on the data as well as the visualisation layer  OSGi framework for extensible architecture  Evaluation with PDF, XML, Text and HTML plug-ins  Dynamic plug-in extensibility  Investigate extensibility in a study with developers  Usability evaluation 21
21. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be October 14, 2014 References  A.A.O Tayeh and Beat Signer, Open Cross- Document Linking and Browsing based on a Visual Plug-in Architecture, Proceedings of WISE 2014,15th International Conference on Web Information System Engineering, Thessaloniki, Greece, October, 2014  http://wise.vub.ac.be/sites/default/files/publications/WISE2014.pdf