Open Cross-Document Linking and Browsing Based on a Visual Plug-in Architecture

Presentation given at WISE 2014, 15th International Conference on Web Information System Engineering, Thessaloniki, Greece.

ABSTRACT: Digital documents often do not exist in isolation but are implicitly or explicitly linked to parts of other documents. Nevertheless,most existing document formats only support links to web resources but not to parts of third-party documents. An open cross-document link service should address the multitude of existing document formats and be extensible to support emerging document formats and models. We present an architecture and prototype of an open cross-document link service and browser that is based on the RSL hypermedia metamodel. A main contribution is the specification and development of a visual plug-in solution that enables the integration of new document formats without requiring changes to the cross-document browser’s main user interface component. The presented visual plug-in mechanism makes use of the Open Service Gateway initiative (OSGi) specification for modularisation and plug-in extensibility and has been validated by developing data as well as visual plug-ins for a number of existing document formats.

Research paper: https://beatsigner.com/publications/open-cross-document-linking-and-browsing-based-on-a-visual-plug-in-architecture.pdf

Transcript

1. 2 December 2005
   Open Cross-Document Linking and Browsing
   based on a Visual Plug-in Architecture
   Ahmed A.O.Tayeh and Beat Signer
   Web & Information Systems Engineering Lab (WISE)
   Department of Computer Science
   Vrije Universiteit Brussel
   WEB & INFORMATION
   SYSTEMS ENGINEERING
   

   View Slide

2. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Isolated Digital Documents
   ×
   1
   HTML document PDF document
   

   View Slide

3. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Isolated Digital Documents …
   2
   ×
   ×
   ×
   Word document PDF document
   

   View Slide

4. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Isolated Digital Documents …
    Limited possibilities to create hyperlinks between snippets
   of different document formats
    document formats mainly address links to web resources (HTML)
    simple embedded unidirectional links
    linked documents are not aware of any links that have been
   defined from other source documents
   3
   

   View Slide

5. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Existing Support for Hyperlinks
   4
   

   View Slide

6. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Cross-Document Linking
   Anchor (page index + ec(x,y,w,h)) Anchor (XPointer-like expression)
   Anchor (start and end indices) Anchor(XPointer expression)
   5
   PDF (linear-like model) HTML (tree model)
   Text (linear model) XML (tree model)
   

   View Slide

7. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Open Hypermedia
    No support for cross-document linking
    Limited extensibility
    on the data as well as
   on the visual level
   6
   MADCOW: A Multimedia Digital Annotation System,
   Paolo Bottoni et al., Proceedings of AVI 2004
   

   View Slide

8. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Requirements for a Linking Service
    No changes required to existing document formats and
   standards
    No assumptions about the document models
    e.g. WYSIWYG, tree models, constrained tree models, …
    Support existing as well as emerging document formats
    Support advanced linking features
   7
   

   View Slide

9. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
   October 14, 2014
   Open Cross-Media Architecture
   9
   Annotation/Link
   Service
   Client Application
   Annotation/Link
   Browser & Editor
   Visual Plug-ins
   Resource Plug-in Repository
   Data Plug-ins
   Visual Plug-ins Data Plug-ins
   Visual Plug-ins
   An Architecture for Open Cross-Media Annotation Services,
   Beat Signer and Moria C. Norrie, Proceeding of WISE 2009
   

   View Slide

10. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Data Level Extensibility (RSL Approach)
    As We May Link: A General Metamodel for Hypermedia Systems,
    Beat Signer and Moria C. Norrie, Proceeding of ER 2007
    …
    …
    11
    file://
    URL
    file://
    URL
    …
    shape
    XPointer
    (s,e)
    XPointer
    …
    

    View Slide

11. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Visualisation Layer Extensibility
    12
    local visual plug-ins
    external visual plug-ins
    communicate
    Link Browser
    

    View Slide

12. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Visualisation Layer Extensibility …
     Local visual plug-ins vs. external visual plug-ins
    Link Browser
    Link visual plug-in Link visual plug-in
    13
    Acrobat Reader Google Chrome
    

    View Slide

13. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Visualisation Layer Extensibility …
     DefaultDocument class
     necessary methods to visualise
    any document format
    - getSelector()
    - openDocument()
    - …
     event listeners and handlers
     Local visual plug-ins
     extend DefaultDocument class
     might use existing visualisation libraries
     Link browser
     instantiates DefaultDocument
    14
    DocFormat1 DocFormat3 DocFormat2
    Visual Plug-ins
    Visualisation
    Gateway DocFormat3
    DocFormat3 Application
    Third-Party Applications
    Visualisation
    DocFormat1
    Visualisation
    DocFormat2
    Browser
    

    View Slide

14. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Visualisation Layer Extensibility …
     External visual plug-ins
     provide methods to handle selections
    in third-party applications
     communicate with the link browser
    via a specific gateway plug-in
     Gateway plug-ins
     launch third-party application
     offer different communication
    protocols
    - TCP sockets
    - WebSockets
    - REST API as a fallback
    DocFormat1 DocFormat3 DocFormat2
    Visual Plug-ins
    Visualisation
    Gateway DocFormat3
    DocFormat3 Application
    Third-Party Applications
    Visualisation
    DocFormat1
    Visualisation
    DocFormat2
    Browser
    15
    

    View Slide

15. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Open Cross-Document Link Service Architecture
    DocFormat3
    Database
    DocFormat2
    DocFormat1
    DocFormat1 DocFormat3 DocFormat2
    Visual Plug-ins
    Visualisation
    Gateway DocFormat3
    RSL
    Database Manager
    DocFormat3 Application
    Third-Party Applications
    Data Plug-ins
    Core
    Data Layer
    Visualisation
    DocFormat1
    Visualisation
    DocFormat2
    Browser
    16
    

    View Slide

16. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Open Service Gateway initiative (OSGi)
     Enhances modularisation of our linking service
     Use OSGi dynamic extensibility for dynamic extensibility
    of the linking service
     OSGi versioning and dependency resolution reduces
    problems in dealing with multiple libraries (“JAR hell”)
     linking service might provide different visual plug-ins (versions) for
    a single document format
    17
    

    View Slide

17. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Open Service Gateway initiative (OSGi) …
    User Interface Manifest
    Manifest-Version: 1.0
    Bundle-ManifestVersion: 2
    Bundle-Name: UserInterface
    Bundle-SymbolicName: com.rsl.userInterface
    Bundle-Version: 1.0.0.qualifier
    Bundle-RequiredExecutionEnvironment: JavaSE-1.7
    Export-Package: org.rsl.userInterface
    Import-Package: org.rsl.core, org.rsl.service,
    org.rsl.databasemanager,
    org.associations.collections,
    org.osgi.framework;version="1.7.0"
    PDF Local Visual Plug-in Manifest
    Manifest-Version: 1.0
    Bundle-ManifestVersion: 2
    Bundle-Name: Visual
    Bundle-SymbolicName: org.rsl.pdf.visual
    Bundle-Version: 1.0.0.qualifier
    Bundle-RequiredExecutionEnvironment: JavaSE-1.7
    Import-Package: org.rsl.core, org.rsl.userInterface,
    org.rsl.pdf.data
    Extension-Class: org.rsl.pdf.visual.Pdf
    Extension-Type: visual
    Extension-Format: application/pdf
    18
    

    View Slide

18. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Link Navigation Scenario
    Browser RSL Registry DocFormat1 Gateway DocFormat3 DocFormat2 DocFormat3
    getSelection()
    selector
    alt
    getPluginType(formatName)
    instantiateClass()
    openDocument(resource, selectors, selector)
    instantiateClass()
    openDocument(resource, selectors, selector)
    openDocument(resource, selectors, selector)
    launchApp()
    [visualPlugin==local]
    [else]
    getLinkTarget(selector)
    getDocument(resource)
    19
    

    View Slide

19. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Evaluation: PDF, XML, Text and HTML Plug-ins
    20
    

    View Slide

20. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    Conclusion and Future Work
     Cross-document link service prototype
     Extensibility on the data as well as the visualisation layer
     OSGi framework for extensible architecture
     Evaluation with PDF, XML, Text and HTML plug-ins
     Dynamic plug-in extensibility
     Investigate extensibility in a study with developers
     Usability evaluation
    21
    

    View Slide

21. Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
    October 14, 2014
    References
     A.A.O Tayeh and Beat Signer, Open Cross-
    Document Linking and Browsing based on a Visual
    Plug-in Architecture, Proceedings of WISE 2014,15th
    International Conference on Web Information System
    Engineering, Thessaloniki, Greece, October, 2014
     http://wise.vub.ac.be/sites/default/files/publications/WISE2014.pdf
    

    View Slide