Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content Composition and Reuse

Beat Signer
November 04, 2010

What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content Composition and Reuse

Presentation given at ER 2010, 29th International Conference on Conceptual Modeling, Vancouver, Canada.

ABSTRACT: Many of today’s digital document formats are strongly based on a digital emulation of printed media. While such a paper simulation
might be appropriate for the visualisation of certain digital content, it is generally not the most effective solution for digitally managing and storing information. The oversimplistic modelling of digital documents as monolithic blocks of linear content, with a lack of structural semantics, does not pay attention to some of the superior features that digital media offers in comparison to traditional paper documents. For example, existing digital document formats adopt the limitations of paper documents by unnecessarily replicating content via copy and paste operations, instead of digitally embedding and reusing parts of digital documents via structural references. We introduce a conceptual model for structural cross-media content composition and highlight how the proposed solution not only enables the reuse of content via structural relationships, but also supports dynamic and context-dependent document adaptation, structural content annotations as well as the integration of arbitrary nontextual media types. We further discuss solutions for the fluid navigation and cross-media content publishing based on the proposed structural cross-media content model.

Research paper: https://beatsigner.com/publications/what-is-wrong-with-digital-documents-a-conceptual-model-for-structural-cross-media-content-composition-and-reuse.pdf

Beat Signer

November 04, 2010
Tweet

More Decks by Beat Signer

Other Decks in Science

Transcript

  1. ER 2010, November 4, 2010
    What is Wrong with Digital Documents?
    Beat Signer, http://www.beatsigner.com
    Web & Information Systems Engineering Lab (WISE)
    Department of Computer Science
    Vrije Universiteit Brussel
    A Conceptual Model for Structural Cross-Media
    Content Composition and Reuse

    View full-size slide

  2. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    PARC User Interface (Mid 70s)
    ▪ Desktop metaphor
    ▪ WIMP interaction
    ▪ "filing cabinets" with
    hierarchical folders
    ▪ Paper paradigm
    ▪ "What You See Is What
    You Get" (WYSIWYG)
    ▪ what about richer
    document formats that
    existed at that time?
    Xerox Star 8010 [http://www.digibarn.com/collections/systems/xerox-8010/]

    View full-size slide

  3. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Digital Documents as a Paper Simulator?
    Vannevar Bush
    Ted Nelson
    Most people don't understand the logic of the
    concept: "What You See Is What You Get" is
    based on printing the document out ("get"
    means "get WHEN YOU PRINT IT OUT"). And that
    means a metaphysical shift: a document can
    only consist of what can be printed! [...] No
    overlays [...] – PAPER UNDER GLASS.
    When data of any sort are placed in storage,
    they are filed alphabetically or numerically,
    and information is found (when it is) by
    tracing it down from subclass to subclass. It
    can be in only one place, unless duplicates
    are used [...] The human mind does not work
    that way. It operates by association.
    As We May Think, Atlantic Monthly, July 1945
    Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009

    View full-size slide

  4. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Paper-based Work Practices
    Marcel Proust, À la recherche du temps perdu
    Du côté de chez Swann [http://expositions.bnf.fr/brouillons/grand/45.htm]
    Marginal
    annotations
    versioning
    Cut and paste
    #

    View full-size slide

  5. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    oN-Line System (NLS)
    ▪ Collaborative System
    by D. Engelbart et al.
    demonstrated in 1968
    ▪ addressing and linking
    parts of files
    ▪ flexible file views
    ▪ overlay file structures
    ▪ cross-file editing
    ▪ ...

    View full-size slide

  6. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    010010001001001
    0101001100010010100010010
    0100100010010001000100001001000
    100010001100001001000110010101010110
    1100010010100010010010010100010101011001
    1010011000011001 010101010100
    11000100101000100 110101010101
    010000100100 01010101000
    01001001001 1111111111
    111 0000000010
    0 11111111
    0101001011
    010101010
    111010101010101
    00101001011010
    Fluid Cross-media Information Spaces

    View full-size slide

  7. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Challenges
    ▪ Representation of cross-media relationships
    ▪ navigational relationships
    ▪ structural relationships
    - within document structure
    - external document structure
    ▪ Content reuse across media boundaries
    ▪ cross-media transclusion
    ▪ Distribution of data and metadata across
    multiple devices and media formats

    View full-size slide

  8. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Structure Metamodel (RSL)
    entity
    selector resource
    (1,*)
    (1,*)
    (1,1) (0,*)
    (0,*) (0,*)
    RefersTo
    HasTarget
    HasSource
    partition
    HasProperties
    parameter
    Properties
    (0,*)
    (0,*)
    HasResolver
    contextResolver
    Context
    Resolvers
    Entities
    (0,*)
    (0,*)
    user
    Users
    parameter
    Preferences
    group
    Groups
    (0,*)
    (0,*)
    (0,*)
    (0,*)
    HasMembers
    AccessibleTo
    individual
    Individuals
    (0,*)
    (0,*)
    (1,1)
    partition
    Has
    Preferences
    (0,*)
    layer
    Layers
    OnLayer |HasLayers|
    (1,1)
    (0,*) (0,*)
    (0,*)
    (1,*)
    (0,*)
    link
    Structural
    Links
    partition
    CreatedBy
    Resources
    Selectors
    |HasChild|
    link
    Navigational
    Links
    link
    Links
    structure
    Structures HasElements
    (0,*) (1,1)
    B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems. In Proc. of ER 2007, Nov 2007
    OM notation
    (developed at ETH Zurich)

    View full-size slide

  9. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Associative File System (RBAF)
    signer
    My Pictures
    Vancouver ER 2010
    Granville
    Island
    Vancouver
    Harbour
    Public
    Market
    Keynote
    J. Thorp
    Olga and
    Lamia
    Dinner
    ER 2010
    multiple classification
    structural link
    navigational link
    My Videos
    Vancouver
    Aquarium
    ER 2010
    Keynote
    cross-media association
    My Presentations
    ER 2010 WISE 2009
    cross-media
    transclusion
    single slide
    transclusion
    (via OOXML
    RSL selector)

    View full-size slide

  10. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Associative File System (RBAF) ...

    View full-size slide

  11. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Structure Metamodel Features
    ▪ Multiple overlay structures can be defined on top
    of a set of entities (views)
    ▪ Structural relationships between parts of
    resources (fragments) via selector concept
    ▪ Structural relationships as first class entities
    ▪ tagging via properties (key/value pairs)
    ▪ navigational links to other resources
    ▪ context-dependent adaptive structures via context
    resolvers

    View full-size slide

  12. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Cross-media Information Browser & Editor
    ▪ How to build a cross-media structure browser & editor?

    View full-size slide

  13. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Conclusions
    ▪ Existing document formats are very much
    based on the simulation of paper affordances
    on static desktop computers
    ▪ Structure metamodel for the representation of
    open and fluid cross-media information spaces
    ▪ associative file system example
    ▪ cross-media information browser & editor
    ▪ Remediation of the "paper simulation" approach
    ▪ WYSIWYG is only one out of many options!

    View full-size slide

  14. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    Acknowledgments
    ▪ Colleagues at ETH Zurich for collaborating on
    the RSL link metamodel; in particular
    Prof. Dr. Moira C. Norrie
    ▪ Gregory Cardone for implementing the initial
    prototype of the RSL-based associative file
    system (RBAF)

    View full-size slide

  15. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - [email protected]
    References
    ▪ B. Signer and M.C. Norrie, As We May Link: A General
    Metamodel for Hypermedia Systems, Proc. of ER 2007,
    Auckland, New Zealand, November 2007, 359–374
    ▪ V. Bush, As We May Think, Atlantic Monthly 176(1), 1945
    ▪ T. Nelson, Geeks Bearing Gifts: How the Computer World
    Got This Way, Mindful Press, 2009
    ▪ B. Signer, Fundamental Concepts for Interactive
    Paper and Cross-Media Information Spaces,
    ISBN 978-3-8370-2713-6, May 2008

    View full-size slide