$30 off During Our Annual Pro Sale. View Details »

ConímbrigAR: A Prototype Augmented Mobile Application for Exploration of Roman Mosaics

ConímbrigAR: A Prototype Augmented Mobile Application for Exploration of Roman Mosaics

Presentation for the "2ªS JORNADAS DOCUMENTAÇÃO E REPRESENTAÇÃO DIGITAL DE BENS CULTURAIS" at the School of Arts, Porto, 2018

Jorge C. S. Cardoso

October 20, 2018
Tweet

More Decks by Jorge C. S. Cardoso

Other Decks in Research

Transcript

  1. ConímbrigAR: A Prototype
    Augmented Mobile
    Application for Exploration of
    Roman Mosaics
    2ªs Jornadas de Documentação e Representação Digital de Bens Culturais,
    19-20 Oct, School of Arts, Porto, 2018
    André Belo, Jorge C. S. Cardoso
    CISUC/DEI, Universidade de Coimbra

    View Slide

  2. Contents
    1. “Mosaico - Conímbriga e Sicó” project
    2. Digital Tools for Exploring Roman Mosaic
    3. Prototype AR Application
    4. AR Framework Testing

    View Slide

  3. “Mosaico -
    Conímbriga e
    Sicó” Project

    View Slide

  4. “Mosaico - Conímbriga e Sicó” Project
    ● Integrated into the CREATOUR
    national project as a pilot
    initiative - Creative Tourism
    Destination Development in
    Small Cities and Rural Areas
    ● “Mosaico – Conímbriga e Sicó”
    is a creative tourism project
    based on the Roman Mosaic
    heritage
    ○ Develops educational, cultural, and
    creative activities around Roman
    Mosaic

    View Slide

  5. “Mosaico - Conímbriga e Sicó” Project
    ● Based on the valuable Roman
    Mosaic Heritage present in the
    geographical axis constituted by
    ○ the Ruins of the Roman city of
    Conímbriga,
    ○ the Roman Villa of Rabaçal, and
    ○ the Monumental Complex of
    Santiago da Guarda.
    ● Headquartered in the
    “Monographic Museum of
    Conímbriga – National Museum”
    ○ an important center for
    archaeological research,
    conservation and restoration of
    Roman Mosaic in Portugal

    View Slide

  6. “Mosaico - Conímbriga e Sicó” Project
    ● Promotes cultural and creative
    activities within the museums,
    interpretative centers and
    archaeological sites included in
    its program of action.
    ● The visitors are invited to be
    involved in alternative
    experiences of sharing
    knowledge about the Roman
    Mosaic Heritage
    ● Mosaic as an expression of
    creativity brought into the
    present and reinterpreted now
    and in the future.

    View Slide

  7. Digital Tools for
    Exploring
    Roman Mosaic
    In the context of the “Mosaico -
    Conímbriga e Sicó” project
    Various planned digital tools to
    support different activities for
    exploring roman mosaic

    View Slide

  8. Mosaic Editor
    ● Support for mosaic workshops
    for non-professionals

    View Slide

  9. Mosaic Programming Environment
    ● Learning computer programming
    by creating mosaic patterns

    View Slide

  10. Innovative Interactive Experiences
    ● Sandbox early prototype
    ● Interactive experiences for kids
    ● Digging mosaics simulations

    View Slide

  11. Augmented
    Reality Mobile
    Application An AR application that provides
    in-place, contextual information
    about the roman mosaics

    View Slide

  12. Augmented Reality Mobile Application
    ● Technical information about the
    mosaics, for example, when they
    were uncovered, what was the
    latest conservation or restoration
    work, etc.
    ● Provide additional conservation
    and restoration information, for
    example, display image overlays
    of the conservation or
    restoration works on mosaics
    over time.

    View Slide

  13. Augmented Reality Mobile Application
    ● Provide a platform for the
    visualization of virtual
    restoration of the existing
    mosaics. The virtual restoration
    images could be created by
    different audiences with different
    purposes. For example, in the
    context of a school visit,
    students could digitally
    manipulate mosaic images and
    creatively “restore” missing parts
    which could then be experienced
    through the AR application.

    View Slide

  14. Augmented Reality Mobile Application
    ● Provide visual indication
    regarding specific aspects of the
    various mosaics.
    ● For example, mosaics could be
    highlighted with graphical
    information regarding the
    various motifs found in the
    mosaics
    ○ geometric patterns, animals, plants,
    compositions, mythological figures,
    etc.

    View Slide

  15. Augmented Reality Mobile Application
    ● Hybrid application (run on
    Android, iOS, etc.)
    ○ Lower development effort
    ● What AR development
    frameworks are most suitable
    for detecting mosaics?

    View Slide

  16. Types of Augmented Reality - Location Based
    ● Based on coordinates (e.g., GPS)
    ● Imprecise, not enough to overlay mosaic details

    View Slide

  17. Types of Augmented Reality - (Structured) Marker Based
    ● Based on a pre-defined, structured visual marker image
    ● Very precise, requires placement of artificial markers on site

    View Slide

  18. Types of Augmented Reality - (Natural Image Features) Marker
    Based
    ● Based on a pre-defined natural images (targets)
    ● Very precise, if targets are good enough
    ● May be used against natural images on site
    ● Usually used with printed images

    View Slide

  19. Types of Augmented Reality - Markerless Based
    ● Based on detection of planar surfaces
    ● Not contextual

    View Slide

  20. AR Development Frameworks
    ● We studied multi-platform AR development frameworks and their
    features
    ○ We wanted natural image features
    ● Narrowed down to 3 frameworks suitable for natural image
    detection

    View Slide

  21. AR Framework Evaluation
    ● Real-world evaluation with roman mosaics at Conímbriga
    ● Targets with different characteristics were captured
    ● A test application was developed using each of the 3 AR
    frameworks

    View Slide

  22. AR Test Application
    ● Test application overlaid graphical shapes over the targets
    ● We screen-captured in video the execution of the application over
    each of the mosaic targets

    View Slide

  23. AR Test Application
    ● We subjectively analysed the various videos on 3 metrics
    ○ Recognition delay
    ○ Minimum required target area
    ○ Visual alignment and stability

    View Slide

  24. AR Framework Evaluation Results - Overall Recognition
    ● Not all targets were
    recognized
    ○ This was expected
    ○ Targets were captured from a
    distance
    ○ Not much effort in capturing
    targets
    ● Wikitude performed very
    poorly
    ○ Unexpected
    ○ Requires further study as to
    why

    View Slide

  25. AR Framework Evaluation Results - Recognition Delay
    ● CraftAR is faster than PixLive
    ○ Almost 0.5 seconds faster

    View Slide

  26. AR Framework Evaluation Results - Minimum required target
    area
    PixLive requires less visible target area

    View Slide

  27. AR Framework Evaluation Results - Visual alignment and
    stability
    -1: bad alignment / stability
    0: ok alignment / stability
    1: good alignment / stability

    View Slide

  28. Conclusion
    ● Study allowed us to understand strong and weak points of these
    AR frameworks
    ● AR frameworks’ performance varies depending on the type of
    image they are recognizing
    ● AR frameworks have different performance compromises
    ○ No single one is best at every performance attribute
    ● Virtual Heritage application developers should test different
    frameworks before commiting to one
    ● We still need to study additional aspects such as the best way to
    capture targets
    ○ Explore alternative AR types

    View Slide