ConímbrigAR: A Prototype Augmented Mobile Application for Exploration of Roman Mosaics

ConímbrigAR: A Prototype Augmented Mobile Application for Exploration of Roman Mosaics

Presentation for the "2ªS JORNADAS DOCUMENTAÇÃO E REPRESENTAÇÃO DIGITAL DE BENS CULTURAIS" at the School of Arts, Porto, 2018

47984b654bec549462b585cad0761c1f?s=128

Jorge C. S. Cardoso

October 20, 2018
Tweet

Transcript

  1. ConímbrigAR: A Prototype Augmented Mobile Application for Exploration of Roman

    Mosaics 2ªs Jornadas de Documentação e Representação Digital de Bens Culturais, 19-20 Oct, School of Arts, Porto, 2018 André Belo, Jorge C. S. Cardoso CISUC/DEI, Universidade de Coimbra
  2. Contents 1. “Mosaico - Conímbriga e Sicó” project 2. Digital

    Tools for Exploring Roman Mosaic 3. Prototype AR Application 4. AR Framework Testing
  3. “Mosaico - Conímbriga e Sicó” Project

  4. “Mosaico - Conímbriga e Sicó” Project • Integrated into the

    CREATOUR national project as a pilot initiative - Creative Tourism Destination Development in Small Cities and Rural Areas • “Mosaico – Conímbriga e Sicó” is a creative tourism project based on the Roman Mosaic heritage ◦ Develops educational, cultural, and creative activities around Roman Mosaic
  5. “Mosaico - Conímbriga e Sicó” Project • Based on the

    valuable Roman Mosaic Heritage present in the geographical axis constituted by ◦ the Ruins of the Roman city of Conímbriga, ◦ the Roman Villa of Rabaçal, and ◦ the Monumental Complex of Santiago da Guarda. • Headquartered in the “Monographic Museum of Conímbriga – National Museum” ◦ an important center for archaeological research, conservation and restoration of Roman Mosaic in Portugal
  6. “Mosaico - Conímbriga e Sicó” Project • Promotes cultural and

    creative activities within the museums, interpretative centers and archaeological sites included in its program of action. • The visitors are invited to be involved in alternative experiences of sharing knowledge about the Roman Mosaic Heritage • Mosaic as an expression of creativity brought into the present and reinterpreted now and in the future.
  7. Digital Tools for Exploring Roman Mosaic In the context of

    the “Mosaico - Conímbriga e Sicó” project Various planned digital tools to support different activities for exploring roman mosaic
  8. Mosaic Editor • Support for mosaic workshops for non-professionals

  9. Mosaic Programming Environment • Learning computer programming by creating mosaic

    patterns
  10. Innovative Interactive Experiences • Sandbox early prototype • Interactive experiences

    for kids • Digging mosaics simulations
  11. Augmented Reality Mobile Application An AR application that provides in-place,

    contextual information about the roman mosaics
  12. Augmented Reality Mobile Application • Technical information about the mosaics,

    for example, when they were uncovered, what was the latest conservation or restoration work, etc. • Provide additional conservation and restoration information, for example, display image overlays of the conservation or restoration works on mosaics over time.
  13. Augmented Reality Mobile Application • Provide a platform for the

    visualization of virtual restoration of the existing mosaics. The virtual restoration images could be created by different audiences with different purposes. For example, in the context of a school visit, students could digitally manipulate mosaic images and creatively “restore” missing parts which could then be experienced through the AR application.
  14. Augmented Reality Mobile Application • Provide visual indication regarding specific

    aspects of the various mosaics. • For example, mosaics could be highlighted with graphical information regarding the various motifs found in the mosaics ◦ geometric patterns, animals, plants, compositions, mythological figures, etc.
  15. Augmented Reality Mobile Application • Hybrid application (run on Android,

    iOS, etc.) ◦ Lower development effort • What AR development frameworks are most suitable for detecting mosaics?
  16. Types of Augmented Reality - Location Based • Based on

    coordinates (e.g., GPS) • Imprecise, not enough to overlay mosaic details
  17. Types of Augmented Reality - (Structured) Marker Based • Based

    on a pre-defined, structured visual marker image • Very precise, requires placement of artificial markers on site
  18. Types of Augmented Reality - (Natural Image Features) Marker Based

    • Based on a pre-defined natural images (targets) • Very precise, if targets are good enough • May be used against natural images on site • Usually used with printed images
  19. Types of Augmented Reality - Markerless Based • Based on

    detection of planar surfaces • Not contextual
  20. AR Development Frameworks • We studied multi-platform AR development frameworks

    and their features ◦ We wanted natural image features • Narrowed down to 3 frameworks suitable for natural image detection
  21. AR Framework Evaluation • Real-world evaluation with roman mosaics at

    Conímbriga • Targets with different characteristics were captured • A test application was developed using each of the 3 AR frameworks
  22. AR Test Application • Test application overlaid graphical shapes over

    the targets • We screen-captured in video the execution of the application over each of the mosaic targets
  23. AR Test Application • We subjectively analysed the various videos

    on 3 metrics ◦ Recognition delay ◦ Minimum required target area ◦ Visual alignment and stability
  24. AR Framework Evaluation Results - Overall Recognition • Not all

    targets were recognized ◦ This was expected ◦ Targets were captured from a distance ◦ Not much effort in capturing targets • Wikitude performed very poorly ◦ Unexpected ◦ Requires further study as to why
  25. AR Framework Evaluation Results - Recognition Delay • CraftAR is

    faster than PixLive ◦ Almost 0.5 seconds faster
  26. AR Framework Evaluation Results - Minimum required target area PixLive

    requires less visible target area
  27. AR Framework Evaluation Results - Visual alignment and stability -1:

    bad alignment / stability 0: ok alignment / stability 1: good alignment / stability
  28. Conclusion • Study allowed us to understand strong and weak

    points of these AR frameworks • AR frameworks’ performance varies depending on the type of image they are recognizing • AR frameworks have different performance compromises ◦ No single one is best at every performance attribute • Virtual Heritage application developers should test different frameworks before commiting to one • We still need to study additional aspects such as the best way to capture targets ◦ Explore alternative AR types