ConímbrigAR: A Prototype Augmented Mobile Application for Exploration of Roman Mosaics

Presentation for the "2ªS JORNADAS DOCUMENTAÇÃO E REPRESENTAÇÃO DIGITAL DE BENS CULTURAIS" at the School of Arts, Porto, 2018

1. ConímbrigAR: A Prototype Augmented Mobile Application for Exploration of Roman Mosaics 2ªs Jornadas de Documentação e Representação Digital de Bens Culturais, 19-20 Oct, School of Arts, Porto, 2018 André Belo, Jorge C. S. Cardoso CISUC/DEI, Universidade de Coimbra
2. Contents 1. “Mosaico - Conímbriga e Sicó” project 2. Digital Tools for Exploring Roman Mosaic 3. Prototype AR Application 4. AR Framework Testing
3. “Mosaico - Conímbriga e Sicó” Project
4. “Mosaico - Conímbriga e Sicó” Project • Integrated into the CREATOUR national project as a pilot initiative - Creative Tourism Destination Development in Small Cities and Rural Areas • “Mosaico – Conímbriga e Sicó” is a creative tourism project based on the Roman Mosaic heritage ◦ Develops educational, cultural, and creative activities around Roman Mosaic
5. “Mosaico - Conímbriga e Sicó” Project • Based on the valuable Roman Mosaic Heritage present in the geographical axis constituted by ◦ the Ruins of the Roman city of Conímbriga, ◦ the Roman Villa of Rabaçal, and ◦ the Monumental Complex of Santiago da Guarda. • Headquartered in the “Monographic Museum of Conímbriga – National Museum” ◦ an important center for archaeological research, conservation and restoration of Roman Mosaic in Portugal
6. “Mosaico - Conímbriga e Sicó” Project • Promotes cultural and creative activities within the museums, interpretative centers and archaeological sites included in its program of action. • The visitors are invited to be involved in alternative experiences of sharing knowledge about the Roman Mosaic Heritage • Mosaic as an expression of creativity brought into the present and reinterpreted now and in the future.
7. Digital Tools for Exploring Roman Mosaic In the context of the “Mosaico - Conímbriga e Sicó” project Various planned digital tools to support different activities for exploring roman mosaic
8. Mosaic Editor • Support for mosaic workshops for non-professionals
9. Mosaic Programming Environment • Learning computer programming by creating mosaic patterns
10. Innovative Interactive Experiences • Sandbox early prototype • Interactive experiences for kids • Digging mosaics simulations
11. Augmented Reality Mobile Application An AR application that provides in-place, contextual information about the roman mosaics
12. Augmented Reality Mobile Application • Technical information about the mosaics, for example, when they were uncovered, what was the latest conservation or restoration work, etc. • Provide additional conservation and restoration information, for example, display image overlays of the conservation or restoration works on mosaics over time.
13. Augmented Reality Mobile Application • Provide a platform for the visualization of virtual restoration of the existing mosaics. The virtual restoration images could be created by different audiences with different purposes. For example, in the context of a school visit, students could digitally manipulate mosaic images and creatively “restore” missing parts which could then be experienced through the AR application.
14. Augmented Reality Mobile Application • Provide visual indication regarding specific aspects of the various mosaics. • For example, mosaics could be highlighted with graphical information regarding the various motifs found in the mosaics ◦ geometric patterns, animals, plants, compositions, mythological figures, etc.
15. Augmented Reality Mobile Application • Hybrid application (run on Android, iOS, etc.) ◦ Lower development effort • What AR development frameworks are most suitable for detecting mosaics?
16. Types of Augmented Reality - Location Based • Based on coordinates (e.g., GPS) • Imprecise, not enough to overlay mosaic details
17. Types of Augmented Reality - (Structured) Marker Based • Based on a pre-defined, structured visual marker image • Very precise, requires placement of artificial markers on site
18. Types of Augmented Reality - (Natural Image Features) Marker Based • Based on a pre-defined natural images (targets) • Very precise, if targets are good enough • May be used against natural images on site • Usually used with printed images
19. Types of Augmented Reality - Markerless Based • Based on detection of planar surfaces • Not contextual
20. AR Development Frameworks • We studied multi-platform AR development frameworks and their features ◦ We wanted natural image features • Narrowed down to 3 frameworks suitable for natural image detection
21. AR Framework Evaluation • Real-world evaluation with roman mosaics at Conímbriga • Targets with different characteristics were captured • A test application was developed using each of the 3 AR frameworks
22. AR Test Application • Test application overlaid graphical shapes over the targets • We screen-captured in video the execution of the application over each of the mosaic targets
23. AR Test Application • We subjectively analysed the various videos on 3 metrics ◦ Recognition delay ◦ Minimum required target area ◦ Visual alignment and stability
24. AR Framework Evaluation Results - Overall Recognition • Not all targets were recognized ◦ This was expected ◦ Targets were captured from a distance ◦ Not much effort in capturing targets • Wikitude performed very poorly ◦ Unexpected ◦ Requires further study as to why
25. AR Framework Evaluation Results - Recognition Delay • CraftAR is faster than PixLive ◦ Almost 0.5 seconds faster
26. AR Framework Evaluation Results - Minimum required target area PixLive requires less visible target area
27. AR Framework Evaluation Results - Visual alignment and stability -1: bad alignment / stability 0: ok alignment / stability 1: good alignment / stability
28. Conclusion • Study allowed us to understand strong and weak points of these AR frameworks • AR frameworks’ performance varies depending on the type of image they are recognizing • AR frameworks have different performance compromises ◦ No single one is best at every performance attribute • Virtual Heritage application developers should test different frameworks before commiting to one • We still need to study additional aspects such as the best way to capture targets ◦ Explore alternative AR types