Evaluation of Multi-Platform Mobile AR Frameworks for Roman Mosaic Augmentation

Presentation for the paper "Evaluation of Multi-Platform Mobile AR Frameworks for Roman Mosaic Augmentation" at the 16th EUROGRAPHICS Workshop on Graphics and Cultural Heritage (EG GCH), Vienna, Austria, 2018.

1. Evaluation of Multi-Platform Mobile AR Frameworks for Roman Mosaic Augmentation EUROGRAPHICS Workshop on Graphics and Cultural Heritage (2018) Vienna, Austria, November 12-15, 2018 Jorge C. S. Cardoso, André Belo CISUC/DEI, Universidade de Coimbra
2. Contents 1. “Mosaico - Conímbriga e Sicó” project a. Digital Tools for Exploring Roman Mosaic 2. Prototype AR Application 3. AR Framework Testing 4. Conclusions
3. “Mosaico - Conímbriga e Sicó” Project
4. “Mosaico - Conímbriga e Sicó” Project • Based on the Roman Mosaic Heritage present in the geographical axis constituted by ◦ the Ruins of the Roman city of Conímbriga, ◦ the Roman Villa of Rabaçal, and ◦ the Monumental Complex of Santiago da Guarda.
5. “Mosaico - Conímbriga e Sicó” Project • Promotes cultural and creative activities within the museums, interpretative centers and archaeological sites ◦ Integrated into the CREATOUR national project as a pilot initiative • Alternative experiences of sharing knowledge about the Roman Mosaic Heritage • Mosaic as a modern expression of creativity brought into the present and reinterpreted
6. Digital Tools for Exploring Roman Mosaic In the context of the “Mosaico - Conímbriga e Sicó” project Various planned digital tools to support different activities for exploring roman mosaic
7. Digital Tools for Exploring Roman Mosaic • Mosaic Editor ◦ Support for mosaic workshops for non-professionals • Programming Environment (Adapted Snap! Environment) ◦ Learning computer programming by creating mosaic patterns • Interactive Sandbox ◦ Interactive experiences for kids ◦ Simulating uncovering mosaics
8. Augmented Reality Mobile Application An AR application that provides in-place, contextual information about the roman mosaics
9. Augmented Reality Mobile Application for Mosaics • Purpose ◦ Display technical information about the mosaics, for example, when they were uncovered, what was the latest conservation or restoration work, etc. ◦ Display image overlays of the conservation or restoration works on mosaics over time. ◦ Provide a platform for the visualization of virtual restoration of the existing mosaics. ◦ Highlight mosaics with graphical information regarding various motifs ▪ geometric patterns, animals, plants, compositions, mythological figures, etc. ◦
10. Types of Augmented Reality Location-based Structured markers “Instant tracking” Natural image features
11. Augmented Reality Mobile Application • Requirement ◦ Multi-platform application (run on Android, iOS, etc.) ▪ Single code base ▪ Lower development effort • What AR development frameworks are available for multi-platform mobile development? • Which AR development frameworks are most suitable for detecting real mosaics?
12. AR Framework Testing
13. AR Development Frameworks • We studied multi-platform AR development frameworks and their features ◦ With natural image feature detection • Narrowed down to 3 frameworks: CraftAR, PixLive, Wikitude
14. AR Framework Evaluation • Real-world evaluation with roman mosaics at Conímbriga • Targets with different characteristics were captured
15. AR Test Application • A simple test application was developed using each of the 3 AR frameworks • Test application overlaid graphical shapes over the targets ◦ Simple shapes, easy to verify alignment
16. AR Test Application • We screen-captured in video the execution of the application over each of the mosaic targets • Three camera movements: ◦ Camera face down, turn up towards the target, then turn left, then right ◦ Horizontal pan left/right ◦ “Zoom in/out”
17. AR Test Application • We screen-captured in video the execution of the application over each of the mosaic targets • We analysed the various videos and extracted 3 metrics ◦ Recognition delay ◦ Minimum required target area ◦ Visual alignment and stability
18. AR Framework Evaluation Results - Overall Recognition • Not all targets were recognized ◦ This was expected ◦ Targets were captured from a distance ◦ Not much effort in capturing targets • Wikitude performed very poorly ◦ Unexpected ◦ Requires further study as to why
19. AR Framework Evaluation Results - Recognition Delay • CraftAR is faster than PixLive ◦ Almost 0.5 seconds faster • PixLive requires less visible target area
20. AR Framework Evaluation Results - Visual alignment and stability 1. Subjectively rated by the two authors a. -1: bad alignment / stability b. 0: ok alignment / stability c. 1: good alignment / stability
21. Conclusion
22. Conclusion • Evaluation in a real scenario setting ◦ Compared three frameworks: CraftAR, PixLive, Wikitude ◦ Wikitude failed, but more testing is required to dismiss it • Study allowed us to understand strong and weak points of these AR frameworks ◦ AR frameworks’ performance varies greatly depending on the type of image they are recognizing ◦ AR frameworks have different performance compromises ▪ No single one is best at every performance attribute • Virtual Heritage application developers should test different frameworks before commiting to one