VR Book: A Tangible Interface for Smartphone-based Virtual Reality

Presentation at the Mobiquitous 2020 conference

Transcript

1. VR Book: A Tangible Interface for
   Smartphone-based Virtual Reality
   Jorge C. S. Cardoso, Jorge M. Ribeiro
   Mobiquitous 2020, December 7-9, N/A, Cyberspace
   

   View Slide

2. Contents
   1. Motivation
   2. Tangible VR Book
   3. Evaluation
   4. Conclusion
   

   View Slide

3. Motivation
   

   View Slide

4. Photo by Fidel Fernando on Unsplash
   Interaction in VR
   Mostly through controllers
   Arbitrary mappings between
   physical and virtual actions
   Low haptic experience
   Difficult to learn
   

   View Slide

5. Photo by Sound On from Pexels
   Interaction in VR
   Gestural interaction is gaining
   traction
   No haptics
   Difficult to learn
   

   View Slide

6. For humans, touch can
   connect you to an object in a
   very personal way, make it
   seem more real.
   Picard, Star Trek: First Contact, 1996
   

   View Slide

7. Photo by Sigmund on Unsplash
   TUIs for Smartphone-based VR
   Tangible interaction VR
   Easy to create and adapt tangibles
   Cheap
   Accessible
   Photo by Sigmund on Unsplash
   

   View Slide

8. Architectural Heritage
   Virtual reconstructions
   No longer existing
   

   View Slide

9. Photo by Sophia Sideri on Unsplash
   Architectural Heritage
   Virtual reconstructions
   Walk-up-and-use situations
   Non-professional use
   Not much time for learning how to
   interact
   Low experience with VR
   

   View Slide

10. Photos by Jorge M. Ribeiro
    Marker-based passive tangibles
    No additional hardware
    instrumentation
    Passive tangibles
    Detected through marker-based
    computer vision
    Easy to adapt and create tangibles
    

    View Slide

11. Design Space
    

    View Slide

12. Based on
    1. Literature analysis
    2. Prototype development
    3. Ideation sessions
    Design space for marker-based
    tangibles for VR
    

    View Slide

13. VR Book
    

    View Slide

14. Tangible VR Book
    Visual markers to identify pages
    Additional markers on interactive
    elements
    A-Frame web-based VR framework
    AR.js component for detection of
    visual markers
    

    View Slide

15. Tangible VR Book - Prototype #1
    Text, Image, Video, 3D model as
    content on the pages
    Page flipping interaction
    Portals triggered by proximity to
    eyes
    

    View Slide

16. Tangible VR Book - Prototype #2
    Gaze- (reticle-) based interaction
    Tower bell ring
    Display photo of location
    Selection of narration language
    

    View Slide

17. Tangible VR Book - Prototype #3
    Push the limits of marker-based
    interactions
    Hand detection
    (Simple) Gesture interaction
    Dynamic markers
    Content outside book
    

    View Slide

18. Evaluation
    

    View Slide

19. Evaluation
    Impacted by Covid-19 pandemic
    Avoid sharing physical prototypes
    

    View Slide

20. Online questionnaire
    Demographics
    Expectations regarding interactions
    Feedback regarding prototype #1
    Overall impression
    Evaluated prototypes #1 and #2
    

    View Slide

21. Twenty-one participants
    20 to 52 years old (average 32)
    14 male, 7 female
    Main results
    

    View Slide

22. Dynamic content
    Trigger content outside the book
    Automatic video reproduction/play when book is brought close to eyes
    Shaking to change video
    Interactive areas activated by touch
    Main results - expectations regarding content and interactions
    

    View Slide

23. Various issues identified: graphical layout, errors in marker detection,
    visual elements not updating fast enough
    Portals are not clear
    Distracting scenario
    Uncomfortable position for watching video
    Main results - specific feedback regarding prototype #1
    

    View Slide

24. Twenty respondents “liked” or “liked very much” the prototypes
    One respondent “disliked” the prototypes
    Main results - overall impression
    

    View Slide

25. Conclusion
    

    View Slide

26. Conclusion
    Exploration of design space of marker-based tangible interaction for VR
    Marker-based tangible interaction is cheap and accessible
    Potential as solution for smartphone-based VR
    Limited evaluation
    Promising results
    

    View Slide

27. VR Book: A Tangible Interface for
    Smartphone-based Virtual Reality
    Jorge C. S. Cardoso, Jorge M. Ribeiro
    Thank you!
    

    View Slide

28. Tangible User Interfaces
    

    View Slide

29. View Slide

30. View Slide

31. View Slide