$30 off During Our Annual Pro Sale. View Details »

VR Book: A Tangible Interface for Smartphone-based Virtual Reality

VR Book: A Tangible Interface for Smartphone-based Virtual Reality

Presentation at the Mobiquitous 2020 conference

Jorge C. S. Cardoso

December 08, 2020
Tweet

More Decks by Jorge C. S. Cardoso

Other Decks in Research

Transcript

  1. VR Book: A Tangible Interface for
    Smartphone-based Virtual Reality
    Jorge C. S. Cardoso, Jorge M. Ribeiro
    Mobiquitous 2020, December 7-9, N/A, Cyberspace

    View Slide

  2. Contents
    1. Motivation
    2. Tangible VR Book
    3. Evaluation
    4. Conclusion

    View Slide

  3. Motivation

    View Slide

  4. Photo by Fidel Fernando on Unsplash
    Interaction in VR
    Mostly through controllers
    Arbitrary mappings between
    physical and virtual actions
    Low haptic experience
    Difficult to learn

    View Slide

  5. Photo by Sound On from Pexels
    Interaction in VR
    Gestural interaction is gaining
    traction
    No haptics
    Difficult to learn

    View Slide

  6. For humans, touch can
    connect you to an object in a
    very personal way, make it
    seem more real.
    Picard, Star Trek: First Contact, 1996

    View Slide

  7. Photo by Sigmund on Unsplash
    TUIs for Smartphone-based VR
    Tangible interaction VR
    Easy to create and adapt tangibles
    Cheap
    Accessible
    Photo by Sigmund on Unsplash

    View Slide

  8. Architectural Heritage
    Virtual reconstructions
    No longer existing

    View Slide

  9. Photo by Sophia Sideri on Unsplash
    Architectural Heritage
    Virtual reconstructions
    Walk-up-and-use situations
    Non-professional use
    Not much time for learning how to
    interact
    Low experience with VR

    View Slide

  10. Photos by Jorge M. Ribeiro
    Marker-based passive tangibles
    No additional hardware
    instrumentation
    Passive tangibles
    Detected through marker-based
    computer vision
    Easy to adapt and create tangibles

    View Slide

  11. Design Space

    View Slide

  12. Based on
    1. Literature analysis
    2. Prototype development
    3. Ideation sessions
    Design space for marker-based
    tangibles for VR

    View Slide

  13. VR Book

    View Slide

  14. Tangible VR Book
    Visual markers to identify pages
    Additional markers on interactive
    elements
    A-Frame web-based VR framework
    AR.js component for detection of
    visual markers

    View Slide

  15. Tangible VR Book - Prototype #1
    Text, Image, Video, 3D model as
    content on the pages
    Page flipping interaction
    Portals triggered by proximity to
    eyes

    View Slide

  16. Tangible VR Book - Prototype #2
    Gaze- (reticle-) based interaction
    Tower bell ring
    Display photo of location
    Selection of narration language

    View Slide

  17. Tangible VR Book - Prototype #3
    Push the limits of marker-based
    interactions
    Hand detection
    (Simple) Gesture interaction
    Dynamic markers
    Content outside book

    View Slide

  18. Evaluation

    View Slide

  19. Evaluation
    Impacted by Covid-19 pandemic
    Avoid sharing physical prototypes

    View Slide

  20. Online questionnaire
    Demographics
    Expectations regarding interactions
    Feedback regarding prototype #1
    Overall impression
    Evaluated prototypes #1 and #2

    View Slide

  21. Twenty-one participants
    20 to 52 years old (average 32)
    14 male, 7 female
    Main results

    View Slide

  22. Dynamic content
    Trigger content outside the book
    Automatic video reproduction/play when book is brought close to eyes
    Shaking to change video
    Interactive areas activated by touch
    Main results - expectations regarding content and interactions

    View Slide

  23. Various issues identified: graphical layout, errors in marker detection,
    visual elements not updating fast enough
    Portals are not clear
    Distracting scenario
    Uncomfortable position for watching video
    Main results - specific feedback regarding prototype #1

    View Slide

  24. Twenty respondents “liked” or “liked very much” the prototypes
    One respondent “disliked” the prototypes
    Main results - overall impression

    View Slide

  25. Conclusion

    View Slide

  26. Conclusion
    Exploration of design space of marker-based tangible interaction for VR
    Marker-based tangible interaction is cheap and accessible
    Potential as solution for smartphone-based VR
    Limited evaluation
    Promising results

    View Slide

  27. VR Book: A Tangible Interface for
    Smartphone-based Virtual Reality
    Jorge C. S. Cardoso, Jorge M. Ribeiro
    Thank you!

    View Slide

  28. Tangible User Interfaces

    View Slide

  29. View Slide

  30. View Slide

  31. View Slide