Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Realities With Arcore.An Introduction ...

Building Realities With Arcore.An Introduction to XR and ARcore

An introduction to the XR world. A comparison between virtual and augmented reality and what is augmented virtuality and how they differ from each other. A go through on ARCore, how it works what are its terminologies and different APIs

Kashif Mehmood

December 19, 2021
Tweet

More Decks by Kashif Mehmood

Other Decks in Programming

Transcript

  1. Building Realities with ARCore
 An Introduction to XR space and

    ARCore Software Engineer at Markaz (YC-21)

  2. Virtual Reality vs Augmented Reality • Computer generated, Virtual Environment

    • No real world view, only virtual environment • Uses body as input • R= Synthetic World • Virtual World merged with Real World • Real + Virtual, composite view • Uses gestures as input • R= Real World AR: Augmented Reality VR: Virtual Reality
  3. Virtual Reality
 • VR Emphasizes Autonomy/ Agency Head Tracking, Body

    as Input • VR Provides Natural Interaction
 Gestures (controller, hand, finger), speech • VR gives a sense of Presence Immersive, multi-sensory ( visual + audio + haptic ) 

  4. Augmented Reality
 • AR combines real + Virtual objects Composite

    view not just virtual objects • AR is interactive in real time Explicit and implicit interactions • AR is registered in 3D Aligns real and virtual objects as part of same world 

  5. Mixed Reality
 1. Reality Virtuality Continuum 2. Synonym of Augmented

    Reality 3. Combination of AR and VR 4. "Stronger" version of AR 5. Type of Collaboration 6. Alignment of Environment 
 Six Working Definition Speicher, Hall, Nebling: What is Mixed Reality? CHI 2019
  6. Reality-Virtuality Continuum
 3. Combination of AR and VR 4. "Stronger"

    version of AR Milgram and Kashino: A taxonomy of mixed reality virtual display: 1994 
 Real Environment Augmented Reality Augmented Virtuality Virtual Reality
 Handhel d Mixed Reality Room sized Tangible Displays Spatial/ Projective Head Mounted

  7. The Terms
 • AR = Augmented reality, enhance physical world


    • VR= Virtual reality, computer generated virtual environment • MR = Mixed reality, (AR + AV, usually not VR) 
 • XR = Extended reality (X is a wild card for A, V and M.)
  8. The 5 W's of AR
 • Who?
 Google, Snap Inc,

    Apple, Facebook, Microsoft, Magic Leap • What? Google glass, AR Core / ARkit, HoloLens, Magic Leap • When?
 Early start, then nothing, then everywhere, sword of Damocles (1968),Glass (2013), HoloLens (2015), AR Core / ARKit(2017) 
 • Where? Early in research labs no consumer products • Why? Blending and virtual environment, technological advancement
  9. Why you should you even Care about AR? • Snap

    inc bought WaveOptics for more than $500 million. • Facebook Inc. has more than 6,000 employees working on its virtual reality and augmented reality projects. • Apple Inc. is said to be developing a virtual reality headset with some AR functionality for a possible release in 2022. • Niantic Inc, has made $4 billion in revenue since the Pokemon Go’s 2016 release. • Shopify Inc. said last September that product pages with AR experiences led to a 94% higher conversion rate compared to traditional listings. According to research from Digi-Capital, means AR could approach an installed base of 3.5 billion by 2023, generating some $85 to $90 billion in revenue
  10. Applications of AR
 • Enhanced Surgical Training And Performance •

    Property Tours And 3D Real Estate Models • Immersive Therapy • Heavy Machinery Training • Ecommerce • Combat training • Education • Tourism • Automobiles
  11. Motion Tracking
 • SLAM ( Simultaneous Localization and Mapping) •

    COM (Concurrent Odometry and Mapping) • Feature Points
 • The Hardware ( Gyroscope and Accelerometer)
  12. Building Apps with AR Core • Cloud Anchors • Augmented

    Faces • Augmented Images • AR Core + ML Kit
  13. Cloud Anchors
 • Shared AR experiences • AR Core Cloud

    Anchor Service • Hosting
 • Resolving

  14. Augmented Faces
 • Uses 2D Face Mesh • Used PNG

    images as face filters
 • Uses 3d Face Mesh 
 • Uses 3d Models ad face filters
 2D 3D Limitation
 AR Core can track only one face at a time
  15. Augmented Faces
 • Uses 2D Face Mesh • Used PNG

    images as face filters
 • Hosting
 • Resolving
 2D 3D
  16. Augmented Images
 • Recognizes Images
 • Returns pose with respect

    to camera
 • Anchors AR Model to the image 
 • Gives 6 degree of freedom

  17. AR Core + ML Kit
 • Integrate ML models to

    AR Core apps
 • Use on device ML 
 • Use custom ML models to enhance AR experiences 
 • Develop AR apps that were not possible without ML Kit

  18. LiDAR Light Detection and Ranging • Helps understand the structure

    and depth
 • Attach different type of content to different parts of environment 
 • Enhance AR experiences 
 • Better Immersion with Enhanced occlusion