Building Realities With Arcore.An Introduction to XR and ARcore

An introduction to the XR world. A comparison between virtual and augmented reality and what is augmented virtuality and how they differ from each other. A go through on ARCore, how it works what are its terminologies and different APIs

Transcript

1. Building Realities with ARCore
 An Introduction to XR space and ARCore Software Engineer at Markaz (YC-21)

2. Virtual Reality vs Augmented Reality • Computer generated, Virtual Environment • No real world view, only virtual environment • Uses body as input • R= Synthetic World • Virtual World merged with Real World • Real + Virtual, composite view • Uses gestures as input • R= Real World AR: Augmented Reality VR: Virtual Reality
3. Virtual Reality
 • VR Emphasizes Autonomy/ Agency Head Tracking, Body as Input • VR Provides Natural Interaction
 Gestures (controller, hand, finger), speech • VR gives a sense of Presence Immersive, multi-sensory ( visual + audio + haptic ) 

4. Augmented Reality
 • AR combines real + Virtual objects Composite view not just virtual objects • AR is interactive in real time Explicit and implicit interactions • AR is registered in 3D Aligns real and virtual objects as part of same world 

5. Mixed Reality
 1. Reality Virtuality Continuum 2. Synonym of Augmented Reality 3. Combination of AR and VR 4. "Stronger" version of AR 5. Type of Collaboration 6. Alignment of Environment 
 Six Working Definition Speicher, Hall, Nebling: What is Mixed Reality? CHI 2019
6. Reality-Virtuality Continuum
 3. Combination of AR and VR 4. "Stronger" version of AR Milgram and Kashino: A taxonomy of mixed reality virtual display: 1994 
 Real Environment Augmented Reality Augmented Virtuality Virtual Reality
 Handhel d Mixed Reality Room sized Tangible Displays Spatial/ Projective Head Mounted

7. The Terms
 • AR = Augmented reality, enhance physical world
 • VR= Virtual reality, computer generated virtual environment • MR = Mixed reality, (AR + AV, usually not VR) 
 • XR = Extended reality (X is a wild card for A, V and M.)
8. The 5 W's of AR
 • Who?
 Google, Snap Inc, Apple, Facebook, Microsoft, Magic Leap • What? Google glass, AR Core / ARkit, HoloLens, Magic Leap • When?
 Early start, then nothing, then everywhere, sword of Damocles (1968),Glass (2013), HoloLens (2015), AR Core / ARKit(2017) 
 • Where? Early in research labs no consumer products • Why? Blending and virtual environment, technological advancement
9. Why you should you even Care about AR? • Snap inc bought WaveOptics for more than $500 million. • Facebook Inc. has more than 6,000 employees working on its virtual reality and augmented reality projects. • Apple Inc. is said to be developing a virtual reality headset with some AR functionality for a possible release in 2022. • Niantic Inc, has made $4 billion in revenue since the Pokemon Go’s 2016 release. • Shopify Inc. said last September that product pages with AR experiences led to a 94% higher conversion rate compared to traditional listings. According to research from Digi-Capital, means AR could approach an installed base of 3.5 billion by 2023, generating some $85 to $90 billion in revenue
10. Applications of AR
 • Enhanced Surgical Training And Performance • Property Tours And 3D Real Estate Models • Immersive Therapy • Heavy Machinery Training • Ecommerce • Combat training • Education • Tourism • Automobiles
11. AR Core Overview
 • Motion Tracking • Anchors • Planes
12. AR Core Overview
 • Light Estimation • Points Cloud • Orientation • Occlusion

13. Motion Tracking
 • SLAM ( Simultaneous Localization and Mapping) • COM (Concurrent Odometry and Mapping) • Feature Points
 • The Hardware ( Gyroscope and Accelerometer)
14. Planes
 • Plans Finding • Anchors • Point Clouds

15. Challenges to AR
 • Plans Finding • Anchors • Point Clouds

16. Building Apps with AR Core • Cloud Anchors • Augmented Faces • Augmented Images • AR Core + ML Kit
17. Cloud Anchors
 • Shared AR experiences • AR Core Cloud Anchor Service • Hosting
 • Resolving

18. Augmented Faces
 • Uses 2D Face Mesh • Used PNG images as face filters
 • Uses 3d Face Mesh 
 • Uses 3d Models ad face filters
 2D 3D Limitation
 AR Core can track only one face at a time
19. Augmented Faces
 • Uses 2D Face Mesh • Used PNG images as face filters
 • Hosting
 • Resolving
 2D 3D
20. Augmented Images
 • Recognizes Images
 • Returns pose with respect to camera
 • Anchors AR Model to the image 
 • Gives 6 degree of freedom

21. AR Core + ML Kit
 • Integrate ML models to AR Core apps
 • Use on device ML 
 • Use custom ML models to enhance AR experiences 
 • Develop AR apps that were not possible without ML Kit

22. LiDAR Light Detection and Ranging • Helps understand the structure and depth
 • Attach different type of content to different parts of environment 
 • Enhance AR experiences 
 • Better Immersion with Enhanced occlusion
23. AR in Pakistan
24. Questions?