AR/VR on Android in 2021

Slide 1

Slide 1 text

AR/VR on Android in 2021 Julien Salvi - Android GDE | Senior Android Engineer @ Aircall GDG Southampton - DevParty @JulienSalvi Bring the Real Reality beyond!

Slide 2

Slide 2 text

Virtual Reality on Android Build new immersive experiences!

Slide 3

Slide 3 text

- John Carmack about Virtual Reality It’s a moral imperative that we must create this!

Slide 4

Slide 4 text

Be immersive

Slide 5

Slide 5 text

Go beyond reality

Slide 6

Slide 6 text

Bring an app to a new level

Slide 7

Slide 7 text

Share your experience, be social

Slide 8

Slide 8 text

VR tools from Google 2015 2016 2017 2018 2019 Cardboard 1.0.0 GoogleVR 1.0.0 Cardboard SDK 1.0.0 (NDK, Unity) Daydream 1.0.0 Daydream discontinued 2014 2020 2021 GoogleVR 1.200.0 archived

Slide 9

Slide 9 text

What about GoogleVR SDK? ● Easy prototyping and integration on existing Android apps. Widgets for 360 videos and more. ● Native SDK (Java, C++) ● No longer supported since November 2019

Slide 10

Slide 10 text

What about GoogleVR SDK? ● Only available on GVR SDK 1.180.0 and earlier. ● Watching 360 videos or panoramic images ● But now deprecated…

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

GVR SDK in action

Slide 13

Slide 13 text

What about Gear VR SDK? ● Gear VR headsets were quite popular back in the days. Optimized for Samsung devices. ● No longer supported since September 2020

Slide 14

Slide 14 text

What about Cardboard SDK? ● Native SDK (C++) for Android, iOS and Unity ● Hard to prototype without a game engine ● Actively supported and maintained

Slide 15

Slide 15 text

VR with Oculus Quest and Go ● High quality standalone VR headset by Oculus. ● Native SDK for Android (NDK, C++) ● Unity/Unreal support

Slide 16

Slide 16 text

Augmented Reality on Android Augment all the things!

Slide 17

Slide 17 text

- from Statista Over 600 millions active users are using AR on mobile devices

Slide 18

Slide 18 text

🛋 Housing, real estate 👾 Gaming and leisure ⚗ Science, education, industry 🏟 A new way to enjoy sport 🗺 Enhance your daily life! Place your screenshot here Building AR apps for…

Slide 19

Slide 19 text

Tools to build AR apps

Slide 20

Slide 20 text

AR tools from Google 2017 2018 2019 2020 2021 ARCore ARCore dev01 ARCore 1.0 ARCore 1.23.0 Sceneform 1.0 Sceneform Sceneform 1.16.0 open-sourced

Slide 21

Slide 21 text

ARCore and Sceneform ARCore & Sceneform are made by Google Many features: motion tracking, light estimation, augmented faces Easy prototyping with Sceneform and you can bring Android UI to AR 250+ devices supported

Slide 22

Slide 22 text

ARCore and Sceneform minSDK 24 if ARCore is required, 14 otherwise OpenGL ES 3.0.0 is required “Google Play Services for AR” must be installed on the device Sceneform requires Java 8

Slide 23

Slide 23 text

Augmented images Bring images to life with ARCore Each Augmented Image database can store up to 1 000 reference images Use the command line arcoreimg to build the image reference database

Slide 24

Slide 24 text

Augmented faces Time to build your own filters! Uses face recognition to map parts of your face into 3D meshes Place your augmented face assets thanks to the identified regions (NOSE_TIP, LEFT_FOREHEAD…)

Slide 25

Slide 25 text

Cloud Anchors Create collaborative AR experiences Each anchor data is stored on the cloud and shared with multiple users You can use Firebase to share your anchors or use your own solution ☁ 📱 📱 ⚓ 🍺 🍺 🍺 🍺

Slide 26

Slide 26 text

Depth API Accurate mapping of the environment Map the depth of your scene so your 3D assets might be hidden behind an object Controlling object oclusion

Slide 27

Slide 27 text

Depth API Provide detailed information about the lighting in your scene Enable immersive and realistic experiences Two lighting mode available: ENVIRONMENTAL_HDR and AMBIENT_INTENSITY

Slide 28

Slide 28 text

Sceneform Sceneform is now open source since March 2020 and no more maintained by Google Hopefully the Android community is awesome and Thomas Gorisse has forked the project and he is actively maintaining the library Uses Filament library inside Sceneform to load glTF assets

Slide 29

Slide 29 text

Sceneform ● 3 basic shapes: cube, cylinder and sphere. ● You must provide a center and a radius or a size to build the shape ● Each shape accepts a Material Basic shapes

Slide 30

Slide 30 text

Sceneform ● Load colors or images texture asynchronously ● Support for transparent colors and textures ● Use MaterialFactory to attach the generated material to the shape Loading textures

Slide 31

Slide 31 text

Sceneform ● You can use Android UI components in AR ● Build XML layouts as you do for your Android apps ● All view interactions are working as usual Android UI in AR

Slide 32

Slide 32 text

Sceneform ● Use glTF assets for complex 3D objects ● Use ObjectAnimator to animate the asset ● Sceneform is using Filament under the hood to load glTF obj. Playing with 3D assets

Slide 33

Slide 33 text

More resources ARCore documentation and codelabs https://developers.google.com/ar/develop https://codelabs.developers.google.com Design guidelines for AR https://designguidelines.withgoogle.com/ar-design Sceneform sample source code https://github.com/Oleur/SheepAR

Slide 34

Slide 34 text

Building AR/VR experiences with Unity A game engine to rule them all!

Slide 35

Slide 35 text

Unity Game Engine Game engine available on Windows, Mac and Linux Build 2D/3D games and AR/VR/MR experiences Fast prototyping and fast scripting Great interoperability with native platforms Supports most of the 3D and 2D asset formats

Slide 36

Slide 36 text

AR Foundation ● Build for ARCore, ARKit, HoloLens and Magic Leap at once. ● Take advantage of Unity features. ● Future-proof AR apps with feature switches. AR Foundation

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

AR Foundation

Slide 39

Slide 39 text

Maps SDK for Unity ● Access to high quality geo data from the Google Maps database ● Renders geographic features such as buildings (footprint + height), monuments, roads, and water areas ● Support for Android and iOS only Google Maps

Slide 40

Slide 40 text

Maps SDK for Unity ● Road lattice to represent the road network of a given map ● Identify and style road intersections. Use the provided metadata to create traﬃc simulation ● Terrain elevation to add fully styleable 3D terrain for a better immersion Google Maps

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

Unity as a library ● Add Unity content directly to your Android apps ● Available since Unity 2019.3 ● Proper library integration as a library

Slide 43

Slide 43 text

Unity as a library ● Fullscreen rendering only ● Only one instance of Unity runtime ● You may need to adapt your plugins to work properly ● Might increase drastically your app size as a library

Slide 44

Slide 44 text

Thank You! Julien Salvi - Android GDE GDG Southampton - DevParty @JulienSalvi Have fun with AR and VR on Android!