AR/VR on Android in 2021

AR/VR have been around for a while in Android now! Since the rise (and fall) of Google Cardboard, a lot of tools have come to help developers build great mobile experiences. In this session, we'll see how the AR ecosystem is strongly evolving with ARCore, Unity, Google Maps and more. And how VR has slowly disappeared from Android. But VR is not dead yet on mobile!

Transcript

1. AR/VR on Android in 2021 Julien Salvi - Android GDE | Senior Android Engineer @ Aircall GDG Southampton - DevParty @JulienSalvi Bring the Real Reality beyond!
2. Virtual Reality on Android Build new immersive experiences!
3. - John Carmack about Virtual Reality It’s a moral imperative that we must create this!
4. Be immersive
5. Go beyond reality
6. Bring an app to a new level
7. Share your experience, be social
8. VR tools from Google 2015 2016 2017 2018 2019 Cardboard 1.0.0 GoogleVR 1.0.0 Cardboard SDK 1.0.0 (NDK, Unity) Daydream 1.0.0 Daydream discontinued 2014 2020 2021 GoogleVR 1.200.0 archived
9. What about GoogleVR SDK? • Easy prototyping and integration on existing Android apps. Widgets for 360 videos and more. • Native SDK (Java, C++) • No longer supported since November 2019
10. What about GoogleVR SDK? • Only available on GVR SDK 1.180.0 and earlier. • Watching 360 videos or panoramic images • But now deprecated… <com.google.vr.sdk.widgets.video.VrVideoView android:id="@+id/panoramic_view" android:layout_height="match_parent" android:layout_width="match_parent" android:scrollbars="@null"/> <com.google.vr.sdk.widgets.pano.VrPanoramaView android:id="@+id/panoramic_view" android:layout_height="match_parent" android:layout_width="match_parent" android:scrollbars="@null"/>
12. GVR SDK in action
13. What about Gear VR SDK? • Gear VR headsets were quite popular back in the days. Optimized for Samsung devices. • No longer supported since September 2020
14. What about Cardboard SDK? • Native SDK (C++) for Android, iOS and Unity • Hard to prototype without a game engine • Actively supported and maintained
15. VR with Oculus Quest and Go • High quality standalone VR headset by Oculus. • Native SDK for Android (NDK, C++) • Unity/Unreal support
16. Augmented Reality on Android Augment all the things!
17. - from Statista Over 600 millions active users are using AR on mobile devices
18. 🛋 Housing, real estate 👾 Gaming and leisure ⚗ Science, education, industry 🏟 A new way to enjoy sport 🗺 Enhance your daily life! Place your screenshot here Building AR apps for…
19. Tools to build AR apps
20. AR tools from Google 2017 2018 2019 2020 2021 ARCore ARCore dev01 ARCore 1.0 ARCore 1.23.0 Sceneform 1.0 Sceneform Sceneform 1.16.0 open-sourced
21. ARCore and Sceneform ARCore & Sceneform are made by Google Many features: motion tracking, light estimation, augmented faces Easy prototyping with Sceneform and you can bring Android UI to AR 250+ devices supported
22. ARCore and Sceneform minSDK 24 if ARCore is required, 14 otherwise OpenGL ES 3.0.0 is required “Google Play Services for AR” must be installed on the device Sceneform requires Java 8
23. Augmented images Bring images to life with ARCore Each Augmented Image database can store up to 1 000 reference images Use the command line arcoreimg to build the image reference database
24. Augmented faces Time to build your own filters! Uses face recognition to map parts of your face into 3D meshes Place your augmented face assets thanks to the identified regions (NOSE_TIP, LEFT_FOREHEAD…)
25. Cloud Anchors Create collaborative AR experiences Each anchor data is stored on the cloud and shared with multiple users You can use Firebase to share your anchors or use your own solution ☁ 📱 📱 ⚓ 🍺 🍺 🍺 🍺
26. Depth API Accurate mapping of the environment Map the depth of your scene so your 3D assets might be hidden behind an object Controlling object oclusion
27. Depth API Provide detailed information about the lighting in your scene Enable immersive and realistic experiences Two lighting mode available: ENVIRONMENTAL_HDR and AMBIENT_INTENSITY
28. Sceneform Sceneform is now open source since March 2020 and no more maintained by Google Hopefully the Android community is awesome and Thomas Gorisse has forked the project and he is actively maintaining the library Uses Filament library inside Sceneform to load glTF assets
29. Sceneform • 3 basic shapes: cube, cylinder and sphere. • You must provide a center and a radius or a size to build the shape • Each shape accepts a Material Basic shapes
30. Sceneform • Load colors or images texture asynchronously • Support for transparent colors and textures • Use MaterialFactory to attach the generated material to the shape Loading textures
31. Sceneform • You can use Android UI components in AR • Build XML layouts as you do for your Android apps • All view interactions are working as usual Android UI in AR
32. Sceneform • Use glTF assets for complex 3D objects • Use ObjectAnimator to animate the asset • Sceneform is using Filament under the hood to load glTF obj. Playing with 3D assets
33. More resources ARCore documentation and codelabs https://developers.google.com/ar/develop https://codelabs.developers.google.com Design guidelines for AR https://designguidelines.withgoogle.com/ar-design Sceneform sample source code https://github.com/Oleur/SheepAR
34. Building AR/VR experiences with Unity A game engine to rule them all!
35. Unity Game Engine Game engine available on Windows, Mac and Linux Build 2D/3D games and AR/VR/MR experiences Fast prototyping and fast scripting Great interoperability with native platforms Supports most of the 3D and 2D asset formats
36. AR Foundation • Build for ARCore, ARKit, HoloLens and Magic Leap at once. • Take advantage of Unity features. • Future-proof AR apps with feature switches. AR Foundation
38. AR Foundation
39. Maps SDK for Unity • Access to high quality geo data from the Google Maps database • Renders geographic features such as buildings (footprint + height), monuments, roads, and water areas • Support for Android and iOS only Google Maps
40. Maps SDK for Unity • Road lattice to represent the road network of a given map • Identify and style road intersections. Use the provided metadata to create traffic simulation • Terrain elevation to add fully styleable 3D terrain for a better immersion Google Maps
42. Unity as a library • Add Unity content directly to your Android apps • Available since Unity 2019.3 • Proper library integration as a library
43. Unity as a library • Fullscreen rendering only • Only one instance of Unity runtime • You may need to adapt your plugins to work properly • Might increase drastically your app size as a library
44. Thank You! Julien Salvi - Android GDE GDG Southampton - DevParty @JulienSalvi Have fun with AR and VR on Android!