Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AR/VR on Android in 2021

AR/VR on Android in 2021

AR/VR have been around for a while in Android now! Since the rise (and fall) of Google Cardboard, a lot of tools have come to help developers build great mobile experiences. In this session, we'll see how the AR ecosystem is strongly evolving with ARCore, Unity, Google Maps and more. And how VR has slowly disappeared from Android. But VR is not dead yet on mobile!

Julien Salvi

May 11, 2021

More Decks by Julien Salvi

Other Decks in Programming


  1. AR/VR on Android in 2021 Julien Salvi - Android GDE

    | Senior Android Engineer @ Aircall GDG Southampton - DevParty @JulienSalvi Bring the Real Reality beyond!
  2. Virtual Reality on Android Build new immersive experiences!

  3. - John Carmack about Virtual Reality It’s a moral imperative

    that we must create this!
  4. Be immersive

  5. Go beyond reality

  6. Bring an app to a new level

  7. Share your experience, be social

  8. VR tools from Google 2015 2016 2017 2018 2019 Cardboard

    1.0.0 GoogleVR 1.0.0 Cardboard SDK 1.0.0 (NDK, Unity) Daydream 1.0.0 Daydream discontinued 2014 2020 2021 GoogleVR 1.200.0 archived
  9. What about GoogleVR SDK? • Easy prototyping and integration on

    existing Android apps. Widgets for 360 videos and more. • Native SDK (Java, C++) • No longer supported since November 2019
  10. What about GoogleVR SDK? • Only available on GVR SDK

    1.180.0 and earlier. • Watching 360 videos or panoramic images • But now deprecated… <com.google.vr.sdk.widgets.video.VrVideoView android:id="@+id/panoramic_view" android:layout_height="match_parent" android:layout_width="match_parent" android:scrollbars="@null"/> <com.google.vr.sdk.widgets.pano.VrPanoramaView android:id="@+id/panoramic_view" android:layout_height="match_parent" android:layout_width="match_parent" android:scrollbars="@null"/>
  11. None
  12. GVR SDK in action

  13. What about Gear VR SDK? • Gear VR headsets were

    quite popular back in the days. Optimized for Samsung devices. • No longer supported since September 2020
  14. What about Cardboard SDK? • Native SDK (C++) for Android,

    iOS and Unity • Hard to prototype without a game engine • Actively supported and maintained
  15. VR with Oculus Quest and Go • High quality standalone

    VR headset by Oculus. • Native SDK for Android (NDK, C++) • Unity/Unreal support
  16. Augmented Reality on Android Augment all the things!

  17. - from Statista Over 600 millions active users are using

    AR on mobile devices
  18. 🛋 Housing, real estate 👾 Gaming and leisure ⚗ Science,

    education, industry 🏟 A new way to enjoy sport 🗺 Enhance your daily life! Place your screenshot here Building AR apps for…
  19. Tools to build AR apps

  20. AR tools from Google 2017 2018 2019 2020 2021 ARCore

    ARCore dev01 ARCore 1.0 ARCore 1.23.0 Sceneform 1.0 Sceneform Sceneform 1.16.0 open-sourced
  21. ARCore and Sceneform ARCore & Sceneform are made by Google

    Many features: motion tracking, light estimation, augmented faces Easy prototyping with Sceneform and you can bring Android UI to AR 250+ devices supported
  22. ARCore and Sceneform minSDK 24 if ARCore is required, 14

    otherwise OpenGL ES 3.0.0 is required “Google Play Services for AR” must be installed on the device Sceneform requires Java 8
  23. Augmented images Bring images to life with ARCore Each Augmented

    Image database can store up to 1 000 reference images Use the command line arcoreimg to build the image reference database
  24. Augmented faces Time to build your own filters! Uses face

    recognition to map parts of your face into 3D meshes Place your augmented face assets thanks to the identified regions (NOSE_TIP, LEFT_FOREHEAD…)
  25. Cloud Anchors Create collaborative AR experiences Each anchor data is

    stored on the cloud and shared with multiple users You can use Firebase to share your anchors or use your own solution ☁ 📱 📱 ⚓ 🍺 🍺 🍺 🍺
  26. Depth API Accurate mapping of the environment Map the depth

    of your scene so your 3D assets might be hidden behind an object Controlling object oclusion
  27. Depth API Provide detailed information about the lighting in your

    scene Enable immersive and realistic experiences Two lighting mode available: ENVIRONMENTAL_HDR and AMBIENT_INTENSITY
  28. Sceneform Sceneform is now open source since March 2020 and

    no more maintained by Google Hopefully the Android community is awesome and Thomas Gorisse has forked the project and he is actively maintaining the library Uses Filament library inside Sceneform to load glTF assets
  29. Sceneform • 3 basic shapes: cube, cylinder and sphere. •

    You must provide a center and a radius or a size to build the shape • Each shape accepts a Material Basic shapes
  30. Sceneform • Load colors or images texture asynchronously • Support

    for transparent colors and textures • Use MaterialFactory to attach the generated material to the shape Loading textures
  31. Sceneform • You can use Android UI components in AR

    • Build XML layouts as you do for your Android apps • All view interactions are working as usual Android UI in AR
  32. Sceneform • Use glTF assets for complex 3D objects •

    Use ObjectAnimator to animate the asset • Sceneform is using Filament under the hood to load glTF obj. Playing with 3D assets
  33. More resources ARCore documentation and codelabs https://developers.google.com/ar/develop https://codelabs.developers.google.com Design guidelines

    for AR https://designguidelines.withgoogle.com/ar-design Sceneform sample source code https://github.com/Oleur/SheepAR
  34. Building AR/VR experiences with Unity A game engine to rule

    them all!
  35. Unity Game Engine Game engine available on Windows, Mac and

    Linux Build 2D/3D games and AR/VR/MR experiences Fast prototyping and fast scripting Great interoperability with native platforms Supports most of the 3D and 2D asset formats
  36. AR Foundation • Build for ARCore, ARKit, HoloLens and Magic

    Leap at once. • Take advantage of Unity features. • Future-proof AR apps with feature switches. AR Foundation
  37. None
  38. AR Foundation

  39. Maps SDK for Unity • Access to high quality geo

    data from the Google Maps database • Renders geographic features such as buildings (footprint + height), monuments, roads, and water areas • Support for Android and iOS only Google Maps
  40. Maps SDK for Unity • Road lattice to represent the

    road network of a given map • Identify and style road intersections. Use the provided metadata to create traffic simulation • Terrain elevation to add fully styleable 3D terrain for a better immersion Google Maps
  41. None
  42. Unity as a library • Add Unity content directly to

    your Android apps • Available since Unity 2019.3 • Proper library integration as a library
  43. Unity as a library • Fullscreen rendering only • Only

    one instance of Unity runtime • You may need to adapt your plugins to work properly • Might increase drastically your app size as a library
  44. Thank You! Julien Salvi - Android GDE GDG Southampton -

    DevParty @JulienSalvi Have fun with AR and VR on Android!