Sceneform for pragmatics

Transcript

1. Sceneform for pragmatics How to embed 3D-rendering in your Android application
2. About me Andrey Dotsenko Android-developer ex-Replika AI
3. The problem
4. 3D-rendering is complicated
5. Table of contents • Solutions for 3D-rendering • What is Sceneform • How to use - simple flow • What we could do with Sceneform • Our experience • Conclusions
6. Solutions for 3D-rendering
7. Solutions for 3D-rendering • GLSurfaceView - native Android approach, raw OpenGL • LibGDX - powerful game development framework • Unity - cross-platform game engine • Filament - real-time physically based rendering engine
9. What is Sceneform?
10. What is Sceneform? • High-level scene graph API • Physically-based 3D renderer - Filament • ARCore integration classes • Sceneform Tools gradle plugin + Android Studio plugin
11. Requirements* • minSdkVersion 24 • OpenGL ES 3.0 or later
12. How to use - simple flow
13. How to use - simple flow Prepare 3D model Load model by ModelRenderable.builder() Add SceneView to layout Convert model to SFB format by plugin Add renderable to the scene
14. 3D model Get: • Poly.google.com • Free3d.com • Turbosquid.com Open: • FBX preview • Maya • Cinema 4D
15. Convert model to SFB build.gradle
16. SFA & Preview
17. Add SceneView to layout
18. Load model
19. It’s alive
20. Scene graph Scene Node Node Node Node
21. What we could do with Sceneform
22. Preloaded or dynamically created models • Convert from files (OBJ, FBX, glTF) by Android Studio Plugin • Create simple shapes at runtime using ShapeFactory and MaterialFactory • Create Views from res/layout at runtime using ViewRenderable
23. Animations Android animations (e.g. FlingAnimation) Predefined (FBX-files only)
24. Change textures at runtime
25. Connect models by bones
26. Move & rotate camera
27. Lighting
28. Our experience
29. True requirements • minSdkVersion 24 • OpenGL ES 3.0 or later
30. Models debugging • Lost textures • Broken animations • Corrupted bones • Position on the screen • Converting converting converting...
31. Converter • Embedded in Android Studio • Command line only • Almost non-configurable • Suppressed or meaningless errors
32. SFB-files & back-compatibility It is not safe to use converted (SFB) models at runtime with different Sceneform version than the converters version. E.g. (FBX-file + Sceneform converter v1.11) + App with Sceneform v1.11 => OK (FBX-file + Sceneform converter v1.11) + App with Sceneform v1.12 => CRASH Bonus - Android Studio crash
33. Lights • Impossible to disable default environment light • Single light source works fine. More than one leads to bugs
34. Broken bones after conversion • Model may be distorted due to bone weight • Distortion depends on the position of the model relative to the center of coordinates
35. Architecture
36. SceneController fun onResume() fun onPause() BaseNodeAnimator fun start() fun stop() fun pause() fun resume() BaseSceneBuilder fun build(): CompletableFuture BaseTransfomation fun transform(modelNode: BaseModelNode): CompletableFuture BaseModelNode fun init(): CompletableFuture
37. Conclusion:
38. Conclusions: • Small amount of supported devices • Problematic tooling • Easy to integrate • Easy to reuse in AR
39. Read: Revolut & 3D cards https://medium.com/revolut/how-we-implemented-3d-cards-in-revolut-fa842 03a8f42 ARCore Cupcakes series of articles https://medium.com/@harivigneshjayapalan/arcore-cupcakes-1-render-a-scen e-without-ar-for-phones-without-arcore-27d61a43a130 Issue tracker & samples https://github.com/google-ar/sceneform-android-sdk
40. Demo project Slides https://github.com/Dosssik/SceneformExamples https://speakerdeck.com/dosssik/sceneform-for-pragmatics
41. Thank you! Questions? @dosssik