Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mobile Dev Japan #3 - Safie Viewer Android & Sc...

Mobile Dev Japan #3 - Safie Viewer Android & Sceneview

Talk about how we implemented Sceneview in Safie Viewer at Mobile Dev Japan in Tokyo

Jerome

July 26, 2024
Tweet

More Decks by Jerome

Other Decks in Programming

Transcript

  1. July 26, 2024 CHA Jerome Safie Inc. Development Division 2 -

    Mobile Team Safie Viewer Android & Sceneview
  2. © Safie Inc.| Self-Introduction - Jerome CHA, French - Joined

    Safie in November 2023 as Android Engineer - Previously worked at Kubell (ex Chatwork) and Enigmo (BUYMA) - Hobbies : Workout (ex-JBBF Bodybuilder) / Making my own robot - Twitter @yujiro45 2
  3. © Safie Inc.| Agenda 1. Sceneview Recap 2. Safie Viewer

    3. Our implementation of Sceneview 4. Problems 3
  4. © Safie Inc.| Agenda 1. Sceneview Recap 2. Safie Viewer

    3. Our implementation of Sceneview 4. Problems 4
  5. © Safie Inc.| Sceneview Recap - https://sceneview.github.io/ - Library that

    uses Google’s ARCore and Filament to display 3D / AR contents - Included 2 views : Sceneview & ARSceneview - My previous presentation @ Mobile Dev Japan #1 https://speakerdeck.com/jeromecha/ar-with-sceneview-on-android 5 ARSceneview Sceneview
  6. © Safie Inc.| Agenda 1. Sceneview Recap 2. Safie Viewer

    3. Our implementation of Sceneview 4. Problems 6
  7. © Safie Inc.| Safie Viewer - Cloud service for security

    cameras - Clear images, easy to setup, easy to use - Videos are in HD 30fps 7
  8. © Safie Inc.| Agenda 1. Sceneview Recap 2. Safie Viewer

    3. Our implementation of Sceneview 4. Problems 8
  9. © Safie Inc.| Safie’s new camera - Safie GO 360

    - Launched in February 1st 2024 - Compact and lightweight, easy to transport - Built-in LTE for cloud recording anytime, anywhere - Horizontal and vertical viewing angle of 182° 9 https://safie.co.jp/news/3033/
  10. © Safie Inc.| 3D model on Android 12 - 3D

    Model created via Blender - iOS : .dae format → convert to .scn format with Xcode - Android : .glb format
  11. © Safie Inc.| Texture Mapping 14 How can we map

    the camera’s stream on the texture of the 3D model
  12. © Safie Inc.| Texture Mapping 18 We need to make

    our own VideoMaterial with ExoPlayer!
  13. © Safie Inc.| Texture Mapping 21 Engine : Google Filament’s

    engine Exoplayer : for the video’s stream materialLoader : for the texture’s creation
  14. © Safie Inc.| Texture Mapping 23 Create a Stream from

    Filament based on the surfaceTexture
  15. © Safie Inc.| Texture Mapping 24 Create a Sceneview Texture

    for our modelNode based on Filament’s Stream
  16. © Safie Inc.| User’s interaction 30 To limit the 3D

    model rotation : - Remove the default’s user interaction - Develop your own behavior from scratch
  17. © Safie Inc.| User’s interaction 31 To remove the default

    behavior : - Create a subclass of SceneView - Setup cameraManipulator to null
  18. © Safie Inc.| User’s interaction - Zoom 35 Zoom in

    a specific point on the 3D model - Pinch in/out : Take the middle point between the two fingers and zoom to that position - Double Tap : Double tap to zoom in/out - If tap outside the 3D model, then zoom in the middle
  19. © Safie Inc.| Agenda 1. Sceneview Recap 2. Safie Viewer

    3. Our implementation of Sceneview 4. Problems 42
  20. © Safie Inc.| Problem 1 : 3D Model hit box

    44 Tapping outside is considered as the edge of the 3D model (but only once)
  21. © Safie Inc.| Problem 1 : 3D Model hit box

    45 Swiping outside also triggered the 3D Model to move
  22. © Safie Inc.| Problem 1 : 3D Model hit box

    46 https://github.com/SceneView/sceneview-android/issues/522
  23. © Safie Inc.| Problem 1 : 3D Model hit box

    47 Resolved by putting a PlaneNode on the top of the 3D model to intercept the hit : Diameter of hemisphere *assuming that the center of the hemisphere is at (0,0) *
  24. © Safie Inc.| Problems 2 : ExoPlayer on multiple surfaces

    50 Switching Fisheye ⟷ 360° view with a button
  25. © Safie Inc.| Problems 2 : ExoPlayer on multiple surfaces

    51 https://github.com/google/ExoPlayer/issues/6939
  26. © Safie Inc.| Problems 2 : ExoPlayer on multiple surfaces

    52 The only solution was to reinit ExoPlayer each time you press the button