Mobile_AR_with_ARCore.pdf

Transcript

1. Mobile AR
   with ARCore
   Moble AR with ARCore - @ErikHellman, Øredev 2018
   

   View Slide

2. https://9to5mac.com/2018/05/16/comment-stop-using-ar-as-a-gimmick/
   

   View Slide

3. Transformative AR
   Moble AR with ARCore - @ErikHellman, Øredev 2018
   

   View Slide

4. Moble AR with ARCore - @ErikHellman, Øredev 2018
   

   View Slide

5. Moble AR with ARCore - @ErikHellman, Øredev 2018
   

   View Slide

6. MERGE Cube
   https://mergevr.com/cube
   

   View Slide

7. Microsoft HoloLens
   https://www.engadget.com/2017/11/01/microsoft-hololens-hard-hat-european-expansion/
   

   View Slide

8. Magic Leap
   Moble AR with ARCore - @ErikHellman, Øredev 2018
   

   View Slide

9. Transformative AR
   Moble AR with ARCore - @ErikHellman, Øredev 2018
   

   View Slide

10. ARCore from Google
    https://developers.google.com/ar
    

    View Slide

11. Motion Tracking
    https://developers.google.com/ar/discover/concepts
    

    View Slide

12. Environmental understanding
    https://developers.google.com/ar/discover/concepts
    

    View Slide

13. Light estimation
    https://developers.google.com/ar/discover/concepts
    

    View Slide

14. User interaction
    Oriented Points
    Anchors and Trackables
    Augmented Images
    Cloud Anchors
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

15. Anchors
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

16. Getting started
    with ARCore
    (Android)
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

17. override fun onResume() {
    super.onResume()
    val arCoreApk = ArCoreApk.getInstance()
    if (!arCoreApk.checkAvailability(this).isSupported()) {
    // Not supported - notify user!
    return
    }
    if (arCoreApk.requestInstall(this, true)
    == ArCoreApk.InstallStatus.INSTALL_REQUESTED)) {
    // Installation needed - activity will pause
    return
    }
    // Everything is ok to go!
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

18. Sceneform (Android)
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

19. Sceneform
    → High-level scenegraph API
    → Realistic Physical Based Rendering using Filament
    → Android Studio plugin
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

20. Filament
    https://github.com/google/filament
    

    View Slide

21. Sceneform
    dependencies {
    // Provides ARCore Session and related resources.
    implementation 'com.google.ar:core:1.5.0'
    // Provides ArFragment, and other UX resources.
    implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.5.1'
    // Alternatively, use ArSceneView without the UX dependency.
    implementation 'com.google.ar.sceneform:core:1.5.1'
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

22. Sceneform
    android:id="@+id/ar_fragment"
    android:name="com.google.ar.sceneform.ux.ArFragment"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

23. Sceneform
    val arFragment = fragmentManager
    .findFragmentById(R.id.ux_fragment) as ArFragment?
    arFragment?.setOnTapArPlaneListener { hitResult, plane, motionEvent ->
    // Create new anchor and node
    val anchor = hitResult.createAnchor()
    val anchorNode = AnchorNode(anchor)
    // Add to the scene
    anchorNode.setParent(arFragment.arSceneView.scene)
    // TODO Add content to the new anchor...
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

24. Scneeform - Display models
    arFragment?.setOnTapArPlaneListener { hitResult, plane, motionEvent ->
    ModelRenderable.builder()
    .setSource(this, Uri.parse("earth.sfb"))
    .build()
    .thenAccept {
    val anchor = hitResult.createAnchor()
    val anchorNode = AnchorNode(anchor)
    val transformableNode = TransformableNode(arFragment.transformationSystem)
    .apply {
    setParent(anchorNode)
    renderable = it
    }
    anchorNode.addChild(transformableNode)
    anchorNode.setParent(arFragment.arSceneView.scene)
    }
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

25. Sceneform - Display Android views
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

26. Sceneform - Display Android views
    arFragment?.setOnTapArPlaneListener { hitResult, plane, motionEvent ->
    ViewRenderable.builder()
    .setView(this, R.layout.simple_form)
    .build()
    .thenAccept { card ->
    val anchor = hitResult.createAnchor()
    val anchorNode = AnchorNode(anchor)
    TransformableNode(arFragment.transformationSystem).apply {
    setParent(anchorNode)
    renderable = card
    select()
    }
    anchorNode.setParent(arFragment.arSceneView.scene)
    card.view.findViewById(R.id.okBtn).setOnClickListener {
    arFragment.arSceneView.scene.removeChild(anchorNode)
    anchor.detach()
    }
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

27. Augmented Images
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

28. Configure Augmented Images
    // ARCore session sestup
    val session = Session(this)
    val config = Config(session)
    // Open image database
    val inputStream = context.assets.open("my_images_db.imgdb")
    val imageDatabase = AugmentedImageDatabase.deserialize(inputStream);
    // Enable image tracking
    config.setAugmentedImageDatabase(imageDatabase);
    session.configure(config);
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

29. Track images
    // in onDrawFrame()
    val frame = session.update()
    val updatedAugmentedImages = frame.getUpdatedTrackables(AugmentedImage::class.java)
    updatedAugmentedImages.forEach {
    when (it.trackingState) {
    TrackingState.PAUSE -> // Detected, but not yet tracked
    TrackingState.TRACKING -> // Tracking! Create an anchor
    TrackingState.STOPPED -> // Remove anchor
    }
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

30. Cloud Anchors
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

31. Cloud Anchors
    android:name="com.google.android.ar.API_KEY"
    android:value="" />
    Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide

32. Moble AR with ARCore - @ErikHellman, Øredev 2018
    

    View Slide