$30 off During Our Annual Pro Sale. View Details »

Mobile_AR_with_ARCore.pdf

Erik Hellman
November 19, 2018
34

 Mobile_AR_with_ARCore.pdf

Erik Hellman

November 19, 2018
Tweet

Transcript

  1. Mobile AR
    with ARCore
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  2. https://9to5mac.com/2018/05/16/comment-stop-using-ar-as-a-gimmick/

    View Slide

  3. Transformative AR
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  4. Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  5. Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  6. MERGE Cube
    https://mergevr.com/cube

    View Slide

  7. Microsoft HoloLens
    https://www.engadget.com/2017/11/01/microsoft-hololens-hard-hat-european-expansion/

    View Slide

  8. Magic Leap
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  9. Transformative AR
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  10. ARCore from Google
    https://developers.google.com/ar

    View Slide

  11. Motion Tracking
    https://developers.google.com/ar/discover/concepts

    View Slide

  12. Environmental understanding
    https://developers.google.com/ar/discover/concepts

    View Slide

  13. Light estimation
    https://developers.google.com/ar/discover/concepts

    View Slide

  14. User interaction
    Oriented Points
    Anchors and Trackables
    Augmented Images
    Cloud Anchors
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  15. Anchors
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  16. Getting started
    with ARCore
    (Android)
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  17. override fun onResume() {
    super.onResume()
    val arCoreApk = ArCoreApk.getInstance()
    if (!arCoreApk.checkAvailability(this).isSupported()) {
    // Not supported - notify user!
    return
    }
    if (arCoreApk.requestInstall(this, true)
    == ArCoreApk.InstallStatus.INSTALL_REQUESTED)) {
    // Installation needed - activity will pause
    return
    }
    // Everything is ok to go!
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  18. Sceneform (Android)
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  19. Sceneform
    → High-level scenegraph API
    → Realistic Physical Based Rendering using Filament
    → Android Studio plugin
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  20. Filament
    https://github.com/google/filament

    View Slide

  21. Sceneform
    dependencies {
    // Provides ARCore Session and related resources.
    implementation 'com.google.ar:core:1.5.0'
    // Provides ArFragment, and other UX resources.
    implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.5.1'
    // Alternatively, use ArSceneView without the UX dependency.
    implementation 'com.google.ar.sceneform:core:1.5.1'
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  22. Sceneform
    android:id="@+id/ar_fragment"
    android:name="com.google.ar.sceneform.ux.ArFragment"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  23. Sceneform
    val arFragment = fragmentManager
    .findFragmentById(R.id.ux_fragment) as ArFragment?
    arFragment?.setOnTapArPlaneListener { hitResult, plane, motionEvent ->
    // Create new anchor and node
    val anchor = hitResult.createAnchor()
    val anchorNode = AnchorNode(anchor)
    // Add to the scene
    anchorNode.setParent(arFragment.arSceneView.scene)
    // TODO Add content to the new anchor...
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  24. Scneeform - Display models
    arFragment?.setOnTapArPlaneListener { hitResult, plane, motionEvent ->
    ModelRenderable.builder()
    .setSource(this, Uri.parse("earth.sfb"))
    .build()
    .thenAccept {
    val anchor = hitResult.createAnchor()
    val anchorNode = AnchorNode(anchor)
    val transformableNode = TransformableNode(arFragment.transformationSystem)
    .apply {
    setParent(anchorNode)
    renderable = it
    }
    anchorNode.addChild(transformableNode)
    anchorNode.setParent(arFragment.arSceneView.scene)
    }
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  25. Sceneform - Display Android views
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  26. Sceneform - Display Android views
    arFragment?.setOnTapArPlaneListener { hitResult, plane, motionEvent ->
    ViewRenderable.builder()
    .setView(this, R.layout.simple_form)
    .build()
    .thenAccept { card ->
    val anchor = hitResult.createAnchor()
    val anchorNode = AnchorNode(anchor)
    TransformableNode(arFragment.transformationSystem).apply {
    setParent(anchorNode)
    renderable = card
    select()
    }
    anchorNode.setParent(arFragment.arSceneView.scene)
    card.view.findViewById(R.id.okBtn).setOnClickListener {
    arFragment.arSceneView.scene.removeChild(anchorNode)
    anchor.detach()
    }
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  27. Augmented Images
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  28. Configure Augmented Images
    // ARCore session sestup
    val session = Session(this)
    val config = Config(session)
    // Open image database
    val inputStream = context.assets.open("my_images_db.imgdb")
    val imageDatabase = AugmentedImageDatabase.deserialize(inputStream);
    // Enable image tracking
    config.setAugmentedImageDatabase(imageDatabase);
    session.configure(config);
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  29. Track images
    // in onDrawFrame()
    val frame = session.update()
    val updatedAugmentedImages = frame.getUpdatedTrackables(AugmentedImage::class.java)
    updatedAugmentedImages.forEach {
    when (it.trackingState) {
    TrackingState.PAUSE -> // Detected, but not yet tracked
    TrackingState.TRACKING -> // Tracking! Create an anchor
    TrackingState.STOPPED -> // Remove anchor
    }
    }
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  30. Cloud Anchors
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  31. Cloud Anchors
    android:name="com.google.android.ar.API_KEY"
    android:value="" />
    Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide

  32. Moble AR with ARCore - @ErikHellman, Øredev 2018

    View Slide