Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building your first Augmented Reality Experiences

Building your first Augmented Reality Experiences

Augmented reality allows you to create richer and more immersive experiences in your apps
In this interactive session, Noble Ackerson, a seasoned Technical Product Lead and GDE, shares insights with examples through a fun "choose your own adventure" technical talk (Bandersnatch-style), on how to build marker-based AR with Augmented Images to create experiences to transform data from the world into practical utility for your users.

Noble Ackerson

August 22, 2019
Tweet

More Decks by Noble Ackerson

Other Decks in Technology

Transcript

  1. Building your first AR Experience
    It’s dangerous to go alone, take AR
    With Noble Ackerson

    View full-size slide

  2. 2
    Help you understand how easy it
    is to build for Augmented
    Reality
    Convey that AR is more than
    “Pokemon-Go” it brings real
    utility to your apps.
    Today’s Goals

    View full-size slide

  3. Green Red
    #ARCoreParkour
    #CYOP
    @nobleackerson
    Feedback

    View full-size slide

  4. Hi, I’m Noble
    Product Strategist, OpenARCloud founding member
    /c/nobleackerson @nobleackerson www.nobles.page

    View full-size slide

  5. The mixture of the real
    world and virtual worlds so
    that one understands the
    other.
    This creates experiences that cannot
    possibly happen anywhere else.
    AR/MR

    View full-size slide

  6. Augmented
    1
    Mixed
    2
    Virtual
    3
    Spatial Cloud

    View full-size slide

  7. Assumes
    ● You have a use-case that is better with AR
    ● You have basic knowledge of Unity3D
    ● You have setup an Android environment before
    ● You are armed with curiosity about AR?

    View full-size slide

  8. So what is ARCore?

    View full-size slide

  9. ARCore fundamentals
    ENVIRONMENTAL
    UNDERSTANDING
    MOTION TRACKING
    (Persistence)
    LIGHT ESTIMATION
    (Environmental HDR)
    /c/nobleackerson @nobleackerson www.nobles.page
    CLOUD ANCHORS
    (Interactions)

    View full-size slide

  10. Social AR experiences with Cloud Anchors
    Fundamentals
    ● Environment
    ○ Reduce UI elements
    ● User
    ○ Encourage user movement when
    interacting with objects but don’t
    require it always.
    ● AR Content
    ○ Realism is important
    ○ Highlighting, Sizing & Movement
    should be familiar gestures
    ● Interface

    The interface is the viewport, & also,
    the hub of interactions for mobile AR.
    ● Communicate the intent of
    camera usage clearly before
    permission prompts.
    ● Be responsive with your UI
    components.
    ● Volumetric UI
    ○ Annotate on focus
    ○ Holograms are good previews

    View full-size slide

  11. Augmented Faces
    ARCore Anchors
    Cloud Anchors
    Augmented Images

    View full-size slide

  12. Demo(s)
    /c/nobleackerson @nobleackerson www.nobles.page

    View full-size slide

  13. EVERYONE HAS A PLAN
    UNTIL THEY GET PUNCHED
    IN THE MOUTH.
    Mike Tyson

    View full-size slide

  14. Development Environment for ARCore
    Unity 2017.4.15f1 +
    2018.3.1f1
    ARCore SDK 1.6.0 +
    A compatible phone
    developers.google.com/ar/discover/supported-devices

    View full-size slide

  15. Building with Unity3D Design Principles

    View full-size slide

  16. How do augmented images work?
    Precompile up to 1000
    images via DB
    1
    Add Images to DB
    2
    Detect Images
    3
    Render Assets
    (video, 3D prefabs, etc.)

    View full-size slide

  17. Add Images to Database
    Precompile up to 1000
    images via DB
    1

    View full-size slide

  18. Create database file

    View full-size slide

  19. Create Session: Use DB File

    View full-size slide

  20. Detect Images
    2

    View full-size slide

  21. myARController.cs
    Update Method

    View full-size slide

  22. myARController.cs
    Update Method

    View full-size slide

  23. Render Assets
    3

    View full-size slide

  24. An example of how the
    Augmented Images API can be
    used with moving targets by
    JD.com
    Credit Google Developer Blog -
    https://developers.googleblog.com/2019/05/AR
    Core-IO19.html

    View full-size slide

  25. Rendering
    ● Utilities
    ○ Blender
    ○ Maya
    etc.
    ○ SketchFab
    ○ Asset Store
    ○ Mixamo
    ● Asset Formats
    ○ OBJ
    ○ FBX
    ○ GLTF 2.0

    View full-size slide

  26. Anatomy of your app
    ➔ ARCore Device
    ➔ Environmental Light
    ➔ A Plane Detector
    ➔ A Raycast from Device
    ➔ A Plane Visualizer
    ➔ System Controller
    ➔ An Event System
    ➔ A 2D Canvas
    ➔ Interactions
    ➔ Some Magic

    View full-size slide

  27. Pro-tips
    ➔ Supports PNG, JPEG and JPG formats
    ➔ Detection is based on points of high contrast
    ➔ Image’s resolution must be at least 300 X 300 pixels
    ➔ Images with repetitive features suck
    ➔ Don’t try to track moving objects.

    View full-size slide

  28. Design Principles for Augmented Reality
    CONTENT
    Avoid separate onscreen
    controls.
    UX
    Use the entire display to engage
    people.
    ACCESSIBILITY
    Be mindful of the user's safety
    and privacy

    View full-size slide

  29. Accessibility

    View full-size slide

  30. UI
    INITIALIZATION PLACEMENT SOUND/HAPTICS

    View full-size slide

  31. Audio enhances
    AR experiences.
    Provide confirmation that a
    virtual object has come into
    contact with a physical surface
    or other virtual object
    Use sound to aid with getting
    interactions with precision for
    your users.

    View full-size slide

  32. User Experience

    View full-size slide

  33. UI: Translation

    View full-size slide

  34. Environmental HDR
    Lighting (ARCore)

    View full-size slide

  35. Lighting sample

    View full-size slide

  36. Privacy, Safety &
    Comfort

    View full-size slide

  37. Safety, Comfort, Happiness, Engagement, Utility
    Image Credit: Luden.io
    Testing & Validation

    View full-size slide

  38. In summary
    ➔ AR has practical use cases & brings immersion
    ➔ Reality in Augmented Reality is Data
    ➔ Have Accessibility, Safety, & Privacy in mind
    /c/nobleackerson @nobleackerson

    View full-size slide

  39. Thank you!
    @nobleackerson github.com/stigsfoot/ARCoreParkour

    View full-size slide