Building your first Augmented Reality Experiences

We took a simple use-case of bringing immersive experiences to online car buying during this DevFest DC talk on building marker-based AR.

Transcript

1. Building your first AR Experience
   It’s dangerous to go alone, take AR
   With Noble Ackerson
   

   View Slide

2. 2
   2
   Help you understand how easy it
   is to build for Augmented
   Reality
   Convey that AR is more than
   “Pokemon-Go” it brings real
   utility to your apps.
   Today’s Goals
   HOW
   

   View Slide

3. Green Red
   #ARCoreParkour
   #CYOP
   @nobleackerson
   Feedback
   

   View Slide

4. Hi, I’m Noble
   Product Strategist, OpenARCloud founding member
   /c/nobleackerson @nobleackerson www.nobles.page
   HOW
   

   View Slide

5. The mixture of the real
   world and virtual worlds so
   that one understands the
   other.
   This creates experiences that cannot
   possibly happen anywhere else.
   AR/MR
   HOW
   

   View Slide

6. Augmented
   1
   Mixed
   2
   Virtual
   3
   HOW
   Spatial Cloud
   

   View Slide

7. Assumes
   ● You have a use-case that is better with AR
   ● You have basic knowledge of Unity3D
   ● You have setup an Android environment before
   ● You are armed with curiosity about AR?
   HOW
   

   View Slide

8. So what is ARCore?
   HOW
   

   View Slide

9. ARCore fundamentals
   ENVIRONMENTAL
   UNDERSTANDING
   MOTION TRACKING
   (Persistence)
   LIGHT ESTIMATION
   (Environmental HDR)
   /c/nobleackerson @nobleackerson www.nobles.page
   HOW
   CLOUD ANCHORS
   (Interactions)
   

   View Slide

10. Source: Alexei Evdokimov Dribbble
    

    View Slide

11. Social AR experiences with Cloud Anchors
    Fundamentals
    ● Environment
    ○ Reduce UI elements
    ● User
    ○ Encourage user movement when
    interacting with objects but don’t
    require it always.
    ● AR Content
    ○ Realism is important
    ○ Highlighting, Sizing & Movement
    should be familiar gestures
    ● Interface
    ○
    The interface is the viewport, & also,
    the hub of interactions for mobile AR.
    ● Communicate the intent of
    camera usage clearly before
    permission prompts.
    ● Be responsive with your UI
    components.
    ● Volumetric UI
    ○ Annotate on focus
    ○ Holograms are good previews
    youtube.com/c/nobleackerson
    

    View Slide

12. Augmented Faces
    ARCore Anchors
    Cloud Anchors
    Augmented Images
    HOW
    

    View Slide

13. Demo(s)
    /c/nobleackerson @nobleackerson www.nobles.page
    

    View Slide

14. EVERYONE HAS A PLAN
    UNTIL THEY GET PUNCHED
    IN THE MOUTH.
    Mike Tyson
    

    View Slide

15. Development Environment for ARCore
    Unity 2017.4.15f1 +
    2018.3.1f1
    ARCore SDK 1.6.0 +
    A compatible phone
    developers.google.com/ar/discover/supported-devices
    

    View Slide

16. Building with Unity3D Design Principles
    

    View Slide

17. How do augmented images work?
    Precompile up to 1000
    images via DB
    1
    Add Images to DB
    2
    Detect Images
    3
    Render Assets
    (video, 3D prefabs, etc.)
    HOW
    

    View Slide

18. Add Images to Database
    Precompile up to 1000
    images via DB
    1
    HOW
    

    View Slide

19. Create database file
    HOW
    

    View Slide

20. HOW
    

    View Slide

21. Create Session: Use DB File
    HOW
    

    View Slide

22. Detect Images
    2
    HOW
    

    View Slide

23. myARController.cs
    Update Method
    

    View Slide

24. myARController.cs
    Update Method
    

    View Slide

25. View Slide

26. Render Assets
    3
    HOW
    

    View Slide

27. An example of how the
    Augmented Images API can be
    used with moving targets by
    JD.com
    Credit Google Developer Blog -
    https://developers.googleblog.com/2019/05/AR
    Core-IO19.html
    

    View Slide

28. Rendering
    ● Utilities
    ○ Blender
    ○ Maya
    etc.
    ○ SketchFab
    ○ Asset Store
    ○ Mixamo
    ● Asset Formats
    ○ OBJ
    ○ FBX
    ○ GLTF 2.0
    

    View Slide

29. Anatomy of your app
    ➔ ARCore Device
    ➔ Environmental Light
    ➔ A Plane Detector
    ➔ A Raycast from Device
    ➔ A Plane Visualizer
    ➔ System Controller
    ➔ An Event System
    ➔ A 2D Canvas
    ➔ Interactions
    ➔ Some Magic
    

    View Slide

30. Pro-tips
    ➔ Supports PNG, JPEG and JPG formats
    ➔ Detection is based on points of high contrast
    ➔ Image’s resolution must be at least 300 X 300 pixels
    ➔ Images with repetitive features suck
    ➔ Don’t try to track moving objects.
    

    View Slide

31. Design Principles for Augmented Reality
    CONTENT
    Avoid separate onscreen
    controls.
    UX
    Use the entire display to engage
    people.
    ACCESSIBILITY
    Be mindful of the user's safety
    and privacy
    

    View Slide

32. Accessibility
    

    View Slide

33. View Slide

34. UI
    INITIALIZATION PLACEMENT SOUND/HAPTICS
    

    View Slide

35. Audio enhances
    AR experiences.
    Provide confirmation that a
    virtual object has come into
    contact with a physical surface
    or other virtual object
    Use sound to aid with getting
    interactions with precision for
    your users.
    

    View Slide

36. User Experience
    

    View Slide

37. UI: Translation
    

    View Slide

38. Environmental HDR
    Lighting (ARCore)
    

    View Slide

39. Lighting sample
    

    View Slide

40. Privacy, Safety &
    Comfort
    

    View Slide

41. Safety, Comfort, Happiness, Engagement, Utility
    Image Credit: Luden.io
    Testing & Validation
    

    View Slide

42. In summary
    ➔ AR has practical use cases
    ➔ Reality in Augmented Reality is Data
    ➔ Have Accessibility, Safety, & Privacy in mind
    /c/nobleackerson @nobleackerson
    

    View Slide

43. Thank you!
    @nobleackerson github.com/stigsfoot/ARCoreParkour
    

    View Slide