GDGDC 2019 - ARCore Parkour | Immersive Experiences and Beyond

Today there are lots of case studies for Spatial Computing (Augmented Reality, Virtual Reality, Mixed Reality etc) in education and entertainment. Unfortunately, the newly released ARCore features like Augmented Images still lack tutorials and practical examples.

This interactive session covers 'the how' about Augmented Reality. We learn how to build using Unity3D and the ARCore SDK

Transcript

1. ARCore Parkour
   A technical talk on building Augmented
   Reality experiences with ARCore
   With Noble Ackerson
   

   View Slide

2. EVERYONE HAS A PLAN
   UNTIL THEY GET PUNCHED IN
   THE MOUTH.
   Mike Tyson
   

   View Slide

3. Sugar Puffs Frosties
   #ARCoreParkour
   #CYOP
   @nobleackerson
   Feedback
   

   View Slide

4. @nobleackerson
   Product Strategist
   Open ARCloud Privacy WGC acting co-chair
   www.nobles.page
   Hi, I’m Noble!
   

   View Slide

5. 5
   5
   Help you understand how easy it is to build
   for Augmented Reality
   Convey that AR is more than
   “Pokemon-Go” it brings real utility to your
   apps.
   Today’s Goals
   

   View Slide

6. Assumes
   ● You have basic knowledge of Unity3D
   ● You have setup an Android environment before
   ● You are armed with curiosity about AR?
   

   View Slide

7. Spatial Computing
   Augmented
   1
   Mixed
   2
   Virtual
   3
   Reality
   /c/nobleackerson @nobleackerson www.nobles.page
   

   View Slide

8. The mixture of the real world
   and virtual worlds so that one
   understands the other.
   This creates experiences that cannot
   possibly happen anywhere else.
   AR/MR
   

   View Slide

9. Image (Slide) credit Rony Abovitz @rabovitz
   

   View Slide

10. AR use-cases About ARCore
    

    View Slide

11. AR: Common business use-cases
    Data capture & markup Entertainment Commerce & Marketing Maintenance and repair
    /c/nobleackerson @nobleackerson www.nobles.page
    

    View Slide

12. Use-case #1: Data
    capture & markup
    

    View Slide

13. View Slide

14. Use-case #2:
    Entertainment
    

    View Slide

15. Magic Leap
    

    View Slide

16. Use case #3:
    Commerce and
    Marketing
    

    View Slide

17. View Slide

18. Use case #4: Aided Maintenance
    and Repair
    

    View Slide

19. View Slide

20. Data Visualization
    

    View Slide

21. Credit: AR Experiments
    

    View Slide

22. LynxFit
    

    View Slide

23. Design Principles About ARCore
    

    View Slide

24. So what is ARCore?
    

    View Slide

25. ARCore fundamentals
    ENVIRONMENTAL
    UNDERSTANDING
    MOTION TRACKING LIGHT ESTIMATION
    /c/nobleackerson @nobleackerson www.nobles.page
    

    View Slide

26. Augmented Faces
    ARCore Anchors
    Cloud Anchors
    Augmented Images
    

    View Slide

27. Demo(s)
    /c/nobleackerson @nobleackerson www.nobles.page
    

    View Slide

28. Development Environment for ARCore
    Unity 2017.4.15f1 +
    2018.3.1f1
    ARCore SDK 1.6.0 +
    A compatible phone
    developers.google.com/ar/discover/supported-devices
    

    View Slide

29. Building with Unity3D Design Principles
    

    View Slide

30. How do augmented images work?
    Precompile up to 1000
    images via DB
    1
    Add Images to DB
    2
    Detect Images
    3
    Render Assets
    (video, 3D prefabs, etc.)
    

    View Slide

31. Add Images to Database
    Precompile up to 1000
    images via DB
    1
    

    View Slide

32. Create database file
    

    View Slide

33. View Slide

34. Create Session: Use DB File
    

    View Slide

35. Detect Images
    2
    

    View Slide

36. myARController.cs
    

    View Slide

37. Render Assets
    3
    

    View Slide

38. Rendering
    ● Utilities
    ○ Blender
    ○ Maya
    etc.
    ○ SketchFab
    ○ Asset Store
    ○ Mixamo
    ● Asset Formats
    ○ OBJ
    ○ FBX
    ○ GLTF 2.0
    

    View Slide

39. Anatomy of your app
    ➔ ARCore Device
    ➔ Environmental Light
    ➔ A Plane Detector
    ➔ A Raycast from Device
    ➔ A Plane Visualizer
    ➔ System Controller
    ➔ An Event System
    ➔ A 2D Canvas
    ➔ Interactions
    ➔ Some Magic
    

    View Slide

40. Pro-tips
    ➔ Supports PNG, JPEG and JPG formats
    ➔ Detection is based on points of high contrast
    ➔ Image’s resolution must be at least 300 X 300 pixels
    ➔ Images with repetitive features suck
    ➔ Don’t try to track moving objects.
    

    View Slide

41. Design Considerations Trends
    

    View Slide

42. Design Principles for Augmented Reality
    CONTENT
    Avoid separate onscreen
    controls.
    UX
    Use the entire display to engage
    people.
    ACCESSIBILITY
    Be mindful of the user's safety
    and privacy
    

    View Slide

43. Accessibility
    

    View Slide

44. View Slide

45. UI
    INITIALIZATION PLACEMENT SOUND/HAPTICS
    

    View Slide

46. Audio enhances
    AR experiences.
    Provide confirmation that a
    virtual object has come into
    contact with a physical surface
    or other virtual object
    Use sound to aid with getting
    interactions with precision for
    your users.
    

    View Slide

47. User Experience
    

    View Slide

48. UI: Translation
    

    View Slide

49. Privacy, Safety &
    Comfort
    

    View Slide

50. Safety, Comfort, Happiness, Engagement, Utility
    Image Credit: Luden.io
    Testing & Validation
    

    View Slide

51. Augmented Images About ARCore
    

    View Slide

52. In summary
    ➔ AR has practical use cases
    ➔ Reality in Augmented Reality is Data
    ➔ Have Accessibility, Safety, & Privacy in mind
    /c/nobleackerson @nobleackerson
    

    View Slide

53. Thank you!
    @nobleackerson www.nobles.page
    

    View Slide

54. Resources
    AR Concepts:
    https://developers.google.com/ar/di
    scover/concepts
    Google Developers ARCore
    https://developers.google.com/ar/
    Github projects for ARCore:
    https://github.com/google-ar
    AR experiments for inspiration and
    to see what could be possible:
    https://experiments.withgoogle.com/ar
    

    View Slide

55. View Slide