DevfestMN - The "R" in AR means data and Augmented Images makes it happen

Today there are lots of case studies for Spatial Computing (Augmented Reality, Virtual Reality, Mixed Reality etc) in education and entertainment. Unfortunately, the newly released ARCore features like Augmented Images still lack tutorials and practical examples.

In this interactive session, Noble Ackerson, a seasoned Technical Product Lead and GDE, shares insights with examples through a fun "choose your own adventure" technical talk (Bandersnatch-style), on how to build marker-based AR with Augmented Images to create experiences to transform data from the world into practical utility for your users.

This audience chose to focus on practical use-cases of Marker Based AR

Transcript

1. How Augmented Images
   Brings data to apps
   A choose your own adventure-style technical talk
   With Noble Ackerson
   

   View Slide

2. Hi, I’m Noble
   Product Strategist, Independent Dev
   /c/nobleackerson @nobleackerson www.nobles.page
   

   View Slide

3. View Slide

4. Spatial Computing
   Augmented
   1
   Mixed
   2
   Virtual
   3
   Reality
   /c/nobleackerson @nobleackerson
   

   View Slide

5. The mixture of the real world
   and virtual worlds so that one
   understands the other.
   This creates experiences that cannot
   possibly happen anywhere else.
   AR/MR
   

   View Slide

6. The R is for Data
   Image Credit: https://dribbble.com/AndrewCoyleDesign
   Spatial computing works
   best when computers
   interpret the environment
   and return responses that
   engage our senses.
   

   View Slide

7. AR: Common business use-cases
   Data capture & markup Entertainment Commerce & Marketing Maintenance and repair
   /c/nobleackerson @nobleackerson
   

   View Slide

8. Use-case #1: Data
   capture & markup
   

   View Slide

9. View Slide

10. Use-case #2: Entertainment
    

    View Slide

11. Magic Leap
    

    View Slide

12. Use case #3: Commerce
    and Marketing
    

    View Slide

13. View Slide

14. Use case #4: Aided Maintenance
    and Repair
    

    View Slide

15. View Slide

16. Data Visualization
    

    View Slide

17. Credit: AR Experiments
    

    View Slide

18. LynxFit
    

    View Slide

19. So what is ARCore?
    /c/nobleackerson @nobleackerson
    

    View Slide

20. ARCore
    Environmental
    Understanding
    Motion
    Tracking
    Light
    Estimation
    

    View Slide

21. Augmented Images
    Marker-based AR where the digital world is
    anchored to the real world
    /c/nobleackerson @nobleackerson
    

    View Slide

22. Development Environment for ARCore
    Unity 2017.4.15f1 +
    2018.3.1f1
    ARCore SDK 1.6.0 +
    A compatible phone
    developers.google.com/ar/discover/supported-devices
    

    View Slide

23. How do augmented images work?
    Precompile up to 1000
    images via DB
    1
    Add Images to DB
    2
    Detect Images
    3
    Render Assets
    (video, 3D prefabs, etc.)
    

    View Slide

24. Create database file
    

    View Slide

25. Detect Images
    

    View Slide

26. Pro-tips
    ● Supports PNG, JPEG and JPG formats
    ● Detection is based on points of high contrast
    ● Image’s resolution must be at least 300 X 300 pixels
    ● Images with repetitive features suck
    ● Don’t try to track moving objects.
    

    View Slide

27. Editor time
    

    View Slide

28. Demos
    

    View Slide

29. Some Design Principles for Augmented Reality
    Be mindful of the user's safety
    and privacy
    Avoid separate onscreen
    controls.
    Use the entire display to
    engage people.
    

    View Slide

30. Safety, Comfort, Happiness, Engagement, Utility
    Image Credit: Luden.io
    Testing & Validation
    

    View Slide

31. What About Audio?
    /c/nobleackerson @nobleackerson
    

    View Slide

32. Audio enhances
    AR experiences.
    Provide confirmation that a
    virtual object has come into
    contact with a physical surface
    or other virtual object
    Use sound to aid with getting
    interactions with precision for
    your users.
    

    View Slide

33. In summary
    ● AR has practical usecases
    ● Reality in Augmented Reality is Data
    ● Make sure your editor/devices are supported
    ● Build for your device (mobile vs head-worn)
    ● Augmented Images have privacy implications
    ● Be mindful of the user's safety
    ● Favor functional experiences that matter
    /c/nobleackerson @nobleackerson
    

    View Slide

34. Thank you!
    @nobleackerson www.nobles.page
    

    View Slide

35. Resources
    AR Concepts:
    https://developers.google.com/ar/di
    scover/concepts
    Google Developers ARCore
    https://developers.google.com/ar/
    Github projects for ARCore:
    https://github.com/google-ar
    AR experiments for inspiration and
    to see what could be possible:
    https://experiments.withgoogle.com/ar
    

    View Slide