Building your first Augmented Reality Experiences

Augmented reality allows you to create richer and more immersive experiences in your apps
In this interactive session, Noble Ackerson, a seasoned Technical Product Lead and GDE, shares insights with examples through a fun "choose your own adventure" technical talk (Bandersnatch-style), on how to build marker-based AR with Augmented Images to create experiences to transform data from the world into practical utility for your users.

Transcript

1. Building your first AR Experience It’s dangerous to go alone, take AR With Noble Ackerson
2. 2 Help you understand how easy it is to build for Augmented Reality Convey that AR is more than “Pokemon-Go” it brings real utility to your apps. Today’s Goals
3. Green Red #ARCoreParkour #CYOP @nobleackerson Feedback
4. Hi, I’m Noble Product Strategist, OpenARCloud founding member /c/nobleackerson @nobleackerson www.nobles.page
5. The mixture of the real world and virtual worlds so that one understands the other. This creates experiences that cannot possibly happen anywhere else. AR/MR
6. Augmented 1 Mixed 2 Virtual 3 Spatial Cloud
7. Assumes • You have a use-case that is better with AR • You have basic knowledge of Unity3D • You have setup an Android environment before • You are armed with curiosity about AR?
8. So what is ARCore?
9. ARCore fundamentals ENVIRONMENTAL UNDERSTANDING MOTION TRACKING (Persistence) LIGHT ESTIMATION (Environmental HDR) /c/nobleackerson @nobleackerson www.nobles.page CLOUD ANCHORS (Interactions)
11. Social AR experiences with Cloud Anchors Fundamentals • Environment ◦ Reduce UI elements • User ◦ Encourage user movement when interacting with objects but don’t require it always. • AR Content ◦ Realism is important ◦ Highlighting, Sizing & Movement should be familiar gestures • Interface ◦ The interface is the viewport, & also, the hub of interactions for mobile AR. • Communicate the intent of camera usage clearly before permission prompts. • Be responsive with your UI components. • Volumetric UI ◦ Annotate on focus ◦ Holograms are good previews
12. Augmented Faces ARCore Anchors Cloud Anchors Augmented Images
13. Demo(s) /c/nobleackerson @nobleackerson www.nobles.page
14. EVERYONE HAS A PLAN UNTIL THEY GET PUNCHED IN THE MOUTH. Mike Tyson
15. Development Environment for ARCore Unity 2017.4.15f1 + 2018.3.1f1 ARCore SDK 1.6.0 + A compatible phone developers.google.com/ar/discover/supported-devices
16. Building with Unity3D Design Principles
17. How do augmented images work? Precompile up to 1000 images via DB 1 Add Images to DB 2 Detect Images 3 Render Assets (video, 3D prefabs, etc.)
18. Add Images to Database Precompile up to 1000 images via DB 1
19. Create database file
21. Create Session: Use DB File
22. Detect Images 2
23. myARController.cs Update Method
24. myARController.cs Update Method
26. Render Assets 3
27. An example of how the Augmented Images API can be used with moving targets by JD.com Credit Google Developer Blog - https://developers.googleblog.com/2019/05/AR Core-IO19.html
28. Rendering • Utilities ◦ Blender ◦ Maya etc. ◦ SketchFab ◦ Asset Store ◦ Mixamo • Asset Formats ◦ OBJ ◦ FBX ◦ GLTF 2.0
29. Anatomy of your app ➔ ARCore Device ➔ Environmental Light ➔ A Plane Detector ➔ A Raycast from Device ➔ A Plane Visualizer ➔ System Controller ➔ An Event System ➔ A 2D Canvas ➔ Interactions ➔ Some Magic
30. Pro-tips ➔ Supports PNG, JPEG and JPG formats ➔ Detection is based on points of high contrast ➔ Image’s resolution must be at least 300 X 300 pixels ➔ Images with repetitive features suck ➔ Don’t try to track moving objects.
31. Design Principles for Augmented Reality CONTENT Avoid separate onscreen controls. UX Use the entire display to engage people. ACCESSIBILITY Be mindful of the user's safety and privacy
32. Accessibility
34. UI INITIALIZATION PLACEMENT SOUND/HAPTICS
35. Audio enhances AR experiences. Provide confirmation that a virtual object has come into contact with a physical surface or other virtual object Use sound to aid with getting interactions with precision for your users.
36. User Experience
37. UI: Translation
38. Environmental HDR Lighting (ARCore)
39. Lighting sample
40. Privacy, Safety & Comfort
41. Safety, Comfort, Happiness, Engagement, Utility Image Credit: Luden.io Testing & Validation
42. In summary ➔ AR has practical use cases & brings immersion ➔ Reality in Augmented Reality is Data ➔ Have Accessibility, Safety, & Privacy in mind /c/nobleackerson @nobleackerson
43. Thank you! @nobleackerson github.com/stigsfoot/ARCoreParkour