ARCore is (still) out there

ARCore is the Android SDK for providing Augmented Reality experiences to our users. In the last 2 years Google seems to not put so much effort in it development, but at the last Google IO they showed new shining features and performance improvements thanks to ML usage. The most important one is the possibility to create georeferenced AR experiences without moving from our desk. In this session we will see these updates using ARFoundation, the Unity3D SDK that wraps ARCore and ARkit (for iOS)

Transcript

1. Mike Trizio ARCore is (still) out there
2. About me Computer Engineer CTO @ Wideverse GDG Bari community lead
3. Immersive computing spectrum Real World Computer-generated Real reality Augmented reality Virtual reality Intro
4. 4 Intro Augmented Reality AR can bring anything to you. It adds computer-generated information and objects to your everyday world.
5. AR can be useful Image Intro
6. AR can be fun Intro
7. classic Vs Mixed Intro
10. ARCore Overview
11. 11 About ARCore Proprietary and Confidential Google’s developer platform for building mobile augmented reality (AR) experiences provides powerful immersion capabilities to help you integrate virtual content realistically on top of your live camera stream Image ARCore Overview
12. Luke, I’m your father !!!!!!!!!!!!!!!!!!!!! ARCore Overview A long time ago…
13. ARCore Overview
15. ARCore Overview Augmented Reality history timeline 1962 Sensorama 1968 Head Mounted Display 2012 Google Glass 2014 Project Tango 2016 Microsoft Hololens 2017-2018 ARkit + ARCore
16. ARCore Skill Tree ARCore Overview Augmentation Post-Production World Annotation Immersion Foundations Cloud Services
17. Immersion
18. 18 Motion Tracking Proprietary and Confidential By tracking the phone’s motion, virtual objects stay anchored in place Image Immersion 17% faster to first plane 15% less tracking reset Improvements
19. 19 Motion Tracking Proprietary and Confidential New machine learning model added Even faster initialization Even fewer tracking resets Immersion Improvements + machine learning
20. Best practice for reliable motion tracking Motion Encourage user motion at the start of Experiences Lightning Encourage well-lit environments to reduce image blur Scene texture Avoid blank walls. Encourage use in interesting environments Immersion
21. 21 Lighting Estimation Proprietary and Confidential ARCore can detect information about the lighting of its environment so you can render your virtual objects under the same conditions as the environment around them. Image Immersion
22. Reflection Shading Shadows Specular highlight Shading Specular highlight Shadows Immersion Ambient Lighting
23. Lightning Immersion
24. Immersion
25. Depth Generate a depth map describing the 3D shape of the world Immersion
26. Occlusion problem Immersion
27. Occlusion Immersion
28. Depth Machine learning 15% faster to initial depth 95% coverage of average depth map Immersion
29. Long Range Depth (new) Works outdoors at distances up to 20 meters Image Immersion
30. Augmentation
31. 31 Planes Proprietary and Confidential ARCore understands the physical structure of the environment, detecting horizontal surfaces, like tables and desks, and makes these surfaces available to your app as planes. Image Augmentation
32. Augmented Images You’ve wanted to have the ability to trigger AR experiences off of real-world images Image Augmentation
34. Augmented images use cases
35. Use Cases Advertising Products Education
36. Use Cases Advertising Products Education
37. Use Cases Advertising Products Education
38. Augmented Faces You’ve wanted to have the ability to trigger AR experiences off of your head Augmentation
39. Instant Placement You’ve wanted to have the ability to trigger AR experiences off of your head Augmentation
40. Hit Test Planes Feature Points Depth Persistent Raycast Augmentation
41. Post Production
42. Recording Post Production
43. Playback Post Production
44. World Annotation
45. Persistent Cloud anchors ARCore's Cloud Anchors API lets you create collaborative or multiplayer apps Image World Annotation
47. Cloud anchors use cases Proprietary and Confidential
48. Use Cases Gaming Shopping Creative Expression Education
49. Use Cases Gaming Shopping Creative Expression Education
50. Use Cases Gaming Shopping Creative Expression Education
51. Gaming Shopping Creative Expression Use cases Education
52. Geospatial API (new) Create AR experiences all across the world World Annotation
53. Geospatial API advantages comparison Proprietary and Confidential GPS + IMU Cloud Anchors Global Localization Global-scale coverage ✔ ✘ ✔ Precise location ✘ ✔ ✔ Precise orientation ✘ ✔ ✔ Remote placement ✔ ✘ ✔ location guidance ✔ ✘ ✔
54. 54 Proprietary and Confidential Cloud services are Cross Platform
55. Some code now…
58. AR Foundation
61. var earthTrackingState = EarthManager.EarthTrackingState; if (earthTrackingState == TrackingState.Tracking){ // camera_geospatial_pose contains geodetic location, rotation, and // confidences values. var cameraGeospatialPose = EarthManager.CameraGeospatialPose; }
63. Quaternion quaternion = Quaternion.AngleAxis(180f - (float)pose.Heading, Vector3.up); if (earthTrackingState == TrackingState.Tracking){ var anchor = AnchorManager.AddAnchor( latitude, longitude, altitude, quaternion); var anchoredAsset = Instantiate(GeospatialAssetPrefab, anchor.transform); }
65. Developing Magic Broom
67. My design skill
68. Poly & Sketchfab
69. Unity
70. Anatomy of an app ➔ ARCore Device ➔ Environmental Light ➔ Plane Detector ➔ Plane Visualizer ➔ Our custom Controller ➔ Event system ➔ 2D Canvas ➔ Buttons
71. Session if (Session.Status != SessionStatus.Tracking) { const int lostTrackingSleepTimeout = 15; Screen.sleepTimeout = lostTrackingSleepTimeout; } else { Screen.sleepTimeout = SleepTimeout.NeverSleep; }
72. Plane Session.GetTrackables<DetectedPlane>(m_NewPlanes, TrackableQueryFilter.New); for (int i = 0; i < m_NewPlanes.Count; i++) { GameObject planeObject = Instantiate(DetectedPlanePrefab, Vector3.zero, Quaternion.identity, transform); planeObject.GetComponent<DetectedPlaneVisualizer>().Initialize(m_NewPlanes[i]); }
73. Plane visualization
74. Raycast TrackableHit hit; TrackableHitFlags raycastFilter = TrackableHitFlags.PlaneWithinPolygon | TrackableHitFlags.FeaturePointWithSurfaceNormal; if (Frame.Raycast(touch.position.x, touch.position.y, raycastFilter, out hit)) { ... }
75. GameObject Instance GameObject magicBroom = Instantiate(magicBroomPrefab, hit.Pose.position, hit.Pose.rotation);
76. Anchor a GameObject var anchor = hit.Trackable.CreateAnchor(hit.Pose); magicBroom.transform.parent = anchor.transform;
77. Putting objects into the scene
78. Light Estimation float lightValue = Shader.GetGlobalFloat("_GlobalLightEstimation"); if (lightValue < 0.4) light.intensity = 1; else light.intensity = 0;
80. What’s next? https://developers.google.com/ar https://github.com/google-ar @GoogleARVR
81. Questions & feedback Thank you! Keep in touch https://sleek.bio/mikelantonio
82. ARCore Android Capabilities (REPLACEEEE) Motion Tracking Environmental Understanding Light Estimation ARCore Overview