Update your reality with ARKit

Transcript

1. UPDATE YOUR REALITY WITH ARKIT CHITARANJAN SAHU
2. CHITARANJAN SAHU ARGMENTED REALITY ▸ The illusion of virtual object in a real world ▸ Place virtual content to real world ▸ Relies on creating a virtual space on a fixed position ▸ Combines motion sensors with video capture
3. CHITARANJAN SAHU DEMO TIME
4. CHITARANJAN SAHU AGENDA ‣ ARkit Basics ‣ ARkit 2.0 ‣ Demo ‣ References
5. CHITARANJAN SAHU
6. CHITARANJAN SAHU ARKIT ‣ Mobile AR platform by Apple in 2017. ‣ High-level , Session based API . ‣ It requires A9 or newer processors to perform all the complex computations in real-time. Thus, you’ll need an iPhone SE, iPhone 6s, iPhone 6s Plus, 5th generation iPad, or a newer device running iOS 11.
7. CHITARANJAN SAHU ARKIT LAYERS Tracking :World Tracking, Visual inertial odometer, Orientation, Position, Scaling object relatively,No external setup required . Scene Understanding : Plane detection, Hit testing, light estimation Rendering : Easy integration , ARView, SceneKit , SpriteKit, Metal,Unity and unreal support for ARKit
8. CHITARANJAN SAHU
9. CHITARANJAN SAHU ARSessionConfiguration
10. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes
11. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ARSessionConfiguration ARWorldTrackingSessionConfiguration
12. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ‣ Enable/Disable Features
13. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ‣ Enable/Disable Features ‣ Availability
14. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ‣ Enable/Disable Features ‣ Availability if ARWorldTrackingSessionConfiguration.isSupported { configuration = ARWorldTrackingSessionConfiguration() } else { configuration = ARSessionConfiguration() }
15. CHITARANJAN SAHU ARSession
16. CHITARANJAN SAHU ARSession ‣ Manage AR processing
17. CHITARANJAN SAHU ARSession ‣ Manage AR processing // Run your session session.run(configuration) // Pause your session session.pause() // Resume your session session.run(session.configuration) // Change your configuration session.run(otherConfiguration)
18. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking // Reset tracking session.run(configuration, options: .resetTracking)
19. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking ‣ Session updates
20. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking ‣ Session updates // Access the latest frame func session(_: ARSession, didUpdate: ARFrame) // Handle session errors func session(_: ARSession, didFailWithError: Error)
21. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking ‣ Session updates ‣ Current frame
22. CHITARANJAN SAHU ARFrame
23. CHITARANJAN SAHU ARFrame ‣ Captured image ‣ Tracking information ‣ Scene information
24. CHITARANJAN SAHU ARAnchor ‣ Real-world position and orientation ‣ ARSession add/remove ‣ ARFrame anchors
25. CHITARANJAN SAHU Tracking
26. CHITARANJAN SAHU World Tracking ‣ Position and orientation ‣ Physical distances ‣ Relative to starting position ‣ 3D-feature points
27. CHITARANJAN SAHU // Create a session let mySession = ARSession() // Set ourselves as the session delegate mySession.delegate = self // Create a world tracking configuration let configuration = ARWorldTrackingSessionConfiguration() // Run the session mySession.run(configuration)
28. CHITARANJAN SAHU ARKit 2.0
29. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps
30. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣ Environment Texturing
31. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣ Environment Texturing ‣ Image Tracking
32. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣ Environment Texturing ‣ Image Tracking ‣ Object Detection
33. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣ Environment Texturing ‣ Image Tracking ‣ Object Detection ‣ Face Tracking Enhancements
34. CHITARANJAN SAHU Saving and Loading Maps // Retrieve world map from session object session.getCurrentWorldMap { worldMap, error in guard let worldMap = worldMap else { showAlert(error) return } }
35. CHITARANJAN SAHU Saving and Loading Maps // Retrieve world map from session object session.getCurrentWorldMap { worldMap, error in guard let worldMap = worldMap else { showAlert(error) return } } // Load world map and run the configuration let configuration = ARWorldTrackingConfiguration() configuration.initialWorldMap = worldMap session.run(configuration)
36. CHITARANJAN SAHU Environment Texturing ‣ Improvisation in reflection, shadow, lighting ‣ Configuration.environmentaltexturing ‣ Gathers scene texture ‣ Cube map representation ‣ Dynamically update texture

37. CHITARANJAN SAHU Image Detection and Tracking Detection ‣ Recognition of known static 2D images ‣ Change position and orientation ‣ Integration with world tracking ‣ Validation Support by Xcode asset catalog
 

38. CHITARANJAN SAHU Image Detection and Tracking Detection ‣ Recognition of known static 2D images ‣ Change position and orientation ‣ Integration with world tracking ‣ Validation Support by Xcode asset catalog Tracking ‣ Multiple image tracking ‣ Images don’t need to be static ‣ Position and orientation for every frame ‣ ARImageTrackingConfiguration
39. CHITARANJAN SAHU
40. CHITARANJAN SAHU 3D Object detection
41. CHITARANJAN SAHU 3D Object detection ‣ Objects need to be scanned first Using ARObjectScanningConfiguration ‣ Detection of a known static 3D object
 configuration.detectionObjects ‣ Feature points,Well-textured, rigid, non-reflective
 Example : museum app,
42. CHITARANJAN SAHU Face tracking ‣ Improvements in face tracking mesh/ARFaceGeometry, light probing ‣ Animoji in Arkit 1.5 ‣ 50+ facial features tracking ‣ New Gaze tracking ‣ Inputs using Gaze tracking ‣ TongueSupport

43. CHITARANJAN SAHU DEMO
44. CHITARANJAN SAHU Reference ‣ https://developer.apple.com/arkit/ ‣ https://developer.apple.com/videos/play/wwdc2017/602/ ‣ https://twitter.com/osfalmer ‣ https://twitter.com/nathangitter

45. CHITARANJAN SAHU THANK YOU