Update your reality with ARKit

Transcript

1. UPDATE YOUR REALITY WITH ARKIT CHITARANJAN SAHU

2. CHITARANJAN SAHU ARGMENTED REALITY ▸ The illusion of virtual object

   in a real world ▸ Place virtual content to real world ▸ Relies on creating a virtual space on a fixed position ▸ Combines motion sensors with video capture

3. CHITARANJAN SAHU DEMO TIME

4. CHITARANJAN SAHU AGENDA ‣ ARkit Basics ‣ ARkit 2.0 ‣

   Demo ‣ References

5. CHITARANJAN SAHU

6. CHITARANJAN SAHU ARKIT ‣ Mobile AR platform by Apple in

   2017. ‣ High-level , Session based API . ‣ It requires A9 or newer processors to perform all the complex computations in real-time. Thus, you’ll need an iPhone SE, iPhone 6s, iPhone 6s Plus, 5th generation iPad, or a newer device running iOS 11.

7. CHITARANJAN SAHU ARKIT LAYERS Tracking :World Tracking, Visual inertial odometer,

   Orientation, Position, Scaling object relatively,No external setup required . Scene Understanding : Plane detection, Hit testing, light estimation Rendering : Easy integration , ARView, SceneKit , SpriteKit, Metal,Unity and unreal support for ARKit

8. CHITARANJAN SAHU

9. CHITARANJAN SAHU ARSessionConfiguration

10. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes

11. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ARSessionConfiguration ARWorldTrackingSessionConfiguration

12. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ‣ Enable/Disable Features

13. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ‣ Enable/Disable Features ‣

    Availability

14. CHITARANJAN SAHU ARSessionConfiguration ‣ Configuration Classes ‣ Enable/Disable Features ‣

    Availability if ARWorldTrackingSessionConfiguration.isSupported { configuration = ARWorldTrackingSessionConfiguration() } else { configuration = ARSessionConfiguration() }

15. CHITARANJAN SAHU ARSession

16. CHITARANJAN SAHU ARSession ‣ Manage AR processing

17. CHITARANJAN SAHU ARSession ‣ Manage AR processing // Run your

    session session.run(configuration) // Pause your session session.pause() // Resume your session session.run(session.configuration) // Change your configuration session.run(otherConfiguration)

18. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking

    // Reset tracking session.run(configuration, options: .resetTracking)

19. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking

    ‣ Session updates

20. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking

    ‣ Session updates // Access the latest frame func session(_: ARSession, didUpdate: ARFrame) // Handle session errors func session(_: ARSession, didFailWithError: Error)

21. CHITARANJAN SAHU ARSession ‣ Manage AR processing ‣ Reset tracking

    ‣ Session updates ‣ Current frame

22. CHITARANJAN SAHU ARFrame

23. CHITARANJAN SAHU ARFrame ‣ Captured image ‣ Tracking information ‣

    Scene information

24. CHITARANJAN SAHU ARAnchor ‣ Real-world position and orientation ‣ ARSession

    add/remove ‣ ARFrame anchors

25. CHITARANJAN SAHU Tracking

26. CHITARANJAN SAHU World Tracking ‣ Position and orientation ‣ Physical

    distances ‣ Relative to starting position ‣ 3D-feature points

27. CHITARANJAN SAHU // Create a session let mySession = ARSession()

    // Set ourselves as the session delegate mySession.delegate = self // Create a world tracking configuration let configuration = ARWorldTrackingSessionConfiguration() // Run the session mySession.run(configuration)

28. CHITARANJAN SAHU ARKit 2.0

29. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps

30. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣

    Environment Texturing

31. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣

    Environment Texturing ‣ Image Tracking

32. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣

    Environment Texturing ‣ Image Tracking ‣ Object Detection

33. CHITARANJAN SAHU ARKit 2.0 ‣ Saving and Loading Maps ‣

    Environment Texturing ‣ Image Tracking ‣ Object Detection ‣ Face Tracking Enhancements

34. CHITARANJAN SAHU Saving and Loading Maps // Retrieve world map

    from session object session.getCurrentWorldMap { worldMap, error in guard let worldMap = worldMap else { showAlert(error) return } }

35. CHITARANJAN SAHU Saving and Loading Maps // Retrieve world map

    from session object session.getCurrentWorldMap { worldMap, error in guard let worldMap = worldMap else { showAlert(error) return } } // Load world map and run the configuration let configuration = ARWorldTrackingConfiguration() configuration.initialWorldMap = worldMap session.run(configuration)

36. CHITARANJAN SAHU Environment Texturing ‣ Improvisation in reflection, shadow, lighting

    ‣ Configuration.environmentaltexturing ‣ Gathers scene texture ‣ Cube map representation ‣ Dynamically update texture


37. CHITARANJAN SAHU Image Detection and Tracking Detection ‣ Recognition of

    known static 2D images ‣ Change position and orientation ‣ Integration with world tracking ‣ Validation Support by Xcode asset catalog
 


38. CHITARANJAN SAHU Image Detection and Tracking Detection ‣ Recognition of

    known static 2D images ‣ Change position and orientation ‣ Integration with world tracking ‣ Validation Support by Xcode asset catalog Tracking ‣ Multiple image tracking ‣ Images don’t need to be static ‣ Position and orientation for every frame ‣ ARImageTrackingConfiguration

39. CHITARANJAN SAHU

40. CHITARANJAN SAHU 3D Object detection

41. CHITARANJAN SAHU 3D Object detection ‣ Objects need to be

    scanned first Using ARObjectScanningConfiguration ‣ Detection of a known static 3D object
 configuration.detectionObjects ‣ Feature points,Well-textured, rigid, non-reflective
 Example : museum app,

42. CHITARANJAN SAHU Face tracking ‣ Improvements in face tracking mesh/ARFaceGeometry,

    light probing ‣ Animoji in Arkit 1.5 ‣ 50+ facial features tracking ‣ New Gaze tracking ‣ Inputs using Gaze tracking ‣ TongueSupport


43. CHITARANJAN SAHU DEMO

44. CHITARANJAN SAHU Reference ‣ https://developer.apple.com/arkit/ ‣ https://developer.apple.com/videos/play/wwdc2017/602/ ‣ https://twitter.com/osfalmer ‣

    https://twitter.com/nathangitter


45. CHITARANJAN SAHU THANK YOU