ARKit for Graphics Beginners

A beginners guide to getting acquainted with ARKit and its capabilities.

Sample Code:
AR with SpriteKit: https://github.com/bgayman/ARSpriteKit
AR with Plane Anchors: https://github.com/bgayman/Wall
AR with Face Anchors: https://github.com/bgayman/ARFaceSceneKit
AR with Image Anchors: https://github.com/bgayman/ARImageSceneKit

Talk given at Women in iOS Meetup

Transcript

1. ARKit F o r G r a p h i c s B e g i n n e r s
2. Augmented Reality Augmented reality (AR) describes user experiences that add 2D or 3D elements to the live view from a device's camera in a way that makes those elements appear to inhabit the real world. —Apple Developer Documentation
3. —Image from engadget.com —Image from thinkmobiles.com Earlier AR
4. ARKit —Image from Apple Developer Documentation
5. SpriteKit 2D and 2.5D Elements SceneKit 3D Elements Content Metal Custom Rendering —Images from Apple Developer Documentation
6. SpriteKit 2D Elements Billboarding Effect
7. —Source code available at https://github.com/bgayman/ARSpriteKit
8. SceneKit
9. Model Space World Space
10. ARKit combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience. —Apple Developer Documentation Visual-Inertial Odometry
11. Anchors ARAnchor ARPlaneAnchor ARFaceAnchor ARImageAnchor
12. Adding Objects
13. ARPlaneAnchor • Works with textured surfaces • Horizontal planes available from iOS 11+ • Vertical planes available from iOS 11.3+ • Rectangular planes on iOS11+ • Irregular meshes iOS 11.3+
14. —Source code available at https://github.com/bgayman/Wall
15. ARFaceAnchor • Requires a front-facing TrueDepth camera (currently iPhone X only) • Only one close, clear face • Not compatible with other anchor types • Requires a privacy policy explaining to the user how you use facial data • Can not be sole basis for app
16. ARFaceAnchor. BlendShapeLocation • eyeBlinkLeft • eyeLookDownLeft • eyeLookInLeft • eyeLookOutLeft • eyeLookUpLeft • eyeSquintLeft • eyeWideLeft • eyeBlinkRight • eyeLookDownRight • eyeLookInRight • eyeLookOutRight • eyeLookUpRight • eyeSquintRight • eyeWideRight • jawForward • jawLeft • jawRight • jawOpen
17. ARFaceAnchor. BlendShapeLocation • mouthClose • mouthFunnel • mouthPucker • mouthLeft • mouthRight • mouthSmileLeft • mouthSmileRight • mouthFrownLeft • mouthFrownRight • mouthDimpleLeft • mouthDimpleRight • mouthStretchLeft • mouthStretchRight • mouthRollLower • mouthRollUpper • mouthShrugLower • mouthShrugUpper • mouthPressLeft
18. ARFaceAnchor. BlendShapeLocation • mouthPressRight • mouthLowerDownLeft • mouthUpperUpLeft • mouthUpperUpRight • browDownLeft • browDownRight • browInnerUp • browOuterUpLeft • browOuterUpRight • cheekPuff • cheekSquintLeft • cheekSquintRight • noseSneerLeft • noseSneerRight
19. let device = MTLCreateSystemDefaultDevice()! let faceGeometry = ARSCNFaceGeometry(device: device) let node = SCNNode(geometry: faceGeometry) node.geometry?.firstMaterial?.diffuse.contents = UIColor.white scene.rootNode.addChildNode(node) —Right Image from Apple Developer Documentation
20. —Source code available at https://github.com/bgayman/ARFaceSceneKit
22. ARImageAnchor • Requires iOS 11.3+ • Works with 2D images only • Does not work well with narrow or wide images (less than 2:1; greater than 1:2)
23. —Source code available at https://github.com/bgayman/Wall
24. Live Demo
25. Questions?