Share
Embed
Transcript
43 slides
-
What is ARKit? iOS11~ Device with A9 chip~ SceneKit or
SpriteKit or Metal Unity ARKit Plugin -
ARKit capable devices Phone X iPhone 8, iPhone 8 Plus
iPhone 7, iPhone 7 Plus iPhone 6s, iPhone 6s Plus iPhone SE iPad Pro (all models) iPad (5th generation) -
AR Configuration Mode Camera iOS Device AROrientationTracking Back Camera iOS
11.0+ ARKit Capable Devices ARWorldTracking Back Camera iOS 11.0+ ARKit Capable Devices ARFaceTracking Front Camera iOS 11.0+ Only iPhone X ARImageTracking Back Camera iOS 12.0+ ARKit Capable Devices ARObjectScanning Back Camera iOS 12.0+ ARKit Capable Devices -
USDZ USD : Universal Scene Description 3D Model Format from
Pixar PDF in 3D world! From iOS12 ~ -
USDZ on iOS12 Check USDZ files on “AR Quick Look
Gallery” https://developer.apple.com/arkit/gallery/ -
USDZ with QuickLook import QuickLook class ViewController: UIViewController { override
func viewDidLoad() { super.viewDidLoad() } // Call Here to show QuickLook Preview Controller!! @IBAction func showQuickLook(_ sender: Any) { let quickViewCon = QLPreviewController() quickViewCon.dataSource = self self.present(quickViewCon, animated: true, completion: nil) } } -
USDZ with QuickLook extension ViewController: QLPreviewControllerDataSource { func numberOfPreviewItems(in controller:
QLPreviewController) -> Int{ return 1 } func previewController(_ controller: QLPreviewController, previewItemAt index: Int) -> QLPreviewItem{ return MyARItem() } } class MyARItem: NSObject, QLPreviewItem{ var previewItemURL: URL? { return Bundle.main.url(forResource: "coffee", withExtension: "usdz") } } -
AR Configuration Mode Camera iOS Device AROrientationTracking Back Camera iOS
11.0+ ARKit Capable Devices ARWorldTracking Back Camera iOS 11.0+ ARKit Capable Devices ARFaceTracking Front Camera iOS 11.0+ Only iPhone X ARImageTracking Back Camera iOS 12.0+ ARKit Capable Devices ARObjectScanning Back Camera iOS 12.0+ ARKit Capable Devices -
AR Configuration Mode Camera iOS Device AROrientationTracking Back Camera iOS
11.0+ ARKit Capable Devices ARWorldTracking Back Camera iOS 11.0+ ARKit Capable Devices ARFaceTracking Front Camera iOS 11.0+ Only iPhone X ARImageTracking Back Camera iOS 12.0+ ARKit Capable Devices ARObjectScanning Back Camera iOS 12.0+ ARKit Capable Devices -
run(_ configuration) Basics of ARKit Recap ARConfiguration ARFrame ARFrame ARFrame
CMMotionManager AVCaptureSession currentFrame ARSession -
Behind the Scenes—Inertial Odometry Motion sensor Image ARFrame Motion +
Motion + Time Image ARFrame Motion + Image ARFrame Motion + Image ARFrame Motion + Motion + Image ARFrame Motion + Motion + -
Orientation Tracking API AROrientationTrackingConfiguration open class ARCamera : NSObject, NSCopying
{ open var transform: simd_float4x4 { get } open var eulerAngles: simd_float3 { get } … } AROrientationTrackingConfiguration -
Behind the Scenes—Visual Inertial Odometry Motion data and computer vision
Image ARFrame Motion + Image ARFrame Motion + Image ARFrame Motion + Image ARFrame Motion CV Image ARFrame Motion CV Time 1 m -
What is “share AR?” “share AR” means share World tracking
information and 3D models Use ARWorldMap class -
ARWorldMap ARWorlMap͕อ࣋͢Δใ 3DͷಛͳͲͷτϥοΩϯά ใͱɺஔ͞ΕͨΞϯΧʔใɻ ARWorldTrackingConfigurationͰͷΈ औಘ͢Δ͜ͱ͕Ͱ͖Δɻ Internal tracking data •
Map of 3D feature points • Local appearance List of named anchors World Map Data A -
Share the World Map // Retrieve world map from ARSession
session.getCurrentWorldMap { worldMap, error in guard let worldMap = worldMap else { showAlert(error) return } // Serialize let data = try NSKeyedArchiver.archivedData(withRootObject: worldMap, requiringSecureCoding: true) } -
Relocalize to World Map Setup configuration // Receive / Load
world map let worldMap : ARWorldMap = ... // Create a session configuration let configuration = ARWorldTrackingConfiguration() configuration.initialWorldMap = worldMap // Run the session session.run(configuration) W -
SwiftShot ARWorldMapΛMultipeerConnectivity্Ͱڞ༗͢ΔήʔϜ SwiftShot https://developer.apple.com/documentation/arkit/ swiftshot_creating_a_game_for_augmented_reality Inside SwiftShot: Creating an AR
Game https://developer.apple.com/videos/play/wwdc2018/605 -
3D Object Scan ARKitͰscanͯ͠3DϞσϧ͕࡞Ͱ͖ Δʁ NO! ARKitͷ scanningɺମ͔Βಛͷ ू߹Λอଘ͢Δ͚ͩɻ ͦͷମΛݕ͢Δ͜ͱͰ͖ͯɺ
ମͷ3DϞσϧ࡞Εͳ͍ɻ How to Acquire an Object? Similar representation as a world map -
How to get ARReferenceObject? AppleͷαϯϓϧίʔυɺScanning and Detecting 3D ObjectsΛ͏ https://developer.apple.com/documentation/arkit/
scanning_and_detecting_3d_objects -
How to use ARReferenceObject? Scanning and Detecting 3D Objects αϯϓϧΛͬͯɺ
ARReferenceObjectΛεΩϟϯɻΤΫεϙʔτͯ͠ɺ*.arobjectΛऔಘɻ XcodeͰɺARͷ৽نϓϩδΣΫτΛ࡞͠ɺAsset CatalogʹAR Resource GroupΛ࡞Γɺ͜ͷϑΝΠϧΛDrag&Dropɻ -
Prepare for Scan override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration() configuration.detectionObjects = ARReferenceObject.referenceObjects(inGroupNamed: "apple", bundle: nil)! sceneView.session.run(configuration) } } -
Detection of Object func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { if let objectAnchor = anchor as? ARObjectAnchor{ let foundObject = objectAnchor.referenceObject // check name" let objectname = foundObject.name // do something to the object! objectAnchor.addTitleNode(on: node) } } -
ARObjectScanningConfiguration ARObjectScanningConfiguration is for use only in development scenarios. High-fidelity
spatial mapping has a high performance and energy cost, and disables ARKit features not necessary for reference object scanning. To produce AR experiences for end users, use ARWorldTrackingConfiguration. Appleͱͯ͠ɺObject ScanϞʔυɺ։ൃϑΣʔζͰ͏ͷͱఆ͠ ͍ͯΔΑ͏Ͱ͢ɻ ։ൃऀ͕ಛఆͷΦϒδΣΫτΛScan͠ɺͦ͜Ͱநग़ͨ͠ ARReferenceObjectΛΞϓϦʹ͍ΕͯϦϦʔεɻ