Slide 1
Slide 1 text
ARKit 2.0
@TachibanaKaoru
2018/6/11 WWDC GoriGori Catch-Up
Slide 2
Slide 2 text
About Me
@TachibanaKaoru
Freelance iOS Engineer
ϘϧμϦϯά͖
ࠓͷςʔϚɿཱྀΛ͠ͳ͕ΒࣄΛ͢Δ
όϦౡɺαϯτϦʔχౡɺόϯίΫ
Slide 3
Slide 3 text
ARKit
Slide 4
Slide 4 text
What is ARKit?
iOS11~
Device with A9 chip~
SceneKit or SpriteKit or Metal
Unity ARKit Plugin
Slide 5
Slide 5 text
ARKit capable devices
Phone X
iPhone 8, iPhone 8 Plus
iPhone 7, iPhone 7 Plus
iPhone 6s, iPhone 6s Plus
iPhone SE
iPad Pro (all models)
iPad (5th generation)
Slide 6
Slide 6 text
ARKit
1.0 ۭؒݕɺฏ໘ʢਫฏ໘ʣݕɺإදݕ
1.5 ฏ໘ʢਨ໘ݕʣɺը૾ݕ
2.0
ը૾ݕϞʔυՃɺARঢ়ଶอଘରԠɺ
Object SacnରԠɺࢹઢɾઉݕ
Slide 7
Slide 7 text
AR Configuration Mode
Camera iOS Device
AROrientationTracking Back Camera iOS 11.0+
ARKit Capable
Devices
ARWorldTracking Back Camera iOS 11.0+
ARKit Capable
Devices
ARFaceTracking Front Camera iOS 11.0+ Only iPhone X
ARImageTracking Back Camera iOS 12.0+
ARKit Capable
Devices
ARObjectScanning Back Camera iOS 12.0+
ARKit Capable
Devices
Slide 8
Slide 8 text
USDZ
Slide 9
Slide 9 text
USDZ
USD : Universal Scene Description
3D Model Format from Pixar
PDF in 3D world!
From iOS12 ~
Slide 10
Slide 10 text
USDZ on iOS12
Available on Safari, memo, Mail and so on.
Slide 11
Slide 11 text
USDZ on iOS12
Check USDZ files on “AR Quick Look Gallery”
https://developer.apple.com/arkit/gallery/
Slide 12
Slide 12 text
USDZ on your App
Available with QLPreviewController, WKWebView, SFSafariViewController
Slide 13
Slide 13 text
USDZ with QuickLook
import QuickLook
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
}
// Call Here to show QuickLook Preview Controller!!
@IBAction func showQuickLook(_ sender: Any) {
let quickViewCon = QLPreviewController()
quickViewCon.dataSource = self
self.present(quickViewCon, animated: true, completion: nil)
}
}
Slide 14
Slide 14 text
USDZ with QuickLook
extension ViewController: QLPreviewControllerDataSource {
func numberOfPreviewItems(in controller: QLPreviewController) -> Int{
return 1
}
func previewController(_ controller: QLPreviewController, previewItemAt index: Int) -> QLPreviewItem{
return MyARItem()
}
}
class MyARItem: NSObject, QLPreviewItem{
var previewItemURL: URL? {
return Bundle.main.url(forResource: "coffee", withExtension: "usdz")
}
}
Slide 15
Slide 15 text
How does ARKit works?
Slide 16
Slide 16 text
AR Configuration Mode
Camera iOS Device
AROrientationTracking Back Camera iOS 11.0+
ARKit Capable
Devices
ARWorldTracking Back Camera iOS 11.0+
ARKit Capable
Devices
ARFaceTracking Front Camera iOS 11.0+ Only iPhone X
ARImageTracking Back Camera iOS 12.0+
ARKit Capable
Devices
ARObjectScanning Back Camera iOS 12.0+
ARKit Capable
Devices
Slide 17
Slide 17 text
AR Configuration Mode
Camera iOS Device
AROrientationTracking Back Camera iOS 11.0+
ARKit Capable
Devices
ARWorldTracking Back Camera iOS 11.0+
ARKit Capable
Devices
ARFaceTracking Front Camera iOS 11.0+ Only iPhone X
ARImageTracking Back Camera iOS 12.0+
ARKit Capable
Devices
ARObjectScanning Back Camera iOS 12.0+
ARKit Capable
Devices
Slide 18
Slide 18 text
run(_ configuration)
Basics of ARKit
Recap
ARConfiguration
ARFrame
ARFrame
ARFrame
CMMotionManager
AVCaptureSession
currentFrame
ARSession
Slide 19
Slide 19 text
Behind the Scenes—Inertial Odometry
Motion sensor
Time
Image
ARFrame
Motion
+
Motion
+
Slide 20
Slide 20 text
Behind the Scenes—Inertial Odometry
Motion sensor
Image
ARFrame
Motion
+
Motion
+ Time
Image
ARFrame
Motion
+
Image
ARFrame
Motion
+
Image
ARFrame
Motion
+
Motion
+
Image
ARFrame
Motion
+
Motion
+
Slide 21
Slide 21 text
Orientation Tracking API
AROrientationTrackingConfiguration
open class ARCamera : NSObject, NSCopying {
open var transform: simd_float4x4 { get }
open var eulerAngles: simd_float3 { get }
…
}
AROrientationTrackingConfiguration
Slide 22
Slide 22 text
Behind the Scenes—Visual Inertial Odometry
Motion data and computer vision
Image
ARFrame
Motion
+
Image
ARFrame
Motion
+
Image
ARFrame
Motion
+
Image
ARFrame
Motion
CV
Image
ARFrame
Motion
CV Time
1 m
Slide 23
Slide 23 text
Behind the Scenes—Visual Inertial Odometry
Extracting robust features
Slide 24
Slide 24 text
Behind the Scenes—Visual Inertial Odometry
Triangulation creates World Map
Slide 25
Slide 25 text
Behind the Scenes—Visual Inertial Odometry
Initialization
C
W
Slide 26
Slide 26 text
AROrientationTracking
ARWorldTracking
Slide 27
Slide 27 text
Share AR!
Slide 28
Slide 28 text
What is “share AR?”
“share AR” means share World tracking information and 3D models
Use ARWorldMap class
Slide 29
Slide 29 text
ARWorldMap
ARWorlMap͕อ࣋͢Δใ
3DͷಛͳͲͷτϥοΩϯά
ใͱɺஔ͞ΕͨΞϯΧʔใɻ
ARWorldTrackingConfigurationͰͷΈ
औಘ͢Δ͜ͱ͕Ͱ͖Δɻ
Internal tracking data
• Map of 3D feature points
• Local appearance
List of named anchors
World Map Data
A
Slide 30
Slide 30 text
Share the World Map
// Retrieve world map from ARSession
session.getCurrentWorldMap { worldMap, error in
guard let worldMap = worldMap else {
showAlert(error)
return
}
// Serialize
let data = try NSKeyedArchiver.archivedData(withRootObject: worldMap,
requiringSecureCoding: true)
}
Slide 31
Slide 31 text
Relocalize to World Map
Setup configuration
// Receive / Load world map
let worldMap : ARWorldMap = ...
// Create a session configuration
let configuration = ARWorldTrackingConfiguration()
configuration.initialWorldMap = worldMap
// Run the session
session.run(configuration)
W
Slide 32
Slide 32 text
SwiftShot
ARWorldMapΛMultipeerConnectivity্Ͱڞ༗͢ΔήʔϜ
SwiftShot
https://developer.apple.com/documentation/arkit/
swiftshot_creating_a_game_for_augmented_reality
Inside SwiftShot: Creating an AR Game
https://developer.apple.com/videos/play/wwdc2018/605
Slide 33
Slide 33 text
3D Object Scan
Slide 34
Slide 34 text
3D Object Scan
ARKit Ͱ 3D ObjectΛScanͯ͠ɺ
ARReferenceObjectΛ࡞Δ
ARReferenceObjectΛͬͯɺͦͷମ͕
Ͳ͜ʹଘࡏ͢Δ͔ݕͰ͖Δ
Slide 35
Slide 35 text
3D Object Scan
ARKitͰscanͯ͠3DϞσϧ͕࡞Ͱ͖
Δʁ
NO!
ARKitͷ scanningɺମ͔Βಛͷ
ू߹Λอଘ͢Δ͚ͩɻ
ͦͷମΛݕ͢Δ͜ͱͰ͖ͯɺ
ମͷ3DϞσϧ࡞Εͳ͍ɻ
How to Acquire an Object?
Similar representation as a world map
Slide 36
Slide 36 text
How to get ARReferenceObject?
AppleͷαϯϓϧίʔυɺScanning and Detecting 3D ObjectsΛ͏
https://developer.apple.com/documentation/arkit/
scanning_and_detecting_3d_objects
Slide 37
Slide 37 text
How to use ARReferenceObject?
Scanning and Detecting 3D Objects αϯϓϧΛͬͯɺ
ARReferenceObjectΛεΩϟϯɻΤΫεϙʔτͯ͠ɺ*.arobjectΛऔಘɻ
XcodeͰɺARͷ৽نϓϩδΣΫτΛ࡞͠ɺAsset CatalogʹAR
Resource GroupΛ࡞Γɺ͜ͷϑΝΠϧΛDrag&Dropɻ
Slide 38
Slide 38 text
Prepare for Scan
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration()
configuration.detectionObjects =
ARReferenceObject.referenceObjects(inGroupNamed: "apple", bundle: nil)!
sceneView.session.run(configuration)
}
}
Slide 39
Slide 39 text
Detection of Object
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if let objectAnchor = anchor as? ARObjectAnchor{
let foundObject = objectAnchor.referenceObject
// check name"
let objectname = foundObject.name
// do something to the object!
objectAnchor.addTitleNode(on: node)
}
}
Slide 40
Slide 40 text
To Scan Efficiently
༷͕͋Δͷ
ςʔϒϧͷ্ʹͷΔ͘Β͍ͷେ͖͞
ॊΒ͔͍ͷͰͳ͘ɺݻ͍ମ
҆ఆͨࣨ͠র໌
Use iPhoneX !
Slide 41
Slide 41 text
Personal Recommendation
Nintendo Amiibo
Slide 42
Slide 42 text
ARObjectScanningConfiguration
ARObjectScanningConfiguration is for use only in development scenarios.
High-fidelity spatial mapping has a high performance and energy cost, and
disables ARKit features not necessary for reference object scanning. To
produce AR experiences for end users, use ARWorldTrackingConfiguration.
Appleͱͯ͠ɺObject ScanϞʔυɺ։ൃϑΣʔζͰ͏ͷͱఆ͠
͍ͯΔΑ͏Ͱ͢ɻ
։ൃऀ͕ಛఆͷΦϒδΣΫτΛScan͠ɺͦ͜Ͱநग़ͨ͠
ARReferenceObjectΛΞϓϦʹ͍ΕͯϦϦʔεɻ
Slide 43
Slide 43 text
·ͱΊ
ARKit 2.0Ͱ͓͠Ζ͍ػೳ͕͍Ζ͍ΖՃ͞Ε͍ͯ·͢ɻ
AR Share
Object Scan
USDZ ϑΝΠϧΛ࣮͕͑΄ͱΜͲ͍Γ·ͤΜɻ
Ͱ͖Δָ͚ͩΛ͠·͠ΐ͏ʂ