ゼロから始める ARKit

Slide 1

Slide 1 text

ARKit αϙʔλʔζColabษڧձ 2018/06/13(ਫ) Keisuke Tatsumi

Slide 2

Slide 2 text

Keisuke Tatsumi • Github, Qiita : cokaholic • Twitter : @TK_u_nya • AppStore : Keisuke Tatsumi

Slide 3

Slide 3 text

iOS engineer at AbemaTV, Inc.

Slide 4

Slide 4 text

ຊ೔ͷςʔϚ͸…

Slide 5

Slide 5 text

ARKit

Slide 6

Slide 6 text

Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary

Slide 7

Slide 7 text

Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary

Slide 8

Slide 8 text

Introduction ARͱ͸ • ʮAugmented Realityʢ֦ுݱ࣮ʣʯͷུ • ࣮ࡍͷ෩ܠʹόʔνϟϧͷࢹ֮৘ใΛॏͶͯදࣔ ͢Δ͜ͱͰɺݱ࣮ͷੈքΛԾ૝తʹ֦ு͢Δٕज़

Slide 9

Slide 9 text

Introduction ֤ࣾͷϞόΠϧARϑϨʔϜϫʔΫ • Apple - ARKit • Android - ARCore

Slide 10

Slide 10 text

Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary

Slide 11

Slide 11 text

What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ

Slide 12

Slide 12 text

What’s ARKit

Slide 13

Slide 13 text

What’s ARKit

Slide 14

Slide 14 text

What’s ARKit

Slide 15

Slide 15 text

What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ • iOS 11Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ͠ ͍ͯΔσόΠε͕ඞཁ

Slide 16

Slide 16 text

Devices supporting ARKit • iPhone6s Ҏ߱ • iPad Proʢୈ1ੈ୅ʣҎ߱ • iPadʢୈ5ੈ୅ʣҎ߱

Slide 17

Slide 17 text

͋Εʁ

Slide 18

Slide 18 text

iPod touch ๨Εͯͳ͍ʁ

Slide 19

Slide 19 text

Ͱ͸ͳ͘ɺ

Slide 20

Slide 20 text

Devices supporting ARKit • iPhone6s Ҏ߱ • iPad Proʢୈ1ੈ୅ʣҎ߱ • iPadʢୈ5ੈ୅ʣҎ߱ • iPod touch ͸શ6ੈ୅͕શͯඇରԠ  ʢiPod touchୈ7ੈ୅ʹظ଴…ʂʣ

Slide 21

Slide 21 text

What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ • iOS 11Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ͠ ͍ͯΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 9Ҏ্͕ඞཁ

Slide 22

Slide 22 text

What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ • iOS 11Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ͠ ͍ͯΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 9Ҏ্͕ඞཁ • macOS Sierra (10.12.4) Ҏ্͕ඞཁ

Slide 23

Slide 23 text

Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary

Slide 24

Slide 24 text

Hello, ARKit World Demo XcodeͷςϯϓϨʔτͰ ࡞੒͞ΕΔARΞϓϦΛى ಈͯ͠Έͨ

Slide 25

Slide 25 text

Augmented Reality App Template ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")! // Set the scene to the view sceneView.scene = scene } override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) // Create a session configuration let configuration = ARWorldTrackingConfiguration() // Run the view's session sceneView.session.run(configuration) } }

Slide 26

Slide 26 text

ARSCNView ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { @IBOutlet var sceneView: ARSCNView! • 3D SceneKitͷίϯςϯπΛΧϝϥͰARͱͯ͠ද ࣔ͢ΔͨΊͷView • AR App Template Ͱ͸ॳظදࣔͷViewController ͷview ͷ subviewͱͯ͠഑ஔ͞Ε͍ͯΔ • ARSCNView ʹ͸ scene: SCNScene ϓϩύςΟ͕ ͋Γɺ͜ͷ scene ʹ ඈߦػͳͲͷ node Λ௥Ճ͢ Δ͜ͱͰɺΦϒδΣΫτΛදࣔͰ͖Δ

Slide 27

Slide 27 text

Set SCNScene To ARSCNView ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")! // Set the scene to the view sceneView.scene = scene } • ඈߦػΦϒδΣΫτͷݩͰ͋Δ ship.scn ϑΝΠϧ Λࢦఆͯ͠ੜ੒ͨ͠ SCNScene Λ ARSCNView ͷ scene ͱͯ͠ઃఆ͢Δ͜ͱͰද͍ࣔͯ͠Δ

Slide 28

Slide 28 text

SCNScene • namedͷҾ਺ͰࢦఆͰ͖Δͷ͸ .scnϑΝΠϧͷΈ // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")!

Slide 29

Slide 29 text

SCNScene • namedͷҾ਺ͰࢦఆͰ͖Δͷ͸ .scnϑΝΠϧͷΈ • ಡΈࠐΜͰ .scnϑΝΠϧ΁ม׵Ͱ͖ΔϑΥʔϚοτ ‣ COLLADA (.dae) ‣ Alembic (.abc) ‣ PLY ϑΝΠϧϑΥʔϚοτ ‣ Standard Triangulated Language (.stl) ‣ Wavefront OBJ (.obj) ‣ Pixar Universal Scene Description (.usd/.usda) // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")!

Slide 30

Slide 30 text

Slide 31

Slide 31 text

How To understand Statistics ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() // Show statistics such as fps and timing information sceneView.showsStatistics = true } } ϑϨʔϜϨʔτήʔδ Metal(Mt) / OpenGL(GL) ϑϨʔϜϨʔτ શମͷϙϦΰϯ਺ υϩʔίʔϧ

Slide 32

Slide 32 text

Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary

Slide 33

Slide 33 text

Image Detection On ARKit Demo ాதώϝͷMMDϞσϧΛ ը૾ݕ஌ͨ͠Ґஔʹදࣔ ͢Δ ※͜ͷޙͷεϥΠυͰը૾ݕ஌ͷ࢓૊ΈΛղઆ͠·͢ɻ

Slide 34

Slide 34 text

Add AR Resource Group Assets.xcassetsΛ։͖ɺ+Ϙλϯ͔ΒʮAR ResourcesʯάϧʔϓΛ࡞੒

Slide 35

Slide 35 text

Add Images Into AR Resources ݕ஌ͤ͞ΔͨΊͷը૾ΛʮAR Resourcesʯάϧʔϓ಺ʹ  υϥοά&υϩοϓͰ௥Ճ͠·͢ɻ © ాதώϝ © (ג)ΞΫτθϩ

Slide 36

Slide 36 text

Set Real Image Size ௥Ճͨ͠ը૾ʹ͸ݱ࣮ͰͷϦΞϧͳαΠζΛ  ೖྗ͢Δඞཁ͕͋Γ·͢ɻ

Slide 37

Slide 37 text

Check Warnings ݕ஌͢Δը૾ͷαΠζΛೖྗͯ͠΋Τϥʔ͕ফ͑ͳ͍৔߹͸ɺ  ͦͷը૾͕ը૾ݕ஌ʹ޲͍͍ͯͳ͍ཧ༝͕දࣔ͞Ε·͢ɻ ͜͏ͳͬͨ৔߹͸ɺΤϥʔ಺༰ʹ४ڌͨ͠ผͷը૾Λ༻ҙ͠·͠ΐ͏ɻ

Slide 38

Slide 38 text

Using Good Images • ը࣭͕ྑ͍ • ಠಛͷಛ௃͕͋Δ • ίϯτϥετ͕͸͖ͬΓ͍ͯ͠Δ • ը࣭͕ߥ͍ • ΧϥʔྖҬ͕ۉҰ͗͢Δ • ৭ຯʹϝϦϋϦ͕ͳ͍ © (ג)ΞΫτθϩ © ాதώϝ

Slide 39

Slide 39 text

Set Detection Images // Create reference images guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else { fatalError("Reference images not found.") } // Create a session configuration let configuration = ARWorldTrackingConfiguration() // Set detection images configuration.detectionImages = referenceImages // Run the view's session sceneView.session.run(configuration) • ݕ஌͍ͤͨ͞ը૾άϧʔϓ໊Ͱ͋ΔʮAR ResourcesʯΛࢦఆͯ͠ ARReferenceImage ΫϥεͷΠϯελϯεΛੜ੒͢Δ • conﬁguration: ARWorldTrackingConﬁguration ͷ detectionImages ʹੜ੒ͨ͠ ARReferenceImage Ληοτ͢Δͱɺ ը૾Λݕ஌Ͱ͖ΔΑ͏ʹͳΔ

Slide 40

Slide 40 text

Get Detected Images extension ViewController: ARSCNViewDelegate { // MARK: - ARSCNViewDelegate func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { // Check detected an image guard let imageAnchor = anchor as? ARImageAnchor, let imageName = imageAnchor.referenceImage.name else { return } // Check the detected image is "hime_tanaka" if imageName == "hime_tanaka" { print("Detected!") } } } • ը૾ͷݕग़͸ ARSCNViewDelegate ͷ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) ϝιου಺ʹͯߦ͏ • ݕग़ͨ͠ը૾ͷ໊લʹΑͬͯɺͲͷը૾͕ݕग़͞Ε͔͕ͨΘ͔ΔͨΊɺ ը૾͝ͱʹ൓ԠΛม͑Δ͜ͱ΋Ͱ͖Δ

Slide 41

Slide 41 text

Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary

Slide 42

Slide 42 text

About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ

Slide 43

Slide 43 text

About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ • iOS 12Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ ͍ͯ͠ΔσόΠε͕ඞཁ

Slide 44

Slide 44 text

Devices supporting ARKit 2 iOS 11͕࢖͑Ε͹ɺiOS 12΋࢖͑Δ

Slide 45

Slide 45 text

Devices supporting ARKit 2 • iPhone6s Ҏ߱ • iPad Proʢୈ1ੈ୅ʣҎ߱ • iPadʢୈ5ੈ୅ʣҎ߱ ॳ୅ARKit ͕࢖͑ΔͳΒɺARKit 2΋࢖͑Δ

Slide 46

Slide 46 text

About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ • iOS 12Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ ͍ͯ͠ΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 10Ҏ্͕ඞཁ

Slide 47

Slide 47 text

About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ • iOS 12Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ ͍ͯ͠ΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 10Ҏ্͕ඞཁ • macOS High Sierra (10.13.4) Ҏ্͕ඞཁ

Slide 48

Slide 48 text

New Features of ARKit 2 • Saving and Loading Maps ‣ Ϛοϐϯάͨ͠ݱ࣮ੈքͷ৘ใΛอଘ͠ɺಡΈࠐΊΔ • Environment Texturing ‣ ޫΛ൓ࣹ͢ΔޮՌΛ࣋ͬͨ3DΦϒδΣΫτʹରͯ͠ɺपΓͷݱ࣮ͷ ෩ܠΛө͢͜ͱ͕Ͱ͖Δ • Image Tracking ‣ ը૾ΛARͷΞϯΧʔͱͯ͠࢖͑Δ • Object Detection ‣ ࣄલʹεΩϟϯ͓͍ͯͨ͠ݱ࣮ͷΦϒδΣΫτΛೝࣝͰ͖Δ • Gaze and Tongue ‣ إೝࣝͰࢹઢͱઉΛೝࣝͰ͖Δ

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Saving and Loading Maps • ARWorldMap ͱ͍͏Ϋϥε͕௥Ճ͞Εɺۭؒͷಛ௃఺ΛϚοϐϯάͨ͠৘ใΛڞ ༗͠ɺෳ਺ͷϢʔβʔͰಉ͡ARମݧ͕Ͱ͖ΔΑ͏ʹͳͬͨ • อଘͨ͠Ϛοϐϯά৘ใ͸ Data ܕʹม׵Ͱ͖ΔͷͰɺMultipeer Connectivity ΍ αʔόʔΛհͯ͠ૹ৴͢Δ͜ͱ͕Ͱ͖Δ • ड͚औͬͨϚοϐϯά৘ใ͸ ARWorldTrackingConﬁguration ͷ initialWorldMap: ARWorldMap ϓϩύςΟʹηοτ͢Δ͜ͱͰಡΈࠐΊΔ

Slide 51

Slide 51 text

Image Tracking • ARImageTrackingConfiguration ͱ͍͏Ϋϥε͕௥Ճ͞ΕɺARSCNViewͷ session ͷ run ΛݺͿࡍͷҾ਺ͷ configuration ͱͯ͠ࢦఆ͢Δ͜ͱͰɺը ૾ೝࣝʴτϥοΩϯά͕Ͱ͖ΔΑ͏ʹͳΔ • τϥοΩϯά͢Δը૾͸ը૾ೝࣝͷ৔߹ͱಉ͘͡ɺconfiguration: ARWorldTrackingConfiguration ͷ detectionImages ʹੜ੒ͨ͠ ARReferenceImage Ληοτ͢Δ

Slide 52

Slide 52 text

Object Detection • ARObjectScanningConﬁguration ͱ͍͏Ϋϥε͕௥Ճ͞ΕɺARSCNView ͷ session ͷ run ΛݺͿࡍͷҾ਺ͷ conﬁguration ͱͯ͠ࢦఆ͢Δ͜ͱͰɺ 3࣍ݩͷΦϒδΣΫτͷಛ௃఺ΛεΩϟϯ͢Δ͜ͱ͕Ͱ͖Δ • εΩϟϯͨ͠ΦϒδΣΫτ͸ ARReferenceObject (.arobject) ͱͯ͠ѻ͏ ͜ͱ͕Ͱ͖ɺasset catalog Ͱ؅ཧͰ͖Δ

Slide 53

Slide 53 text

Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary

Slide 54

Slide 54 text

Summary • ARKit Λ࢖͑͹ɺগͳ͍ίʔυྔͰARମݧΛ ࣮ݱͰ͖Δ • ARKit 2 ͷొ৔ʹΑΓɺ࣮૷Ͱ͖ΔARମݧͷ ෯͕޿͕ͬͨ • ARٕज़Λ࢖ͬͨΞϓϦ͸ࠓޙ΋ͲΜͲΜ૿͑ ͍ͯ͘ͱߟ͑ΒΕΔ

Slide 55

Slide 55 text

Thank youʂ

Slide 56

Slide 56 text

ࢀߟϦϯΫ • Introducing ARKit: Augmented Reality for iOS - WWDC 2017 - Videos - Apple Developer  https://developer.apple.com/videos/play/wwdc2017/602 • What’s New in ARKit 2 - WWDC 2018 - Videos - Apple Developer  https://developer.apple.com/videos/play/wwdc2018/602 • [iOS 11] ͸͡ΊͯͷARKit #WWDC2017 | Developers.IO  https://dev.classmethod.jp/smartphone/ios-11-arkit/ • iOS Ͱ SceneKit Λࢼ͢(Swift 3) ͦͷ1 - ͸͡Ίʹ - Apple Engine  http://appleengine.hatenablog.com/entry/2017/05/29/184901 • iOS Ͱ SceneKit Λࢼ͢(Swift 3) ͦͷ57 - SceneKit Ͱ࢖༻Ͱ͖ΔΦϒδΣΫτϑΝΠϧ - Apple Engine  http://appleengine.hatenablog.com/entry/2017/08/08/183538 • ARKit Tutorial: Working with 2D Image Recognition  https://www.appcoda.com/arkit-image-recognition/ • ʮాதώϝʯ / ాதώϝˍླ໦ώφ ͞Μͷ࡞඼ - χίχཱମ  http://3d.nicovideo.jp/works/td35317 • powroupi/blender_mmd_tools  https://github.com/powroupi/blender_mmd_tools