ゼロから始める ARKit

【サポーターズCoLab勉強会】まだ間に合う！ゼロから始めるARKit入門！（ https://supporterzcolab.com/event/423/ ）での発表資料です。

Transcript

1. ARKit αϙʔλʔζColabษڧձ 2018/06/13(ਫ) Keisuke Tatsumi
2. Keisuke Tatsumi • Github, Qiita : cokaholic • Twitter : @TK_u_nya • AppStore : Keisuke Tatsumi
3. iOS engineer at AbemaTV, Inc.
4. ຊ೔ͷςʔϚ͸…
5. ARKit
6. Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary
7. Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary
8. Introduction ARͱ͸ • ʮAugmented Realityʢ֦ுݱ࣮ʣʯͷུ • ࣮ࡍͷ෩ܠʹόʔνϟϧͷࢹ֮৘ใΛॏͶͯදࣔ ͢Δ͜ͱͰɺݱ࣮ͷੈքΛԾ૝తʹ֦ு͢Δٕज़
9. Introduction ֤ࣾͷϞόΠϧARϑϨʔϜϫʔΫ • Apple - ARKit • Android - ARCore
10. Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary
11. What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ
12. What’s ARKit
13. What’s ARKit
14. What’s ARKit
15. What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ • iOS 11Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ͠ ͍ͯΔσόΠε͕ඞཁ
16. Devices supporting ARKit • iPhone6s Ҏ߱ • iPad Proʢୈ1ੈ୅ʣҎ߱ • iPadʢୈ5ੈ୅ʣҎ߱
17. ͋Εʁ
18. iPod touch ๨Εͯͳ͍ʁ
19. Ͱ͸ͳ͘ɺ
20. Devices supporting ARKit • iPhone6s Ҏ߱ • iPad Proʢୈ1ੈ୅ʣҎ߱ • iPadʢୈ5ੈ୅ʣҎ߱ • iPod touch ͸શ6ੈ୅͕શͯඇରԠ
 ʢiPod touchୈ7ੈ୅ʹظ଴…ʂʣ
21. What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ • iOS 11Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ͠ ͍ͯΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 9Ҏ্͕ඞཁ
22. What’s ARKit • ࡢ೥ͷWWDC2017ʹͯApple͔Βൃද͞Εͨ ARΞϓϦ։ൃͷͨΊͷϑϨʔϜϫʔΫ • iOS 11Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ͠ ͍ͯΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 9Ҏ্͕ඞཁ • macOS Sierra (10.12.4) Ҏ্͕ඞཁ
23. Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary
24. Hello, ARKit World Demo XcodeͷςϯϓϨʔτͰ ࡞੒͞ΕΔARΞϓϦΛى ಈͯ͠Έͨ
25. Augmented Reality App Template ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")! // Set the scene to the view sceneView.scene = scene } override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) // Create a session configuration let configuration = ARWorldTrackingConfiguration() // Run the view's session sceneView.session.run(configuration) } }
26. ARSCNView ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { @IBOutlet var sceneView: ARSCNView! • 3D SceneKitͷίϯςϯπΛΧϝϥͰARͱͯ͠ද ࣔ͢ΔͨΊͷView • AR App Template Ͱ͸ॳظදࣔͷViewController ͷview ͷ subviewͱͯ͠഑ஔ͞Ε͍ͯΔ • ARSCNView ʹ͸ scene: SCNScene ϓϩύςΟ͕ ͋Γɺ͜ͷ scene ʹ ඈߦػͳͲͷ node Λ௥Ճ͢ Δ͜ͱͰɺΦϒδΣΫτΛදࣔͰ͖Δ
27. Set SCNScene To ARSCNView ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")! // Set the scene to the view sceneView.scene = scene } • ඈߦػΦϒδΣΫτͷݩͰ͋Δ ship.scn ϑΝΠϧ Λࢦఆͯ͠ੜ੒ͨ͠ SCNScene Λ ARSCNView ͷ scene ͱͯ͠ઃఆ͢Δ͜ͱͰද͍ࣔͯ͠Δ
28. SCNScene • namedͷҾ਺ͰࢦఆͰ͖Δͷ͸ .scnϑΝΠϧͷΈ // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")!
29. SCNScene • namedͷҾ਺ͰࢦఆͰ͖Δͷ͸ .scnϑΝΠϧͷΈ • ಡΈࠐΜͰ .scnϑΝΠϧ΁ม׵Ͱ͖ΔϑΥʔϚοτ ‣ COLLADA (.dae) ‣ Alembic (.abc) ‣ PLY ϑΝΠϧϑΥʔϚοτ ‣ Standard Triangulated Language (.stl) ‣ Wavefront OBJ (.obj) ‣ Pixar Universal Scene Description (.usd/.usda) // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")!
30. Augmented Reality App Template ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() // Create a new scene let scene = SCNScene(named: "art.scnassets/ship.scn")! // Set the scene to the view sceneView.scene = scene } override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) // Create a session configuration let configuration = ARWorldTrackingConfiguration() // Run the view's session sceneView.session.run(configuration) } }
31. How To understand Statistics ViewController.swift import UIKit import SceneKit import ARKit class ViewController: UIViewController { override func viewDidLoad() { super.viewDidLoad() // Show statistics such as fps and timing information sceneView.showsStatistics = true } } ϑϨʔϜϨʔτήʔδ Metal(Mt) / OpenGL(GL) ϑϨʔϜϨʔτ શମͷϙϦΰϯ਺ υϩʔίʔϧ
32. Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary
33. Image Detection On ARKit Demo ాதώϝͷMMDϞσϧΛ ը૾ݕ஌ͨ͠Ґஔʹදࣔ ͢Δ ※͜ͷޙͷεϥΠυͰը૾ݕ஌ͷ࢓૊ΈΛղઆ͠·͢ɻ
34. Add AR Resource Group Assets.xcassetsΛ։͖ɺ+Ϙλϯ͔ΒʮAR ResourcesʯάϧʔϓΛ࡞੒
35. Add Images Into AR Resources ݕ஌ͤ͞ΔͨΊͷը૾ΛʮAR Resourcesʯάϧʔϓ಺ʹ
 υϥοά&υϩοϓͰ௥Ճ͠·͢ɻ © ాதώϝ © (ג)ΞΫτθϩ
36. Set Real Image Size ௥Ճͨ͠ը૾ʹ͸ݱ࣮ͰͷϦΞϧͳαΠζΛ
 ೖྗ͢Δඞཁ͕͋Γ·͢ɻ
37. Check Warnings ݕ஌͢Δը૾ͷαΠζΛೖྗͯ͠΋Τϥʔ͕ফ͑ͳ͍৔߹͸ɺ
 ͦͷը૾͕ը૾ݕ஌ʹ޲͍͍ͯͳ͍ཧ༝͕දࣔ͞Ε·͢ɻ ͜͏ͳͬͨ৔߹͸ɺΤϥʔ಺༰ʹ४ڌͨ͠ผͷը૾Λ༻ҙ͠·͠ΐ͏ɻ
38. Using Good Images • ը࣭͕ྑ͍ • ಠಛͷಛ௃͕͋Δ • ίϯτϥετ͕͸͖ͬΓ͍ͯ͠Δ • ը࣭͕ߥ͍ • ΧϥʔྖҬ͕ۉҰ͗͢Δ • ৭ຯʹϝϦϋϦ͕ͳ͍ © (ג)ΞΫτθϩ © ాதώϝ
39. Set Detection Images // Create reference images guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else { fatalError("Reference images not found.") } // Create a session configuration let configuration = ARWorldTrackingConfiguration() // Set detection images configuration.detectionImages = referenceImages // Run the view's session sceneView.session.run(configuration) • ݕ஌͍ͤͨ͞ը૾άϧʔϓ໊Ͱ͋ΔʮAR ResourcesʯΛࢦఆͯ͠ ARReferenceImage ΫϥεͷΠϯελϯεΛੜ੒͢Δ • configuration: ARWorldTrackingConfiguration ͷ detectionImages ʹੜ੒ͨ͠ ARReferenceImage Ληοτ͢Δͱɺ ը૾Λݕ஌Ͱ͖ΔΑ͏ʹͳΔ
40. Get Detected Images extension ViewController: ARSCNViewDelegate { // MARK: - ARSCNViewDelegate func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { // Check detected an image guard let imageAnchor = anchor as? ARImageAnchor, let imageName = imageAnchor.referenceImage.name else { return } // Check the detected image is "hime_tanaka" if imageName == "hime_tanaka" { print("Detected!") } } } • ը૾ͷݕग़͸ ARSCNViewDelegate ͷ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) ϝιου಺ʹͯߦ͏ • ݕग़ͨ͠ը૾ͷ໊લʹΑͬͯɺͲͷը૾͕ݕग़͞Ε͔͕ͨΘ͔ΔͨΊɺ ը૾͝ͱʹ൓ԠΛม͑Δ͜ͱ΋Ͱ͖Δ
41. Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary
42. About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ
43. About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ • iOS 12Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ ͍ͯ͠ΔσόΠε͕ඞཁ
44. Devices supporting ARKit 2 iOS 11͕࢖͑Ε͹ɺiOS 12΋࢖͑Δ
45. Devices supporting ARKit 2 • iPhone6s Ҏ߱ • iPad Proʢୈ1ੈ୅ʣҎ߱ • iPadʢୈ5ੈ୅ʣҎ߱ ॳ୅ARKit ͕࢖͑ΔͳΒɺARKit 2΋࢖͑Δ
46. About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ • iOS 12Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ ͍ͯ͠ΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 10Ҏ্͕ඞཁ
47. About ARKit 2 • ࠓ೥ͷWWDC2018ʹͯApple͔Βൃද͞Εͨ ARKitͷ࠷৽όʔδϣϯ • iOS 12Ҏ্͔ͭɺA9Ҏ߱ͷϓϩηοαΛ౥ࡌ ͍ͯ͠ΔσόΠε͕ඞཁ • ౷߹։ൃ؀ڥ͸Xcode 10Ҏ্͕ඞཁ • macOS High Sierra (10.13.4) Ҏ্͕ඞཁ
48. New Features of ARKit 2 • Saving and Loading Maps ‣ Ϛοϐϯάͨ͠ݱ࣮ੈքͷ৘ใΛอଘ͠ɺಡΈࠐΊΔ • Environment Texturing ‣ ޫΛ൓ࣹ͢ΔޮՌΛ࣋ͬͨ3DΦϒδΣΫτʹରͯ͠ɺपΓͷݱ࣮ͷ ෩ܠΛө͢͜ͱ͕Ͱ͖Δ • Image Tracking ‣ ը૾ΛARͷΞϯΧʔͱͯ͠࢖͑Δ • Object Detection ‣ ࣄલʹεΩϟϯ͓͍ͯͨ͠ݱ࣮ͷΦϒδΣΫτΛೝࣝͰ͖Δ • Gaze and Tongue ‣ إೝࣝͰࢹઢͱઉΛೝࣝͰ͖Δ
49. New Features of ARKit 2 • Saving and Loading Maps ‣ Ϛοϐϯάͨ͠ݱ࣮ੈքͷ৘ใΛอଘ͠ɺಡΈࠐΊΔ • Environment Texturing ‣ ޫΛ൓ࣹ͢ΔޮՌΛ࣋ͬͨ3DΦϒδΣΫτʹରͯ͠ɺपΓͷݱ࣮ͷ ෩ܠΛө͢͜ͱ͕Ͱ͖Δ • Image Tracking ‣ ը૾ΛARͷΞϯΧʔͱͯ͠࢖͑Δ • Object Detection ‣ ࣄલʹεΩϟϯ͓͍ͯͨ͠ݱ࣮ͷΦϒδΣΫτΛೝࣝͰ͖Δ • Gaze and Tongue ‣ إೝࣝͰࢹઢͱઉΛೝࣝͰ͖Δ
50. Saving and Loading Maps • ARWorldMap ͱ͍͏Ϋϥε͕௥Ճ͞Εɺۭؒͷಛ௃఺ΛϚοϐϯάͨ͠৘ใΛڞ ༗͠ɺෳ਺ͷϢʔβʔͰಉ͡ARମݧ͕Ͱ͖ΔΑ͏ʹͳͬͨ • อଘͨ͠Ϛοϐϯά৘ใ͸ Data ܕʹม׵Ͱ͖ΔͷͰɺMultipeer Connectivity ΍ αʔόʔΛհͯ͠ૹ৴͢Δ͜ͱ͕Ͱ͖Δ • ड͚औͬͨϚοϐϯά৘ใ͸ ARWorldTrackingConfiguration ͷ initialWorldMap: ARWorldMap ϓϩύςΟʹηοτ͢Δ͜ͱͰಡΈࠐΊΔ
51. Image Tracking • ARImageTrackingConfiguration ͱ͍͏Ϋϥε͕௥Ճ͞ΕɺARSCNViewͷ session ͷ run ΛݺͿࡍͷҾ਺ͷ configuration ͱͯ͠ࢦఆ͢Δ͜ͱͰɺը ૾ೝࣝʴτϥοΩϯά͕Ͱ͖ΔΑ͏ʹͳΔ • τϥοΩϯά͢Δը૾͸ը૾ೝࣝͷ৔߹ͱಉ͘͡ɺconfiguration: ARWorldTrackingConfiguration ͷ detectionImages ʹੜ੒ͨ͠ ARReferenceImage Ληοτ͢Δ
52. Object Detection • ARObjectScanningConfiguration ͱ͍͏Ϋϥε͕௥Ճ͞ΕɺARSCNView ͷ session ͷ run ΛݺͿࡍͷҾ਺ͷ configuration ͱͯ͠ࢦఆ͢Δ͜ͱͰɺ 3࣍ݩͷΦϒδΣΫτͷಛ௃఺ΛεΩϟϯ͢Δ͜ͱ͕Ͱ͖Δ • εΩϟϯͨ͠ΦϒδΣΫτ͸ ARReferenceObject (.arobject) ͱͯ͠ѻ͏ ͜ͱ͕Ͱ͖ɺasset catalog Ͱ؅ཧͰ͖Δ
53. Agenda • Introduction • What’s ARKit • Hello, ARKit World • Image Detection On ARKit • About ARKit 2 • Summary
54. Summary • ARKit Λ࢖͑͹ɺগͳ͍ίʔυྔͰARମݧΛ ࣮ݱͰ͖Δ • ARKit 2 ͷొ৔ʹΑΓɺ࣮૷Ͱ͖ΔARମݧͷ ෯͕޿͕ͬͨ • ARٕज़Λ࢖ͬͨΞϓϦ͸ࠓޙ΋ͲΜͲΜ૿͑ ͍ͯ͘ͱߟ͑ΒΕΔ
55. Thank youʂ
56. ࢀߟϦϯΫ • Introducing ARKit: Augmented Reality for iOS - WWDC 2017 - Videos - Apple Developer
 https://developer.apple.com/videos/play/wwdc2017/602 • What’s New in ARKit 2 - WWDC 2018 - Videos - Apple Developer
 https://developer.apple.com/videos/play/wwdc2018/602 • [iOS 11] ͸͡ΊͯͷARKit #WWDC2017 | Developers.IO
 https://dev.classmethod.jp/smartphone/ios-11-arkit/ • iOS Ͱ SceneKit Λࢼ͢(Swift 3) ͦͷ1 - ͸͡Ίʹ - Apple Engine
 http://appleengine.hatenablog.com/entry/2017/05/29/184901 • iOS Ͱ SceneKit Λࢼ͢(Swift 3) ͦͷ57 - SceneKit Ͱ࢖༻Ͱ͖ΔΦϒδΣΫτϑΝΠϧ - Apple Engine
 http://appleengine.hatenablog.com/entry/2017/08/08/183538 • ARKit Tutorial: Working with 2D Image Recognition
 https://www.appcoda.com/arkit-image-recognition/ • ʮాதώϝʯ / ాதώϝˍླ໦ώφ ͞Μͷ࡞඼ - χίχཱମ
 http://3d.nicovideo.jp/works/td35317 • powroupi/blender_mmd_tools
 https://github.com/powroupi/blender_mmd_tools