ARKit + SceneKitでMinesweeperを作ってみた

ARKit + SceneKitでマインスイーパーをつくってみました

Transcript

1. Create AR Minesweeper by using ARKit 2017.12.2 ୈ75ճ Cocoaษڧձؔ੢ @matuyuji
2. @matuyuji safx-dev.blogspot.jp
4. Minesweeper
5. Minesweeper
6. AR Minesweeper
7. AR Minesweeper 1 1
9. AR Minesweeper 1 • ϓϨΠϠʔͷ͍Δηϧ͕։͔Εɺര஄ͷ਺Λදࣔ͢Δ • ը໘্ͷηϧͷλοϓͰɺηϧʹʮϚʔΫʯͰ͖Δ 1 ʮࠓ͍ΔηϧʯΈ͍ͨͳ ֓೦͕Ͱ͖Δ఺͕ैདྷͱ ҟͳΔ
10. Motivation • ARKitΛ࢖ͬͯԿ͔࡞ͬͯΈ͍ͨ • ͦͦ͜͜؆୯ͳήʔϜ͕Αͦ͞͏ • ฏ໘͕औΕͦ͏ & ࣗ෼ͷ৔ॴ͕Θ͔Γͦ͏ → ஍໘͕࢖͑ΔͳΒMinesweeperͱ͔Αͦ͞͏ → ARKit + SceneKitͰMinesweeperΛ࡞ͬͯΈΔ
11. Augmented Reality
12. –ARKIT | Apple Developer Documentation “Augmented reality (AR) describes user experiences that add 2D or 3D elements to the live view from a device's camera in a way that makes those elements appear to inhabit the real world. ”
14. ARKit + SceneKit
15. ARKit • ϫʔϧυτϥοΩϯά • ฏ໘ݕग़ɺিಥ൑ఆɺޫݯ༧ଌ • ߴ౓ͳAPI (ࡉ͔͍ઃఆ͕ෆཁ) • ϞόΠϧͰಈ࡞ • SceneKit, SpriteKit, Metal ARKit combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.
18. class ViewController: UIViewController, ARSCNViewDelegate { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() sceneView.delegate = self sceneView.showsStatistics = true let scene = SCNScene(named: "art.scnassets/ship.scn")! sceneView.scene = scene } override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) let configuration = ARWorldTrackingConfiguration() sceneView.session.run(configuration) } override func viewWillDisappear(_ animated: Bool) { super.viewWillDisappear(animated) sceneView.session.pause() } }
19. ARSCNView ARSession ARCamera ARFrame ARConfiguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors
20. ARSCNView ARSession ARCamera ARFrame ARConfiguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors • ܧঝؔ܎: UIView ← SCNView ← ARSCNView • σόΠεΧϝϥʹө͍ͬͯΔ಺༰Λγʔϯͷഎܠ ͱͯ͠දࣔ • ϢʔβͷҐஔ΍޲͖ͷมߋͰΧϝϥҐஔ͕มΘͬ ͨͱ͖ʹදࣔΛࣗಈͰมߋͯ͘͠ΕΔ
21. ARSCNView ARSession ARCamera ARFrame ARConfiguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors • σόΠεΧϝϥͱϞʔγϣϯ ॲཧΛߦ͏ • ཪͰ͸AVCaptureSessionͱ CMMotionManagerΛ࢖͏ • ηογϣϯͷઃఆΛ͢Δந৅Ϋϥε • ฏ໘ݕग़ͱϙδγϣϯτϥοΩϯάΛ͍ͨ͠ ͳΒ͹ARWorldTrackingConfigurationΛ࢖͏ ֤ϑϨʔϜ͝ͱͷσόΠε Χϝϥͷը૾ͱϙδγϣϯ τϥοΩϯά৘ใΛ࣋ͭ Χϝϥ ϙδγϣϯτϥοΩ ϯά͞Ε͍ͯΔ΋ͷ
22. class ViewController: UIViewController, ARSCNViewDelegate { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() sceneView.delegate = self sceneView.showsStatistics = true let scene = SCNScene(named: "art.scnassets/ship.scn")! sceneView.scene = scene } override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) let configuration = ARWorldTrackingConfiguration() sceneView.session.run(configuration) } override func viewWillDisappear(_ animated: Bool) { super.viewWillDisappear(animated) sceneView.session.pause() } }
23. Plain Detection • ݕग़͞ΕΔͱARSCNViewDelegateͷ renderer(_:didAdd:for:)ͳͲ͕ݺ͹ΕΔ override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = .horizontal sceneView.session.run(configuration) }
24. renderer(_:didAdd:for:) The view calls this method once for each new anchor. ARKit also calls this method to provide visual content for any ARAnchor objects you manually add using the session's add(anchor:) method. optional func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) SCNSceneRendererDelegate func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { guard let planeAnchor = anchor as? ARPlaneAnchor else { return } → Planeݕग़࣌Ҏ֎ʹ΋ࣗ෼͕௥Ճͨ͠ͱ͖΋ݺ͹ΕΔ
25. renderer(_:updateAtTime:) SceneKit calls this method exactly once per frame, so long as the SCNView object (or other SCNSceneRenderer object) displaying the scene is not paused. → γʔϯ͕ఀࢭ͠ͳ͍ݶΓɺຖϑϨʔϜݺ͹ΕΔ optional func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) SCNSceneRendererDelegate let planes = currentFrame.anchors.filter { $0 is ARPlaneAnchor }
28. Plane Detection • ஍໘΋ݕग़Ͱ͖Δ • ͔͠͠ɺݕग़ʹ͕͔͔࣌ؒΔ • ෦԰ͷܗঢ়ʹਖ਼͘͠Ԋͬͯݕग़͞Εͳ͍ • ్தͰฏ໘ͷϚʔδ͕ൃੜͨ͠Γ͢Δ • ྑ͍λΠϛϯάͰฏ໘ݕग़ΛࢭΊͨ΄͏͕Αͦ͞͏
29. Hit Test for Plane • σόΠεεΫϦʔϯΛλοϓͨ͠ͱ͖ͳͲʹͲͷฏ໘ʹ৮ Ε͔ͨΛಘΒΕΔ • Χϝϥ͔Β͍ۙ΋ͷॱʹιʔτ͞Εͨ݁ՌΛฦ͢ @IBAction func handleTapGesture(_ tap: UITapGestureRecognizer) { let point = tap.location(in: sceneView) let results = sceneView.hitTest(point, types: .estimatedHorizontalPlane) ɿ }
30. Hit Test • ARSCNView͸3ͭͷhitTestΛ࣋ͭͷͰ஫ҙ • ARSCNViewࣗମ͕࣋ͭhitTest(_:types:) • ARฏ໘ͱͷώοτςετ • SceneKitͷSCNSceneRenderer͕࣋ͭhitTest(_:options:) • scene্ʹ͋ΔSceneKitΦϒδΣΫτͱͷώοτςετ • UIViewͷhitTest(_:with:)
31. ΦϒδΣΫτͷ௥Ճ • SCNNodeΛARSCNViewͷscene΁௥Ճ͢Δ • δΦϝτϦͷ୯Ґ͸ϝʔτϧ • ΦϒδΣΫτͷҐஔ͕ۭؒ಺ʹݻఆ͞ΕΔ let geom = SCNBox(width: 1, height: 1, length: 1, chamferRadius: 0) geom.firstMaterial!.diffuse.contents = UIColor.red geom.firstMaterial!.specular.contents = UIColor.white let node = SCNNode(geometry: geom) sceneView.scene.rootNode.addChildNode(node)
32. ΦϒδΣΫτΞχϝʔγϣϯ node.runAction(SCNAction.moveBy(x: 0.5, y: 0, z: 0, duration: 0.2))
33. AR Minesweeper
34. ஌Δඞཁ͕͋ͬͨ͜ͱ • ϢʔβʔͷҐஔΛͲ͏औΔ͔ • ARͰ஍໘ΛऔͬͯɺήʔϜʹ࢖͑Δ͔ • MinesweeperΛͲ͏࡞Δ͔
35. Camera Position • ARSCNView.session.currentFrame.cameraͰ ARCamera͕औΕΔ • ARCamera͔ΒiPhoneͷ (τϥοΩϯά։͔࢝Βͷ૬ରͷ) ࠲ඪ͕औΕΔ
36. ARSCNView ARSession ARCamera ARFrame ARConfiguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors
37. ARCamera guard let currentFrame = sceneView.session.currentFrame else { return } let camera = currentFrame.camera ɿ
38. Camera Position —ARKitExample func positionFromTransform(_ transform: matrix_float4x4) -> SCNVector3 { return SCNVector3Make(transform.columns.3.x, transform.columns.3.y, transform.columns.3.z) } // Usage let position = positionFromTransform(camera.transform) let angles = camera.eulerAngles
39. Position for Minesweeper • ΧϝϥͷҐஔ (= iPhoneΛ͍࣋ͬͯΔҐஔ) ͔Β஍໘ʹ ਨ௚ʹ߱Ζͨ͠఺ Λ ݱࡏͷҐஔ ͱ͢Δ • ࣮ࡍʹ͸30~40cm΄Ͳલʹདྷͯ͠·͏
 (ࠓճ͸ͦͷิਖ਼Λ͠ͳ͍)
40. Ϣʔβͷ͍Δηϧ • ݱࡏͷҐஔ ͔ΒҰ൪͍ۙηϧΛϢʔβͷ͍Δηϧͱ ͢Δ • ࣮ࡍʹ͸30~40cm΄Ͳલʹདྷͯ͠·͏
 (ࠓճ͸ͦͷิਖ਼Λ͠ͳ͍) • ୅ΘΓʹɺࠓ͍Δηϧʹ৭Λ෇͚ͨΓɺ ηϧ΁ͷҠಈΛ3ඵ଴ͭΑ͏ʹͨ͠Γͨ͠
41. ஌Δඞཁ͕͋ͬͨ͜ͱ • ϢʔβʔͷҐஔΛͲ͏औΔ͔ • ARͰ஍໘ΛऔͬͯɺήʔϜʹ࢖͑Δ͔ • MinesweeperΛͲ͏࡞Δ͔
42. Plane Detection (࠶ܝ) • ஍໘΋ݕग़Ͱ͖Δ • ͔͠͠ɺݕग़ʹ͕͔͔࣌ؒΔ • ෦԰ͷܗঢ়ʹਖ਼͘͠Ԋͬͯݕग़͞Εͳ͍ • ్தͰฏ໘ͷϚʔδ͕ൃੜͨ͠Γ͢Δ • ྑ͍λΠϛϯάͰฏ໘ݕग़ΛࢭΊͨ΄͏͕Αͦ͞͏
43. Game field for Minesweeper • Ұ൪େ͖͍ฏ໘Λ࢖͏ • Ұ൪௿͍Ґஔʹ͋Δฏ໘Λ࢖͏ • ࣗಈݕग़ΛఘΊͨ • δΣενϟ (ύϯɺϐϯνɺϩʔςʔτ) Ͱࣗ෼ͰϑΟʔϧυ ͷҐஔɺେ͖͞ɺճసΛௐ੔͢ΔΑ͏ʹͨ͠ • ࢦఆͨ͠αΠζͷηϧͰϑΟʔϧυΛຒΊΔΑ͏ʹͨ͠ ͦ΋ͦ΋্ख͘ݕग़Ͱ ͖͍ͯͳ͍ͷͰ্ख͘ ͍͔ͳ͍
    
45. ஌Δඞཁ͕͋ͬͨ͜ͱ • ϢʔβʔͷҐஔΛͲ͏औΔ͔ • ARͰ஍໘ΛऔͬͯɺήʔϜʹ࢖͑Δ͔ • MinesweeperΛͲ͏࡞Δ͔
46. AR Minesweeper • δΣενϟͰϑΟʔϧυͷେ͖͞ͳͲΛܾΊͯ։࢝ • ݱࡏ͍Δηϧ͕දࣔ͞ΕΔ • ηϧʹ3ඵ͍ͨΒͦͷηϧ͕։͘ • ര஄Ҏ֎ͷηϧΛશ෦։͍ͨΒΫϦΞ • ը໘্ͷηϧͷλοϓͰɺηϧʹʮϚʔΫʯͰ͖Δ
47. AR Minesweeper 1 1
48. Tips
49. Simd Methods • SIMD໋ྩΛར༻ͯ͠ૣ͍ (Α͏ͩ) ← υΩϡϝϯτͳ͍ • Xcode 9͔Βར༻Ͱ͖Δ • simdTransformͳͲ͕͋Δ
50. ײ૝
51. Wireless Development ARKitΛ࢖ͬͨΞϓϦͩͱඞਢ
52. Wireless Development
53. Wireless Development ͜͏ͳΔͱɺXcode࠶ىಈͳͲ͍Ζ͍ΖؤுΒͳ͍ͱ͍͚ͳ͍
54. ײ૝ • ஍໘ͷݕग़ਫ਼౓͕ѱ͍ • ฏ໘͕Ϛʔδ͞ΕͨΓ͢ΔͷͰ్தͰࢭΊͨ΄͏͕Α͍ • ήʔϜϓϨΠʹ޿͍৔ॴ͕ཁΔ • 10×10Ͱ΋5mඞཁ • ͦ΋ͦ΋ήʔϜͱ͓ͯ͠΋͠Ζ͘ͳ͍…
55. ·ͱΊ
56. Recap • ARKit + SceneKitͰMinesweeperΛͭͬͨ͘ • ARKit͸σόΠεΧϝϥҐஔͷτϥοΩϯά΍ฏ໘ݕग़ͳ ͲͰ͖Δ • SceneKitͷΦϒδΣΫτΛஔ͍ͯΠϯλϥΫγϣϯͰ͖Δ