ARKit + SceneKitでMinesweeperを作ってみた

Slide 1

Slide 1 text

Create AR Minesweeper by using ARKit 2017.12.2 ୈ75ճ Cocoaษڧձؔ੢ @matuyuji

Slide 2

Slide 2 text

@matuyuji safx-dev.blogspot.jp

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Minesweeper

Slide 5

Slide 5 text

Minesweeper

Slide 6

Slide 6 text

AR Minesweeper

Slide 7

Slide 7 text

AR Minesweeper 1 1

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

AR Minesweeper 1 • ϓϨΠϠʔͷ͍Δηϧ͕։͔Εɺര஄ͷ਺Λදࣔ͢Δ • ը໘্ͷηϧͷλοϓͰɺηϧʹʮϚʔΫʯͰ͖Δ 1 ʮࠓ͍ΔηϧʯΈ͍ͨͳ ֓೦͕Ͱ͖Δ఺͕ैདྷͱ ҟͳΔ

Slide 10

Slide 10 text

Motivation • ARKitΛ࢖ͬͯԿ͔࡞ͬͯΈ͍ͨ • ͦͦ͜͜؆୯ͳήʔϜ͕Αͦ͞͏ • ฏ໘͕औΕͦ͏ & ࣗ෼ͷ৔ॴ͕Θ͔Γͦ͏ → ஍໘͕࢖͑ΔͳΒMinesweeperͱ͔Αͦ͞͏ → ARKit + SceneKitͰMinesweeperΛ࡞ͬͯΈΔ

Slide 11

Slide 11 text

Augmented Reality

Slide 12

Slide 12 text

–ARKIT | Apple Developer Documentation “Augmented reality (AR) describes user experiences that add 2D or 3D elements to the live view from a device's camera in a way that makes those elements appear to inhabit the real world. ”

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

ARKit + SceneKit

Slide 15

Slide 15 text

ARKit • ϫʔϧυτϥοΩϯά • ฏ໘ݕग़ɺিಥ൑ఆɺޫݯ༧ଌ • ߴ౓ͳAPI (ࡉ͔͍ઃఆ͕ෆཁ) • ϞόΠϧͰಈ࡞ • SceneKit, SpriteKit, Metal ARKit combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

class ViewController: UIViewController, ARSCNViewDelegate { @IBOutlet var sceneView: ARSCNView! override func viewDidLoad() { super.viewDidLoad() sceneView.delegate = self sceneView.showsStatistics = true let scene = SCNScene(named: "art.scnassets/ship.scn")! sceneView.scene = scene } override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) let configuration = ARWorldTrackingConfiguration() sceneView.session.run(configuration) } override func viewWillDisappear(_ animated: Bool) { super.viewWillDisappear(animated) sceneView.session.pause() } }

Slide 19

Slide 19 text

ARSCNView ARSession ARCamera ARFrame ARConﬁguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors

Slide 20

Slide 20 text

ARSCNView ARSession ARCamera ARFrame ARConﬁguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors • ܧঝؔ܎: UIView ← SCNView ← ARSCNView • σόΠεΧϝϥʹө͍ͬͯΔ಺༰Λγʔϯͷഎܠ ͱͯ͠දࣔ • ϢʔβͷҐஔ΍޲͖ͷมߋͰΧϝϥҐஔ͕มΘͬ ͨͱ͖ʹදࣔΛࣗಈͰมߋͯ͘͠ΕΔ

Slide 21

Slide 21 text

ARSCNView ARSession ARCamera ARFrame ARConﬁguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors • σόΠεΧϝϥͱϞʔγϣϯ ॲཧΛߦ͏ • ཪͰ͸AVCaptureSessionͱ CMMotionManagerΛ࢖͏ • ηογϣϯͷઃఆΛ͢Δந৅Ϋϥε • ฏ໘ݕग़ͱϙδγϣϯτϥοΩϯάΛ͍ͨ͠ ͳΒ͹ARWorldTrackingConﬁgurationΛ࢖͏ ֤ϑϨʔϜ͝ͱͷσόΠε Χϝϥͷը૾ͱϙδγϣϯ τϥοΩϯά৘ใΛ࣋ͭ Χϝϥ ϙδγϣϯτϥοΩ ϯά͞Ε͍ͯΔ΋ͷ

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Plain Detection • ݕग़͞ΕΔͱARSCNViewDelegateͷ renderer(_:didAdd:for:)ͳͲ͕ݺ͹ΕΔ override func viewWillAppear(_ animated: Bool) { super.viewWillAppear(animated) let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = .horizontal sceneView.session.run(configuration) }

Slide 24

Slide 24 text

renderer(_:didAdd:for:) The view calls this method once for each new anchor. ARKit also calls this method to provide visual content for any ARAnchor objects you manually add using the session's add(anchor:) method. optional func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) SCNSceneRendererDelegate func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { guard let planeAnchor = anchor as? ARPlaneAnchor else { return } → Planeݕग़࣌Ҏ֎ʹ΋ࣗ෼͕௥Ճͨ͠ͱ͖΋ݺ͹ΕΔ

Slide 25

Slide 25 text

renderer(_:updateAtTime:) SceneKit calls this method exactly once per frame, so long as the SCNView object (or other SCNSceneRenderer object) displaying the scene is not paused. → γʔϯ͕ఀࢭ͠ͳ͍ݶΓɺຖϑϨʔϜݺ͹ΕΔ optional func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) SCNSceneRendererDelegate let planes = currentFrame.anchors.filter { $0 is ARPlaneAnchor }

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

Plane Detection • ஍໘΋ݕग़Ͱ͖Δ • ͔͠͠ɺݕग़ʹ͕͔͔࣌ؒΔ • ෦԰ͷܗঢ়ʹਖ਼͘͠Ԋͬͯݕग़͞Εͳ͍ • ్தͰฏ໘ͷϚʔδ͕ൃੜͨ͠Γ͢Δ • ྑ͍λΠϛϯάͰฏ໘ݕग़ΛࢭΊͨ΄͏͕Αͦ͞͏

Slide 29

Slide 29 text

Hit Test for Plane • σόΠεεΫϦʔϯΛλοϓͨ͠ͱ͖ͳͲʹͲͷฏ໘ʹ৮ Ε͔ͨΛಘΒΕΔ • Χϝϥ͔Β͍ۙ΋ͷॱʹιʔτ͞Εͨ݁ՌΛฦ͢ @IBAction func handleTapGesture(_ tap: UITapGestureRecognizer) { let point = tap.location(in: sceneView) let results = sceneView.hitTest(point, types: .estimatedHorizontalPlane) ɿ }

Slide 30

Slide 30 text

Hit Test • ARSCNView͸3ͭͷhitTestΛ࣋ͭͷͰ஫ҙ • ARSCNViewࣗମ͕࣋ͭhitTest(_:types:) • ARฏ໘ͱͷώοτςετ • SceneKitͷSCNSceneRenderer͕࣋ͭhitTest(_:options:) • scene্ʹ͋ΔSceneKitΦϒδΣΫτͱͷώοτςετ • UIViewͷhitTest(_:with:)

Slide 31

Slide 31 text

ΦϒδΣΫτͷ௥Ճ • SCNNodeΛARSCNViewͷscene΁௥Ճ͢Δ • δΦϝτϦͷ୯Ґ͸ϝʔτϧ • ΦϒδΣΫτͷҐஔ͕ۭؒ಺ʹݻఆ͞ΕΔ let geom = SCNBox(width: 1, height: 1, length: 1, chamferRadius: 0) geom.firstMaterial!.diffuse.contents = UIColor.red geom.firstMaterial!.specular.contents = UIColor.white let node = SCNNode(geometry: geom) sceneView.scene.rootNode.addChildNode(node)

Slide 32

Slide 32 text

ΦϒδΣΫτΞχϝʔγϣϯ node.runAction(SCNAction.moveBy(x: 0.5, y: 0, z: 0, duration: 0.2))

Slide 33

Slide 33 text

AR Minesweeper

Slide 34

Slide 34 text

஌Δඞཁ͕͋ͬͨ͜ͱ • ϢʔβʔͷҐஔΛͲ͏औΔ͔ • ARͰ஍໘ΛऔͬͯɺήʔϜʹ࢖͑Δ͔ • MinesweeperΛͲ͏࡞Δ͔

Slide 35

Slide 35 text

Camera Position • ARSCNView.session.currentFrame.cameraͰ ARCamera͕औΕΔ • ARCamera͔ΒiPhoneͷ (τϥοΩϯά։͔࢝Βͷ૬ରͷ) ࠲ඪ͕औΕΔ

Slide 36

Slide 36 text

ARSCNView ARSession ARCamera ARFrame ARConﬁguration SCNScene run(_:options:) session scene currentFrame camera ARAnchor anchors

Slide 37

Slide 37 text

ARCamera guard let currentFrame = sceneView.session.currentFrame else { return } let camera = currentFrame.camera ɿ

Slide 38

Slide 38 text

Camera Position —ARKitExample func positionFromTransform(_ transform: matrix_float4x4) -> SCNVector3 { return SCNVector3Make(transform.columns.3.x, transform.columns.3.y, transform.columns.3.z) } // Usage let position = positionFromTransform(camera.transform) let angles = camera.eulerAngles

Slide 39

Slide 39 text

Position for Minesweeper • ΧϝϥͷҐஔ (= iPhoneΛ͍࣋ͬͯΔҐஔ) ͔Β஍໘ʹ ਨ௚ʹ߱Ζͨ͠఺ Λ ݱࡏͷҐஔ ͱ͢Δ • ࣮ࡍʹ͸30~40cm΄Ͳલʹདྷͯ͠·͏  (ࠓճ͸ͦͷิਖ਼Λ͠ͳ͍)

Slide 40

Slide 40 text

Ϣʔβͷ͍Δηϧ • ݱࡏͷҐஔ ͔ΒҰ൪͍ۙηϧΛϢʔβͷ͍Δηϧͱ ͢Δ • ࣮ࡍʹ͸30~40cm΄Ͳલʹདྷͯ͠·͏  (ࠓճ͸ͦͷิਖ਼Λ͠ͳ͍) • ୅ΘΓʹɺࠓ͍Δηϧʹ৭Λ෇͚ͨΓɺ ηϧ΁ͷҠಈΛ3ඵ଴ͭΑ͏ʹͨ͠Γͨ͠

Slide 41

Slide 41 text

஌Δඞཁ͕͋ͬͨ͜ͱ • ϢʔβʔͷҐஔΛͲ͏औΔ͔ • ARͰ஍໘ΛऔͬͯɺήʔϜʹ࢖͑Δ͔ • MinesweeperΛͲ͏࡞Δ͔

Slide 42

Slide 42 text

Plane Detection (࠶ܝ) • ஍໘΋ݕग़Ͱ͖Δ • ͔͠͠ɺݕग़ʹ͕͔͔࣌ؒΔ • ෦԰ͷܗঢ়ʹਖ਼͘͠Ԋͬͯݕग़͞Εͳ͍ • ్தͰฏ໘ͷϚʔδ͕ൃੜͨ͠Γ͢Δ • ྑ͍λΠϛϯάͰฏ໘ݕग़ΛࢭΊͨ΄͏͕Αͦ͞͏

Slide 43

Slide 43 text

Game ﬁeld for Minesweeper • Ұ൪େ͖͍ฏ໘Λ࢖͏ • Ұ൪௿͍Ґஔʹ͋Δฏ໘Λ࢖͏ • ࣗಈݕग़ΛఘΊͨ • δΣενϟ (ύϯɺϐϯνɺϩʔςʔτ) Ͱࣗ෼ͰϑΟʔϧυ ͷҐஔɺେ͖͞ɺճసΛௐ੔͢ΔΑ͏ʹͨ͠ • ࢦఆͨ͠αΠζͷηϧͰϑΟʔϧυΛຒΊΔΑ͏ʹͨ͠ ͦ΋ͦ΋্ख͘ݕग़Ͱ ͖͍ͯͳ͍ͷͰ্ख͘ ͍͔ͳ͍

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

஌Δඞཁ͕͋ͬͨ͜ͱ • ϢʔβʔͷҐஔΛͲ͏औΔ͔ • ARͰ஍໘ΛऔͬͯɺήʔϜʹ࢖͑Δ͔ • MinesweeperΛͲ͏࡞Δ͔

Slide 46

Slide 46 text

AR Minesweeper • δΣενϟͰϑΟʔϧυͷେ͖͞ͳͲΛܾΊͯ։࢝ • ݱࡏ͍Δηϧ͕දࣔ͞ΕΔ • ηϧʹ3ඵ͍ͨΒͦͷηϧ͕։͘ • ര஄Ҏ֎ͷηϧΛશ෦։͍ͨΒΫϦΞ • ը໘্ͷηϧͷλοϓͰɺηϧʹʮϚʔΫʯͰ͖Δ

Slide 47

Slide 47 text

AR Minesweeper 1 1

Slide 48

Slide 48 text

Tips

Slide 49

Slide 49 text

Simd Methods • SIMD໋ྩΛར༻ͯ͠ૣ͍ (Α͏ͩ) ← υΩϡϝϯτͳ͍ • Xcode 9͔Βར༻Ͱ͖Δ • simdTransformͳͲ͕͋Δ

Slide 50

Slide 50 text

ײ૝

Slide 51

Slide 51 text

Wireless Development ARKitΛ࢖ͬͨΞϓϦͩͱඞਢ

Slide 52

Slide 52 text

Wireless Development

Slide 53

Slide 53 text

Wireless Development ͜͏ͳΔͱɺXcode࠶ىಈͳͲ͍Ζ͍ΖؤுΒͳ͍ͱ͍͚ͳ͍

Slide 54

Slide 54 text

ײ૝ • ஍໘ͷݕग़ਫ਼౓͕ѱ͍ • ฏ໘͕Ϛʔδ͞ΕͨΓ͢ΔͷͰ్தͰࢭΊͨ΄͏͕Α͍ • ήʔϜϓϨΠʹ޿͍৔ॴ͕ཁΔ • 10×10Ͱ΋5mඞཁ • ͦ΋ͦ΋ήʔϜͱ͓ͯ͠΋͠Ζ͘ͳ͍…

Slide 55

Slide 55 text

·ͱΊ

Slide 56

Slide 56 text

Recap • ARKit + SceneKitͰMinesweeperΛͭͬͨ͘ • ARKit͸σόΠεΧϝϥҐஔͷτϥοΩϯά΍ฏ໘ݕग़ͳ ͲͰ͖Δ • SceneKitͷΦϒδΣΫτΛஔ͍ͯΠϯλϥΫγϣϯͰ͖Δ