ARKit in visionOS

「Mobile勉強会 Wantedly × チームラボ × Sansan #12」での発表資料です。
https://sansan.connpass.com/event/303229/

Transcript

1. ARKit in visionOS ΞϨ͸Vision ProͰ࣮ݱͰ͖Δͷ͔ అ मҰʢ@shu223ʣ
2. ࣗݾ঺հ • అ मҰ • @shu223 (GitHub, Qiita, Zenn, note, , YouTube, Podcast, etc...) • ॻ੶ʢ঎ۀग़൛4࡭ɺݸਓग़൛ଟ਺ @BOOTHʣ:
3. ARKit in visionOSͷαϯϓϧू Appleެࣜͷ ARKit in visionOSνϡʔτϦΞϧ Λ ࣮ࡍʹಈ͘Ξ ϓϦͱ࣮ͯ͠૷ ͨ͠αϯϓϧɿ github.com/shu223/visionOS-Sampler
4. ຊ೔ͷΞδΣϯμ • visionOSʹ͓͚ΔARKit͕iOSͰͲ͏มΘ͔ͬͨ • visionOSͰʮΞϨʯ͸࣮ݱͰ͖Δͷ͔
5. visionOSΞϓϦͷ࡞Γํ • Swift, SwiftUIΛ࢖ͬͯॻ͚Δ • ಉ͡ίʔυϕʔεͰiOS / visionOS྆ϦϦʔε΋Մೳ • iOSͷ΄ͱΜͲͷϑϨʔϜϫʔΫ͸visionOSͰ΋࢖͑Δ • ྫɿ Core ML, Core Bluetooth, Vision, etc...
6. visionOSͰ࢖͑ͳ͍ϑϨʔϜϫʔΫ 1 ActivityKit AdSupport AppClip AutomatedDeviceEnrollment BusinessChat CarKey CarPlay Cinematic ClockKit CoreLocationUI CoreMediaIO CoreNFC CoreTelephony DeviceActivity DockKit ExposureNotification FamilyControls FinanceKit FinanceKitUI ManagedSettings ManagedSettingsUI Messages MLCompute NearbyInteraction OpenAL ProximityReader RoomPlan SafetyKit ScreenTime SensorKit ServiceManagement Social Twitter WidgetKit WorkoutKit 1 Bringing your existing apps to visionOS | Apple Developer Documentation
7. ʢ౰વʣARKit΋࢖͑Δ
8. iOSͰARKit΍ͬͯͨ͠༨༟ʂ
9. ✋
10. iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ੒ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal, .vertical] // ηογϣϯ։࢝ sceneView.session.run(configuration)
11. iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕௥Ճ͞ΕΔͱσϦήʔτϝιου͕ݺ͹ΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... }
12. iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ੒ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal, .vertical] ⚠ ARWorldTrackingConfiguration ɾɾɾvisionOSͰ͸ ࢖͑ͳ͍ʢARConfiguration ࣗମ͕࢖͑ͳ͍ʣ
13. iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯ։࢝ sceneView.session.run(configuration) ⚠ ARSCNView ɾɾɾvisionOSͰ࢖͑ͳ͍ ⚠ ARSession ɾɾɾಉ্
14. iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕௥Ճ͞ΕΔͱσϦήʔτϝιου͕ݺ͹ΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... } ⚠ ARSCNViewDelegate ɾɾɾvisionOSͰ࢖͑ͳ͍ ⚠ ARAnchor, ARPlaneAnchor ɾɾɾಉ্
15. ⚠ ARKit in iOSͷAPI͸΄΅࢖͑ͳ͍
16. visionOSͰͷฏ໘ݕग़ͷίʔυ // ηογϣϯͱDataProviderͷॳظԽ let session = ARKitSession() let provider = PlaneDetectionProvider( alignments: [.horizontal, .vertical] ) // ηογϣϯ։࢝ try await session.run([provider])
17. visionOSͰͷฏ໘ݕग़ͷίʔυ // Data Provider͔ΒΞϯΧʔ͕ಘΒΕΔ for await update in provider.anchorUpdates { let planeAnchor = update.anchor ... }
18. ͢΂ͯvisionOSઐ༻ͷ৽API
19. Ͳ͏มΘͬͨͷ͔ʁ
20. // iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
21. // iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider]) ͲͪΒ΋run΍stopͱ͍ͬͨϝιουΛ࣋ͭ
22. // iOS: ARSession session.run(configuration) // ARConfigurationΛ౉͢ // visionOS: ARKitSession try await session.run([provider]) // DataProviderΛ౉͢
23. • ARKitSession ʢvisionOSʣ΋ ARSession ʢiOSʣ΋Ͳͪ Β΋ηογϣϯΛ؅ཧ • ࢖͍ํ΋ݶΓͳ͘ࣅ͍ͯΔ • Anchor ʢvisionOSʣͱ ARAnchor ʢiOSʣ΋͍͍ͩͨࣅͨ Α͏ͳ΋ͷ 2 2 visionOSʹ͓͚ΔARKit͸iOS൛ͱͲ͏มΘ͔ͬͨ
24. Ͳ͏ͯ͜͠͏ͳͬͨ... • ಉ͡ػೳͳͷʹ... • ࣮૷΋͍͍ͩͨࣅͨΑ͏ͳ΋Μͳͷʹ... ಉ͡APIʹ͠ͱ͍ͨํ͕iOSͱίʔυΛڞ௨ԽͰ͖ͯྑ͔ͬͨͷ Ͱ͸...ʁ
25. ARKit in visionOSͷʮؾ࣋ͪʯΛཧղ͢Δ
26. ϙΠϯτͦͷ1: Vision Proʹ͓͍ͯARKitͷػೳ͸ৗ࣌ON
27. Debug Visualizationػೳ • Xcode 15.1 beta ͷػೳ • visionOSͷॾʑΛՄࢹԽͯ͘͠ΕΔ
28. Debug Visualization Λ༗ޮʹͯ͠৭ʑݟͯΈΔͱ...
29. ARKitͷػೳ͸ৗ࣌ON • ΞϓϦ͕ARKitͷػೳΛ࢖͍ͬͯΑ͏ ͱ͍·͍ͱɺWorld Tracking΍Scene Understanding͕ৗ࣌ߦΘΕ͍ͯΔ • ϋϯυτϥοΩϯά΋ಉ༷
30. ϙΠϯτͦͷ2: ϓϥΠόγʔ
31. ARKitͱϓϥΠόγʔ ARKit͸ɺϢʔβʔڐՄͳ͠ʹΞϓϦʹ σʔλΛఏڙ͠ͳ͍
32. ϓϥΠόγʔͱData Provider Ϣʔβʔ͕ڋ൱͍ͯ͠ΔσʔλΛఏڙ͢ ΔData ProviderͰηογϣϯ࣮ߦ͠Α ͏ͱ͢Δͱɺηογϣϯ͕ࣦഊ͢Δ
33. • Vision Pro͸͍͏ͳΕ͹ARKitσόΠε • ΞϓϦ։ൃऀ͕ARKitػೳΛ੍ޚ͢ΔͷͰ͸ͳ͘ɺৗ࣌ಈ ͍͍ͯΔ • ϓϥΠόγʔ΁ͷ഑ྀ • σʔλΛෆ༻ҙʹ͸։ൃऀʹ౉͞ͳ͍
34. // iOS: // ConfigurationͰʮ࢖͍͍ͨARKitػೳΛߏ੒ʯ session.run(configuration) // visionOS: // Data ProviderΛ௨ͯ͡ʮ࢖͍͍ͨARKitσʔλʹΞΫηεʯ try await session.run([provider])
35. ͜͜·Ͱͷ·ͱΊ • ARKit in visionOS͸iOSͱ͸·ͬͨ͘ҧ͏API܈Λ࢖͏ • ͦͷઃܭࢥ૝͸ɺVision Pro͕ARKitσόΠεͰ͋Δͱߟ͑Δ ͱᡰʹམͪΔ
36. ʙΞϨ͸Vision ProͰ࣮ݱͰ͖Δͷ͔ʙ
37. ʢͨͿΜଟ͘ͷ։ൃऀ͕ʣ΍Γ͍ͨ͜ͱ • ϓϩϑΟʔϧσʔλΛਓͷ಄্ʹද ࣔͨ͠Γ • Ϟϊͷ৘ใ΍ΠϯετϥΫγϣϯΛ දࣔͨ͠Γ
38. ARKit in iOS • People Occulusion • Pose Detection • Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͍͘ΒͰ΋खஈ͕͋Δ
39. ARKit in visionOS • People Occulusion • Pose Detection • Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͲΕ΋ෆՄ 4 4 น΍ఱҪ౳ͷʮ໘ʯΛݕग़͠छผΛಛఆ͢Δ͜ͱ͸Մೳ
40. ۤ೑ͷࡦ • DeviceAnchorΛར༻ • BLE΍UWB౳ͷҐஔਪఆٕज़Λར༻ • ϚΠΫΞϨΠʹΑΔԻݯํ޲ਪఆ
41. DeviceAnchorΛར༻ DeviceAnchor The position and orientation of Apple Vision Pro. • ૬ख΋Vision ProΛண͚͍ͯΕ͹Ґஔɾ޲͖͕Θ͔Δʁ • ͨͿΜࣗ෼ͷ͔͠औΕͳ͍ • Ծʹଞਓͷ΋औΕΔͱͯ͠ɺʮVision ProΛண͚͍ͯΔਓ ͚ͩʯ͸ଟ͘ͷϢʔεέʔεʹ߹Θͳ͍
42. BLE΍UWB౳ͷҐஔਪఆٕज़Λར༻ • Core Bluetooth͸ར༻Մೳ • ΦϒδΣΫτΛݻఆ͢Δ΄Ͳͷਫ਼౓͸ظ଴Ͱ͖ͳ͍ • iBeaconͷΑ͏ʹʮۙ෇͍ͨΒʙ͢Δʯఔ౓ͷϢʔεέʔ ε͕ଥ౰ • Nearby Interaction͸ར༻ෆՄ
43. ϚΠΫΞϨΠʹΑΔԻݯํ޲ਪఆ • Vision ProͷϚΠΫ͸ϞϊϥϧͳͷͰແཧͦ͏
44. ݁࿦ɿ visionOS 2.0 ଴ͪ 5 5 c.f. ࠓͦ͜watchOS
45. ·ͱΊ • ARKit in visionOSͷ৽APIΛཧղ͢ΔϙΠϯτ͸ʮVision Pro ͸ARKitσόΠεʯʮϓϥΠόγʔ΁ͷ഑ྀʯ • visionOS 1.0Ͱ͸ਓ΍෺ମʹରͯ͠ϓϩϑΟʔϧ౳ͷԿΒ͔ ͷΦϒδΣΫτΛઃஔ͢ΔϢʔεέʔε͸ݫͦ͠͏