Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ARKit in visionOS
Search
shu223
December 19, 2023
Programming
0
3
ARKit in visionOS
「Mobile勉強会 Wantedly × チームラボ × Sansan #12」での発表資料です。
https://sansan.connpass.com/event/303229/
shu223
December 19, 2023
Tweet
Share
More Decks by shu223
See All by shu223
GPT-4o with iOS
shu223
0
18
WhisperKit がだいぶ良いので紹介する
shu223
0
46
Animating Shapes with Simple Equations
shu223
0
8
ローカルLLM on iOS の現状まとめ
shu223
0
24
Core ML版Stable DiffusionをiOSで快適に動かす
shu223
0
28
iOSではじめるフォトグラメトリ #iOSDC
shu223
0
8
MLOps for Core ML #iOSDC 2022
shu223
0
9
slidify-sample
shu223
1
1.1k
機械学習のブルーオーシャン Core ML by 堤 修一 #iOSDC Japan 2020
shu223
12
5.5k
Other Decks in Programming
See All in Programming
Harnessing Large Language Models for Training-free Video Anomaly Detection
tereka114
1
1.3k
はしめてのプログラミングとロボット制御
watawatavoltage
0
290
Mastering Developer Experience: A Roadmap for Success 【開発生産性Conference 2024】
findyinc
1
380
I/O Extended Android in Korea 2024 ~ Whats new in Android development tools
pluu
0
250
Namespace on read
tagomoris
2
370
スクラムマスターって孤独じゃないですか?
yoshitaroyoyo
1
140
CSC307 Lecture 10
javiergs
PRO
0
310
TiDB Serverless ~理想のServerless DBを考える~
soso_15315
1
160
CSC307 Lecture 11
javiergs
PRO
0
240
12年前の『型システム入門』翻訳の思い出話
mame
11
1.2k
Prompt FlowによるLLMアプリケーション開発
yuto2000
1
1k
Advanced App Shrinking Techniques
cbeyls
2
150
Featured
See All Featured
Designing for Performance
lara
604
67k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
26
1.8k
Navigating Team Friction
lara
181
13k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
35
6.3k
Designing for humans not robots
tammielis
247
25k
Mobile First: as difficult as doing things right
swwweet
219
8.8k
The World Runs on Bad Software
bkeepers
PRO
63
11k
Building Effective Engineering Teams - LeadDev
addyosmani
47
2.2k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
224
21k
jQuery: Nuts, Bolts and Bling
dougneiner
61
7.4k
Building a Scalable Design System with Sketch
lauravandoore
458
32k
Build your cross-platform service in a week with App Engine
jlugia
227
17k
Transcript
ARKit in visionOS ΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ అ मҰʢ@shu223ʣ
ࣗݾհ • అ मҰ • @shu223 (GitHub, Qiita, Zenn, note,
, YouTube, Podcast, etc...) • ॻ੶ʢۀग़൛4ɺݸਓग़൛ଟ @BOOTHʣ:
ARKit in visionOSͷαϯϓϧू Appleެࣜͷ ARKit in visionOSνϡʔτϦΞϧ Λ ࣮ࡍʹಈ͘Ξ ϓϦͱ࣮ͯ͠
ͨ͠αϯϓϧɿ github.com/shu223/visionOS-Sampler
ຊͷΞδΣϯμ • visionOSʹ͓͚ΔARKit͕iOSͰͲ͏มΘ͔ͬͨ • visionOSͰʮΞϨʯ࣮ݱͰ͖Δͷ͔
visionOSΞϓϦͷ࡞Γํ • Swift, SwiftUIΛͬͯॻ͚Δ • ಉ͡ίʔυϕʔεͰiOS / visionOS྆ϦϦʔεՄೳ • iOSͷ΄ͱΜͲͷϑϨʔϜϫʔΫvisionOSͰ͑Δ
• ྫɿ Core ML, Core Bluetooth, Vision, etc...
visionOSͰ͑ͳ͍ϑϨʔϜϫʔΫ 1 ActivityKit AdSupport AppClip AutomatedDeviceEnrollment BusinessChat CarKey CarPlay Cinematic
ClockKit CoreLocationUI CoreMediaIO CoreNFC CoreTelephony DeviceActivity DockKit ExposureNotification FamilyControls FinanceKit FinanceKitUI ManagedSettings ManagedSettingsUI Messages MLCompute NearbyInteraction OpenAL ProximityReader RoomPlan SafetyKit ScreenTime SensorKit ServiceManagement Social Twitter WidgetKit WorkoutKit 1 Bringing your existing apps to visionOS | Apple Developer Documentation
ʢવʣARKit͑Δ
iOSͰARKitͬͯͨ͠༨༟ʂ
✋
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] // ηογϣϯ։࢝ sceneView.session.run(configuration)
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... }
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯͷίϯϑΟΪϡϨʔγϣϯΛੜ let configuration = ARWorldTrackingConfiguration() configuration.planeDetection = [.horizontal,
.vertical] ⚠ ARWorldTrackingConfiguration ɾɾɾvisionOSͰ ͑ͳ͍ʢARConfiguration ࣗମ͕͑ͳ͍ʣ
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ηογϣϯ։࢝ sceneView.session.run(configuration) ⚠ ARSCNView ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARSession ɾɾɾಉ্
iOSͰͷARKitͷฏ໘ݕग़ͷίʔυ // ΞϯΧʔ͕Ճ͞ΕΔͱσϦήʔτϝιου͕ݺΕΔ func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode,
for anchor: ARAnchor) { // ίϯϑΟΪϡϨʔγϣϯʹԠͨ͡ΞϯΧʔ͕ಘΒΕΔ guard let planeAnchor = anchor as? ARPlaneAnchor else { ... } ... } ⚠ ARSCNViewDelegate ɾɾɾvisionOSͰ͑ͳ͍ ⚠ ARAnchor, ARPlaneAnchor ɾɾɾಉ্
⚠ ARKit in iOSͷAPI΄΅͑ͳ͍
visionOSͰͷฏ໘ݕग़ͷίʔυ // ηογϣϯͱDataProviderͷॳظԽ let session = ARKitSession() let provider =
PlaneDetectionProvider( alignments: [.horizontal, .vertical] ) // ηογϣϯ։࢝ try await session.run([provider])
visionOSͰͷฏ໘ݕग़ͷίʔυ // Data Provider͔ΒΞϯΧʔ͕ಘΒΕΔ for await update in provider.anchorUpdates {
let planeAnchor = update.anchor ... }
ͯ͢visionOSઐ༻ͷ৽API
Ͳ͏มΘͬͨͷ͔ʁ
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
// iOS: ARSession session.run(configuration) // visionOS: ARKitSession try await session.run([provider])
ͲͪΒrunstopͱ͍ͬͨϝιουΛ࣋ͭ
// iOS: ARSession session.run(configuration) // ARConfigurationΛ͢ // visionOS: ARKitSession try
await session.run([provider]) // DataProviderΛ͢
• ARKitSession ʢvisionOSʣ ARSession ʢiOSʣͲͪ ΒηογϣϯΛཧ • ͍ํݶΓͳ͘ࣅ͍ͯΔ • Anchor
ʢvisionOSʣͱ ARAnchor ʢiOSʣ͍͍ͩͨࣅͨ Α͏ͳͷ 2 2 visionOSʹ͓͚ΔARKitiOS൛ͱͲ͏มΘ͔ͬͨ
Ͳ͏ͯ͜͠͏ͳͬͨ... • ಉ͡ػೳͳͷʹ... • ࣮͍͍ͩͨࣅͨΑ͏ͳΜͳͷʹ... ಉ͡APIʹ͠ͱ͍ͨํ͕iOSͱίʔυΛڞ௨ԽͰ͖ͯྑ͔ͬͨͷ Ͱ...ʁ
ARKit in visionOSͷʮؾ࣋ͪʯΛཧղ͢Δ
ϙΠϯτͦͷ1: Vision Proʹ͓͍ͯARKitͷػೳৗ࣌ON
Debug Visualizationػೳ • Xcode 15.1 beta ͷػೳ • visionOSͷॾʑΛՄࢹԽͯ͘͠ΕΔ
Debug Visualization Λ༗ޮʹͯ͠৭ʑݟͯΈΔͱ...
ARKitͷػೳৗ࣌ON • ΞϓϦ͕ARKitͷػೳΛ͍ͬͯΑ͏ ͱ͍·͍ͱɺWorld TrackingScene Understanding͕ৗ࣌ߦΘΕ͍ͯΔ • ϋϯυτϥοΩϯάಉ༷
ϙΠϯτͦͷ2: ϓϥΠόγʔ
ARKitͱϓϥΠόγʔ ARKitɺϢʔβʔڐՄͳ͠ʹΞϓϦʹ σʔλΛఏڙ͠ͳ͍
ϓϥΠόγʔͱData Provider Ϣʔβʔ͕ڋ൱͍ͯ͠ΔσʔλΛఏڙ͢ ΔData ProviderͰηογϣϯ࣮ߦ͠Α ͏ͱ͢Δͱɺηογϣϯ͕ࣦഊ͢Δ
• Vision Pro͍͏ͳΕARKitσόΠε • ΞϓϦ։ൃऀ͕ARKitػೳΛ੍ޚ͢ΔͷͰͳ͘ɺৗ࣌ಈ ͍͍ͯΔ • ϓϥΠόγʔͷྀ • σʔλΛෆ༻ҙʹ։ൃऀʹ͞ͳ͍
// iOS: // ConfigurationͰʮ͍͍ͨARKitػೳΛߏʯ session.run(configuration) // visionOS: // Data ProviderΛ௨ͯ͡ʮ͍͍ͨARKitσʔλʹΞΫηεʯ
try await session.run([provider])
͜͜·Ͱͷ·ͱΊ • ARKit in visionOSiOSͱ·ͬͨ͘ҧ͏API܈Λ͏ • ͦͷઃܭࢥɺVision Pro͕ARKitσόΠεͰ͋Δͱߟ͑Δ ͱᡰʹམͪΔ
ʙΞϨVision ProͰ࣮ݱͰ͖Δͷ͔ʙ
ʢͨͿΜଟ͘ͷ։ൃऀ͕ʣΓ͍ͨ͜ͱ • ϓϩϑΟʔϧσʔλΛਓͷ಄্ʹද ࣔͨ͠Γ • ϞϊͷใΠϯετϥΫγϣϯΛ දࣔͨ͠Γ
ARKit in iOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͍͘ΒͰखஈ͕͋Δ
ARKit in visionOS • People Occulusion • Pose Detection •
Face Tracking • Object Detection • ϑϨʔϜը૾ʹରͯ͠Core MLར༻ ͲΕෆՄ 4 4 นఱҪͷʮ໘ʯΛݕग़͠छผΛಛఆ͢Δ͜ͱՄೳ
ۤͷࡦ • DeviceAnchorΛར༻ • BLEUWBͷҐஔਪఆٕज़Λར༻ • ϚΠΫΞϨΠʹΑΔԻݯํਪఆ
DeviceAnchorΛར༻ DeviceAnchor The position and orientation of Apple Vision Pro.
• ૬खVision ProΛண͚͍ͯΕҐஔɾ͖͕Θ͔Δʁ • ͨͿΜࣗͷ͔͠औΕͳ͍ • ԾʹଞਓͷऔΕΔͱͯ͠ɺʮVision ProΛண͚͍ͯΔਓ ͚ͩʯଟ͘ͷϢʔεέʔεʹ߹Θͳ͍
BLEUWBͷҐஔਪఆٕज़Λར༻ • Core Bluetoothར༻Մೳ • ΦϒδΣΫτΛݻఆ͢Δ΄Ͳͷਫ਼ظͰ͖ͳ͍ • iBeaconͷΑ͏ʹʮ͍ۙͨΒʙ͢ΔʯఔͷϢʔεέʔ ε͕ଥ •
Nearby Interactionར༻ෆՄ
ϚΠΫΞϨΠʹΑΔԻݯํਪఆ • Vision ProͷϚΠΫϞϊϥϧͳͷͰແཧͦ͏
݁ɿ visionOS 2.0 ͪ 5 5 c.f. ࠓͦ͜watchOS
·ͱΊ • ARKit in visionOSͷ৽APIΛཧղ͢ΔϙΠϯτʮVision Pro ARKitσόΠεʯʮϓϥΠόγʔͷྀʯ • visionOS 1.0Ͱਓମʹରͯ͠ϓϩϑΟʔϧͷԿΒ͔
ͷΦϒδΣΫτΛઃஔ͢ΔϢʔεέʔεݫͦ͠͏