Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
iPhoneでコスプレをする技術
Search
Tatsuya Tanaka
September 13, 2018
Programming
3
1.1k
iPhoneでコスプレをする技術
iPhoneでコスプレをする技術
俺コン 2018 Summer
#orecon_ios
Tatsuya Tanaka
September 13, 2018
Tweet
Share
More Decks by Tatsuya Tanaka
See All by Tatsuya Tanaka
iPhoneのセンサー情報をmacOSアプリでリアルタイム活用するための技術
tattn
1
650
Better use of SwiftUI
tattn
0
450
Swift Concurrencyによる安全で快適な非同期処理
tattn
2
1.3k
iOSアプリの技術選択2022
tattn
7
3.9k
Widget Suggestions 対応と ヤフーの新OS対応
tattn
1
1.3k
WidgetKitで良い体験を作るには / Good experience with WidgetKit
tattn
2
1.7k
既存アプリにSwiftUIをどう組み込んでいくか
tattn
8
2.4k
iOS 14からのアプリ内課金
tattn
5
2.9k
iOS 14の位置情報系アップデート
tattn
0
22k
Other Decks in Programming
See All in Programming
Android 16 × Jetpack Composeで縦書きテキストエディタを作ろう / Vertical Text Editor with Compose on Android 16
cc4966
2
250
Navigating Dependency Injection with Metro
zacsweers
3
1k
Putting The Genie in the Bottle - A Crash Course on running LLMs on Android
iurysza
0
140
基礎から学ぶ大画面対応(Learning Large-Screen Support from the Ground Up)
tomoya0x00
0
1.6k
意外と簡単!?フロントエンドでパスキー認証を実現する WebAuthn
teamlab
PRO
2
770
AIでLINEスタンプを作ってみた
eycjur
1
230
Laravel Boost 超入門
fire_arlo
3
220
もうちょっといいRubyプロファイラを作りたい (2025)
osyoyu
1
450
モバイルアプリからWebへの横展開を加速した話_Claude_Code_実践術.pdf
kazuyasakamoto
0
330
概念モデル→論理モデルで気をつけていること
sunnyone
2
270
Ruby Parser progress report 2025
yui_knk
1
450
Navigation 2 を 3 に移行する(予定)ためにやったこと
yokomii
0
290
Featured
See All Featured
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
36
2.5k
How to Ace a Technical Interview
jacobian
279
23k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.7k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
How to train your dragon (web standard)
notwaldorf
96
6.2k
Code Review Best Practice
trishagee
70
19k
Transcript
iPhoneͰίεϓϨΛ͢Δٕज़ ాதୡ (@tattn) Զίϯ 2018 Summer #orecon_ios
ాத ୡ (@tattn) • Yahoo!Ҋ • iOSΞϓϦΤϯδχΞ @tattn @tanakasan2525 @tattn
ࡢͷApple Event؍·͔ͨ͠ʁ
iOS12ͷհϖʔδ IUUQTXXXBQQMFDPNKQJPTJPT
Ҋ͕ϖʔδʹࡌΓ·ͨ͠ IUUQTXXXBQQMFDPNKQJPTJPT
iPhoneͰίεϓϨΛ͢Δٕज़ IUUQTGPSUFFKQJPTEDKBQBOQSPQPTBMECGCCGCBBGCDGF
όʔνϟϧYouTuber ͍ͬͯ·͔͢ʁ
όʔνϟϧYouTuber (VTuber) Βͳ͍ਓϠϑʔͰݕࡧʂʮVTuberʯ YouTubeͳͲͷಈը৴αʔϏε্Ͱ 2D3DͷΞόλʔͰ৴͍ͯ͠Δਓͷ͜ͱɻ ϦΞϧͱόʔνϟϧͷମΛಉظ͢Δٕज़Λ͍ͬͯΔ͜ͱ͕ଟ͍ɻ
VTuberΛࢧ͑Δٕज़ • ମΛಈ͔͢ • खಈɺKinectɺVRػثɺϞʔγϣϯΩϟϓνϟʔػث • දΛಈ͔͢ • खಈɺFaceRigɺDlibɺVisionɺARKit •
(Λม͢Δ) • ιϑτΣΞ (࿀ͳͲ)ɺϋʔυΣΞ (VT-3ͳͲ) • දࣔ͢Δ • UnityɺFaceRigɺSceneKit (ҰྫͰ͢)
ٕज़తʹ໘ന͘ɺ৭ʑ࡞ͬͯΈ·ͨ͠ ※͜ΕOculus RiftͱUnityΛ͍ͬͯ·͢ VRδΣϯΨ VRΩϟονϘʔϧ VTuberͬΆ͍ࢹ ©Kizuna AI IUUQTUXJUUFSDPNUBOBLBTBOTUBUVT
εϚϗͰVTuberʹͳΕΔ
ͦ͏ɺiPhoneͳΒͶɻ
͜ΜͳΞϓϦΛ࡞Γ·ͨ͠ Α͔ͬͨΒ༡ΜͰΈͯͶˠ iPhone X × ARKit × SceneKit CoreAnimation ×
Accelerate ʮVTuberʯͰݕࡧ
༻ٕज़ Face Tracking ϨϯμϦϯάɾϞʔγϣϯ TrueDepthΧϝϥ ܭࢉ iPhone X × ARKit
× SceneKit CoreAnimation × Accelerate
ARKit ARFaceAnchorͰإͷ࢟ΛऔಘΛऔಘ BlendShapeLocationͰإͷύʔπͷঢ়ଶΛऔಘ IUUQTEFWFMPQFSBQQMFDPNEPDVNFOUBUJPOBSLJUBSGBDFBODIPS
Face Trackingͷ࣮ߦ guard ARFaceTrackingConfiguration.isSupported else { return } let configuration
= ARFaceTrackingConfiguration() configuration.isLightEstimationEnabled = true let arSession = ARSession() arSession.delegate = self arSession.run(configuration, options: [.resetTracking, .removeExistingAnchors])
إͷύʔπͷঢ়ଶऔಘ extension ViewController: ARSessionDelegate { func session(_ session: ARSession, didUpdate
anchors: [ARAnchor]) { guard let faceAnchor = anchors .lazy .compactMap({ $0 as? ARFaceAnchor }) .first else { return } if let mouthOpenness = faceAnchor.blendShapes[.jawOpen]?.doubleValue { let e = 0.05 // ޱΛด͘͢͢͡Δ let mouthOpennessX8 = mouthOpenness * 8 // ޱΛ։͚͘͢͢Δ let value = mouthOpennessX8 < e ? 0 : mouthOpennessX8 > 1 - e ? 1 : mouthOpennessX8 // Ϟσϧͷߋ৽ }
SceneKit SCNSceneʹΩϟϥΫλʔΛஔ͠ SCNViewͰSceneΛදࣔ ©Kizuna AI
ϞσϧͷಡΈࠐΈ ϑϦʔͷϞσϧ͕ଟ͘ɺ දͷϞʔϑΟϯάϞʔγϣϯͳͲͷ ޓੑ͕ߴ͍MMDϞσϧΛࠓճ༻ ※MMD=MikuMikuDance IUUQEJDOJDPWJEFPKQBNJLVNJLVEBODF
MMDͷಡΈࠐΈ ϑΥʔϚοτͷ༷ࣗମެ։͞Ε͍ͯΔ IUUQLLILTFFTBBOFUDBUFHPSZIUNM ↑zipͷதʹ༷ॻ͕͋Δ IUUQTHJUIVCDPNNBHJDJFO..%4DFOF,JU ࠓճMMDSceneKitΛ༻
Ωϟϥͷύʔπͷऔಘ let model: SCNNode = try loadMMDModel(fromName: "kizunaai/kizunaai.pmx") let neck
= model.childNode(withName: "ट", recursively: true) let leftEye = model.childNode(withName: "ࠨ", recursively: true) let rightEye = model.childNode(withName: "ӈ", recursively: true) MMDͷNode (Ϙʔϯ) ໊ʹ҉తͳϓϩτίϧ͕͋ΔͷͰ ผͷϞσϧͦͷ··Ͱେମಈ͘ (ͨ·ʹશ֯ΧλΧφͱ֯ΧλΧφͷදهΏΕ͕...ţŜŖŪͳͲ)
CoreAnimation / SceneKit let keyPath = "morpher.weights.eye" var animation =
CAKeyframeAnimation(keyPath: keyPath) animation.values = [AnyObject]() animation.keyTimes = [NSNumber]() animation.timingFunctions = [CAMediaTimingFunction]() ... let animationGroup = CAAnimationGroup() animationGroup.animations = [CAAnimation]() ... animationGroup.animations?.append(animation) ... let animation = SCNAnimation(caAnimation: animationGroup) let player = SCNAnimationPlayer(animation: animation) model.addAnimationPlayer(player, forKey: nil) player.play()
Accelerate.framework ύϑΥʔϚϯεʹಛԽͨ͠ࢉज़ԋࢉϑϨʔϜϫʔΫ ࠓճ simd ͱ͍͏ খ͞ͳϕΫτϧͱߦྻΛѻ͏ϞδϡʔϧΛར༻ ※ simd = Single
Instruction Multiple Data
औಘͨ͠σʔλΛମͱटʹө͢Δ func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) { guard
let faceAnchor = anchors .lazy.compactMap({ $0 as? ARFaceAnchor }).first else { return } let transform: simd_float4x4 = faceAnchor.transform let quaternion = simd_quaternion(transform) neck.simdWorldOrientation = quaternion let translate4: simd_float4 = transform.columns.3 let translate = simd_float3(translate4.x * 5, translate4.y * 5, translate4.z * 50 + 20) // w = 1 model.simdPosition = translate ࠨӈ͋·Γಈ͔ͣɺલޙಈ͖͘͢͢Δ ARKit/SceneKitӈखܥ
͜Ε·Ͱઆ໌ٕͨ͠ज़ΛΈ߹ΘͤΔͱ όʔνϟϧͳମΛಘΔ͜ͱ͕Ͱ͖·͢ʂ
VRM
VRMͱ IUUQTEXBOHPHJUIVCJPWSN υϫϯΰ͕࡞ͬͨVR༻ͷ3DϞσϧϑΥʔϚοτ 4݄ʹެ։͞ΕɺVRք۾Ͱʹͳ͍ͬͯΔ ɾχίχཱମ ɾόʔνϟϧΩϟετ ɾcluster ɾVDraw ɾVΧπ ͳͲͰར༻͞Ε͍ͯΔ
VRMΛiOSͰಡΈࠐΊΔϥΠϒϥϦ VRMΛiOSͰಡΈࠐΊΔϥΠϒϥϦΛ࡞Γ·ͨ͠ IUUQTHJUIVCDPNUBUUO73.,JU VRMKit
VRMΛiOSͰಡΈࠐΊΔϥΠϒϥϦ IUUQTHJUIVCDPNUBUUO73.,JU VRMKit ϝλσʔλͷऔಘͱ Ϟσϧͷද͕ࣔՄೳ
͊͞ɺΈͳ͞Μόʔνϟϧͳੈք
͋Γ͕ͱ͏͍͟͝·ͨ͠