SwiftでVRMファイルを表示してみた話

https://github.com/tattn/VTuberKit
https://github.com/tattn/VRMKit

UnityなしでVRMを読み込んでVTuber機能をアプリに組み込めるライブラリを作りました
https://qiita.com/tattn/items/c2d4505064f78da93ef3

#potatotips 59
https://potatotips.connpass.com/event/119277/

Transcript

1. SwiftͰVRMϑΝΠϧΛදࣔͯ͠Έͨ࿩ ాதୡ໵ (@tattn) #potatotips 59
2. ాத ୡ໵ / ͨͳͨͭ (@tattn) • Yahoo!৐׵Ҋ಺ • iOSΞϓϦΤϯδχΞ • Unity΋΍ͬͯ·͢ @tattn @tanakasan2525 @tattn
3. VRMͱ͸
4. VRMͱ͸ IUUQTEXBOHPHJUIVCJPWSN VR޲͚ͷਓܕ3DΞόλʔϑΥʔϚοτ
 ϙʔλϏϦςΟ͕ߴͯ͘ѻ͍΍͍͢
 ެࣜͰఏڙ͞Ε͍ͯΔͷ͸Unity࣮૷ͷΈ ɾχίχཱମ
 ɾόʔνϟϧΩϟετ
 ɾcluster
 ɾVDraw
 ɾVRoid ͳͲͰར༻͞Ε͍ͯΔ
5. SwiftͰVRMΛදࣔͰ͖Δ
 ϥΠϒϥϦΛ࡞Γ·ͨ͠
6. DEMO
7. PDF / DEMOࣦഊ༻εΫγϣ IUUQTUXJUUFSDPNUBOBLBTBOTUBUVT ಈըˠ © Kizuna AI © SSS LLC. © DWANGO Co., Ltd.
8. VTuberKit / VRMKit / VRMSceneKit VTuberKit IUUQTHJUIVCDPNUBUUO75VCFS,JU IUUQTHJUIVCDPNUBUUO73.,JU VTuberͬΆ͍͜ͱ͕Ͱ͖Δ (؆୯ʹΩϟϥΛදࣔͰ͖Δ) VRMKit VRMSceneKit VRMͷಡΈࠐΈ͕Ͱ͖Δ ಡΈࠐΜͩVRMΛSceneKitͰදࣔͰ͖Δ ґଘάϥϑ
9. ࣮૷ (ϥΠϒϥϦͷ࢖͍ํ͸GitHubΛݟͯͶ)
10. VRMͷಡΈࠐΈ
11. VRM (GLTF/GLB) ͷσʔλߏ଄ IUUQTHJUIVCDPN,ISPOPT(SPVQHM5'CMPCNBTUFSTQFDJpDBUJPO3&"%.&NE magic = ϑΝΠϧܗࣜͷνΣοΫ༻
 version = GLBͷόʔδϣϯ (VRMͷ৔߹ɺ2)
 length = σʔλͷαΠζ
 Chunk = Ϟσϧσʔλ (0͸JSONɺ1͸όΠφϦ)
12. SwiftͰόΠφϦσʔλΛಡΈࠐΉ let data = try Data(contentsOf: url) let version: UInt32 = data.subdata(in: 0..<4) .withUnsafeBytes { $0.pointee } subdataͰόΠφϦσʔλΛ੾Γग़ͯ͠
 withUnsafeBytesͰద੾ͳσʔλܕͱͯ͠औΓग़͢
13. Chunk 0͸JSON
14. Chunk 0ͷJSONΛಡΈࠐΉ let endOffset = offset + Int(length) let jsonData = data.subdata(in: offset..<endOffset) let decoder = JSONDecoder() self.jsonData = try decoder.decode(GLTF.self, from: jsonData) JSON෦෼ͷσʔλΛ੾Γग़ͯ͠ɺ
 JSONDecoderʹ౤͛Δ
15. GLTFܕ public struct GLTF: Codable { let extensionsUsed: [String]? let extensionsRequired: [String]? let accessors: [Accessor]? let animations: [Animation]? let asset: Asset let buffers: [Buffer]? let bufferViews: [BufferView]? let cameras: [Camera]? let images: [Image]? let materials: [Material]? let meshes: [Mesh]? let nodes: [Node]? let samplers: [Sampler]? let _scene: Int? var scene: Int { return _scene ?? 0 } let scenes: [Scene]? let skins: [Skin]? let textures: [Texture]? let extensions: Extension? let extras: Extras? private enum CodingKeys: String, CodingKey { case extensionsUsed case extensionsRequired case accessors case animations case asset case buffers case bufferViews case cameras case images case materials case meshes case nodes case samplers case _scene = "scene" case scenes case skins case textures case extensions case extras } } εϥΠυͰ͸঺հ͕೉͍͠ͷͰ
 GitHubͰݟͯͶ IUUQTHJUIVCDPNUBUUO73.,JUCMPCNBTUFS4PVSDFT73.,JU73.(-5'TXJGU
16. ࢓༷ॻΛݟͳ͕ΒCodableͳϞσϧΛ࡞Δ ࢓༷ॻ͕ΊͪΌΊͪΌ
 Θ͔Γ΍͍͢ʂ IUUQTHJUIVCDPN,ISPOPT(SPVQHM5'CMPCNBTUFSTQFDJpDBUJPO 3&"%.&NESFGFSFODFBDDFTTPS
17. VRMͷදࣔ
18. ϝογϡͷ࡞੒ (ൈਮ) let meshNode = SCNNode() for primitive in mesh.primitives { // primitive = GLTF.Mesh.Primitive let attributes: [SCNGeometrySource] = <௖఺ɾ๏ઢσʔλͷಡΈࠐΈ> let elements: [SCNGeometryElement] = <௖఺ΠϯσοΫεͷಡΈࠐΈ> let geometry = SCNGeometry(sources: attributes, elements: elements) geometry.materials = <ϚςϦΞϧͷಡΈࠐΈ> let primitiveNode = SCNNode() primitiveNode.geometry = geometry meshNode.addChildNode(primitiveNode) }
19. ௖఺σʔλͷ࡞੒ (ൈਮ) let buffer = vrm.gltf.binaryBuffer let bufferView = buffer.subdata(in: byteOffset..<byteOffset + byteLength) let source = SCNGeometrySource(data: bufferView, semantic: .vertex, vectorCount: accessor.count, usesFloatComponents: true, componentsPerVector: 3, // x, y, z bytesPerComponent: 4, // float dataOffset: accessor.byteOffset, dataStride: 3 * 4)
20. ීஈ͋·Γ࢖Θͳ͍໘ന͍Ϋϥε SCNGeometrySource = ௖఺΍๏ઢɺϘʔϯͳͲͷσʔλ SCNGeometryElement = ௖఺ΠϯσοΫε SCNMaterial / SCNMaterialProperty = ςΫενϟ΍ը૾ͳͲͷσʔλ SCNMorpher = ϞʔϑΟϯά (ද৘ͷมߋ) SCNSkinner = εΩχϯά (ϘʔϯΞχϝʔγϣϯ) CAKeyframeAnimation = ΩʔϑϨʔϜΞχϝʔγϣϯ
21. ͕࣌ؒͳ͍ͷͰ
 ࠷ޙʹ
22. ࠷ޙʹ QiitaʹϥΠϒϥϦͷ࢖͍ํΛॻ͖·ͨ͠ IUUQTRJJUBDPNUBUUOJUFNTDEGEBFG ͨ͘͞Μͷ͍͍ͶΛ௖͍ͨͷͰ
 ػೳ֦ுΛؤுΖ͏ͱࢥ͍·͢ IUUQTUXJUUFSDPNUBOBLBTBOTUBUVT
23. ࠓ͸UnityͷAnimationΛimportͰ͖Δ࢓૊ΈΛ࡞ͬͯ·͢ ↑͋ͱগͬ͠Ά͍ ࣦഊͯ͠δϣδϣཱͪʁʹ Mecanimͷ
 σϑΥϧτϙʔζ͸
 Ҡ২੒ޭ ˜
    6OJUZ
    5FDIOPMPHJFT
    +BQBO6$- Unityͷ๛෋ͳΞηοτ͕࢖͑ΔΑ͏ʹͳͬͨΒ࠷ߴͰ͢