SwiftでVRMファイルを表示してみた話

 SwiftでVRMファイルを表示してみた話

https://github.com/tattn/VTuberKit
https://github.com/tattn/VRMKit

UnityなしでVRMを読み込んでVTuber機能をアプリに組み込めるライブラリを作りました
https://qiita.com/tattn/items/c2d4505064f78da93ef3

#potatotips 59
https://potatotips.connpass.com/event/119277/

0620564f0125b8b3b7f4fe40c10b8b4e?s=128

Tatsuya Tanaka

February 28, 2019
Tweet

Transcript

  1. SwiftͰVRMϑΝΠϧΛදࣔͯ͠Έͨ࿩ ాதୡ໵ (@tattn) #potatotips 59

  2. ాத ୡ໵ / ͨͳͨͭ (@tattn) • Yahoo!৐׵Ҋ಺ • iOSΞϓϦΤϯδχΞ •

    Unity΋΍ͬͯ·͢ @tattn @tanakasan2525 @tattn
  3. VRMͱ͸

  4. VRMͱ͸ IUUQTEXBOHPHJUIVCJPWSN VR޲͚ͷਓܕ3DΞόλʔϑΥʔϚοτ
 ϙʔλϏϦςΟ͕ߴͯ͘ѻ͍΍͍͢
 ެࣜͰఏڙ͞Ε͍ͯΔͷ͸Unity࣮૷ͷΈ ɾχίχཱମ
 ɾόʔνϟϧΩϟετ
 ɾcluster
 ɾVDraw
 ɾVRoid

    ͳͲͰར༻͞Ε͍ͯΔ
  5. SwiftͰVRMΛදࣔͰ͖Δ
 ϥΠϒϥϦΛ࡞Γ·ͨ͠

  6. DEMO

  7. PDF / DEMOࣦഊ༻εΫγϣ IUUQTUXJUUFSDPNUBOBLBTBOTUBUVT ಈըˠ © Kizuna AI © SSS

    LLC. © DWANGO Co., Ltd.
  8. VTuberKit / VRMKit / VRMSceneKit VTuberKit IUUQTHJUIVCDPNUBUUO75VCFS,JU IUUQTHJUIVCDPNUBUUO73.,JU VTuberͬΆ͍͜ͱ͕Ͱ͖Δ (؆୯ʹΩϟϥΛදࣔͰ͖Δ)

    VRMKit VRMSceneKit VRMͷಡΈࠐΈ͕Ͱ͖Δ ಡΈࠐΜͩVRMΛSceneKitͰදࣔͰ͖Δ ґଘάϥϑ
  9. ࣮૷ (ϥΠϒϥϦͷ࢖͍ํ͸GitHubΛݟͯͶ)

  10. VRMͷಡΈࠐΈ

  11. VRM (GLTF/GLB) ͷσʔλߏ଄ IUUQTHJUIVCDPN,ISPOPT(SPVQHM5'CMPCNBTUFSTQFDJpDBUJPO3&"%.&NE magic = ϑΝΠϧܗࣜͷνΣοΫ༻
 version = GLBͷόʔδϣϯ

    (VRMͷ৔߹ɺ2)
 length = σʔλͷαΠζ
 Chunk = Ϟσϧσʔλ (0͸JSONɺ1͸όΠφϦ)
  12. SwiftͰόΠφϦσʔλΛಡΈࠐΉ let data = try Data(contentsOf: url) let version: UInt32

    = data.subdata(in: 0..<4) .withUnsafeBytes { $0.pointee } subdataͰόΠφϦσʔλΛ੾Γग़ͯ͠
 withUnsafeBytesͰద੾ͳσʔλܕͱͯ͠औΓग़͢
  13. Chunk 0͸JSON

  14. Chunk 0ͷJSONΛಡΈࠐΉ let endOffset = offset + Int(length) let jsonData

    = data.subdata(in: offset..<endOffset) let decoder = JSONDecoder() self.jsonData = try decoder.decode(GLTF.self, from: jsonData) JSON෦෼ͷσʔλΛ੾Γग़ͯ͠ɺ
 JSONDecoderʹ౤͛Δ
  15. GLTFܕ public struct GLTF: Codable { let extensionsUsed: [String]? let

    extensionsRequired: [String]? let accessors: [Accessor]? let animations: [Animation]? let asset: Asset let buffers: [Buffer]? let bufferViews: [BufferView]? let cameras: [Camera]? let images: [Image]? let materials: [Material]? let meshes: [Mesh]? let nodes: [Node]? let samplers: [Sampler]? let _scene: Int? var scene: Int { return _scene ?? 0 } let scenes: [Scene]? let skins: [Skin]? let textures: [Texture]? let extensions: Extension? let extras: Extras? private enum CodingKeys: String, CodingKey { case extensionsUsed case extensionsRequired case accessors case animations case asset case buffers case bufferViews case cameras case images case materials case meshes case nodes case samplers case _scene = "scene" case scenes case skins case textures case extensions case extras } } εϥΠυͰ͸঺հ͕೉͍͠ͷͰ
 GitHubͰݟͯͶ IUUQTHJUIVCDPNUBUUO73.,JUCMPCNBTUFS4PVSDFT73.,JU73.(-5'TXJGU
  16. ࢓༷ॻΛݟͳ͕ΒCodableͳϞσϧΛ࡞Δ ࢓༷ॻ͕ΊͪΌΊͪΌ
 Θ͔Γ΍͍͢ʂ IUUQTHJUIVCDPN,ISPOPT(SPVQHM5'CMPCNBTUFSTQFDJpDBUJPO 3&"%.&NESFGFSFODFBDDFTTPS

  17. VRMͷදࣔ

  18. ϝογϡͷ࡞੒ (ൈਮ) let meshNode = SCNNode() for primitive in mesh.primitives

    { // primitive = GLTF.Mesh.Primitive let attributes: [SCNGeometrySource] = <௖఺ɾ๏ઢσʔλͷಡΈࠐΈ> let elements: [SCNGeometryElement] = <௖఺ΠϯσοΫεͷಡΈࠐΈ> let geometry = SCNGeometry(sources: attributes, elements: elements) geometry.materials = <ϚςϦΞϧͷಡΈࠐΈ> let primitiveNode = SCNNode() primitiveNode.geometry = geometry meshNode.addChildNode(primitiveNode) }
  19. ௖఺σʔλͷ࡞੒ (ൈਮ) let buffer = vrm.gltf.binaryBuffer let bufferView = buffer.subdata(in:

    byteOffset..<byteOffset + byteLength) let source = SCNGeometrySource(data: bufferView, semantic: .vertex, vectorCount: accessor.count, usesFloatComponents: true, componentsPerVector: 3, // x, y, z bytesPerComponent: 4, // float dataOffset: accessor.byteOffset, dataStride: 3 * 4)
  20. ීஈ͋·Γ࢖Θͳ͍໘ന͍Ϋϥε SCNGeometrySource = ௖఺΍๏ઢɺϘʔϯͳͲͷσʔλ SCNGeometryElement = ௖఺ΠϯσοΫε SCNMaterial / SCNMaterialProperty

    = ςΫενϟ΍ը૾ͳͲͷσʔλ SCNMorpher = ϞʔϑΟϯά (ද৘ͷมߋ) SCNSkinner = εΩχϯά (ϘʔϯΞχϝʔγϣϯ) CAKeyframeAnimation = ΩʔϑϨʔϜΞχϝʔγϣϯ
  21. ͕࣌ؒͳ͍ͷͰ
 ࠷ޙʹ

  22. ࠷ޙʹ QiitaʹϥΠϒϥϦͷ࢖͍ํΛॻ͖·ͨ͠ IUUQTRJJUBDPNUBUUOJUFNTDEGEBFG ͨ͘͞Μͷ͍͍ͶΛ௖͍ͨͷͰ
 ػೳ֦ுΛؤுΖ͏ͱࢥ͍·͢ IUUQTUXJUUFSDPNUBOBLBTBOTUBUVT

  23. ࠓ͸UnityͷAnimationΛimportͰ͖Δ࢓૊ΈΛ࡞ͬͯ·͢ ↑͋ͱগͬ͠Ά͍ ࣦഊͯ͠δϣδϣཱͪʁʹ Mecanimͷ
 σϑΥϧτϙʔζ͸
 Ҡ২੒ޭ ˜6OJUZ5FDIOPMPHJFT+BQBO6$- Unityͷ๛෋ͳΞηοτ͕࢖͑ΔΑ͏ʹͳͬͨΒ࠷ߴͰ͢