Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SwiftでVRMファイルを表示してみた話

Tatsuya Tanaka
February 28, 2019

 SwiftでVRMファイルを表示してみた話

https://github.com/tattn/VTuberKit
https://github.com/tattn/VRMKit

UnityなしでVRMを読み込んでVTuber機能をアプリに組み込めるライブラリを作りました
https://qiita.com/tattn/items/c2d4505064f78da93ef3

#potatotips 59
https://potatotips.connpass.com/event/119277/

Tatsuya Tanaka

February 28, 2019
Tweet

More Decks by Tatsuya Tanaka

Other Decks in Technology

Transcript

  1. SwiftͰVRMϑΝΠϧΛදࣔͯ͠Έͨ࿩
    ాதୡ໵ (@tattn)
    #potatotips 59

    View Slide

  2. ాத ୡ໵ / ͨͳͨͭ (@tattn)
    • Yahoo!৐׵Ҋ಺
    • iOSΞϓϦΤϯδχΞ
    • Unity΋΍ͬͯ·͢
    @tattn
    @tanakasan2525
    @tattn

    View Slide

  3. VRMͱ͸

    View Slide

  4. VRMͱ͸
    IUUQTEXBOHPHJUIVCJPWSN
    VR޲͚ͷਓܕ3DΞόλʔϑΥʔϚοτ

    ϙʔλϏϦςΟ͕ߴͯ͘ѻ͍΍͍͢

    ެࣜͰఏڙ͞Ε͍ͯΔͷ͸Unity࣮૷ͷΈ
    ɾχίχཱମ

    ɾόʔνϟϧΩϟετ

    ɾcluster

    ɾVDraw

    ɾVRoid ͳͲͰར༻͞Ε͍ͯΔ

    View Slide

  5. SwiftͰVRMΛදࣔͰ͖Δ

    ϥΠϒϥϦΛ࡞Γ·ͨ͠

    View Slide

  6. DEMO

    View Slide

  7. PDF / DEMOࣦഊ༻εΫγϣ
    IUUQTUXJUUFSDPNUBOBLBTBOTUBUVT
    ಈըˠ
    © Kizuna AI
    © SSS LLC.
    © DWANGO Co., Ltd.

    View Slide

  8. VTuberKit / VRMKit / VRMSceneKit
    VTuberKit
    IUUQTHJUIVCDPNUBUUO75VCFS,JU
    IUUQTHJUIVCDPNUBUUO73.,JU
    VTuberͬΆ͍͜ͱ͕Ͱ͖Δ (؆୯ʹΩϟϥΛදࣔͰ͖Δ)
    VRMKit
    VRMSceneKit
    VRMͷಡΈࠐΈ͕Ͱ͖Δ
    ಡΈࠐΜͩVRMΛSceneKitͰදࣔͰ͖Δ
    ґଘάϥϑ

    View Slide

  9. ࣮૷
    (ϥΠϒϥϦͷ࢖͍ํ͸GitHubΛݟͯͶ)

    View Slide

  10. VRMͷಡΈࠐΈ

    View Slide

  11. VRM (GLTF/GLB) ͷσʔλߏ଄
    IUUQTHJUIVCDPN,ISPOPT(SPVQHM5'CMPCNBTUFSTQFDJpDBUJPO3&"%.&NE
    magic = ϑΝΠϧܗࣜͷνΣοΫ༻

    version = GLBͷόʔδϣϯ (VRMͷ৔߹ɺ2)

    length = σʔλͷαΠζ

    Chunk = Ϟσϧσʔλ (0͸JSONɺ1͸όΠφϦ)

    View Slide

  12. SwiftͰόΠφϦσʔλΛಡΈࠐΉ
    let data = try Data(contentsOf: url)
    let version: UInt32 = data.subdata(in: 0..<4)
    .withUnsafeBytes { $0.pointee }
    subdataͰόΠφϦσʔλΛ੾Γग़ͯ͠

    withUnsafeBytesͰద੾ͳσʔλܕͱͯ͠औΓग़͢

    View Slide

  13. Chunk 0͸JSON

    View Slide

  14. Chunk 0ͷJSONΛಡΈࠐΉ
    let endOffset = offset + Int(length)
    let jsonData = data.subdata(in: offset..let decoder = JSONDecoder()
    self.jsonData = try decoder.decode(GLTF.self, from: jsonData)
    JSON෦෼ͷσʔλΛ੾Γग़ͯ͠ɺ

    JSONDecoderʹ౤͛Δ

    View Slide

  15. GLTFܕ
    public struct GLTF: Codable {
    let extensionsUsed: [String]?
    let extensionsRequired: [String]?
    let accessors: [Accessor]?
    let animations: [Animation]?
    let asset: Asset
    let buffers: [Buffer]?
    let bufferViews: [BufferView]?
    let cameras: [Camera]?
    let images: [Image]?
    let materials: [Material]?
    let meshes: [Mesh]?
    let nodes: [Node]?
    let samplers: [Sampler]?
    let _scene: Int?
    var scene: Int { return _scene ?? 0 }
    let scenes: [Scene]?
    let skins: [Skin]?
    let textures: [Texture]?
    let extensions: Extension?
    let extras: Extras?
    private enum CodingKeys: String, CodingKey {
    case extensionsUsed
    case extensionsRequired
    case accessors
    case animations
    case asset
    case buffers
    case bufferViews
    case cameras
    case images
    case materials
    case meshes
    case nodes
    case samplers
    case _scene = "scene"
    case scenes
    case skins
    case textures
    case extensions
    case extras
    }
    }
    εϥΠυͰ͸঺հ͕೉͍͠ͷͰ

    GitHubͰݟͯͶ
    IUUQTHJUIVCDPNUBUUO73.,JUCMPCNBTUFS4PVSDFT73.,JU73.(-5'TXJGU

    View Slide

  16. ࢓༷ॻΛݟͳ͕ΒCodableͳϞσϧΛ࡞Δ
    ࢓༷ॻ͕ΊͪΌΊͪΌ

    Θ͔Γ΍͍͢ʂ
    IUUQTHJUIVCDPN,ISPOPT(SPVQHM5'CMPCNBTUFSTQFDJpDBUJPO
    3&"%.&NESFGFSFODFBDDFTTPS

    View Slide

  17. VRMͷදࣔ

    View Slide

  18. ϝογϡͷ࡞੒ (ൈਮ)
    let meshNode = SCNNode()
    for primitive in mesh.primitives { // primitive = GLTF.Mesh.Primitive
    let attributes: [SCNGeometrySource] = <௖఺ɾ๏ઢσʔλͷಡΈࠐΈ>
    let elements: [SCNGeometryElement] = <௖఺ΠϯσοΫεͷಡΈࠐΈ>
    let geometry = SCNGeometry(sources: attributes, elements: elements)
    geometry.materials = <ϚςϦΞϧͷಡΈࠐΈ>
    let primitiveNode = SCNNode()
    primitiveNode.geometry = geometry
    meshNode.addChildNode(primitiveNode)
    }

    View Slide

  19. ௖఺σʔλͷ࡞੒ (ൈਮ)
    let buffer = vrm.gltf.binaryBuffer
    let bufferView = buffer.subdata(in: byteOffset..let source = SCNGeometrySource(data: bufferView,
    semantic: .vertex,
    vectorCount: accessor.count,
    usesFloatComponents: true,
    componentsPerVector: 3, // x, y, z
    bytesPerComponent: 4, // float
    dataOffset: accessor.byteOffset,
    dataStride: 3 * 4)

    View Slide

  20. ීஈ͋·Γ࢖Θͳ͍໘ന͍Ϋϥε
    SCNGeometrySource = ௖఺΍๏ઢɺϘʔϯͳͲͷσʔλ
    SCNGeometryElement = ௖఺ΠϯσοΫε
    SCNMaterial / SCNMaterialProperty = ςΫενϟ΍ը૾ͳͲͷσʔλ
    SCNMorpher = ϞʔϑΟϯά (ද৘ͷมߋ)
    SCNSkinner = εΩχϯά (ϘʔϯΞχϝʔγϣϯ)
    CAKeyframeAnimation = ΩʔϑϨʔϜΞχϝʔγϣϯ

    View Slide

  21. ͕࣌ؒͳ͍ͷͰ

    ࠷ޙʹ

    View Slide

  22. ࠷ޙʹ
    QiitaʹϥΠϒϥϦͷ࢖͍ํΛॻ͖·ͨ͠
    IUUQTRJJUBDPNUBUUOJUFNTDEGEBFG
    ͨ͘͞Μͷ͍͍ͶΛ௖͍ͨͷͰ

    ػೳ֦ுΛؤுΖ͏ͱࢥ͍·͢
    IUUQTUXJUUFSDPNUBOBLBTBOTUBUVT

    View Slide

  23. ࠓ͸UnityͷAnimationΛimportͰ͖Δ࢓૊ΈΛ࡞ͬͯ·͢
    ↑͋ͱগͬ͠Ά͍
    ࣦഊͯ͠δϣδϣཱͪʁʹ
    Mecanimͷ

    σϑΥϧτϙʔζ͸

    Ҡ২੒ޭ
    ˜6OJUZ5FDIOPMPHJFT+BQBO6$-
    Unityͷ๛෋ͳΞηοτ͕࢖͑ΔΑ͏ʹͳͬͨΒ࠷ߴͰ͢

    View Slide