Non-DOM components with WebGL in Vue.js

Both in English and Japanese.
The example code can be seen below:
https://github.com/yomotsu/vue-webgl-non-dom-component-example

Transcript

2. Frontend Engineer at PixelGrid, Inc. Akihiro Oyamada @yomotsu
4. https://ff-wear.com/?template=2_59832150-1541043364
5. https://twitter.com/aniplex_plus/status/779261285783392256 http://yomotsu.net/blog/assets/2016-12-25-xmas/ Made with +
7. Excuse: This topic is considered as
 Experimental
8. ͝஫ҙɿ 
 ࢼͯ͠Έͨྫͷ঺հͰ͢
   ʢϕετϓϥΫςΟεͰ͸͋Γ·ͤΜʣ
9. • High optimized rendering performances • Advanced effects with GLSL Filters beyond CSS • Yet Vue can handle UI, data and APIs Why with WebGL?
10. w ߴ଎ͳϨϯμϦϯάύϑΥʔϚϯε
    w $44Ͱ͸Ͱ͖ͳ͍ɺ
 (-4-ʹΑΔΞυόϯευͳඳըޮՌ
    w ͨͩ͠ɺ7VFͰ6*΍σʔλͷྲྀΕ͸࡞Δ ͳͥ8FC(-Ͱʁ
11. How?
12. Ͳ͏΍ͬͯʁ
13. Failure story…
14. ࣦഊஊʜ
15. • WebGL is async. 
 WebGL Rendering is not synced with DOM Rendering. • Made a pure class,
 independent of Vue. Separate WebGL and Vue…
16. w 8FC(-͸ඇಉظ
 8FC(-ͷϨϯμϦϯά͸
 %0.ϨϯμϦϯάͱ͸ผλΠϛϯά
    w 7VFʹґଘ͠ͳ͍ɺ
 ϐϡΞͳDMBTTΛ࡞ͬͨ 8FC(-ͱ7VFΛ੾Γ཭͢
17. Okay, I will separate them!
18. Α͠ɺ෼͚Α͏ʂ
19. • Same data in 2 places.
 (Vuex and the pure class) • Infinite loop in two-way binding.
 (Due to number precision in JS) • Unreadable code… Separate WebGL and Vue…
20. w ಉ͡σʔλ͕̎Օॴʹ
 ʢ7VFYͱ8FC(-༻ϐϡΞΫϥεʣ
    w ૒ํ޲όΠϯυͰແݶϧʔϓ
 ʢ+4ͷ਺஋ޡࠩͷృΓ߹͍ͳͲʣ
    w ಡΈͮΒ͍ίʔυ΁ʜ ෼͚ͨ݁Ռʜ
21. Solutions and Technics
22. ղܾྫ
23. <template> less Component
24. <template>͕ͳ͍ 
 ίϯϙʔωϯτ
25. Demo WebGL Canvas
 https://github.com/yomotsu/vue-webgl-non-dom-component-example 25
26. <template> <canvas :width="width" :height="height" @click="rayPick" ></canvas> </template>
27. mounted() { this.glRender = this.glRender.bind( this ); this.renderer = new THREE.WebGLRenderer( { canvas: this.$el, stencil: false, alpha: true, } ); ……
28. • <canvas> for <template> • Init the WebGL Renderer at mounted • Destroy the Renderer at destroyed WebGL Canvas component
29. w DBOWBT
    ͷΈͷ
    UFNQMBUF
    w 8FC(-
    3FOEFSFSΛNPVOUFEͰॳظԽ
    w %FTUSPZ࣌ʹ8FC(-
    3FOEFSFSΛഇغ 8FC(-
    $BOWBTίϯϙʔωϯτ
30. Demo DisplayObjects 30
31. export default { name: ‘DisplayObject3D’, …… render() { // template がないので、 // 空の render を明示して // `Failed to mount component: template or render function not defined.` // のエラーを防ぐ } }
32. created() { var loader = new THREE.GLTFLoader(); loader.load( './vue.glb', ( gltf ) => { this.mesh = gltf.scene; this.mesh.userData.vuexId = this.id; this.mesh.traverse( ( obj ) => obj.userData.vuexId = this.id ); this.scene.add( this.mesh ); this.scene.add( this.transformControl ); this.transformControl.attach( this.mesh ); this.$emit( 'changed' ); } ...
33. • WebGL objects are not DOM. • <template> is not needed. • But Vue props. Display Object components
34. w 8FC(-಺ΦϒδΣΫτ͸%0.Ͱ͸ͳ ͍
    w UFNQMBUF͸ෆཁ
    w Ͱ΋1SPQT͸΄͍͠ දࣔΦϒδΣΫτίϯϙʔωϯτ
35. • omit <template> • Void render • Init the component at created. • Dispose at destroyed. Display object components
36. w UFNQMBUF͸φγ
    w SFOEFS͸Կ΋ͤ͞ͳ͍
    w DSFBUFEͰίϯϙʔωϯτΛॳظԽ
    w EFTUSPZFEͰഇغ දࣔΦϒδΣΫτίϯϙʔωϯτ
37. • WebGL root component listen events. • Then, WebGL render. • Doesn't rely on Virtual-DOM. Render them
38. w 8FC(-
    $BOWBT
    DPNQPOFOU͕
 ΠϕϯτΛ؂ࢹ
    w ΠϕϯτʹԠͯ͡8FC(-ͷϨϯμϦϯά
    w όʔνϟϧ%0.ʹ͸པΒͳ͍ Ϩϯμʔ͢Δ
39. <template> <canvas …(snip) > <template v-if="renderReady"> <template v-for=“object in objects"> {{ /* 必ず一意のkeyを入れる */ }} <Object3D @changed="glRender" …(snip) />
40. methods: { glRender() { if ( this.willRender ) return; this.willRender = true; requestAnimationFrame( () => { if ( ! this.renderer ) return; this.renderer.render( this.scene, this.camera ); delete this.willRender; } ); }, …(snip)
41. Other points
42. ͦͷଞ
 ΍ͬͯΈͨ͜ͱ
43. • Avoid multiple update at the sometime. • Combine multiple props into one object. • “Watch” will be also merged into one. Use computed for combined-props
44. w ಉ࣌ෳ਺ճͷߋ৽Λճආ
    w QSPQTͷ૊Έ߹Θͤ͸DPNQVUFEͰҰͭʹ
    w ͜ΕʹΑΓɺ8BUDI΋Ϛʔδ͞ΕΔ ࠞ߹QSPQT͸DPNQVUFEͰ
45. watch: { positionX() { ... }, positionY() { ... },
46. watch: { transform() { ... },
47. computed: { transform() { return { positionX: this.positionX, positionY: this.positionY, positionZ: this.positionZ, } }
48. • Texture upload (from CPU to GPU) • Mesh upload • Rendering WebGL is async
49. • ςΫενϟͷΞοϓϩʔυ(CPU͔ΒGPU) • ϝογϡ৘ใͷΞοϓσʔτ • ϨϯμϦϯά WebGL͸ඇಉظલఏ
50. • DOM is easier to build UI • Since WebGL doesn’t make DOM,
 You cannot use DOM inspector. • (May makes more difficult to inspect.) Consider using DOM
51. w %0.ʹΑΔ6*ߏங͸؆୯
    w 8FC(-͸%0.Λ࡞Βͳ͍ͷͰ
 %0.ΠϯεϖΫλʔ͕࢖͑ͳ͍
    w ։ൃ͕େมʹͳΔ %0.͕࢖͑ͳ͍͔Λ࠶ݕ౼
52. • Mount can be done without <template> • Init and $mount the component manually.
 (But it usually for testing.) Manual Mounting
53. • Mount can be done without <template> • Init and $mount the component manually.
 (But it usually for testing.) खಈϚ΢ϯτ
54. Demo https://github.com/yomotsu/vue-webgl-non-dom-component-example 54
55. // CanvasComponent(親)内で、手動でマウントする const displayObject3D = new DisplayObject3d( { propsData: { id: object.id, positionX: object.positionX, positionY: object.positionY, …(snip) } } ); displayObject3D.$on( 'changed', () => { this.glRender(); } );
56. The new hope:
 Custom render
 comes with Vue 3
58. Conclusion
59. WebGL is just a tool
 not the purpose.
60. • <template> less component
 could be a solution
 for None-DOM instances • Then, your data flow would be clearer
61. • DOMΛ࣋ͨͳ͍Πϯελϯεͷ
 ղܾࡦͱͯ͠ͷ
 <template> Ϩείϯϙʔωϯτ • σʔλͷྲྀΕ͸ΑΓ໌֬ʹ
62. gl.finish(); @yomotsu All demos can be found:
 https://github.com/yomotsu/vue-webgl-non-dom-component-example