Non-DOM components with WebGL in Vue.js

Both in English and Japanese.
The example code can be seen below:
https://github.com/yomotsu/vue-webgl-non-dom-component-example

Transcript

1. View Slide

2. Frontend Engineer
   at PixelGrid, Inc.
   Akihiro Oyamada
   @yomotsu
   

   View Slide

3. View Slide

4. https://ff-wear.com/?template=2_59832150-1541043364
   

   View Slide

5. https://twitter.com/aniplex_plus/status/779261285783392256
   http://yomotsu.net/blog/assets/2016-12-25-xmas/
   Made with +
   

   View Slide

6. View Slide

7. Excuse: This topic is considered as

   Experimental
   

   View Slide

8. ͝஫ҙɿ
   

   ࢼͯ͠Έͨྫͷ঺հͰ͢
   ʢϕετϓϥΫςΟεͰ͸͋Γ·ͤΜʣ
   

   View Slide

9. • High optimized rendering performances
   • Advanced effects with GLSL Filters
   beyond CSS
   • Yet Vue can handle UI, data and APIs
   Why with WebGL?
   

   View Slide

10. w ߴ଎ͳϨϯμϦϯάύϑΥʔϚϯε
    w $44Ͱ͸Ͱ͖ͳ͍ɺ

    (-4-ʹΑΔΞυόϯευͳඳըޮՌ
    w ͨͩ͠ɺ7VFͰ6*΍σʔλͷྲྀΕ͸࡞Δ
    ͳͥ8FC(-Ͱʁ
    

    View Slide

11. How?
    

    View Slide

12. Ͳ͏΍ͬͯʁ
    

    View Slide

13. Failure story…
    

    View Slide

14. ࣦഊஊʜ
    

    View Slide

15. • WebGL is async. 

    WebGL Rendering is not synced with DOM Rendering.
    • Made a pure class,

    independent of Vue.
    Separate WebGL and Vue…
    

    View Slide

16. w 8FC(-͸ඇಉظ

    8FC(-ͷϨϯμϦϯά͸

    %0.ϨϯμϦϯάͱ͸ผλΠϛϯά
    w 7VFʹґଘ͠ͳ͍ɺ

    ϐϡΞͳDMBTTΛ࡞ͬͨ
    8FC(-ͱ7VFΛ੾Γ཭͢
    

    View Slide

17. Okay, I will separate them!
    

    View Slide

18. Α͠ɺ෼͚Α͏ʂ
    

    View Slide

19. • Same data in 2 places.

    (Vuex and the pure class)
    • Infinite loop in two-way binding.

    (Due to number precision in JS)
    • Unreadable code…
    Separate WebGL and Vue…
    

    View Slide

20. w ಉ͡σʔλ͕̎Օॴʹ

    ʢ7VFYͱ8FC(-༻ϐϡΞΫϥεʣ
    w ૒ํ޲όΠϯυͰແݶϧʔϓ

    ʢ+4ͷ਺஋ޡࠩͷృΓ߹͍ͳͲʣ
    w ಡΈͮΒ͍ίʔυ΁ʜ
    ෼͚ͨ݁Ռʜ
    

    View Slide

21. Solutions and Technics
    

    View Slide

22. ղܾྫ
    

    View Slide

23. less Component
    

    View Slide

24. ͕ͳ͍
    

    ίϯϙʔωϯτ
    

    View Slide

25. Demo
    WebGL Canvas

    https://github.com/yomotsu/vue-webgl-non-dom-component-example
    25
    

    View Slide

26. 
    :width="width"
    :height="height"
    @click="rayPick"
    >
    
    

    View Slide

27. mounted() {
    this.glRender = this.glRender.bind( this );
    this.renderer = new THREE.WebGLRenderer( {
    canvas: this.$el,
    stencil: false,
    alpha: true,
    } );
    ……
    

    View Slide

28. • for
    • Init the WebGL Renderer at mounted
    • Destroy the Renderer at destroyed
    WebGL Canvas component
    

    View Slide

29. w DBOWBTͷΈͷUFNQMBUF
    w 8FC(-3FOEFSFSΛNPVOUFEͰॳظԽ
    w %FTUSPZ࣌ʹ8FC(-3FOEFSFSΛഇغ
    8FC(-$BOWBTίϯϙʔωϯτ
    

    View Slide

30. Demo
    DisplayObjects
    30
    

    View Slide

31. export default {
    name: ‘DisplayObject3D’,
    ……
    render() {
    // template がないので、
    // 空の render を明示して
    // `Failed to mount component: template
    or render function not defined.`
    // のエラーを防ぐ
    }
    }
    

    View Slide

32. created() {
    var loader = new THREE.GLTFLoader();
    loader.load(
    './vue.glb',
    ( gltf ) => {
    this.mesh = gltf.scene;
    this.mesh.userData.vuexId = this.id;
    this.mesh.traverse( ( obj ) => obj.userData.vuexId = this.id );
    this.scene.add( this.mesh );
    this.scene.add( this.transformControl );
    this.transformControl.attach( this.mesh );
    this.$emit( 'changed' );
    }
    ...
    

    View Slide

33. • WebGL objects are not DOM.
    • is not needed.
    • But Vue props.
    Display Object components
    

    View Slide

34. w 8FC(-಺ΦϒδΣΫτ͸%0.Ͱ͸ͳ
    ͍
    w UFNQMBUF͸ෆཁ
    w Ͱ΋1SPQT͸΄͍͠
    දࣔΦϒδΣΫτίϯϙʔωϯτ
    

    View Slide

35. • omit
    • Void render
    • Init the component at created.
    • Dispose at destroyed.
    Display object components
    

    View Slide

36. w UFNQMBUF͸φγ
    w SFOEFS͸Կ΋ͤ͞ͳ͍
    w DSFBUFEͰίϯϙʔωϯτΛॳظԽ
    w EFTUSPZFEͰഇغ
    දࣔΦϒδΣΫτίϯϙʔωϯτ
    

    View Slide

37. • WebGL root component listen events.
    • Then, WebGL render.
    • Doesn't rely on Virtual-DOM.
    Render them
    

    View Slide

38. w 8FC(-$BOWBTDPNQPOFOU͕

    ΠϕϯτΛ؂ࢹ
    w ΠϕϯτʹԠͯ͡8FC(-ͷϨϯμϦϯά
    w όʔνϟϧ%0.ʹ͸པΒͳ͍
    Ϩϯμʔ͢Δ
    

    View Slide

39. 
    …(snip)
    >
    
    
    {{ /* 必ず一意のkeyを入れる */ }}
    @changed="glRender"
    …(snip)
    />
    

    View Slide

40. methods: {
    glRender() {
    if ( this.willRender ) return;
    this.willRender = true;
    requestAnimationFrame( () => {
    if ( ! this.renderer ) return;
    this.renderer.render( this.scene, this.camera );
    delete this.willRender;
    } );
    },
    …(snip)
    

    View Slide

41. Other points
    

    View Slide

42. ͦͷଞ

    ΍ͬͯΈͨ͜ͱ
    

    View Slide

43. • Avoid multiple update at the sometime.
    • Combine multiple props into one object.
    • “Watch” will be also merged into one.
    Use computed for combined-props
    

    View Slide

44. w ಉ࣌ෳ਺ճͷߋ৽Λճආ
    w QSPQTͷ૊Έ߹Θͤ͸DPNQVUFEͰҰͭʹ
    w ͜ΕʹΑΓɺ8BUDI΋Ϛʔδ͞ΕΔ
    ࠞ߹QSPQT͸DPNQVUFEͰ
    

    View Slide

45. watch: {
    positionX() {
    ...
    },
    positionY() {
    ...
    },
    

    View Slide

46. watch: {
    transform() {
    ...
    },
    

    View Slide

47. computed: {
    transform() {
    return {
    positionX: this.positionX,
    positionY: this.positionY,
    positionZ: this.positionZ,
    }
    }
    

    View Slide

48. • Texture upload (from CPU to GPU)
    • Mesh upload
    • Rendering
    WebGL is async
    

    View Slide

49. • ςΫενϟͷΞοϓϩʔυ(CPU͔ΒGPU)
    • ϝογϡ৘ใͷΞοϓσʔτ
    • ϨϯμϦϯά
    WebGL͸ඇಉظલఏ
    

    View Slide

50. • DOM is easier to build UI
    • Since WebGL doesn’t make DOM,

    You cannot use DOM inspector.
    • (May makes more difficult to inspect.)
    Consider using DOM
    

    View Slide

51. w %0.ʹΑΔ6*ߏங͸؆୯
    w 8FC(-͸%0.Λ࡞Βͳ͍ͷͰ

    %0.ΠϯεϖΫλʔ͕࢖͑ͳ͍
    w ։ൃ͕େมʹͳΔ
    %0.͕࢖͑ͳ͍͔Λ࠶ݕ౼
    

    View Slide

52. • Mount can be done without
    • Init and $mount the component
    manually.

    (But it usually for testing.)
    Manual Mounting
    

    View Slide

53. • Mount can be done without
    • Init and $mount the component
    manually.

    (But it usually for testing.)
    खಈϚ΢ϯτ
    

    View Slide

54. Demo
    https://github.com/yomotsu/vue-webgl-non-dom-component-example
    54
    

    View Slide

55. // CanvasComponent(親)内で、手動でマウントする
    const displayObject3D = new DisplayObject3d( {
    propsData: {
    id: object.id,
    positionX: object.positionX,
    positionY: object.positionY,
    …(snip)
    }
    } );
    displayObject3D.$on( 'changed', () => {
    this.glRender();
    } );
    

    View Slide

56. The new hope:

    Custom render

    comes with Vue 3
    

    View Slide

57. View Slide

58. Conclusion
    

    View Slide

59. WebGL is just a tool

    not the purpose.
    

    View Slide

60. • less component

    could be a solution

    for None-DOM instances
    • Then, your data flow would be
    clearer
    

    View Slide

61. • DOMΛ࣋ͨͳ͍Πϯελϯεͷ

    ղܾࡦͱͯ͠ͷ

    Ϩείϯϙʔωϯτ
    • σʔλͷྲྀΕ͸ΑΓ໌֬ʹ
    

    View Slide

62. gl.finish();
    @yomotsu
    All demos can be found:

    https://github.com/yomotsu/vue-webgl-non-dom-component-example
    

    View Slide