WebGL Libs for WebApp Frameworks

all demos can be found here:
https://www.dropbox.com/sh/5l4nt9q6w6f7x8g/AABN93DZm2ryN49m8890mj9ka?dl=0

Transcript

1. Presented by Akihiro Oyamada (@yomotsu)
   Sep 24, 2017
   WebGL Libs for

   WebApp Frameworks
   

   View Slide

2. Frontend Engineer
   at PixelGrid, Inc.
   Akihiro Oyamada
   @yomotsu
   https://github.com/mrdoob/three.js/graphs/contributors
   

   View Slide

3. https://twitter.com/aniplex_plus/status/779261285783392256
   http://yomotsu.net/blog/assets/2016-12-25-xmas/
   Made with +
   

   View Slide

4. webpack three.js
   pixi.js
   etc
   React
   Vue.js
   Angular
   etc
   three.js logo: https://github.com/mrdoob/three.js/issues/2789
   

   View Slide

5. Tips for development with frameworks + WebGL libs:
   1. Import three.js plugins in webpack.
   2. Handle Canvas and Renderer as a WebApp view.
   3. Remove Canvas and GL context appropriately.
   4. Assets compressing.
   Agenda
   

   View Slide

6. Import three.js plugins

   in webpack
   

   View Slide

7. • OrbitControls
   • EffectComposer and Passes
   • CSS3DRenderer
   • Loaders
   and others.
   Plugins for three.js
   

   View Slide

8. Plugins do not support

   es6 module import/export
   Problem
   so far.
   

   View Slide

9. webpack imports-loader
   Solution
   https://github.com/mrdoob/three.js/issues/9562#issuecomment-307092696
   

   View Slide

10. import * as THREE from 'three';
    THREE.OrbitControls = require( 'imports-loader?THREE!
    exports-loader?THREE.OrbitControls!../node_modules/
    three/examples/js/controls/OrbitControls.js' );
    // snip
    const controls = new THREE.OrbitControls( camera );
    

    View Slide

11. import * as THREE from 'three';
    THREE.OrbitControls = require( 'imports-loader?THREE!
    exports-loader?THREE.OrbitControls!../node_modules/
    three/examples/js/controls/OrbitControls.js' );
    // snip
    const controls = new THREE.OrbitControls( camera );
    

    View Slide

12. import * as THREE from 'three';
    THREE.OrbitControls = require( 'imports-loader?THREE!
    exports-loader?THREE.OrbitControls!../node_modules/
    three/examples/js/controls/OrbitControls.js' );
    // snip
    const controls = new THREE.OrbitControls( camera );
    

    View Slide

13. import * as THREE from 'three';
    THREE.OrbitControls = require( 'imports-loader?THREE!
    exports-loader?THREE.OrbitControls!../node_modules/
    three/examples/js/controls/OrbitControls.js' );
    // snip
    const controls = new THREE.OrbitControls( camera );
    

    View Slide

14. import * as THREE from 'three';
    THREE.OrbitControls = require( 'imports-loader?THREE!
    exports-loader?THREE.OrbitControls!../node_modules/
    three/examples/js/controls/OrbitControls.js' );
    // snip
    const controls = new THREE.OrbitControls( camera );
    

    View Slide

15. import * as THREE from 'three';
    THREE.OrbitControls = require( 'imports-loader?THREE!
    exports-loader?THREE.OrbitControls!../node_modules/
    three/examples/js/controls/OrbitControls.js' );
    // snip
    const controls = new THREE.OrbitControls( camera );
    

    View Slide

16. import * as THREE from 'three';
    THREE.OrbitControls = require( 'imports-loader?THREE!
    exports-loader?THREE.OrbitControls!../node_modules/
    three/examples/js/controls/OrbitControls.js' );
    // snip
    const controls = new THREE.OrbitControls( camera );
    

    View Slide

17. demo
    http://localhost:8080/
    

    View Slide

18. Warnings will be ignored in the production build.
    http://localhost:8000/demo1-webpack/
    

    View Slide

19. Handle Canvas

    as a WebApp view
    

    View Slide

20. Canvas element has to be handled

    by the WebApp framework.
    Problem
    

    View Slide

21. Use mounted canvas,

    then bind Renderer in

    mounted or

    componentDidMount hock.
    Solution
    

    View Slide

22. 
    
    
    <br/>import * as THREE from 'three';<br/>export default {<br/>name: 'view-3d',<br/>data () {<br/>return {<br/>width: 200,<br/>height: 200<br/>mount a canvas element<br/>

    View Slide

23. }
    },
    mounted() {
    const width = this.width;
    const height = this.height;
    const loader = new THREE.JSONLoader();
    // (snip)
    this.renderer = new THREE.WebGLRenderer( {
    canvas: this.$el, // <-- bind to mouted canvas
    antialias: true,
    stencil: false
    } );
    

    View Slide

24. 
    
    
    <br/>import * as PIXI from 'pixi.js';<br/>export default {<br/>data () {<br/>return {}<br/>},<br/>mounted() {<br/>

    View Slide

25. return {}
    },
    mounted() {
    this.renderer = new PIXI.WebGLRenderer( {
    width: 300,
    height: 300,
    view: this.$el,
    antialias: true,
    transparent: false,
    backgroundColor: 0xcccccc
    } );
    

    View Slide

26. demo
    http://localhost:8000/demo2-mount-three/
    http://localhost:8000/demo2-mount-pixi/
    http://localhost:8000/demo2-mount-chart/
    

    View Slide

27. Removing

    Canvas and GL context
    

    View Slide

28. If canvas and renderer were removed
    inappropriately.
    It causes memory leaking.
    Problem
    

    View Slide

29. Clear cache and unbind GL context in

    beforeDestroy or

    componentWillUnmount hock.
    Solution
    

    View Slide

30. tick() {
    if ( ! this.running ) return;
    requestAnimationFrame( () => this.tick() );
    this.mesh.rotation.y += 0.03;
    this.renderer.render( this.scene, this.camera );
    }
    Your render loop
    

    View Slide

31. beforeDestroy() {
    this.running = false;
    while ( this.scene.children.length > 0 ) {
    const object = this.scene.children[ this.scene.children.length - 1 ];
    deepDispose( object );
    this.scene.remove( object );
    }
    this.renderer.dispose();
    this.renderer.forceContextLoss();
    this.renderer.context = undefined;
    this.renderer.domElement = undefined;
    // until next garbage collection
    this.$el.width = 1;
    this.$el.height = 1;
    },
    Dispose all objects
    

    View Slide

32. beforeDestroy() {
    this.running = false;
    while ( this.scene.children.length > 0 ) {
    const object = this.scene.children[ this.scene.children.length - 1 ];
    deepDispose( object );
    this.scene.remove( object );
    }
    this.renderer.dispose();
    this.renderer.forceContextLoss();
    this.renderer.context = undefined;
    this.renderer.domElement = undefined;
    // until next garbage collection
    this.$el.width = 1;
    this.$el.height = 1;
    },
    Children of scene consist of
    • Meshes
    • Lights
    • Object3d
    

    View Slide

33. function deepDispose( object3D ) {
    object3D.traverse( object3D => dispose( object3D ) );
    }
    function dispose ( object3D ) {
    if ( !! object3D.geometry ) {
    object3D.geometry.dispose();
    object3D.geometry = undefined;
    }
    if ( !! object3D.material && object3D.material instanceof Array ) {
    object3D.material.forEach( material => disposeMaterial( material ) );
    

    View Slide

34. function deepDispose( object3D ) {
    object3D.traverse( object3D => dispose( object3D ) );
    }
    function dispose ( object3D ) {
    if ( !! object3D.geometry ) {
    object3D.geometry.dispose();
    object3D.geometry = undefined;
    }
    if ( !! object3D.material && object3D.material instanceof Array ) {
    object3D.material.forEach( material => disposeMaterial( material ) );
    

    View Slide

35. function dispose ( object3D ) {
    if ( !! object3D.geometry ) {
    object3D.geometry.dispose();
    object3D.geometry = undefined;
    }
    if ( !! object3D.material && object3D.material instanceof Array ) {
    object3D.material.forEach( material => disposeMaterial( material ) );
    } else if ( !! object3D.material ) {
    disposeMaterial( material );
    }
    

    View Slide

36. function disposeMaterial( material ) {
    if ( !! material.map ) {
    material.map.dispose();
    material.map = undefined;
    }
    // do that for normalMap, specularMap and bumpMap too
    material.dispose();
    material = undefined;
    }
    

    View Slide

37. Before unmount:
    • Stop render loop.
    • Dispose all geoms, materials and
    textures recursively.
    • Release renderer and context.
    • Resize canvas to 1px * 1px.
    

    View Slide

38. demo
    http://localhost:8000/demo3-unmount/
    

    View Slide

39. tick() {
    if ( ! this.running ) return;
    requestAnimationFrame( () => this.tick() );

    this.movieClip.rotation += 0.03;
    this.renderer.render( this.stage );
    }
    

    View Slide

40. beforeDestroy() {
    this.running = false;
    const destroyChildren = true;
    this.stage.destroy( destroyChildren );
    this.renderer.destroy();
    this.renderer.view = undefined;
    // until next garbage collection
    this.$el.width = 1;
    this.$el.height = 1;
    },
    

    View Slide

41. Before unmount:
    • Stop render loop.
    • Destroy the stage with deep option.
    • Release renderer and context.
    • Resize canvas to 1px * 1px.
    

    View Slide

42. FYI: May not necessary to remove the canvas.
    It may cause flickering.
    Consider to just hide it.
    

    View Slide

43. 
    
    toggleShow
    
    
    
    

    View Slide

44. 
    
    toggleShow
    
    
    
    

    View Slide

45. demo
    http://localhost:8000/demo3-show-toggle/
    

    View Slide

46. Assets compressing
    

    View Slide

47. The file size of 3D objects are big.
    Problem
    

    View Slide

48. Compress them.
    Solution
    

    View Slide

49. View Slide

50. original 16,513KB
    zgip 4,585KB
    draco 344KB
    

    View Slide

51. • gZip and Zip reduce 70%.
    • Draco reduces 98%.
    

    View Slide

52. Draco loader for three.js
    

    View Slide

53. Prepare draco libs
    https://github.com/mrdoob/three.js/tree/dev/examples/js/loaders/draco
    

    View Slide

54. Load only DRACOLoader.js into your JS.
    Others will be loaded by Loader.
    

    View Slide

55. const DRACO_PATH = './libs/';
    const DRACO_TYPE = !! window.WebAssembly ? 'wasm' : 'js';
    const dracoLoader = new THREE.DRACOLoader(
    DRACO_PATH,
    { type: DRACO_TYPE }
    );
    dracoLoader.load( './model/rameses.drc', ( geometry ) => {
    const material = new THREE.MeshStandardMaterial( {
    map: texture
    } );
    const mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    } );
    

    View Slide

56. const DRACO_PATH = './libs/';
    const DRACO_TYPE = !! window.WebAssembly ? 'wasm' : 'js';
    const dracoLoader = new THREE.DRACOLoader(
    DRACO_PATH,
    { type: DRACO_TYPE }
    );
    dracoLoader.load( './model/rameses.drc', ( geometry ) => {
    const material = new THREE.MeshStandardMaterial( {
    map: texture
    } );
    const mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    The decoder will be loaded from external file.

    You need set the path to it.
    

    View Slide

57. const DRACO_PATH = './libs/';
    const DRACO_TYPE = !! window.WebAssembly ? 'wasm' : 'js';
    const dracoLoader = new THREE.DRACOLoader(
    DRACO_PATH,
    { type: DRACO_TYPE }
    );
    dracoLoader.load( './model/rameses.drc', ( geometry ) => {
    const material = new THREE.MeshStandardMaterial( {
    map: texture
    } );
    const mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    The recorder is available in both Wasm and JS.
    Use preferred version.
    

    View Slide

58. const DRACO_PATH = './libs/';
    const DRACO_TYPE = !! window.WebAssembly ? 'wasm' : 'js';
    const dracoLoader = new THREE.DRACOLoader(
    DRACO_PATH,
    { type: DRACO_TYPE }
    );
    dracoLoader.load( './model/rameses.drc', ( geometry ) => {
    const material = new THREE.MeshStandardMaterial( {
    map: texture
    } );
    const mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    Make a loader instance for three.js
    

    View Slide

59. const DRACO_PATH = './libs/';
    const DRACO_TYPE = !! window.WebAssembly ? 'wasm' : 'js';
    const dracoLoader = new THREE.DRACOLoader(
    DRACO_PATH,
    { type: DRACO_TYPE }
    );
    dracoLoader.load( './model/rameses.drc', ( geometry ) => {
    const material = new THREE.MeshStandardMaterial( {
    map: texture
    } );
    const mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    } );
    Load a drc file
    

    View Slide

60. const DRACO_PATH = './libs/';
    const DRACO_TYPE = !! window.WebAssembly ? 'wasm' : 'js';
    const dracoLoader = new THREE.DRACOLoader(
    DRACO_PATH,
    { type: DRACO_TYPE }
    );
    dracoLoader.load( './model/rameses.drc', ( geometry ) => {
    const material = new THREE.MeshStandardMaterial( {
    map: texture
    } );
    const mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    } );
    

    View Slide

61. demo
    http://yomotsu.net/blog/assets/2017-09-18-draco/demo/demo1.html
    

    View Slide

62. For more info, read my blog post:

    http://yomotsu.net/blog/2017/09/18/draco.html
    

    View Slide

63. Unzip JS lib is available on NPM as well:

    https://www.npmjs.com/package/zip-loader
    

    View Slide

64. • Pass renderer.render() if not needed.
    • Store as numbers: colour, time etc…
    Other things
    

    View Slide

65. three.js / pixi.js and others
    can be

    a part of WebApp!
    Not only for artistic projects.
    Conclusion
    

    View Slide

66. For Viewer, Simulator, DataVisualization
    and more!
    Conclusion
    

    View Slide

67. gl.finish();
    @yomotsu
    All demos can be found:
    https://www.dropbox.com/sh/hx2zgyh38i1n4z3/AACdSlmGcI3OOZ9Hv_wMU1kua?dl=0
    

    View Slide