WebGL / WebVR for FrontEnd Engineer

Transcript

1. WebGL / WebVR

   for FrontEnd Engineer
   Presented by Akihiro Oyamada (@yomotsu)
   Feb 25, 2017
   

   View Slide

2. Frontend Engineer
   at PixelGrid, Inc.
   Akihiro Oyamada
   @yomotsu
   

   View Slide

3. I don’t talk about WebVR much…
   Disclaimer...
   

   View Slide

4. View Slide

5. http://yomotsu.net/blog/assets/2016-12-25-xmas/ http://www.welcometofillory.com/
   https://plus360degrees.com/uxcars/ http://www.playkeepout.com/
   

   View Slide

6. Almost native app looks!

   Cuz it works on top of native graphic APIs
   

   View Slide

7. View Slide

8. Sounds difficult?
   There are libraries!
   

   View Slide

9. View Slide

10. Pros
    • Strong and large community
    • Docs and examples
    • JS friendly easy APIs
    • Low level WebGL access
    Cons
    • Doesn't follow semver,

    Breaking changes sometime
    • Tricky garbage collection
    

    View Slide

11. View Slide

12. View Slide

13. View Slide

14. var scene = new THREE.Scene();
    var camera = new THREE.PerspectiveCamera( 60, width / height, 1, 1000 );
    camera.position.set( 0, 0, 5 );
    var renderer = new THREE.WebGLRenderer();
    renderer.setSize( width, height );
    document.body.appendChild( renderer.domElement );
    var light = new THREE.HemisphereLight( 0x443333, 0x332222, 2 );
    scene.add( light );
    var geometry = new THREE.SphereGeometry( 2, 2, 2 );
    var material = new THREE.MeshPhongMaterial( { color: 0xff0000 } );
    var mesh = new THREE.Mesh( geometry, material );
    scene.add( mesh );
    renderer.render( scene, camera );
    

    View Slide

15. let’s move onto the
    main topics for today
    

    View Slide

16. • Avoid unnecessary rendering
    • Reduce drawcalls
    • Avoid frequent texture uploading
    • Consider to use GPU
    • Compress 3D assets
    

    View Slide

17. Avoid unnecessary rendering
    

    View Slide

18. Continuously re-rendering Better
    

    View Slide

19. demo
    19
    http://localhost:8000/1_re-rendering/bad.html
    http://localhost:8000/1_re-rendering/better.html
    

    View Slide

20. ( function anim () {
    requestAnimationFrame( anim );
    renderer.render( scene, camera );
    } )();
    Re-rendering may unnecessary
    

    View Slide

21. window.addEventListener( 'resize', function () {
    const width = window.innerWidth;
    const height = window.innerHeight;
    camera.aspect = width / height;
    camera.updateProjectionMatrix();
    renderer.setSize( width, height );

    renderer.render( scene, camera ); // <- don't do this!
    } );
    Re-rendering may unnecessary
    

    View Slide

22. ( function anim () {
    requestAnimationFrame( anim );
    if ( !myApp.needsUpdate ) { return; }
    renderer.render( scene, camera );
    } )();
    Pass re-rendering if unnecessary
    

    View Slide

23. var prevPosition = new THREE.Vector3();
    var checkCameraPosition = function () {
    if ( !approximatelyEqual( camera.position, prevPosition ) ) {
    myApp.needsUpdate = true;
    }
    prevPosition.copy( camera.position );
    };
    Checking “needsUpdate”
    

    View Slide

24. window.addEventListener( 'resize', function () {
    const width = window.innerWidth;
    const height = window.innerHeight;
    camera.aspect = width / height;
    camera.updateProjectionMatrix();
    renderer.setSize( width, height );

    myApp.needsUpdate = true; // for next rAF
    } );
    Checking “needsUpdate”
    

    View Slide

25. Reduce drawcalls
    

    View Slide

26. Continuously repainting Better
    8 drawcalls 1 drawcall
    

    View Slide

27. demo
    27
    http://localhost:8000/2_texture-atlas/1_house1.html
    http://localhost:8000/2_texture-atlas/2_house2.html
    

    View Slide

28. .FSHFNBUFSJBMT
    

    View Slide

29. 3 drawcalls for 3 objects 1 drawcall 3 objects
    

    View Slide

30. demo
    30
    http://localhost:8000/2_texture-atlas/3_house-multi1.html
    http://localhost:8000/2_texture-atlas/4_house-multi2.html
    http://localhost:8000/2_texture-atlas/5_house-multi3.html
    

    View Slide

31. var houseGeometry = ...;
    var houseGeometry2 = ...;

    houseGeometry2.applyMatrix(
    new THREE.Matrix4().makeTranslation( -10, 0, 0 )
    );
    houseGeometry.merge( houseGeometry2 );
    Merge geometries
    

    View Slide

32. Avoid frequent

    texture uploading
    

    View Slide

33. View Slide

34. demo
    34
    http://localhost:8000/3_spriteAnim/bad.html
    http://localhost:8000/3_spriteAnim/better.html
    

    View Slide

35. View Slide

36. View Slide

37. $16BOE(16

    BSFQIZTJDBMMZTFQBSBUFE
    

    View Slide

38. ( function anim () {
    var elapsed = clock.getElapsedTime();
    requestAnimationFrame( anim );
    texture.image = images[ frame ];
    texture.needsUpdate = true;
    renderer.render( scene, camera );
    frame = ( elapsed * 20 % images.length ) | 0;
    } )();
    Bad: Replacing image
    

    View Slide

39. ( function anim () {
    var elapsed = clock.getElapsedTime();
    requestAnimationFrame( anim );
    texture.offset.set(
    frame % frameCol / frameCol,
    ( frameRow - 1 ) / frameRow - ( ( frame / frameCol ) | 0 ) / frameRow
    );
    renderer.render( scene, camera );
    frame = ( elapsed * 20 % frameLength ) | 0;
    } )();
    Better: Use UV offset
    

    View Slide

40. View Slide

41. var rtt = new THREE.WebGLRenderTarget(
    512, 512,

    {
    minFilter: THREE.LinearFilter,
    magFilter: THREE.NearestFilter,
    format: THREE.RGBFormat
    }
    );
    renderer.render( sceneRTT, cameraRTT, rtt, true );
    RTT
    

    View Slide

42. GPU may be utilised
    

    View Slide

43. View Slide

44. demo
    44
    http://localhost:8000/4_particle/bad.html
    http://localhost:8000/4_particle/better.html
    

    View Slide

45. requestAnimationFrame( anim );
    for ( var i = 0, l = particleLength; i < l; i ++ ) {
    if ( geometry.attributes.position.array[ i * 3 + 1 ] > - boxHeight * 0.5 ) {
    geometry.attributes.position.array[ i * 3 + 1 ] -= delta;
    } else {
    geometry.attributes.position.array[ i * 3 + 1 ] = boxHeight * 0.5;
    }
    };
    geometry.attributes.position.needsUpdate = true;
    Calc in CPU
    

    View Slide

46. uniform float time;
    uniform float height;
    uniform float scale;
    void main () {
    vec3 pos = vec3( position.x, position.y - time * 1.0, position.z );
    pos.y = mod( pos.y, height ) - ( height * 0.5 );
    vec4 mvPosition = modelViewMatrix * vec4( pos, 1.0 );
    gl_Position = projectionMatrix * mvPosition;
    gl_PointSize = 1.0;
    }
    Calc in GPU (Shader)
    

    View Slide

47. Compress 3D assets
    

    View Slide

48. 3D Model file consists of mostly numbers
    

    View Slide

49. 3D scanned(ish) Model file
    

    View Slide

50. http://localhost:8000/5_compress/bad.html
    

    View Slide

51. View Slide

52. Save 70% of size!
    

    View Slide

53. View Slide

54. zip is an

    archive format
    

    View Slide

55. Multiple files

    in one zip file
    

    View Slide

56. • Contains multi files
    • Compressed One file
    • Arrows you to make

    a progressive bar
    • UnZip JS libs are

    available on GitHub
    

    View Slide

57. View Slide

58. View Slide

59. https://opensource.googleblog.com/2017/01/introducing-draco-compression-for-3d.html
    As of Jan 13, 2017
    

    View Slide

60. • Compressing lib for 3D assets
    • Developed by Google
    • Decompressor is available in JS

    (but 800kb emscripten)
    Might be still early…?
    

    View Slide

61. Conclusion
    

    View Slide

62. WebGL is…
    • Click, then it will open
    • No installing, but loading
    • Devices/Browsers are unspecified
    

    View Slide

63. • Write your code efficiently
    • CPU and GPU are separated
    • Compress 3D assets
    Conclusion
    

    View Slide

64. WebVR?
    

    View Slide

65. https://www.youtube.com/watch?v=cBvCS78ZC1c
    

    View Slide

66. https://w3c.github.io/webvr/
    

    View Slide

67. Navigator.getVRDisplays()
    params and states
    

    View Slide

68. Navigator.getVRDisplays()
    params and states
    •d[ i ].VREyeParameters
    •FOV
    •offset (eye transition)
    •d[ i ].getPose()
    •position
    •orientation (quaternion)
    and etc…
    

    View Slide

69. WebVR is a small API!
    

    View Slide

70. View Slide

71. View Slide

72. One more thing
    

    View Slide

73. View Slide

74. ...
    mounted () {
    const scene = this.getScene();
    scene.add( this.mesh );
    },
    beforeDestroy () {
    const scene = this.getScene();
    // optional
    // this.mesh.material.map.dispose();
    // this.mesh.geometry.dispose();
    // this.mesh.material.dispose();
    scene.remove( this.mesh );
    },
    

    View Slide

75. WebGL can be used

    with WebApp frameworks!
    

    View Slide

76. 

    for FrontEnd Engineer
    Presented by Akihiro Oyamada (@yomotsu)
    Feb 25, 2017
    WebGL / WebVR

    

    View Slide

77. three.js

    for FrontEnd Engineer
    Presented by Akihiro Oyamada (@yomotsu)
    Feb 25, 2017
    

    View Slide

78. gl.finish();
    @yomotsu
    

    View Slide

79. Ξϯέʔτʹ

    ͝ڠྗ͍ͩ͘͞
    http://bit.ly/2meQNnL
    gl.finish();
    @yomotsu
    

    View Slide