WebGL: The graphics API for web developers

my slide at GREE Tech Talk #08 : WebGL http://techtalk.labs.gree.jp/08/

Transcript

1. WebGL:

   The graphics API

   for web developers
   Presented by Akihiro Oyamada (@yomotsu)
   Jun 18, 2015
   

   View Slide

2. Frontend Engineer
   at PixelGrid, Inc.
   Akihiro Oyamada
   @yomotsu
   

   View Slide

3. Limitations of the Web 3D
   

   View Slide

4. 4
   www.nike.com/gb/en_gb/c/nikeid
   

   View Slide

5. 5
   http://www2.asics.co.jp/baseball/3dorder
   

   View Slide

6. 6
   http://www2.asics.co.jp/io/help/help.html
   They also provide

   an installation guide…
   

   View Slide

7. There were

   no graphics APIs

   for the Web
   

   View Slide

8. The history

   of the web 3d
   

   View Slide

9. View Slide

10. • A Web Standerd
    • A JavaScript feature
    • Great performance
    • Programable Shaders in GLSL
    

    View Slide

11. Click a link,

    Just it will open!
    

    View Slide

12. Maintained by Khronos Group,
    not W3C
    

    View Slide

13. 13
    http://caniuse.com/#feat=webgl
    

    View Slide

14. Most of browsers support WebGL
    

    View Slide

15. Hardware-

    accelerated
    

    View Slide

16. View Slide

17. View Slide

18. vec3(-1, -1, 0)
    vec3( 1, -1, 0)
    vec3(-1, 1, 0)
    vec3(-1, 1, 0)
    vec3( 1, -1, 0)
    vec3( 1, 1, 0)
    ...
    1 0 0 0 1
    0 1 0 0 0
    0 0 1 0 0
    0 0 0 1 0
    0 0 0 0 1
    ✕
    position
    matrix
    

    View Slide

19. for ( var i = 0, l = position.length; i < l; i ++ ) {
    !
    matrix.multiplyVector( position[ i ] );
    !
    }
    

    View Slide

20. Scalar Operation SIMD Operation
    

    View Slide

21. Scalar Operation SIMD Operation
    

    View Slide

22. Scalar Operation SIMD Operation
    

    View Slide

23. CPU GPU
    

    View Slide

24. CPU GPU
    

    View Slide

25. Super fast!
    because of the parallel processing
    

    View Slide

26. • Tech definitions
    • Verbose initialization of WebGL
    • Shaders in GLSL
    • Math: Linear algebra, physics, etc…
    :PVTUJMMIBWFUPMFBSO
    

    View Slide

27. three.js
    PIXI.js
    TWGL
    

    View Slide

28. three.js
    

    View Slide

29. 29
    http://threejs.org/
    

    View Slide

30. 30
    http://threejs.org/docs/
    

    View Slide

31. 31
    http://www.apple.com/macbook/design/
    

    View Slide

32. 32
    https://www.ana-flightconnections.com/
    

    View Slide

33. 33
    https://www.ana-flightconnections.com/
    

    View Slide

34. 34
    http://fr.special-t.com/media/ogreen/en/
    

    View Slide

35. 35
    https://yomotsu.github.io/walkthrough
    

    View Slide

36. make a scene
    

    View Slide

37. make a camera
    

    View Slide

38. add a light
    

    View Slide

39. add an object
    

    View Slide

40. http://localhost:8000/threejs/threejs_0_basic.html
    

    View Slide

41. 
    
    
    
    =^.^=
    
    
    
    <br/>var width = window.innerWidth,<br/>height = window.innerHeight;<br/>!<br/>var scene = new THREE.Scene();<br/>!<br/>var camera = new THREE.PerspectiveCamera( 40, width / height, 1, 1000 );<br/>camera.position.set( 0, 0, 3 );<br/>!<br/>var renderer = new THREE.WebGLRenderer();<br/>renderer.setSize( width, height );<br/>document.body.appendChild( renderer.domElement );<br/>!<br/>

    View Slide

42. 
    
    
    
    =^.^=
    
    
    
    <br/>var width = window.innerWidth,<br/>height = window.innerHeight;<br/>!<br/>var scene = new THREE.Scene();<br/>!<br/>var camera = new THREE.PerspectiveCamera( 40, width / height, 1, 1000 );<br/>camera.position.set( 0, 0, 3 );<br/>!<br/>var renderer = new THREE.WebGLRenderer();<br/>renderer.setSize( width, height );<br/>document.body.appendChild( renderer.domElement );<br/>!<br/>load pixi.js<br/>

    View Slide

43. <br/>var width = window.innerWidth,<br/>height = window.innerHeight;<br/>!<br/>var scene = new THREE.Scene();<br/>!<br/>var camera = new THREE.PerspectiveCamera( 40, width / height, 1, 1000 );<br/>camera.position.set( 0, 0, 3 );<br/>!<br/>var renderer = new THREE.WebGLRenderer();<br/>renderer.setSize( width, height );<br/>document.body.appendChild( renderer.domElement );<br/>!<br/>var directionalLight = new THREE.DirectionalLight( 0xffffff );<br/>directionalLight.position.set( 0, 1, 1 ).normalize();<br/>scene.add( directionalLight );<br/>!<br/>var box = new THREE.Mesh(<br/>new THREE.BoxGeometry( 1, 1, 1 ),<br/>new THREE.MeshPhongMaterial( { color: 0xff0000 } )<br/>);<br/>make a scene<br/>

    View Slide

44. <br/>var width = window.innerWidth,<br/>height = window.innerHeight;<br/>!<br/>var scene = new THREE.Scene();<br/>!<br/>var camera = new THREE.PerspectiveCamera( 40, width / height, 1, 1000 );<br/>camera.position.set( 0, 0, 3 );<br/>!<br/>var renderer = new THREE.WebGLRenderer();<br/>renderer.setSize( width, height );<br/>document.body.appendChild( renderer.domElement );<br/>!<br/>var directionalLight = new THREE.DirectionalLight( 0xffffff );<br/>directionalLight.position.set( 0, 1, 1 ).normalize();<br/>scene.add( directionalLight );<br/>!<br/>var box = new THREE.Mesh(<br/>new THREE.BoxGeometry( 1, 1, 1 ),<br/>new THREE.MeshPhongMaterial( { color: 0xff0000 } )<br/>);<br/>make a camera<br/>

    View Slide

45. var scene = new THREE.Scene();
    !
    var camera = new THREE.PerspectiveCamera( 40, width / height, 1, 1000 );
    camera.position.set( 0, 0, 3 );
    !
    var renderer = new THREE.WebGLRenderer();
    renderer.setSize( width, height );
    document.body.appendChild( renderer.domElement );
    !
    var directionalLight = new THREE.DirectionalLight( 0xffffff );
    directionalLight.position.set( 0, 1, 1 ).normalize();
    scene.add( directionalLight );
    !
    var box = new THREE.Mesh(
    new THREE.BoxGeometry( 1, 1, 1 ),
    new THREE.MeshPhongMaterial( { color: 0xff0000 } )
    );
    scene.add( box );
    !
    ( function renderLoop () {
    !
    make a renderer
    

    View Slide

46. !
    var renderer = new THREE.WebGLRenderer();
    renderer.setSize( width, height );
    document.body.appendChild( renderer.domElement );
    !
    var directionalLight = new THREE.DirectionalLight( 0xffffff );
    directionalLight.position.set( 0, 1, 1 ).normalize();
    scene.add( directionalLight );
    !
    var box = new THREE.Mesh(
    new THREE.BoxGeometry( 1, 1, 1 ),
    new THREE.MeshPhongMaterial( { color: 0xff0000 } )
    );
    scene.add( box );
    !
    ( function renderLoop () {
    !
    requestAnimationFrame( renderLoop );
    box.rotation.y += 0.01;
    renderer.render( scene, camera );
    !
    make a light
    

    View Slide

47. var directionalLight = new THREE.DirectionalLight( 0xffffff );
    directionalLight.position.set( 0, 1, 1 ).normalize();
    scene.add( directionalLight );
    !
    var box = new THREE.Mesh(
    new THREE.BoxGeometry( 1, 1, 1 ),
    new THREE.MeshPhongMaterial( { color: 0xff0000 } )
    );
    scene.add( box );
    !
    ( function renderLoop () {
    !
    requestAnimationFrame( renderLoop );
    box.rotation.y += 0.01;
    renderer.render( scene, camera );
    !
    } )();
    
    !
    

    View Slide

48. scene.add( directionalLight );
    !
    var box = new THREE.Mesh(
    new THREE.BoxGeometry( 1, 1, 1 ),
    new THREE.MeshPhongMaterial( { color: 0xff0000 } )
    );
    scene.add( box );
    !
    ( function renderLoop () {
    !
    requestAnimationFrame( renderLoop );
    box.rotation.y += 0.01;
    renderer.render( scene, camera );
    !
    } )();
    
    !
    

    View Slide

49. http://localhost:8000/threejs/threejs_0_basic.html
    

    View Slide

50. 3D Models
    http://localhost:8000/threejs/threejs_1_3dmodel.html
    

    View Slide

51. skeletal animation
    http://localhost:8000/threejs/threejs_2_animation.html
    

    View Slide

52. events
    http://localhost:8000/threejs/threejs_3_event.html
    

    View Slide

53. custom shader
    http://yomotsu.github.io/VolumetricFire/examples/example2.html
    

    View Slide

54. • 3D model import
    • Animations
    • Math functions ( such as vector and matrix )
    • Build-in Shaders
    • Post processing effects
    and more!
    

    View Slide

55. View Slide

56. 56
    http://www.pixijs.com/
    

    View Slide

57. 57
    http://pixijs.github.io/docs/
    

    View Slide

58. 58
    http://www.html5gamedevs.com/forum/15-pixijs/
    

    View Slide

59. 59
    http://www.goodboydigital.com/runpixierun/
    

    View Slide

60. 60
    http://kaatje.ketnet.be/html/
    

    View Slide

61. 61
    http://www.wolverineunleashed.com/
    

    View Slide

62. 62
    http://archive.revolutionsinsound.com/#/explore/clubnights/bugged-out
    

    View Slide

63. http://localhost:8000/pixijs/pixi_0_basic.html
    

    View Slide

64. 
    
    
    
    =^.^=
    <br/>body{margin: 0;}<br/>canvas{display: block;}<br/>
    
    
    !
    
    !
    <br/>!<br/>var renderer = new PIXI.autoDetectRenderer( 600, 400, {<br/>backgroundColor : 0xCCCCCC<br/>} );<br/>document.body.appendChild( renderer.view );<br/>!<br/>

    View Slide

65. <br/>body{margin: 0;}<br/>canvas{display: block;}<br/>
    
    
    !
    
    !
    <br/>!<br/>var renderer = new PIXI.autoDetectRenderer( 600, 400, {<br/>backgroundColor : 0xCCCCCC<br/>} );<br/>document.body.appendChild( renderer.view );<br/>!<br/>var stage = new PIXI.Container();<br/>!<br/>var monster = new PIXI.Sprite.fromImage( 'monster.png' );<br/>monster.position.set( 100, 50 );<br/>stage.addChild( monster );<br/>load pixi.js<br/>

    View Slide

66. 
    !
    <br/>!<br/>var renderer = new PIXI.autoDetectRenderer( 600, 400, {<br/>backgroundColor : 0xCCCCCC<br/>} );<br/>document.body.appendChild( renderer.view );<br/>!<br/>var stage = new PIXI.Container();<br/>!<br/>var monster = new PIXI.Sprite.fromImage( 'monster.png' );<br/>monster.position.set( 100, 50 );<br/>stage.addChild( monster );<br/>!<br/>var animate = ( function animate() {<br/>!<br/>requestAnimationFrame( animate );<br/>renderer.render( stage );<br/>!<br/>} )();<br/>make a renderer<br/>

    View Slide

67. backgroundColor : 0xCCCCCC
    } );
    document.body.appendChild( renderer.view );
    !
    var stage = new PIXI.Container();
    !
    var monster = new PIXI.Sprite.fromImage( 'monster.png' );
    monster.position.set( 100, 50 );
    stage.addChild( monster );
    !
    var animate = ( function animate() {
    !
    requestAnimationFrame( animate );
    renderer.render( stage );
    !
    } )();
    !
    
    !
    

    View Slide

68. backgroundColor : 0xCCCCCC
    } );
    document.body.appendChild( renderer.view );
    !
    var stage = new PIXI.Container();
    !
    var monster = new PIXI.Sprite.fromImage( 'monster.png' );
    monster.position.set( 100, 50 );
    stage.addChild( monster );
    !
    var animate = ( function animate() {
    !
    requestAnimationFrame( animate );
    renderer.render( stage );
    !
    } )();
    !
    
    !
    

    View Slide

69. var stage = new PIXI.Container();
    !
    var monster = new PIXI.Sprite.fromImage( 'monster.png' );
    monster.position.set( 100, 50 );
    stage.addChild( monster );
    !
    var animate = ( function animate() {
    !
    requestAnimationFrame( animate );
    renderer.render( stage );
    !
    } )();
    !
    
    !
    

    View Slide

70. http://localhost:8000/pixijs/pixi_0_basic.html
    

    View Slide

71. filters
    http://localhost:8000/pixijs/pixi_1_blend.html
    

    View Slide

72. sprite animations
    http://localhost:8000/pixijs/pixi_2_sprite.html
    

    View Slide

73. events
    http://localhost:8000/pixijs/pixi_3_event.html
    

    View Slide

74. shaders
    http://localhost:8000/pixijs/pixi_4_shader.html
    

    View Slide

75. custom shaders
    http://localhost:8000/pixijs/pixi_5_shader2.html
    

    View Slide

76. • Sprite and spine animations
    • Mouse and touch events
    • Pixel manipulation with WebGL
    • Custom shaders
    and more
    

    View Slide

77. • Stage has been deprecated, now container
    • Name spaces has been changed
    • EventListener has been supported
    • Color matrix has been changed to 5x4 from 4x4
    and many changes
    %JGGFSFODFTCFUXFFO7BOE7
    

    View Slide

78. TWGL.js
    by the author of WebGL Fundamentals
    

    View Slide

79. 79
    http://twgljs.org/
    

    View Slide

80. http://localhost:8000/twgljs/twgl_1.html
    

    View Slide

81. 
    
    
    
    =^.^=
    <br/>body{margin: 0;}<br/>canvas{display: block;}<br/>
    
    
    !
    
    !
    !
    
    <br/>!<br/>var canvas = document.getElementById( 'canvas' );<br/>canvas.width = window.innerWidth;<br/>canvas.height = window.innerHeight;<br/>

    View Slide

82. 
    
    !
    
    !
    !
    
    <br/>!<br/>var canvas = document.getElementById( 'canvas' );<br/>canvas.width = window.innerWidth;<br/>canvas.height = window.innerHeight;<br/>!<br/>var gl = twgl.getWebGLContext( canvas );<br/>!<br/>var vs = [<br/>'attribute vec3 position;',<br/>'attribute vec2 texcoord;',<br/>'uniform mat4 projectionMatrix;',<br/>'uniform mat4 viewMatrix;',<br/>prepare a canvas<br/>

    View Slide

83. 
    
    !
    
    !
    !
    
    <br/>!<br/>var canvas = document.getElementById( 'canvas' );<br/>canvas.width = window.innerWidth;<br/>canvas.height = window.innerHeight;<br/>!<br/>var gl = twgl.getWebGLContext( canvas );<br/>!<br/>var vs = [<br/>'attribute vec3 position;',<br/>'attribute vec2 texcoord;',<br/>'uniform mat4 projectionMatrix;',<br/>'uniform mat4 viewMatrix;',<br/>load twgl.js<br/>

    View Slide

84. !
    
    <br/>!<br/>var canvas = document.getElementById( 'canvas' );<br/>canvas.width = window.innerWidth;<br/>canvas.height = window.innerHeight;<br/>!<br/>var gl = twgl.getWebGLContext( canvas );<br/>!<br/>var vs = [<br/>'attribute vec3 position;',<br/>'attribute vec2 texcoord;',<br/>'uniform mat4 projectionMatrix;',<br/>'uniform mat4 viewMatrix;',<br/>'uniform mat4 modelMatrix;',<br/>'varying vec2 vUv;',<br/>'void main () {',<br/>get gl context<br/>

    View Slide

85. !
    var vs = [
    'attribute vec3 position;',
    'attribute vec2 texcoord;',
    'uniform mat4 projectionMatrix;',
    'uniform mat4 viewMatrix;',
    'uniform mat4 modelMatrix;',
    'varying vec2 vUv;',
    'void main () {',
    'gl_Position = projectionMatrix * viewMatrix * modelMatrix *
    vec4(position, 1.0 );',
    'vUv = texcoord;',
    !
    '}'
    ].join( '\n' );
    !
    var fs = [
    prepare VS source
    

    View Slide

86. ].join( '\n' );
    !
    var fs = [
    'precision mediump float;',
    !
    'uniform sampler2D diffuse;',
    !
    'varying vec2 vUv;',
    !
    'void main () {',
    !
    'vec4 diffuseColor = texture2D( diffuse, vUv );',
    'gl_FragColor = diffuseColor;',
    // 'gl_FragColor = vec4( 0, 0, 0, 1 );',
    !
    '}'
    ].join( '\n' );
    !
    var programInfo = twgl.createProgramInfo( gl, [ vs, fs ] );
    !
    var arrays = {
    position: {
    prepare FS source
    

    View Slide

87. !
    '}'
    ].join( '\n' );
    !
    var programInfo = twgl.createProgramInfo( gl, [ vs, fs ] );
    !
    var arrays = {
    position: {
    numComponents: 3,
    data: [
    -1, -1, 0,
    1, -1, 0,
    -1, 1, 0,
    !
    -1, 1, 0,
    1, -1, 0,
    1, 1, 0
    ]
    },
    indices: {
    numComponents: 3,
    data: [
    compile those and

    create a program
    

    View Slide

88. !
    var arrays = {
    position: {
    numComponents: 3,
    data: [
    -1, -1, 0,
    1, -1, 0,
    -1, 1, 0,
    !
    -1, 1, 0,
    1, -1, 0,
    1, 1, 0
    ]
    },
    indices: {
    numComponents: 3,
    data: [
    0, 1, 2,
    3, 4, 5
    ]
    },
    prepare attributes
    

    View Slide

89. },
    indices: {
    numComponents: 3,
    data: [
    0, 1, 2,
    3, 4, 5
    ]
    },
    texcoord: {
    numComponents: 2,
    data: [
    0, 1,
    1, 1,
    0, 0,
    !
    0, 0,
    1, 1,
    1, 0
    ]
    }
    };
    !
    

    View Slide

90. };
    !
    var bufferInfo = twgl.createBufferInfoFromArrays( gl, arrays );
    !
    !
    var projectionMatrix = twgl.m4.perspective(
    30 * Math.PI / 180,
    gl.canvas.clientWidth / gl.canvas.clientHeight,
    1,
    1000
    );
    var viewMatrix = twgl.m4.translate( twgl.m4.identity(), [ 0, 0 ,-5 ] );
    var modelMatrix = twgl.m4.rotationY( 1 );
    !
    !
    var tex = twgl.createTexture( gl, {
    min: gl.NEAREST,
    mag: gl.NEAREST,
    src: './uv.png'
    }, function () {} );
    !
    make a buffer
    

    View Slide

91. !
    var projectionMatrix = twgl.m4.perspective(
    30 * Math.PI / 180,
    gl.canvas.clientWidth / gl.canvas.clientHeight,
    1,
    1000
    );
    var viewMatrix = twgl.m4.translate( twgl.m4.identity(), [ 0, 0 ,-5 ] );
    var modelMatrix = twgl.m4.rotationY( 1 );
    !
    !
    var tex = twgl.createTexture( gl, {
    min: gl.NEAREST,
    mag: gl.NEAREST,
    src: './uv.png'
    }, function () {} );
    !
    var uniforms = {
    projectionMatrix: projectionMatrix,
    viewMatrix : viewMatrix,
    modelMatrix : modelMatrix,
    prepare unifroms
    

    View Slide

92. }, function () {} );
    !
    var uniforms = {
    projectionMatrix: projectionMatrix,
    viewMatrix : viewMatrix,
    modelMatrix : modelMatrix,
    diffuse : tex
    };
    !
    var render = function ( time ) {
    !
    twgl.resizeCanvasToDisplaySize( gl.canvas );
    gl.viewport( 0, 0, gl.canvas.width, gl.canvas.height );
    uniforms.modelMatrix = twgl.m4.rotationY( time * 0.001 );
    !
    gl.useProgram( programInfo.program );
    twgl.setBuffersAndAttributes( gl, programInfo, bufferInfo );
    twgl.setUniforms( programInfo, uniforms );
    gl.drawElements( gl.TRIANGLES, bufferInfo.numElements,
    gl.UNSIGNED_SHORT, 0 );
    !
    

    View Slide

93. !
    var render = function ( time ) {
    !
    twgl.resizeCanvasToDisplaySize( gl.canvas );
    gl.viewport( 0, 0, gl.canvas.width, gl.canvas.height );
    uniforms.modelMatrix = twgl.m4.rotationY( time * 0.001 );
    !
    gl.useProgram( programInfo.program );
    twgl.setBuffersAndAttributes( gl, programInfo, bufferInfo );
    twgl.setUniforms( programInfo, uniforms );
    gl.drawElements( gl.TRIANGLES, bufferInfo.numElements,
    gl.UNSIGNED_SHORT, 0 );
    !
    requestAnimationFrame( render );
    !
    };
    !
    render( 0 );
    !
    
    !
    render it!
    

    View Slide

94. http://localhost:8000/twgljs/twgl_1.html
    

    View Slide

95. http://localhost:8000/twgljs/twgl_2.html
    

    View Slide

96. • Less verbose initialization
    • Math functions
    That’s it!
    

    View Slide

97. WebGL is not PS4
    

    View Slide

98. • WebUI for 3D contents

    Still not exist a lot
    • Multiple devices

    High-End Gaming PC, Laptop, Mobile and others
    • Internet Connection
    8IBUXFOFFEUPUIJOLBCPVU
    

    View Slide

99. • Number of polygons

    esp. file size
    • Texture size
    • Resolution
    • Number of Draw-calls
    8IBUXFOFFEUPDBSFBCPVU
    

    View Slide

100. Like 3DS, PSP, PSVita
     FYI: Playable characters in FINAL FANTASY TYPE-0 consist of

     about 1000 tris and 76KB textures
     http://hexadrive.jp/lab/case/1573/
     

     View Slide

101. Video games could be

     great knowledge
     

     View Slide

102. View Slide

103. View Slide

104. WebVR
     

     View Slide

105. d = navigator.

     getVRDevices()
     

     View Slide

106. d = navigator.

     getVRDevices()
     •d[ i ].VRDevice (Type of the device)
     •d[ i ].VREyeParameters
     •FOV
     •eyeTranslation
     •d[ i ].VRPositionState
     •position
     •velocity
     •orientation
     •angularVelocity
     othres...
     

     View Slide

107. vrHMD.getEyeTranslation( ‘left' );
     vrHMD.getEyeTranslation( 'right' );
     vrHMD.getRecommendedEyeFieldOfView( 'left' );
     vrHMD.getRecommendedEyeFieldOfView( 'right' );
     

     View Slide

108. Chromium
     WebVR Builds
     Firefox Nightly Builds
     WebVR Oculus Rift Enabler
     as of Jun, 2015
     

     View Slide

109. Virtuix Omni
     Locomotion
     HMD
     Cyberith Virtualizer
     OCULUS Rift SONY Morpheus HTC Vine
     • Leap Motion
     • PrioVR
     • Control VR
     • Sixense Stem
     • Razer Hydra
     Others
     

     View Slide

110. 110
     https://www.youtube.com/watch?v=aTtfAQEeAJI
     

     View Slide

111. View Slide

112. Drop-in Phone

     Viewers
     

     View Slide

113. Drop-in Phone

     Viewers
     • Different APIs

     (It will use DeviceOrientation API instead)
     • Limited performance

     (Stereo rendering on mobile device)
     

     View Slide

114. Conclusion
     

     View Slide

115. WebGL is

     a cross-platform,

     web standard, fast,

     powerful graphics API
     

     View Slide

116. Several JS libraries

     are available

     for different purposes
     

     View Slide

117. WebGL is
     still expanding
     WebGL2, WebVR and others
     

     View Slide

118. gl.finish();
     @yomotsu
     

     View Slide