WebGL in Ember

If you know JavaScript and how to develop web applications, you're well on your way to being able to draw 3D scenes in the browser. If you're an Ember developer, life is even easier because the framework reduces the friction in programming for the web. In this talk, I'll go over what WebGL is and how you can start learning it while taking advantage of what you already know about Ember.

Transcript

1. WebGL in Ember Ember-SF Meetup January 24, 2017 https://speakerdeck.com/mtmckenna/webgl-in-ember matt@mtmckenna.com @mattmckenna
2. In This Talk • WebGL in general • Why Ember is helpful when developing WebGL apps • Simplest way to integrate Three.js into Ember • A few ways Ember can help you develop Three.js apps
3. What is WebGL? • WebGL is a web API that... • ... can draw 3D graphics on an HTML <canvas>. • ... renders through graphics hardware (i.e. it renders fast). • Screenshot from A Journey Through Middle Earth
4. Let's Look at Something! • Impact of Diabetes
5. Ways to WebGL • Use the WebGL API directly • Use a 3D JavaScript library like Three.js, Babylon.js, or PlayCanvas • Use a native 3D engine that transpiles into JS (e.g. Unity or Unreal)
6. Using the WebGL API Directly Can Be Rough Because... • The API hasn't been designed for app developer speed. • It is easy to get stuck on math/graphics issues.
7. Using a Native Engine Can Be Rough Because... • There's a lot of overhead when running apps built with native engines (file size, etc.). • You know, it totally might be fine--I just have no experience with native 3D engines.
8. You Probably Want to Use a JavaScript Library • Three.js hits the sweet spot for developer productivity and customizability. • Look how small a starter app can be!
9. Creating a Cube in Three.js var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth/window.innerHeight, 0.1, 1000 ); var renderer = new THREE.WebGLRenderer(); renderer.setSize( window.innerWidth, window.innerHeight ); document.body.appendChild( renderer.domElement ); var geometry = new THREE.BoxGeometry( 1, 1, 1 ); var material = new THREE.MeshBasicMaterial( { color: 0x00ff00 } ); var cube = new THREE.Mesh( geometry, material ); scene.add( cube ); camera.position.z = 5; var animate = function () { requestAnimationFrame( animate ); cube.rotation.x += 0.1; cube.rotation.y += 0.1; renderer.render(scene, camera); }; animate();
10. Sample Cube!
11. • Production-level WebGL apps (like all production-level JavaScript apps!) have to deal with boring boilerplate like compiling assets, routing, persisting data, maintaining state, testing, and deploying. Sample Cube is Cool, but Production Apps are Harder: Part 1 of 2
12. Build a Game in Ember Starring Your Cat (Emberconf 2016) • Let Ember handle the boring parts: • Building full-screen, nested UIs • Testing • Deploying to web/mobile/desktop • Croissant the Pizza Cat
13. Create the Sample Cube in Ember $ npm install -g ember-cli $ ember new ember-threejs-cube $ cd ember-threejs-cube $ npm install --save-dev ember-browserify three $ ember g component sample-cube $ ember serve Install everything with ember-cli: {{sample-cube}} app/templates/application.hbs: import Ember from 'ember'; import THREE from 'npm:three'; export default Ember.Component.extend({ didInsertElement() { this._super(...arguments); <Paste in sample cube code!> } }); app/components/sample-cube.js:
14. Sample Cube in Ember!
15. • Production-level WebGL apps also have to deal with... • Interactivity • Async stuff (downloading things, component lifecycles) • Custom shaders Sample Cube is Cool, but Production Apps are Harder: Part 2 of 2
16. Three.js in Ember Demo App • Three.js in Ember • threejs-in-ember.netlify.com • GitHub
17. Interactivity • Using an Ember component's built in event handling is handy for making the scene interactive. • Ember's data-down-actions-up (DDAU) pattern still applies when updating properties of the scene.
18. Handling Events touchStart() { this.set('rotating', true); }, touchMove(event) { this.handleUserRotation(event); }, touchEnd() { this.set('rotating', false); }, app/components/three-container.js
19. Updating a Scene with DDAU • Data down, actions up! • Can use standard Ember/HTML UI controls to modify the Three.js scene. {{three-container scale=scale partyMode=partyMode shapeData=model}} <input class="scale-slider" type="range" value={{scale}} oninput={{action "updateScale"}} /> app/templates/application.hbs: actions: { updateScale(event) { this.set('scale', event.target.value); } } app/controllers/application.js:
20. Async Stuff • ember-concurrency solves two problems in the demo app: • Removes the need to track the download state of textures (e.g. no need for an "isDownloading" flag, etc.) • Gracefully cancels the animation callback when the component has been destroyed
21. requestAnimationFrame without ember-concurrency animate() { this.rotateShape(); requestAnimationFrame(() => { this.animate(); }); } Blows up if the component is destroyed before the callback is called, which is bad: So you have to add a guard conditional, which is annoying: animate() { this.rotateShape(); requestAnimationFrame(() => { if (!this.isDestroyed) { this.animate(); } }); }
22. requestAnimationFrame with ember-concurrency animateTask: task(function * () { this.rotateShape(); requestAnimationFrame(() => { this.get('animateTask').perform(); }); }) Let ember-concurrency handle animation and stop worrying about the component lifecyle:
23. Shaders • Programs written in GLSL • Run directly on the GPU (graphics processing unit) • Responsible for figuring out what your 3D objects look like • Two types of shaders: vertex, fragment
24. Fragment Shaders • Fragment shaders determine the color of things on the screen. • Here's a fragment shader that makes things green: void main() { gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); }
25. Passing Shaders Around in JS is a Pain • Shaders are passed into WebGL as strings and later compiled, which is sort of logistically annoying. • What's the best way to configure your environment so passing the shader string into the above statement that doesn't feel super hacky? new THREE.ShaderMaterial({ vertexShader: vertexShaderString, fragmentShader: fragmentShaderString });
26. Formatting Shaders: Script Tags <script id="fragment-shader" type="x-shader/x-fragment"> void main() { gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); } </script> var fragmentShaderString = document.getElementById('fragment-shader').innerText;
27. Formatting Shaders: Concatenated Strings var fragmentShaderString = 'varying vec3 vColor; \n' + 'void main() { \n' + ' gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); \n' + '}';
28. Formatting Shaders: Joined Array var fragmentShaderString = [ 'varying vec3 vColor; \n', 'void main() { \n', ' gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); \n', '}' ].join('');
29. Formatting Shaders: Imported Module export default ` void main() { gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); } `; import fragmentShaderString from './fragment-shader';
30. Formatting Shaders: Importing via ember-stringify void main() { gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0); } import shaders from 'threejs-in-ember/ember-stringify'; var fragmentShaderString = shaders['fragment.glsl']; shaders/fragment.glsl: import the shader:
31. Party Mode
32. Recap • Combine Ember's best practices with 3D JS libraries like Three.js to have a great time creating 3D scenes for the web.
33. WebGL Resources • WebGL Fundamentals • WebGL Programming Guide (book) • MDN: Learn WebGL • MDN: Lighting In WebGL
34. Thank you! https://speakerdeck.com/mtmckenna/webgl-in-ember