Starmap - Journey through a WebGL project

The team from Turbulent in Montreal will go through the lessons learned building a large-scale WebGL project for Star Citizen. Anecdotes, stack & tech choices, the good and the bad.

Transcript

1. STARMAP Journey through a WebGL project Turbulent, Gamerizon, JS Montreal, April 2016 Benoit Beauséjour, Roger Cyr, Martin Lizée, Robert Lizée b@turbulent.ca
2. STARMAP Designed, developed @ Turbulent & Gamerizon Started in 2014 Voted on by the Star Citizen Community Initial prototyping using Unity 3D Rewritten from scratch with WebGL ~3000 hours of work ~20K lines of code ~ 4 month production
3. “ Let’s see it. https://robertsspaceindustries.com/starmap
4. Project anatomy
5. Project anatomy Must produce an interactive image (fast!) Split responsibility between 2 components Split technology, split concerns Focus on the talent of each team Parallel development tracks In-Browser Bowser!
6. User interface
7. User interface In charge of applicative logic Routing and site navigation Manages Web Audio integration Client persistence (settings) Fullscreen mode
8. Viewer
9. Viewer WebGL application Render 3D world in 2D canvas Loading process (models, textures) Shader and effects Handle direct mouse interactions REST API controller Data caching and loading
10. The Stack
11. People breakdown UI Developer UI Developer 2D Artist 3D Artist Shader Artist Game Developer
12. Viewer
13. Development Stack Package Management Builder & Bundler Language & Transpiler npm gulp, webpack Typescript, tsc
14. Why Typescript? Superset of Javascript Optionally typed Transpiles to ES5 Does not require a runtime In the browser Bowser!
15. Why Typescript? Type inference Type annotations ES6 Class system with inheritance Generics, Enums Function expressions Feels like C# Low-cost productivity boost
16. IntelliSense with Visual Studio code +
17. three.js 3D Renderer Abstract WebGL for us (yay!) Extremely lightweight “Battle Tested” on multiple platforms Full access to low-level GLSL Support for 3D formats like Collada Solid math support classes
18. “ But...
19. three.js Animation support for Collada was incomplete. Cannot instance animated objects. support (json/binary model format) was also incomplete. ...but hey! source available!
20. “ Make tools.
21. Importance of the pipeline tools Developed a minimal client with drag & drop functionality. Go from artist to screen rapidly. Allow fast iteration of art assets before export from 3D tools Ensures the core work is done before artist can work Ensures WYSIWYG
22. Tools Data drive most of your systems. Libraries like dat.gui make it easy to add control UIs. Insist your backend API persist your tools data.
23. “ Performance is in The Metal.
24. Where is the metal? “The Metal” Shader Programs
25. What is the metal? Compiled Shader Programs Javascript Vertex Shader Fragment Shader Rasterization Shape Assembly Buffer Object
26. Where is the metal? Javascript Vertex Shader Fragment Shader Rasterization Shape Assembly Buffer Object Shape Assembly Rasterization Fragment Vertex
27. Testing the limits: 262,000 animated particles! Fully interactive CSS3D pages shown with transparency and reflexion to provide a smooth integration with the 3D environment Experimentation
28. The Black Hole is a simple Quad with a complex Fragment Shader The Fragment Shader handles everything, including the animation and the 3D perspective view. No Bitmap! The Black Hole is entirely procedural. Complex shapes made simple
29. WebGL Adoption
30. Lessons Push as much work as possible to the GPU Maximize use of Fragment Shaders even for full 3D objects ThreeJS is not yet ready for full-on games development. GLSL Effects. Procedural textures. Noise formulas to avoid bitmaps. Billboards, Fragment shaders with simple geometry - Almost FREE Antialiasing & Animation Animation system and art pipeline not yet mature enough
31. Resources to touch the Metal ShaderToy: https://www.shadertoy.com/ GLSL Sandbox: http://glslsandbox.com/ Interactive 3D Graphics (CS291 @ Udacity) https://www.udacity. com/course/interactive-3d-graphics--cs291 WebGL Fundamentals: http://webglfundamentals.com/
32. Too close to the Metal
33. User Interface
34. Development Stack Package Management Builder & Bundler Language & Transpiler npm gulp, webpack ES6, Babel
35. Application Stack Framework Animation Download marionette Shifty fetch
36. Building a bridge Viewer initially written using TypeScript's native module system but later migrated to the CommonJS module system for usage with Webpack Makes it possible to use NPM as the package manager. Makes TypeScript autocompletion not as efficient Viewer compiled as a reusable JavaScript library, to be included by the main application Token replacement (dev-only code paths using the Define Plugin & UglifyJS’s dead- code elimination) Slower build time even when using incremental builds (Watcher) Allows multiple compilation targets (Standalone / Library) and environments (Development / Production) GLSL loader allows GLSL shaders in separate files. Less error prone, allows syntax highlighting and autocompletion.
37. ES6 Viewer API The API provided by the Viewer library was well designed but low level We created an ES6 Viewer API Defined a simple event-driven interface Layer between the viewer and the UI Easier communication Easier synchronization
38. The Tower of Babel (ES6 to ES5 transpiler) Typescript Babel Vendor locking (MS) Poor ES6 support (at the time) Symbols Generators for .. of Computed properties De-structuring Spread Operator ... Supports most interesting parts of ES6 Includes Polyfills Future-proof (ECMAScript 6 is the current JS Standard) Optional typing if required (ie using FB Flowtype) Previously used internally on smaller projects Other ES5 to 6 compilers were not as mature Last year!
39. Web Audio API The alternative to Flash for having sound in a web application. - Low-level but powerful - No-sound fallback for unsupported browsers - Documentation is bare at the moment.
40. Animating those SVGs Before exporting - a checklist - Remove unused elements, hiding the layers is not enough - Create necessary masks and clip paths - Move all the animatable elements to their animation start position.
41. Animating those SVGs After exporting - another checklist Clean up SVGs exports from Illustrator manually : - Remove useless elements (SVG groups <g>, Comments) - Make sure that it contains no filter-created and unwanted base64-encoded rasterized pngs - Add classes/ids to elements to be animated
42. Animating those SVGs Animating - CSS usable for most browsers but JS has better support - Create path : - Set "stroke-dasharray" to its length - Animate its "stroke-dashoffset" - To get path length call SVGPathElement#getTotalLength() - Transformations - Point of origin at the top left of the element - Makes some transformations like rotation more painful to handle.
43. “ Uh oh... -Me
44. Troubleshooting - a year in review Firefox inline SVG rendering glitches What happened What we did Some SVG filters would not render properly in Firefox Use an object tag to embed the SVG and access its DOM using HTMLObjectElement#contentDocument
45. Troubleshooting - a year in review Firefox slow SVG rendering What happened What we did SVG animations cause layout trashing in Firefox Disable costly animations when Firefox is detected using **cough** user agent sniffing **cough**
46. Troubleshooting - a year in review The loading animation What happened What we did Collada model file parser blocks event loop for SVG + JS, freezing the animation for short periods of time. Could not move the parser to a web worker because of its reliance on DOM. Write a simpler CSS-only animation that would not be blocked by the event loop.
47. Troubleshooting - a year in review Firefox Web Audio fadings What happened What we did (proposed by a Mozilla Web Audio engineer) Using Shifty to do volume fadings would cause crackling noises. Use the combination of AudioParam#setValueAtTime and AudioParam#linearramptovalueattime to produce a linear fade in/out.
48. “ Arggggshghghghghg. -Me again
49. Lessons It’s possible to make kickass large 3D interactive projects on the web. Ally the versatility of the web platform with the power of GL to elevate the user experience. Compile-time checks make huge difference in the workflow. Using web tools for UI allows fast iterations. Real-time 3D graphics on the web: Still not a walk in the park.
50. Thanks! Hey! We’re hiring! Hit us up athr@turbulent.ca