Growing a Language for Graphics - DotJS

"Growing a Language for Graphics" is about JavaScript as seen from the eyes of a data visualization / graphics person. By having a look at GLSL - a domain specific language (DSL) for graphics used in WebGL - I talk about what language features would help graphic developers on the Web and I show the long history of trying to standardize operator overloading into JavaScript.
This presentation was given at DotJS in Paris, France in December 2013.

Transcript

1. [Growing] a Language for Graphics @philogb dotJS Dec. 2013
4. WebGL Canvas SVG
5. WebGL
6. OpenGL OpenGL ES WebGL WebGL
7. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html WebGL - Pipeline
8. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html Triangle Assembly WebGL - Pipeline
9. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html Triangle Assembly Rasterization WebGL - Pipeline
10. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html Triangle Assembly Rasterization Fragment Shader WebGL - Pipeline
11. JavaScript GPU (Compiled Program) WebGL JS API WebGL - API
12. JavaScript Fragment Shader WebGL JS API Vertex Shader GLSL API GLSL API WebGL - API
13. GLSL A DSL for Graphics •C-like •Built-in types, functions for graphics •Operator overloading
14. 1 vec4 vector = vec4( 0, 1, vec2(0, 0) ); 2 vec3 point = vector.xyz; 3 vec3 rgb = vector.rgb; 4 mat4 m = mat4(vector); 5 vec4 ans = vector * m; 6 7 float delta = 0.3; 8 vec4 from = vec4(0); 9 vec4 to = vec4(1); 10 vec4 current = from + (to - from) * delta; 11 vec4 current2 = mix(from, to, delta); 12 GLSL Syntax
15. GLSL Built-in functions radians degrees sin cos tan asin acos atan pow exp log exp2 log2 sqrt inversesqrt abs sign floor ceil fract mod min max clamp mix step smoothstep length distance dot cross normalize faceforward reflect refract matrixCompMult
16. GLSL & JS Simple interpolation vec4 current = from + (to - from) * delta; var current = (to.add(from.scale(-1))).scale(delta).add(from);
17. vec4 current = from + (to - from) * delta; var current = (to.add(from.scale(-1))).scale(delta).add(from); GLSL & JS Simple interpolation
20. •The Lambda Papers (Scheme)
21. •The Lambda Papers (Scheme) •Java (1994), Sun fellow
22. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39 (1st Edition)
23. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39 (1st Edition) •The C Language (X3J11)
24. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39 (1st Edition) •The C Language (X3J11) •Fortran (X3J3)
25. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39 (1st Edition) •The C Language (X3J11) •Fortran (X3J3) •Common Lisp (X3J13)
26. Growing a language 1998 - OOPSLA
27. General Rationale Some people want •Complex Number •Rational Number •Interval •Vector{2,3,4} •Matrix Should these be added to the language?
28. Value Objects
29. Value Objects •Many use-cases (int64, complex, decimal...)
30. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators
31. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving Boolean Algebra
32. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving Boolean Algebra •Preserving Relational Relations (trichotomy)
33. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving Boolean Algebra •Preserving Relational Relations (trichotomy) •Strict Equality Operators unchanged
34. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving Boolean Algebra •Preserving Relational Relations (trichotomy) •Strict Equality Operators unchanged •Literal Syntax
35. Value Objects But how did we get here?
36. Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Operator Overloading
37. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic FunctionsDouble Dispatch Revisited Value Types Value Proxies Value Objects ES4 draft 1 class Complex! { ... } 2 3 generic intrinsic function +( a: Complex, b: Complex ) 4 new Complex( a.real + b.real, a.imag + b.imag ) 5 6 generic intrinsic function +( a: Complex, b: AnyNumber ) 7 a + Complex(b) 8 9 generic intrinsic function +( a: AnyNumber, b: Complex ) 10 Complex(a) + b
38. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic FunctionsDouble Dispatch Revisited Value Types Value Proxies Value Objects Dynamic Dispatch - How does it work? 1 class Person { 2 public void eat (Food f) { println ("eating food"); } 3 public void eat(Pasta p) { println ("eating pasta"); } 4 } 5 /*...*/ 6 Food food = new Pasta(); 7 somePerson.eat(food); Java does not have dynamic dispatch in its method arguments
39. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects But what happens with “instances”? 1 class John extends Person { 2 public void eat(Food f) { println ("mmm...."); } 3 public void eat(Pasta p) { println ("Yummy!"); } 4 } 5 6 Person me = new John(); 7 Food pasta = new Pasta(); 8 me.eat(pasta); Dynamic Dispatch - How does it work?
40. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Dynamic Dispatch Java: First arg. is dynamically dispatched 1 public void function eat(John john, Food f) { ... } 2 public void function eat(John john, Pasta p) { ... } 3 4 public void function eat(Person person, Food f) { ... } 5 public void function eat(Person person, Pasta p) { ... }
41. Generic Functions Dynamic Dispatch CLOS: All args. are dynamically dispatched 1 (defmethod eat ((john John) (f Food)) ... ) 2 (defmethod eat ((john John) (p Pasta)) ... ) 3 4 (defmethod eat ((person Person) (f Food)) ... ) 5 (defmethod eat ((person Person) (p Pasta)) ... ) Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects
42. Double Dispatch Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Same principle as Smalltalk 1 Number.prototype['+'] = function(arg) { 2 if (isNumber(arg)) { 3 return primAdd(this, arg); 4 } else { 5 return arg['reverse+'](this); 6 } 7 }; 8 9 Number.prototype['reverse+'] = function(receiver) { 10 if (isNumber(receiver)) { 11 return primAdd(receiver, this); 12 } else { 13 throw ...; 14 } 15 }; 16
43. Double Dispatch Was proposed before based on ExtendScript 1 Array.prototype['*'] = function(k, rev) { 2 if( k.constructor != Number ) return; 3 4 var i = this.length, 5 r = this.concat(); 6 7 while( i-- ) r[i] *= k; 8 return r; 9 }; 10 11 // Sample code: 12 var a = [1,2,3]; 13 alert( a * 5 ); // [5,10,15] 14 // (reversed == true) 15 alert( 5 * a ); // [5,10,15] (works also!) Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects
44. Double Dispatch Problems on extensibility 1 Point.prototype['+'] = function (that) { 2 if (that instanceof Point) { 3 return new Point(this.x + that.x, this.y + that.y); 4 } else if (that instanceof Number) { 6 return new Point(this.x + that, this.y + that); 7 } else if (...) { 8 //x 10000 9 } else { 10 return that['reverse+'](this); 11 } 12 }; Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects
45. OO Revisited Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects 1 function pointPlusNumber(a, b) { 2 return new Point(a.x + b, a.y + b); 3 } 4 5 Function.defineOperator('+', Point, Number, pointPlusNumber); 6 7 function numberPlusPoint(a, b) { 8 return new Point(a + b.x, a + b.y); 9 } 10 11 Function.defineOperator('+', Number, Point, numberPlusPoint); 12 13 function addPoints(a, b) { 14 return new Point(a.x + b.x, a.y + b.y); 15 } 16 17 Function.defineOperator('+', Point, Point, addPoints);
46. Value Types Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects “Value types would be objects that behave like primitives such as number and string: they are immutable or appear to be (...) In order to be usable, value types must "overload" (...) operator syntax; they also need to extend literal syntax (e.g. 1.1m for decimal 1.1).”
47. Value Proxies Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Proxies 1 var p = new Proxy(target, handler); delete proxy.name deleteProperty Object.freeze(proxy) freeze name in proxy has proxy.name get proxy.name = val set for (prop in proxy) enumerate proxy.apply(thisValue, args) apply new proxy(...args) construct
48. Value Objects Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects ...are not user-defined
49. After 7 years still no consensus on how users can perform operator overloading
51. Thanks @philogb http://philogb.github.io/
52. # Bibliography ## Why operator overloading http://www.jroller.com/cpurdy/entry/the_seven_habits_of_highly1 ## ES4 overview http://www.ecmascript.org/es4/spec/overview.pdf ## Generic functions https://mail.mozilla.org/pipermail/es-discuss/2007-November/005043.html https://mail.mozilla.org/pipermail/es-discuss/2007-November/005049.html https://mail.mozilla.org/pipermail/es-discuss/2007-November/005052.html https://mail.mozilla.org/pipermail/es-discuss/2007-November/005053.html ## 2009 Operator Overloading proposal https://mail.mozilla.org/pipermail/es-discuss/2009-January/008535.html https://mail.mozilla.org/pipermail/es-discuss/2009-January/008541.html http://wiki.ecmascript.org/doku.php?id=strawman:operator_overloading_with_double_dispatch ## ExtendScript OO http://forums.adobe.com/thread/646268 http://www.indiscripts.com/post/2010/05/operator-overloading-with-extendscript ## Operator Overloading revisited https://mail.mozilla.org/pipermail/es-discuss/2009-June/009603.html https://mail.mozilla.org/pipermail/es-discuss/2009-June/009604.html https://mail.mozilla.org/pipermail/es-discuss/2009-June/ ## Value types http://wiki.ecmascript.org/doku.php?id=strawman:value_types Decimal type: http://intertwingly.net/stories/2008/09/20/estest.html https://mail.mozilla.org/pipermail/es-discuss/2009-January/008646.html Value types page started: https://mail.mozilla.org/pipermail/es-discuss/2010-January/010677.html ## Value objects http://wiki.ecmascript.org/doku.php?id=strawman:value_objects https://mail.mozilla.org/pipermail/es-discuss/2012-March/021411.html http://www.slideshare.net/BrendanEich/value-objects https://bugzilla.mozilla.org/show_bug.cgi?id=749786 ## Value proxies http://wiki.ecmascript.org/doku.php?id=strawman:value_proxies ## ES next https://mail.mozilla.org/pipermail/es-discuss/2013-April/029949.html