Growing a Language for Graphics - DotJS

Growing a Language for Graphics - DotJS

"Growing a Language for Graphics" is about JavaScript as seen from the eyes of a data visualization / graphics person. By having a look at GLSL - a domain specific language (DSL) for graphics used in WebGL - I talk about what language features would help graphic developers on the Web and I show the long history of trying to standardize operator overloading into JavaScript.
This presentation was given at DotJS in Paris, France in December 2013.

A2b51961d5cb9140c6b949d2c9bfe0a7?s=128

Nicolas Garcia Belmonte

December 02, 2013
Tweet

Transcript

  1. [Growing] a Language for Graphics @philogb dotJS Dec. 2013

  2. None
  3. None
  4. WebGL Canvas SVG

  5. WebGL

  6. OpenGL OpenGL ES WebGL WebGL

  7. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html WebGL - Pipeline

  8. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html Triangle Assembly WebGL - Pipeline

  9. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html Triangle Assembly Rasterization WebGL -

    Pipeline
  10. Vertex Shader image source: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Chapter-1:-The-Graphics-Pipeline.html Triangle Assembly Rasterization Fragment Shader

    WebGL - Pipeline
  11. JavaScript GPU (Compiled Program) WebGL JS API WebGL - API

  12. JavaScript Fragment Shader WebGL JS API Vertex Shader GLSL API

    GLSL API WebGL - API
  13. GLSL A DSL for Graphics •C-like •Built-in types, functions for

    graphics •Operator overloading
  14. 1 vec4 vector = vec4( 0, 1, vec2(0, 0) );

    2 vec3 point = vector.xyz; 3 vec3 rgb = vector.rgb; 4 mat4 m = mat4(vector); 5 vec4 ans = vector * m; 6 7 float delta = 0.3; 8 vec4 from = vec4(0); 9 vec4 to = vec4(1); 10 vec4 current = from + (to - from) * delta; 11 vec4 current2 = mix(from, to, delta); 12 GLSL Syntax
  15. GLSL Built-in functions radians degrees sin cos tan asin acos

    atan pow exp log exp2 log2 sqrt inversesqrt abs sign floor ceil fract mod min max clamp mix step smoothstep length distance dot cross normalize faceforward reflect refract matrixCompMult
  16. GLSL & JS Simple interpolation vec4 current = from +

    (to - from) * delta; var current = (to.add(from.scale(-1))).scale(delta).add(from);
  17. vec4 current = from + (to - from) * delta;

    var current = (to.add(from.scale(-1))).scale(delta).add(from); GLSL & JS Simple interpolation
  18. None
  19. None
  20. •The Lambda Papers (Scheme)

  21. •The Lambda Papers (Scheme) •Java (1994), Sun fellow

  22. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39

    (1st Edition)
  23. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39

    (1st Edition) •The C Language (X3J11)
  24. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39

    (1st Edition) •The C Language (X3J11) •Fortran (X3J3)
  25. •The Lambda Papers (Scheme) •Java (1994), Sun fellow •ECMA TC39

    (1st Edition) •The C Language (X3J11) •Fortran (X3J3) •Common Lisp (X3J13)
  26. Growing a language 1998 - OOPSLA

  27. General Rationale Some people want •Complex Number •Rational Number •Interval

    •Vector{2,3,4} •Matrix Should these be added to the language?
  28. Value Objects

  29. Value Objects •Many use-cases (int64, complex, decimal...)

  30. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators

  31. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving

    Boolean Algebra
  32. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving

    Boolean Algebra •Preserving Relational Relations (trichotomy)
  33. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving

    Boolean Algebra •Preserving Relational Relations (trichotomy) •Strict Equality Operators unchanged
  34. Value Objects •Many use-cases (int64, complex, decimal...) •Overloadable Operators •Preserving

    Boolean Algebra •Preserving Relational Relations (trichotomy) •Strict Equality Operators unchanged •Literal Syntax
  35. Value Objects But how did we get here?

  36. Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012

    Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Operator Overloading
  37. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic FunctionsDouble Dispatch Revisited Value Types Value Proxies Value Objects ES4 draft 1 class Complex! { ... } 2 3 generic intrinsic function +( a: Complex, b: Complex ) 4 new Complex( a.real + b.real, a.imag + b.imag ) 5 6 generic intrinsic function +( a: Complex, b: AnyNumber ) 7 a + Complex(b) 8 9 generic intrinsic function +( a: AnyNumber, b: Complex ) 10 Complex(a) + b
  38. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic FunctionsDouble Dispatch Revisited Value Types Value Proxies Value Objects Dynamic Dispatch - How does it work? 1 class Person { 2 public void eat (Food f) { println ("eating food"); } 3 public void eat(Pasta p) { println ("eating pasta"); } 4 } 5 /*...*/ 6 Food food = new Pasta(); 7 somePerson.eat(food); Java does not have dynamic dispatch in its method arguments
  39. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects But what happens with “instances”? 1 class John extends Person { 2 public void eat(Food f) { println ("mmm...."); } 3 public void eat(Pasta p) { println ("Yummy!"); } 4 } 5 6 Person me = new John(); 7 Food pasta = new Pasta(); 8 me.eat(pasta); Dynamic Dispatch - How does it work?
  40. Generic Functions Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Dynamic Dispatch Java: First arg. is dynamically dispatched 1 public void function eat(John john, Food f) { ... } 2 public void function eat(John john, Pasta p) { ... } 3 4 public void function eat(Person person, Food f) { ... } 5 public void function eat(Person person, Pasta p) { ... }
  41. Generic Functions Dynamic Dispatch CLOS: All args. are dynamically dispatched

    1 (defmethod eat ((john John) (f Food)) ... ) 2 (defmethod eat ((john John) (p Pasta)) ... ) 3 4 (defmethod eat ((person Person) (f Food)) ... ) 5 (defmethod eat ((person Person) (p Pasta)) ... ) Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects
  42. Double Dispatch Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Same principle as Smalltalk 1 Number.prototype['+'] = function(arg) { 2 if (isNumber(arg)) { 3 return primAdd(this, arg); 4 } else { 5 return arg['reverse+'](this); 6 } 7 }; 8 9 Number.prototype['reverse+'] = function(receiver) { 10 if (isNumber(receiver)) { 11 return primAdd(receiver, this); 12 } else { 13 throw ...; 14 } 15 }; 16
  43. Double Dispatch Was proposed before based on ExtendScript 1 Array.prototype['*']

    = function(k, rev) { 2 if( k.constructor != Number ) return; 3 4 var i = this.length, 5 r = this.concat(); 6 7 while( i-- ) r[i] *= k; 8 return r; 9 }; 10 11 // Sample code: 12 var a = [1,2,3]; 13 alert( a * 5 ); // [5,10,15] 14 // (reversed == true) 15 alert( 5 * a ); // [5,10,15] (works also!) Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects
  44. Double Dispatch Problems on extensibility 1 Point.prototype['+'] = function (that)

    { 2 if (that instanceof Point) { 3 return new Point(this.x + that.x, this.y + that.y); 4 } else if (that instanceof Number) { 6 return new Point(this.x + that, this.y + that); 7 } else if (...) { 8 //x 10000 9 } else { 10 return that['reverse+'](this); 11 } 12 }; Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010 Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects
  45. OO Revisited Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects 1 function pointPlusNumber(a, b) { 2 return new Point(a.x + b, a.y + b); 3 } 4 5 Function.defineOperator('+', Point, Number, pointPlusNumber); 6 7 function numberPlusPoint(a, b) { 8 return new Point(a + b.x, a + b.y); 9 } 10 11 Function.defineOperator('+', Number, Point, numberPlusPoint); 12 13 function addPoints(a, b) { 14 return new Point(a.x + b.x, a.y + b.y); 15 } 16 17 Function.defineOperator('+', Point, Point, addPoints);
  46. Value Types Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects “Value types would be objects that behave like primitives such as number and string: they are immutable or appear to be (...) In order to be usable, value types must "overload" (...) operator syntax; they also need to extend literal syntax (e.g. 1.1m for decimal 1.1).”
  47. Value Proxies Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects Proxies 1 var p = new Proxy(target, handler); delete proxy.name deleteProperty Object.freeze(proxy) freeze name in proxy has proxy.name get proxy.name = val set for (prop in proxy) enumerate proxy.apply(thisValue, args) apply new proxy(...args) construct
  48. Value Objects Oct. 2007 Jan. 2009 Jun. 2009 Jan. 2010

    Mar. 2012 Dec. 2010 Generic Functions Double Dispatch Revisited Value Types Value Proxies Value Objects ...are not user-defined
  49. After 7 years still no consensus on how users can

    perform operator overloading
  50. None
  51. Thanks @philogb http://philogb.github.io/

  52. # Bibliography ## Why operator overloading http://www.jroller.com/cpurdy/entry/the_seven_habits_of_highly1 ## ES4 overview

    http://www.ecmascript.org/es4/spec/overview.pdf ## Generic functions https://mail.mozilla.org/pipermail/es-discuss/2007-November/005043.html https://mail.mozilla.org/pipermail/es-discuss/2007-November/005049.html https://mail.mozilla.org/pipermail/es-discuss/2007-November/005052.html https://mail.mozilla.org/pipermail/es-discuss/2007-November/005053.html ## 2009 Operator Overloading proposal https://mail.mozilla.org/pipermail/es-discuss/2009-January/008535.html https://mail.mozilla.org/pipermail/es-discuss/2009-January/008541.html http://wiki.ecmascript.org/doku.php?id=strawman:operator_overloading_with_double_dispatch ## ExtendScript OO http://forums.adobe.com/thread/646268 http://www.indiscripts.com/post/2010/05/operator-overloading-with-extendscript ## Operator Overloading revisited https://mail.mozilla.org/pipermail/es-discuss/2009-June/009603.html https://mail.mozilla.org/pipermail/es-discuss/2009-June/009604.html https://mail.mozilla.org/pipermail/es-discuss/2009-June/ ## Value types http://wiki.ecmascript.org/doku.php?id=strawman:value_types Decimal type: http://intertwingly.net/stories/2008/09/20/estest.html https://mail.mozilla.org/pipermail/es-discuss/2009-January/008646.html Value types page started: https://mail.mozilla.org/pipermail/es-discuss/2010-January/010677.html ## Value objects http://wiki.ecmascript.org/doku.php?id=strawman:value_objects https://mail.mozilla.org/pipermail/es-discuss/2012-March/021411.html http://www.slideshare.net/BrendanEich/value-objects https://bugzilla.mozilla.org/show_bug.cgi?id=749786 ## Value proxies http://wiki.ecmascript.org/doku.php?id=strawman:value_proxies ## ES next https://mail.mozilla.org/pipermail/es-discuss/2013-April/029949.html