Creating own language made easy

Transcript

1. Creating own language made easy Ingvar Stepanyan Universal Mind case

   ‘email’: return ‘me@rreverser.com’; default: return ‘RReverser’;

2. Everything sucks

3. Everything sucks – human version

4. Everything sucks – developer version

5. Scary magic ? ?? ???

6. None

7. Not so scary magic JS JS JS Parser Generator

8. Parsers This is not code you are looking for

9. Parser generators • jison Bison in javascript, used by Coffeescript

   • PEG.js parser generator for JavaScript based on the parsing expression grammar formalism • JS/CC LALR(1) parser generator • OMeta/JS (source) metacompiler for many languages to many targets, including js. • languagejs - PEG parser written in JavaScript with first class errors • Canopy Self-hosting PEG parser compiler for JavaScript, influenced by Ruby parser generators • jsparse PEG by Grandmaster Chris Double • ReParse parser combinator library for Javascript like Haskell's Parsec • p4js Monadic parser library for JavaScript • JSGLR Scannerless, Generalized Left-to-right Rightmost (SGLR) derivation parser for JavaScript • antlr has a javascript target • Cruiser.Parse LL(k) parser • ...

10. Parser generators PEG (PEG.js) LALR(1) (Jison) start = additive additive

    = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;

11. Choice ordering PEG (PEG.js) LALR(1) (Jison) start = additive additive

    = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;

12. Ambiguity PEG (PEG.js) LALR(1) (Jison) start = additive additive =

    left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ; if a then (if b then s) else s2 or if a then (if b then s else s2) if a then if b then s else s2:

13. Left recursion: 1-2-3 PEG (PEG.js) LALR(1) (Jison) [ "x", "=",

    [ "1", "-", [ "2", "-", "3" ] ] ] [ "x", "=", [ [ "1", "-", "2" ], "-", "3" ] ]

14. Left recursion: 1-2-3 PEG (PEG.js) LALR(1) (Jison) start = additive

    additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;

15. Summary choice PEG (PEG.js) LALR(1) (Jison) start = additive additive

    = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;

16. Jison Bison in JavaScript

17. Jison syntax: Helpers %{ var scope = {}; %} …

18. Jison syntax: Lexer … %lex %% \s+ /* skip whitespace

    */ [A-Za-z_]\w+ return 'ID'; \d+ return ‘NUMBER’; [+*;=] return yytext; <<EOF>> return 'EOF'; /lex …

19. Jison syntax: Operator prededence … %left ';‘ %right ‘=‘ %left

    ‘+’ %left ‘*’ … /* “x=a*b+c” -> assign(“x”, “a*b+c”) -> assign(“x”, add(“a*b”, “c”)) -> assign(“x”, add(mul(“a”, “b”), “c”)) */

20. Jison syntax: Parsing rules %start program program : stmt* EOF

    { return $1 } ; stmt : expr ‘;’ -> $1 ; expr : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | NUMBER -> $1 ;

21. Code generation

22. Methods • string concatenation • building AST object + escodegen

    (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)

23. Debugging: source maps

24. Methods • string concatenation • building AST object + escodegen

    (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)

25. AST way

26. Methods • string concatenation • building AST object + escodegen

    (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)

27. new SourceNode(line, column, filename, jsChunk) // line, column – position

    in original file // filename – name of original file // jsChunk – JavaScript code string, another SourceNode // instance or array of those SourceNode

28. Resulting stack JS JS JS Jison source-map

29. Live demo