Creating own language made easy

Ингвар Степанян

Transcript

1. Creating own language made easy Ingvar Stepanyan Universal Mind case ‘email’: return ‘me@rreverser.com’; default: return ‘RReverser’;
2. Everything sucks
3. Everything sucks – human version
4. Everything sucks – developer version
5. Scary magic ? ?? ???
7. Not so scary magic JS JS JS Parser Generator
8. Parsers This is not code you are looking for
9. Parser generators • jison Bison in javascript, used by Coffeescript • PEG.js parser generator for JavaScript based on the parsing expression grammar formalism • JS/CC LALR(1) parser generator • OMeta/JS (source) metacompiler for many languages to many targets, including js. • languagejs - PEG parser written in JavaScript with first class errors • Canopy Self-hosting PEG parser compiler for JavaScript, influenced by Ruby parser generators • jsparse PEG by Grandmaster Chris Double • ReParse parser combinator library for Javascript like Haskell's Parsec • p4js Monadic parser library for JavaScript • JSGLR Scannerless, Generalized Left-to-right Rightmost (SGLR) derivation parser for JavaScript • antlr has a javascript target • Cruiser.Parse LL(k) parser • ...
10. Parser generators PEG (PEG.js) LALR(1) (Jison) start = additive additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
11. Choice ordering PEG (PEG.js) LALR(1) (Jison) start = additive additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
12. Ambiguity PEG (PEG.js) LALR(1) (Jison) start = additive additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ; if a then (if b then s) else s2 or if a then (if b then s else s2) if a then if b then s else s2:
13. Left recursion: 1-2-3 PEG (PEG.js) LALR(1) (Jison) [ "x", "=", [ "1", "-", [ "2", "-", "3" ] ] ] [ "x", "=", [ [ "1", "-", "2" ], "-", "3" ] ]
14. Left recursion: 1-2-3 PEG (PEG.js) LALR(1) (Jison) start = additive additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
15. Summary choice PEG (PEG.js) LALR(1) (Jison) start = additive additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
16. Jison Bison in JavaScript
17. Jison syntax: Helpers %{ var scope = {}; %} …
18. Jison syntax: Lexer … %lex %% \s+ /* skip whitespace */ [A-Za-z_]\w+ return 'ID'; \d+ return ‘NUMBER’; [+*;=] return yytext; <<EOF>> return 'EOF'; /lex …
19. Jison syntax: Operator prededence … %left ';‘ %right ‘=‘ %left ‘+’ %left ‘*’ … /* “x=a*b+c” -> assign(“x”, “a*b+c”) -> assign(“x”, add(“a*b”, “c”)) -> assign(“x”, add(mul(“a”, “b”), “c”)) */
20. Jison syntax: Parsing rules %start program program : stmt* EOF { return $1 } ; stmt : expr ‘;’ -> $1 ; expr : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | NUMBER -> $1 ;
21. Code generation
22. Methods • string concatenation • building AST object + escodegen (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)
23. Debugging: source maps
24. Methods • string concatenation • building AST object + escodegen (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)
25. AST way
26. Methods • string concatenation • building AST object + escodegen (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)
27. new SourceNode(line, column, filename, jsChunk) // line, column – position in original file // filename – name of original file // jsChunk – JavaScript code string, another SourceNode // instance or array of those SourceNode
28. Resulting stack JS JS JS Jison source-map
29. Live demo