Creating own language made easy

Dd3f18c87b851137000c7427d7bd5d32?s=47 fwdays
April 29, 2014

Creating own language made easy

Ингвар Степанян

Dd3f18c87b851137000c7427d7bd5d32?s=128

fwdays

April 29, 2014
Tweet

Transcript

  1. Creating own language made easy Ingvar Stepanyan Universal Mind case

    ‘email’: return ‘me@rreverser.com’; default: return ‘RReverser’;
  2. Everything sucks

  3. Everything sucks – human version

  4. Everything sucks – developer version

  5. Scary magic ? ?? ???

  6. None
  7. Not so scary magic JS JS JS Parser Generator

  8. Parsers This is not code you are looking for

  9. Parser generators • jison Bison in javascript, used by Coffeescript

    • PEG.js parser generator for JavaScript based on the parsing expression grammar formalism • JS/CC LALR(1) parser generator • OMeta/JS (source) metacompiler for many languages to many targets, including js. • languagejs - PEG parser written in JavaScript with first class errors • Canopy Self-hosting PEG parser compiler for JavaScript, influenced by Ruby parser generators • jsparse PEG by Grandmaster Chris Double • ReParse parser combinator library for Javascript like Haskell's Parsec • p4js Monadic parser library for JavaScript • JSGLR Scannerless, Generalized Left-to-right Rightmost (SGLR) derivation parser for JavaScript • antlr has a javascript target • Cruiser.Parse LL(k) parser • ...
  10. Parser generators PEG (PEG.js) LALR(1) (Jison) start = additive additive

    = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  11. Choice ordering PEG (PEG.js) LALR(1) (Jison) start = additive additive

    = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  12. Ambiguity PEG (PEG.js) LALR(1) (Jison) start = additive additive =

    left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ; if a then (if b then s) else s2 or if a then (if b then s else s2) if a then if b then s else s2:
  13. Left recursion: 1-2-3 PEG (PEG.js) LALR(1) (Jison) [ "x", "=",

    [ "1", "-", [ "2", "-", "3" ] ] ] [ "x", "=", [ [ "1", "-", "2" ], "-", "3" ] ]
  14. Left recursion: 1-2-3 PEG (PEG.js) LALR(1) (Jison) start = additive

    additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  15. Summary choice PEG (PEG.js) LALR(1) (Jison) start = additive additive

    = left:multiplicative "+" right:additive { return left + right; } / multiplicative multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %left "+" %left "*" %start program; %% program : expression { return $$ } ; expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  16. Jison Bison in JavaScript

  17. Jison syntax: Helpers %{ var scope = {}; %} …

  18. Jison syntax: Lexer … %lex %% \s+ /* skip whitespace

    */ [A-Za-z_]\w+ return 'ID'; \d+ return ‘NUMBER’; [+*;=] return yytext; <<EOF>> return 'EOF'; /lex …
  19. Jison syntax: Operator prededence … %left ';‘ %right ‘=‘ %left

    ‘+’ %left ‘*’ … /* “x=a*b+c” -> assign(“x”, “a*b+c”) -> assign(“x”, add(“a*b”, “c”)) -> assign(“x”, add(mul(“a”, “b”), “c”)) */
  20. Jison syntax: Parsing rules %start program program : stmt* EOF

    { return $1 } ; stmt : expr ‘;’ -> $1 ; expr : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | NUMBER -> $1 ;
  21. Code generation

  22. Methods • string concatenation • building AST object + escodegen

    (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)
  23. Debugging: source maps

  24. Methods • string concatenation • building AST object + escodegen

    (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)
  25. AST way

  26. Methods • string concatenation • building AST object + escodegen

    (http://github.com/Constellation/escodegen) • using SourceNode from source-map (https://github.com/mozilla/source-map)
  27. new SourceNode(line, column, filename, jsChunk) // line, column – position

    in original file // filename – name of original file // jsChunk – JavaScript code string, another SourceNode // instance or array of those SourceNode
  28. Resulting stack JS JS JS Jison source-map

  29. Live demo