Tweet
Tweet

Slide 1

Slide 1 text

Implementing a Simple Interpreter Ray Song May 27, 2012 Ray Song Implementing a Simple Interpreter

Slide 2

Slide 2 text

Flow sheet Lexical analysis Grammatical analysis Figure : Flow Ray Song Implementing a Simple Interpreter

Slide 3

Slide 3 text

Flow sheet Lexical analysis Grammatical analysis Figure : Flow Ray Song Implementing a Simple Interpreter

Slide 4

Slide 4 text

Lexical Analysis manual flex Ray Song Implementing a Simple Interpreter

Slide 5

Slide 5 text

Lexical Analysis manual flex Ray Song Implementing a Simple Interpreter

Slide 6

Slide 6 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 7

Slide 7 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 8

Slide 8 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 9

Slide 9 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 10

Slide 10 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 11

Slide 11 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 12

Slide 12 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 13

Slide 13 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 14

Slide 14 text

Tokens identifier integer string while if else print NEWLINE NO WHITESPACE Ray Song Implementing a Simple Interpreter

Slide 15

Slide 15 text

Off-side rule def hello(): print ‘world’ indentation sensitive Ex: ISWIM(1966), occam(1983), Miranda(1985), Haskell(1990), Python(1991) Ray Song Implementing a Simple Interpreter

Slide 16

Slide 16 text

Off-side rule def hello(): print ‘world’ indentation sensitive Ex: ISWIM(1966), occam(1983), Miranda(1985), Haskell(1990), Python(1991) Ray Song Implementing a Simple Interpreter

Slide 17

Slide 17 text

Off-side rule - Cont. represented as virtual tokens INDENT DEDENT Ray Song Implementing a Simple Interpreter

Slide 18

Slide 18 text

Off-side rule - Cont. represented as virtual tokens INDENT DEDENT Ray Song Implementing a Simple Interpreter

Slide 19

Slide 19 text

Off-side rule - Cont. represented as virtual tokens INDENT DEDENT Ray Song Implementing a Simple Interpreter

Slide 20

Slide 20 text

Example if a > 2: print 5 print a IF a > 2 : NEWLINE INDENT PRINT 5 NEWLINE DEDENT PRINT a NEWLINE Ray Song Implementing a Simple Interpreter

Slide 21

Slide 21 text

Grammatical analysis Cocke–Younger–Kasami algorithm Earley’s algorithm LR parser Recursive-descent parser Ray Song Implementing a Simple Interpreter

Slide 22

Slide 22 text

Grammatical analysis Cocke–Younger–Kasami algorithm Earley’s algorithm LR parser Recursive-descent parser Ray Song Implementing a Simple Interpreter

Slide 23

Slide 23 text

Grammatical analysis Cocke–Younger–Kasami algorithm Earley’s algorithm LR parser Recursive-descent parser Ray Song Implementing a Simple Interpreter

Slide 24

Slide 24 text

Grammatical analysis Cocke–Younger–Kasami algorithm Earley’s algorithm LR parser Recursive-descent parser Ray Song Implementing a Simple Interpreter

Slide 25

Slide 25 text

Tools Bison Can generate C, C++ and Java codes ANTLR Ray Song Implementing a Simple Interpreter

Slide 26

Slide 26 text

Tools Bison Can generate C, C++ and Java codes ANTLR Ray Song Implementing a Simple Interpreter

Slide 27

Slide 27 text

Tools Bison Can generate C, C++ and Java codes ANTLR Ray Song Implementing a Simple Interpreter

Slide 28

Slide 28 text

Expression Precedence climbing method Shunting-yard algorithm Parsing expressions in infix notation Output in Reverse Polish notation (RPN) Output in Abstract syntax tree (AST) Operator precedence parser Ray Song Implementing a Simple Interpreter

Slide 29

Slide 29 text

Expression Precedence climbing method Shunting-yard algorithm Parsing expressions in infix notation Output in Reverse Polish notation (RPN) Output in Abstract syntax tree (AST) Operator precedence parser Ray Song Implementing a Simple Interpreter

Slide 30

Slide 30 text

Expression Precedence climbing method Shunting-yard algorithm Parsing expressions in infix notation Output in Reverse Polish notation (RPN) Output in Abstract syntax tree (AST) Operator precedence parser Ray Song Implementing a Simple Interpreter

Slide 31

Slide 31 text

Expression Precedence climbing method Shunting-yard algorithm Parsing expressions in infix notation Output in Reverse Polish notation (RPN) Output in Abstract syntax tree (AST) Operator precedence parser Ray Song Implementing a Simple Interpreter

Slide 32

Slide 32 text

Expression Precedence climbing method Shunting-yard algorithm Parsing expressions in infix notation Output in Reverse Polish notation (RPN) Output in Abstract syntax tree (AST) Operator precedence parser Ray Song Implementing a Simple Interpreter

Slide 33

Slide 33 text

Expression Precedence climbing method Shunting-yard algorithm Parsing expressions in infix notation Output in Reverse Polish notation (RPN) Output in Abstract syntax tree (AST) Operator precedence parser Ray Song Implementing a Simple Interpreter

Slide 34

Slide 34 text

Parser combinator Straightforward to construct Readability Parsec Ray Song Implementing a Simple Interpreter

Slide 35

Slide 35 text

Parser combinator Straightforward to construct Readability Parsec Ray Song Implementing a Simple Interpreter

Slide 36

Slide 36 text

Parser combinator Straightforward to construct Readability Parsec Ray Song Implementing a Simple Interpreter

Slide 37

Slide 37 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 38

Slide 38 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 39

Slide 39 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 40

Slide 40 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 41

Slide 41 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 42

Slide 42 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 43

Slide 43 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 44

Slide 44 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 45

Slide 45 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 46

Slide 46 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 47

Slide 47 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 48

Slide 48 text

Impl Two top-level nonterminals: STMT and EXPR STMT: SIMPLE STMT ‘\n’ | COMPOUND SIMPLE STMT: EXPR SIMPLE STMT: IDENT ‘=’ EXPR SIMPLE STMT: BREAK SIMPLE STMT: print EXPR COMPOUND: if EXPR ’:’ SUITE OPT ELSE COMPOUND: while EXPR ’:’ SUITE SUITE: many1(‘\n’) INDENT many1(STMT) DEDENT SUITE: SIMPLE STMT ‘\n’ OPT ELSE: ELSE ’:’ SUITE OPT ELSE: /* empty */ Ray Song Implementing a Simple Interpreter

Slide 49

Slide 49 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 50

Slide 50 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 51

Slide 51 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 52

Slide 52 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 53

Slide 53 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 54

Slide 54 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 55

Slide 55 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 56

Slide 56 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 57

Slide 57 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 58

Slide 58 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 59

Slide 59 text

Impl - Cont. EXPR: EXPR ‘==’ TERM EXPR: EXPR ’!=’ TERM EXPR: TERM TERM: TERM ‘+’ FACTOR TERM: FACTOR FACTOR: FACTOR ‘*’ ATOM FACTOR: ATOM ATOM: identifier ATOM: literal integer ATOM: literal string ATOM: ‘(’ EXPR ’)’ Ray Song Implementing a Simple Interpreter

Slide 60

Slide 60 text

Example if a > 2: print 5 print a Ray Song Implementing a Simple Interpreter

Slide 61

Slide 61 text

Example - Cont. Figure : Parse Ray Song Implementing a Simple Interpreter

Slide 62

Slide 62 text

Interpreting Abstract syntax tree (AST). Define semantics for each class of nodes object(atom): trivial binary operator BinOP(operator, lhs, rhs, RESULT) :- obj1 = eval(lhs), obj2 = eval(rhs), calc(op, obj1, obj2, RESULT). Object & BinOP inherit from Expr Ray Song Implementing a Simple Interpreter

Slide 63

Slide 63 text

Interpreting Abstract syntax tree (AST). Define semantics for each class of nodes object(atom): trivial binary operator BinOP(operator, lhs, rhs, RESULT) :- obj1 = eval(lhs), obj2 = eval(rhs), calc(op, obj1, obj2, RESULT). Object & BinOP inherit from Expr Ray Song Implementing a Simple Interpreter

Slide 64

Slide 64 text

Interpreting Abstract syntax tree (AST). Define semantics for each class of nodes object(atom): trivial binary operator BinOP(operator, lhs, rhs, RESULT) :- obj1 = eval(lhs), obj2 = eval(rhs), calc(op, obj1, obj2, RESULT). Object & BinOP inherit from Expr Ray Song Implementing a Simple Interpreter

Slide 65

Slide 65 text

Interpreting Abstract syntax tree (AST). Define semantics for each class of nodes object(atom): trivial binary operator BinOP(operator, lhs, rhs, RESULT) :- obj1 = eval(lhs), obj2 = eval(rhs), calc(op, obj1, obj2, RESULT). Object & BinOP inherit from Expr Ray Song Implementing a Simple Interpreter

Slide 66

Slide 66 text

Interpreting Abstract syntax tree (AST). Define semantics for each class of nodes object(atom): trivial binary operator BinOP(operator, lhs, rhs, RESULT) :- obj1 = eval(lhs), obj2 = eval(rhs), calc(op, obj1, obj2, RESULT). Object & BinOP inherit from Expr Ray Song Implementing a Simple Interpreter

Slide 67

Slide 67 text

Subclasses of Stmt - Cont. Assign eval() need a parameter: Binding (which variable holds which object) ExprStmt Print Continue (throwing an exception) Ray Song Implementing a Simple Interpreter

Slide 68

Slide 68 text

Subclasses of Stmt - Cont. Assign eval() need a parameter: Binding (which variable holds which object) ExprStmt Print Continue (throwing an exception) Ray Song Implementing a Simple Interpreter

Slide 69

Slide 69 text

Subclasses of Stmt - Cont. Assign eval() need a parameter: Binding (which variable holds which object) ExprStmt Print Continue (throwing an exception) Ray Song Implementing a Simple Interpreter

Slide 70

Slide 70 text

Subclasses of Stmt - Cont. Assign eval() need a parameter: Binding (which variable holds which object) ExprStmt Print Continue (throwing an exception) Ray Song Implementing a Simple Interpreter

Slide 71

Slide 71 text

Subclasses of Stmt - Cont. Assign eval() need a parameter: Binding (which variable holds which object) ExprStmt Print Continue (throwing an exception) Ray Song Implementing a Simple Interpreter