Writing a Lisp in Python

I gave a roughly 60-minute walkthrough of a Pythonic implementation of a subset of the Lisp programming language. An ancient, powerful beast was awakened.

Transcript

1. Writing a Lisp in Python Brian N. Muhia.
2. About Brian... • Trained in Mathematics. • Python coder since 2009, started out doing number theory and beginner crypto problem sets on Windows. • Lisper since 2009. Really got into it in 2011, got started coding simple text-adventure games.
3. If I'm going too slow, read this: http://norvig.com/lispy.html
4. Purpose of this talk: 1. How to implement languages in general.
5. Purpose of this talk: 2. How to implement a subset of Lisp in Python.
6. Why Lisp? “LISP is worth learning for a different reason — the profound enlightenment experience you will have when you finally get it. That experience will make you a better programmer for the rest of your days, even if you never actually use LISP itself a lot.” - Eric S. Raymond
7. Why Lisp? • A working subset of Lisp can be implemented in ~90 lines of Python. • It has very little syntax, which is good.
8. Syntax & Semantics • Syntax: What a language looks like. • Semantics: What correctly-written expressions in that language mean.
9. 2 + 2 is mathematical syntax.
10. The semantics (meaning) of the expresson “2 + 2” is “4”.
11. But..wait. How does Lisp code even look?
12. Lisp lispy> (define my-variable 50) ...compare... Python >>> my-variable = 50
13. Computing factorials... 5! = 5 x 4 x 3 x 2 x 1 n! = n x (n – 1) x (n – 2) x … x 3 x 2 x 1 1! = 1 0! = 1 (define factorial (number) (if (<= number 1) 1 (* number (factorial (- number 1)))))
14. How is all this computed?
15. Parser >> program = "(begin (define r 3) (* 3.141592653 (* r r)))" >> parse(program) ['begin', ['define', 'r', 3], ['*', 3.141592653, ['*', 'r', 'r']]] >> eval(parse(program)) 28.274333877
16. Program Representation Output Parser Evaluator
17. Let's write some code.
18. A few things... from __future__ import division isa = isinstance Symbol = str
19. Our Interpreter Begins with the beautiful Miss Eva Lu Ator... def eval(x, env=global_env): "Evaluate an expression in an environment. The parameter env is a dictionary."
20. Lisp concepts.
21. Variable reference A symbol is interpreted as a variable name; its value is the variable's value. Example: x, var, my_name...
22. Code: if isa(x, Symbol): return env.find(x)[x]
23. Constant literal Syntax: number Semantics: A number evaluates to itself. Example: 12, 3e+8, 6.042e+23
24. Code: elif not isa(x, list): return x
25. Quotation Syntax: (quote exp) Semantics: Return the exp literally; do not evaluate it. Example: (quote (a b c)) => (a b c)
26. Code elif x[0] == 'quote': (_, exp) = x return exp
27. Conditional • Syntax: (if test If-true if-false) • Semantics: Evaluate test; if true, return if-true. If false, return if-false. Example: (if (< 10 20) (+ 1 1) (+ 3 3)) 2 ⇒
28. Code elif x[0] == 'if': (_, test, if_true, if_false) = x return eval((if_true if eval(test, env) else if_false), env)
29. Assignment • Syntax: (set! var exp) • Semantics: Evaluate exp and assign that value to var, which must have been previously defined. The '!' is part of the name.
30. Code elif x[0] == 'set!': (_, var, exp) = x env.find(var)[var] = eval(exp, env)
31. Definition Syntax: (define var exp) Semantics: Define a new variable and give it the value of evaluating the expression exp. Example: (define r 3) or (define square (lambda (x) (* x x)))
32. Code elif x[0] == 'define': (_, var, exp) = x env[var] = eval(exp, env)
33. Procedure Syntax: (lambda (var...) exp) Semantics: Create a procedure with parameter(s) named var... and the exp as the body. Example: (lambda (r) (* 3.141592653 r r))
34. Code elif x[0] == 'lambda': (_, vars, exp) = x return lambda *args: eval(exp, Env(vars, args, env))
35. Sequencing Syntax: (begin exp...) Semantics: Evaluate each of the expressions, in left-to-right order, and return the final value. Example: (begin (set! x 1) (set! x (+ x 1)) (* x 2)) 4 ⇒
36. Code elif x[0] == 'begin': for exp in x[1:]: val = eval(exp, env) return val
37. Procedure call Syntax: (proc exp...) Semantics: If proc is anything other than the symbols if, set!, define, lambda, begin, or quote then it is treated as a procedure and is evaluated using the same rules defined above. Example: (square 12) 144 ⇒
38. Code else: exps = [eval(exp, env) for exp in x] proc = exps.pop(0) return proc(*exps)
39. Phew!
40. Had enough?
41. An example to explain eval's operation:
42. Consider >> program = "(define area (lambda (r) (* 3.141592653 (* r r)))" >>> parse(program) ['define', 'area', ['lambda', ['r'], ['*', 3.141592653, ['*', 'r', 'r']]]] >>> eval(parse(program)) None
44. Now for some Environments.
45. Mappings (dictionary) of variable names (symbols) to the values (data objects held by them).
46. class Env(dict): "An environment: a dict of {'var':val} pairs, with an outer Env." def __init__(self, parms=(), args=(), outer=None): self.update(zip(parms,args)) self.outer = outer def find(self, var): "Find the innermost Env where var appears." return self if var in self else self.outer.find(var)
47. (define make­account (lambda (balance) (lambda (amt) (begin (set! balance (+ balance amt)) balance)))) (define a1 (make­account 100.00)) (a1 ­20.00)
48. Lexical scoping The process of looking first in inner environments and then in outer ones in order to find bindings to referenced variables.
49. Now to define the global environment.
50. def add_globals(env): "Add some Scheme standard procedures to an environment." import math, operator as op env.update(vars(math)) # sin, sqrt... env.update( {'+':op.add, '-':op.sub, '*':op.mul, '/':op.div, 'not':op.not_, '>':op.gt, '<':op.lt, '>=':op.ge, '<=':op.le, '=':op.eq, 'equal?':op.eq, 'eq?':op.is_, 'length':len, 'cons':lambda x,y:[x]+y, 'first':lambda x:x[0],'rest':lambda x:x[1:], 'append':op.add, 'list':lambda *x:list(x), 'list?': lambda x:isa(x,list), 'null?':lambda x:x==[], 'symbol?':lambda x: isa(x, Symbol)}) return env
51. Parsing... • Lexical analysis... • Syntactic analysis...
52. def tokenize(s): "Convert a string into a list of tokens." return s.replace('(',' ( ').replace(')',' ) ').split() >>> program = "(set! twox (* x 2))" >>> tokenize(program) # lexical analysis ['(', 'set!', 'twox', '(', '*', 'x', '2', ')', ')']
53. def read(s): "Read a Scheme expression from a string." return read_from(tokenize(s)) parse = read >>> parse(program) ['set!', 'twox', ['*', 'x', 2]]
54. def read_from(tokens): "Read an expression from a sequence of tokens." if len(tokens) == 0: raise SyntaxError('unexpected EOF while reading') token = tokens.pop(0) if '(' == token: L = [] while tokens[0] != ')': L.append(read_from(tokens)) tokens.pop(0) # pop off ')' return L elif ')' == token: raise SyntaxError('unexpected )') else: return atom(token)
55. def atom(token): "Numbers become numbers; every other token is a symbol." try: return int(token) except ValueError: try: return float(token) except ValueError: return Symbol(token)
56. def to_string(exp): "Convert a Python object back into a Lisp- readable string." return '(' + ' '.join(map(to_string,\ exp))+')' if isa(exp, list) else str(exp)
57. def repl(prompt='lis.py> '): "A prompt read-eval-print loop." while True: val = eval(parse(raw_input(prompt))) if val is not None: print to_string(val)
58. Now, load up $ python ­i lis.py, run repl(), and enjoy your new Lisp!
59. #TeamLambda
61. Thanks to: Peter Norvig, for permission.