Let’s create a programming language! [RUG::B edition]

Transcript

1. Let’s create a programming
 language! DENIS DEFREYNE RUG : :B

   JUNE 2, 2016

2. 2 dynamic vs. static

3. 3 dynamic vs. static

4. 4 compiler interpreter vs.

5. 5 compiler interpreter vs.

6. but why? 6

7. incremental 7

8. Language 1 8 Numbers

9. % 9 cat stuff.cke 4173 5 27 2016

10. % 10 cat stuff.cke | ./clarke 4173 5 27 2016

11. 11 module Grammar extend DParse::DSL DIGIT = char_in('0'..'9') PROGRAM =

    DIGIT end

12. 12 res = Grammar::PROGRAM.apply($stdin.read) 
 case res when DParse::Success handle_success(res)

    when DParse::Failure handle_failure(res) end

13. 13 def handle_success(success) p success.data end

14. 14 DIGIT = char_in('0'..'9') 
 
 
 
 
 


    


15. 15 DIGIT = … NUMBER = repeat1(DIGIT) 
 
 


    


16. 16 DIGIT = … NUMBER = repeat1(DIGIT) .capture 
 


    
 


17. 17 DIGIT = … NUMBER = repeat1(DIGIT) .capture .map {

    |d| d.to_i } 
 
 
 


18. 18 DIGIT = … NUMBER = … EXPRESSION = NUMBER

    
 
 
 
 


19. 19 DIGIT = … NUMBER = … EXPRESSION = …

    EXPRESSIONS = intersperse( EXPRESSION, char("\n").ignore, ).compact

20. 20 DIGIT = … NUMBER = …
 EXPRESSION = …

    EXPRESSIONS = … PROGRAM = seq(EXPRESSIONS, eof) .first
 


21. % 21 cat stuff.cke 4173 5 27 2016

22. % 22 cat stuff.cke | ./clarke [4173, 5, 27, 2016]

23. 23 def handle_success(success) p success.data end

24. 24 def handle_success(success) success.data.each do |expr| p eval_expr(expr) end end

25. 25 def eval_expr(expr) expr end

26. % 26 cat stuff.cke | ./clarke 4173 5 27 2016

27. % 27 echo "aa" | ./clarke expected digit at line

    1, column 1 aa ↑

28. Language 2 28 Functions

29. % 29 cat stuff.cke 4173 add(2, 5)

30. % 30 cat stuff.cke | ./clarke 4173 7

31. 31 LETTER = char_in('a'..'z') FUNCTION_NAME = repeat1(LETTER).capture

32. 32 FUNCTION_CALL = seq( FUNCTION_NAME, char('('), intersperse( lazy { EXPRESSION

    }, char(','), ), char(')'), )

33. 33 FUNCTION_CALL = seq( FUNCTION_NAME, char('('), intersperse( lazy { EXPRESSION

    }, char(',').ignore, ).compact, char(')'), )

34. 34 FUNCTION_CALL = seq( FUNCTION_NAME, char('(').ignore, intersperse( lazy { EXPRESSION

    }, char(',').ignore, ).compact, char(')').ignore, ).compact

35. 35 EXPRESSION = NUMBER

36. 36 EXPRESSION = alt(NUMBER, FUNCTION_CALL)

37. % 37 cat stuff.cke 4173 add(2, 5)

38. % 38 cat stuff.cke | ./clarke 4173 ["add", [2, 5]]

39. 39 def eval_expr(expr) expr end

40. 40 def eval_expr(expr) case expr when Integer expr when Array

    eval_function_call(expr) end end

41. 41 def eval_function_call(expr) case expr[0] when 'add' expr[1] .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end

42. % 42 cat stuff.cke 4173 add(2, 5)

43. % 43 cat stuff.cke | ./clarke 4173 7

44. Ugh, we forgot about whitespace. 44

45. Cleanup 1 45 Abstract syntax

46. 46 class IntNode < Struct.new(:value) end class FunCallNode < Struct.new(:name,

    :arguments) end

47. 47 DIGIT = … NUMBER = repeat1(DIGIT) .capture .map {

    |d| d.to_i } 
 
 
 


48. 48 DIGIT = … NUMBER = repeat1(DIGIT) .capture .map {

    |d| IntNode.new(d.to_i) } 
 
 
 


49. 49 FUNCTION_CALL = seq( FUNCTION_NAME, char('(').ignore, intersperse(…), char(')').ignore, ).compact

50. 50 FUNCTION_CALL = seq( FUNCTION_NAME, char('(').ignore, intersperse(…), char(')').ignore, ).compact.map do

    |data| FunCallNode.new(data[0], data[1]) end

51. 51 def eval_expr(expr) case expr when Integer expr when Array

    eval_function_call(expr) end end

52. 52 def eval_expr(expr) case expr when IntNode expr.value when FunCallNode

    eval_function_call(expr) end end

53. 53 def eval_function_call(expr) case expr[0] when 'add' expr[1] .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end

54. 54 def eval_function_call(expr) case expr.name when 'add' expr.args .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end

55. Cleanup 2 55 Environment

56. 56 class FunDef < Struct.new(:args, :body) end

57. 57 $env = {} $env['add'] = FunDef.new( ['a', 'b'], ->(a,

    b) { a + b }, )

58. 58 def eval_function_call(expr) case expr.name when 'add' expr.args .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end

59. 59 def eval_function_call(expr) fun = $env[expr.name] values = expr.args.map {

    |e| eval_expr(e) } fun.body.call(*values) end

60. … with error handling, of course. 60

61. Language 3 61 Variables

62. % 62 cat stuff.cke 4173 add(2,amount)

63. % 63 cat stuff.cke | ./clarke 4173 7

64. 64 class VarNode < Struct.new(:name) end

65. 65 VARIABLE_NAME = repeat1(LETTER).capture

66. 66 VARIABLE_REFERENCE = VARIABLE_NAME .map { |d| VarNode.new(d) }

67. 67 EXPRESSION = alt(
 NUMBER,
 FUNCTION_CALL,
 )


68. 68 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, )

69. 69 def eval_expr(expr) case expr when IntNode expr.value when FunCallNode

    eval_function_call(expr) end end
 


70. 70 def eval_expr(expr) case expr when IntNode expr.value when FunCallNode

    eval_function_call(expr) when VarNode eval_var(expr) end end

71. 71 def eval_var(expr) $env[expr.name] end

72. 72 $env['amount'] = 5

73. Language 3.1 73 Variable assignment

74. % 74 cat stuff.cke 4173 amount=5 add(2,amount)

75. % 75 cat stuff.cke | ./clarke 4173 7

76. 76 class AssignNode < Struct.new(:name, :rhs) end

77. 77 ASSIGNMENT = seq( VARIABLE_NAME, char('='), lazy { EXPRESSION },

    )

78. 78 ASSIGNMENT = seq( VARIABLE_NAME, char('=').ignore, lazy { EXPRESSION },

    ).compact

79. 79 ASSIGNMENT = seq( VARIABLE_NAME, char('=').ignore, lazy { EXPRESSION },

    ).compact.map do |data| AssignNode.new(data[0], data[1]) end

80. 80 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, )


81. 81 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, ASSIGNMENT, )

82. 82 def eval_expr(expr) case expr when … … when AssignNode

    eval_assign(expr) end end

83. 83 def eval_assign(expr) $env[expr.name] = eval_expr(expr.rhs) end

84. Cleanup 3 84 Print function

85. 85 def handle_success(success) success.data.each do |expr| p eval_expr(expr) end end

86. 86 def handle_success(success) success.data.each do |expr| eval_expr(expr) end end

87. 87 $env['print'] = FunDef.new( ['a'], ->(a) { p a },

    )

88. % 88 cat stuff.cke 4173 amount=5 print(add(2,amount))

89. % 89 cat stuff.cke | ./clarke 7

90. Cleanup 4 90 Immutable env

91. 91 def eval_expr(expr) case expr when … … when AssignNode

    eval_assign(expr) end end

92. 92 def eval_expr(expr, env) case expr when … … when

    AssignNode eval_assign(expr, env) end end

93. 93 def eval_assign(expr) $env[expr.name] = eval_expr(expr.rhs) end
 


94. 94 def eval_assign(expr, env) $env[expr.name] = eval_expr(expr.rhs) end
 


95. 95 def eval_assign(expr, env) val = eval_expr(expr.rhs) new_env = env.merge(expr.name

    => val) ValAndEnv.new(val, new_env) end

96. 96 class ValAndEnv < Struct.new(:val, :env) end

97. 97 def handle_success(success) success.data.each do |expr| eval_expr(expr) end end

98. 98 def handle_success(success) init = ValAndEv.new(0, INITIAL_ENV) 
 
 


99. 99 def handle_success(success) init = ValAndEv.new(0, INITIAL_ENV) success.data.reduce(init) do |result,

    e| eval_expr(e, result.env) end end

100. 100 INITIAL_ENV = { 'add' => FunDef.new( ['a', 'b'], ->(a,

     b) { a + b }, ) }

101. Language 4 101 Scopes

102. % 102 cat stuff.cke 4173 amount = { a =

     2 b = 3 add(a, b) } print(add(2, amount))

103. % 103 cat stuff.cke | ./clarke 7

104. 104 class ScopeNode < Struct.new(:exprs) end

105. 105 SCOPE = seq( char('{'), repeat1(lazy { EXPRESSION }), char('}'),

     )

106. 106 SCOPE = seq( char('{').ignore, repeat1(lazy { EXPRESSION }), char('}').ignore,

     ).compact

107. 107 SCOPE = seq( char('{').ignore, repeat1(lazy { EXPRESSION }), char('}').ignore,

     ).compact.map do |data| ScopeNode.new(data[0]) end

108. 108 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, ASSIGNMENT, )


109. 109 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, ASSIGNMENT, SCOPE, )

110. 110 def eval_expr(expr, env) case expr when … … when

     ScopeNode eval_scope(expr, env) end end

111. 111 def eval_scope(expr, env) init = ValAndEnv.new(0, env) result =

     expr.exprs.reduce(init) do |result, e| eval_expr(e, result.env) end ValAndEnv.new(result.val, env) end

112. 112 def eval_scope(expr, env) init = ValAndEnv.new(0, env) result =

     expr.exprs.reduce(init) do |result, e| eval_expr(e, result.env) end ValAndEnv.new(result.val, env) end

113. % 113 cat stuff.cke 4173 amount = { a =

     2 b = 3 add(a, b) } print(add(2, amount))

114. Language 5 114 Conditionals

115. % 115 cat stuff.cke if (0) { print(100) } else

     { print(200) }

116. % 116 cat stuff.cke | ./clarke 200

117. SKIP 117

118. Language 6 118 Function definitions

119. % 119 cat stuff.cke fun multiply(a, b) { if (b)

     { add(a, multiply(a, sub(b, 1))) } else { 0 } } print(multiply(3, 5))

120. 120 class FunDefNode < Struct.new( :name, :argument_names, :body) end

121. 121 FUNCTION_DEF = seq( string('fun'), FUNCTION_NAME, char('('), intersperse( VARIABLE_NAME, char(','),

     ), char(')'), SCOPE, )
 


122. 122 FUNCTION_DEF = seq( string('fun').ignore, FUNCTION_NAME, char('(').ignore, intersperse( VARIABLE_NAME, char(',').ignore,

     ).compact, char(')').ignore, SCOPE, ).compact
 


123. 123 FUNCTION_DEF = seq( string('fun').ignore, FUNCTION_NAME, char('(').ignore, intersperse( VARIABLE_NAME, char(',').ignore,

     ).compact, char(')').ignore, SCOPE, ).compact.map do |data| FunDefNode.new(data[0], data[1], data[2]) end

124. 124 class FunDef < Struct.new(:argument_names, :body) end

125. 125

126. 125 def eval_function_def(expr, env)

127. 125 def eval_function_def(expr, env) fun_def = FunDef.new(
 expr.argument_names, expr.body)


128. 125 def eval_function_def(expr, env) fun_def = FunDef.new(
 expr.argument_names, expr.body)
 ValAndEnv.new(


     fun_def,
 env.merge(expr.name => fun_def),
 )
 end

129. 126 def eval_function_call(expr, env) fun = env[expr.name] values = expr.args.map

     { |e| eval_expr(e) } 
 
 
 
 fun.body.call(*values)

130. 127 def eval_function_call(expr, env) fun = env[expr.name] values = expr.args.map

     { |e| eval_expr(e) } case fun.body when Proc when Scope … fun.body.call(*values)

131. 127 def eval_function_call(expr, env) fun = env[expr.name] values = expr.args.map

     { |e| eval_expr(e) } case fun.body when Proc when Scope … fun.body.call(*values)

132. 128

133. 128 when Scope
 new_env = env.merge(
 Hash[fun.argument_names.zip(values)])


134. 128 when Scope
 new_env = env.merge(
 Hash[fun.argument_names.zip(values)])
 eval_scope(fun.body, new_env)
 end

135. % 129 cat stuff.cke fun multiply(a, b) { if (b)

     { add(a, multiply(a, sub(b, 1))) } else { 0 } } print(multiply(3, 5))

136. Language 7 130 Operators

137. % 131 cat stuff.cke fun multiply(a, b) { if (b)

     { a + multiply(a, b - 1) } else { 0 } } print(multiply(3, 5))

138. 132 10 - 2 - 5 * 2 + 2

     ^ 3 ^ 4

139. SKIP 133

140. etc.

141. Types! Loops! Closures! Objects!
 Classes! Inheritance! Tuples! Records! Typechecking! Multimethods!

     Enumerations! Pattern matching! Currying! Modules! 135

142. 136

143. 136 L1 Numbers

144. 136 L1 Numbers L2 Functions

145. 136 L1 Numbers L2 Functions C1 Abstract syntax

146. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

147. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

     L3 Variables

148. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

     L3 Variables C3 Print function

149. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

     L3 Variables C3 Print function C4 Immutable environment

150. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

     L3 Variables C3 Print function C4 Immutable environment L4 Scopes

151. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

     L3 Variables C3 Print function C4 Immutable environment L4 Scopes L5 Conditionals

152. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

     L3 Variables C3 Print function C4 Immutable environment L4 Scopes L5 Conditionals L6 Function definitions

153. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

     L3 Variables C3 Print function C4 Immutable environment L4 Scopes L5 Conditionals L6 Function definitions L7 Operators

154. 137 github.com/ddfreyne/clarke

155. 137 github.com/ddfreyne/clarke SOON

156. 138 Ready to interpret your questions. DENIS@STONESHIP.ORG @DDFREYNE