Pro Yearly is on sale from $80 to $50! »

Let’s create a programming language! [RUG::B edition]

Let’s create a programming language! [RUG::B edition]

Be732ee41fd3038aa98a0a7e7b7be081?s=128

Denis Defreyne

June 02, 2016
Tweet

Transcript

  1. Let’s create a programming
 language! DENIS DEFREYNE RUG : :B

    JUNE 2, 2016
  2. 2 dynamic vs. static

  3. 3 dynamic vs. static

  4. 4 compiler interpreter vs.

  5. 5 compiler interpreter vs.

  6. but why? 6

  7. incremental 7

  8. Language 1 8 Numbers

  9. % 9 cat stuff.cke 4173 5 27 2016

  10. % 10 cat stuff.cke | ./clarke 4173 5 27 2016

  11. 11 module Grammar extend DParse::DSL DIGIT = char_in('0'..'9') PROGRAM =

    DIGIT end
  12. 12 res = Grammar::PROGRAM.apply($stdin.read) 
 case res when DParse::Success handle_success(res)

    when DParse::Failure handle_failure(res) end
  13. 13 def handle_success(success) p success.data end

  14. 14 DIGIT = char_in('0'..'9') 
 
 
 
 
 


  15. 15 DIGIT = … NUMBER = repeat1(DIGIT) 
 
 


  16. 16 DIGIT = … NUMBER = repeat1(DIGIT) .capture 
 


    
 

  17. 17 DIGIT = … NUMBER = repeat1(DIGIT) .capture .map {

    |d| d.to_i } 
 
 
 

  18. 18 DIGIT = … NUMBER = … EXPRESSION = NUMBER

    
 
 
 
 

  19. 19 DIGIT = … NUMBER = … EXPRESSION = …

    EXPRESSIONS = intersperse( EXPRESSION, char("\n").ignore, ).compact
  20. 20 DIGIT = … NUMBER = …
 EXPRESSION = …

    EXPRESSIONS = … PROGRAM = seq(EXPRESSIONS, eof) .first
 

  21. % 21 cat stuff.cke 4173 5 27 2016

  22. % 22 cat stuff.cke | ./clarke [4173, 5, 27, 2016]

  23. 23 def handle_success(success) p success.data end

  24. 24 def handle_success(success) success.data.each do |expr| p eval_expr(expr) end end

  25. 25 def eval_expr(expr) expr end

  26. % 26 cat stuff.cke | ./clarke 4173 5 27 2016

  27. % 27 echo "aa" | ./clarke expected digit at line

    1, column 1 aa ↑
  28. Language 2 28 Functions

  29. % 29 cat stuff.cke 4173 add(2, 5)

  30. % 30 cat stuff.cke | ./clarke 4173 7

  31. 31 LETTER = char_in('a'..'z') FUNCTION_NAME = repeat1(LETTER).capture

  32. 32 FUNCTION_CALL = seq( FUNCTION_NAME, char('('), intersperse( lazy { EXPRESSION

    }, char(','), ), char(')'), )
  33. 33 FUNCTION_CALL = seq( FUNCTION_NAME, char('('), intersperse( lazy { EXPRESSION

    }, char(',').ignore, ).compact, char(')'), )
  34. 34 FUNCTION_CALL = seq( FUNCTION_NAME, char('(').ignore, intersperse( lazy { EXPRESSION

    }, char(',').ignore, ).compact, char(')').ignore, ).compact
  35. 35 EXPRESSION = NUMBER

  36. 36 EXPRESSION = alt(NUMBER, FUNCTION_CALL)

  37. % 37 cat stuff.cke 4173 add(2, 5)

  38. % 38 cat stuff.cke | ./clarke 4173 ["add", [2, 5]]

  39. 39 def eval_expr(expr) expr end

  40. 40 def eval_expr(expr) case expr when Integer expr when Array

    eval_function_call(expr) end end
  41. 41 def eval_function_call(expr) case expr[0] when 'add' expr[1] .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end
  42. % 42 cat stuff.cke 4173 add(2, 5)

  43. % 43 cat stuff.cke | ./clarke 4173 7

  44. Ugh, we forgot about whitespace. 44

  45. Cleanup 1 45 Abstract syntax

  46. 46 class IntNode < Struct.new(:value) end class FunCallNode < Struct.new(:name,

    :arguments) end
  47. 47 DIGIT = … NUMBER = repeat1(DIGIT) .capture .map {

    |d| d.to_i } 
 
 
 

  48. 48 DIGIT = … NUMBER = repeat1(DIGIT) .capture .map {

    |d| IntNode.new(d.to_i) } 
 
 
 

  49. 49 FUNCTION_CALL = seq( FUNCTION_NAME, char('(').ignore, intersperse(…), char(')').ignore, ).compact

  50. 50 FUNCTION_CALL = seq( FUNCTION_NAME, char('(').ignore, intersperse(…), char(')').ignore, ).compact.map do

    |data| FunCallNode.new(data[0], data[1]) end
  51. 51 def eval_expr(expr) case expr when Integer expr when Array

    eval_function_call(expr) end end
  52. 52 def eval_expr(expr) case expr when IntNode expr.value when FunCallNode

    eval_function_call(expr) end end
  53. 53 def eval_function_call(expr) case expr[0] when 'add' expr[1] .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end
  54. 54 def eval_function_call(expr) case expr.name when 'add' expr.args .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end
  55. Cleanup 2 55 Environment

  56. 56 class FunDef < Struct.new(:args, :body) end

  57. 57 $env = {} $env['add'] = FunDef.new( ['a', 'b'], ->(a,

    b) { a + b }, )
  58. 58 def eval_function_call(expr) case expr.name when 'add' expr.args .map {

    |e| eval_expr(e) } .reduce(:+) when 'mul' … end end
  59. 59 def eval_function_call(expr) fun = $env[expr.name] values = expr.args.map {

    |e| eval_expr(e) } fun.body.call(*values) end
  60. … with error handling, of course. 60

  61. Language 3 61 Variables

  62. % 62 cat stuff.cke 4173 add(2,amount)

  63. % 63 cat stuff.cke | ./clarke 4173 7

  64. 64 class VarNode < Struct.new(:name) end

  65. 65 VARIABLE_NAME = repeat1(LETTER).capture

  66. 66 VARIABLE_REFERENCE = VARIABLE_NAME .map { |d| VarNode.new(d) }

  67. 67 EXPRESSION = alt(
 NUMBER,
 FUNCTION_CALL,
 )


  68. 68 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, )

  69. 69 def eval_expr(expr) case expr when IntNode expr.value when FunCallNode

    eval_function_call(expr) end end
 

  70. 70 def eval_expr(expr) case expr when IntNode expr.value when FunCallNode

    eval_function_call(expr) when VarNode eval_var(expr) end end
  71. 71 def eval_var(expr) $env[expr.name] end

  72. 72 $env['amount'] = 5

  73. Language 3.1 73 Variable assignment

  74. % 74 cat stuff.cke 4173 amount=5 add(2,amount)

  75. % 75 cat stuff.cke | ./clarke 4173 7

  76. 76 class AssignNode < Struct.new(:name, :rhs) end

  77. 77 ASSIGNMENT = seq( VARIABLE_NAME, char('='), lazy { EXPRESSION },

    )
  78. 78 ASSIGNMENT = seq( VARIABLE_NAME, char('=').ignore, lazy { EXPRESSION },

    ).compact
  79. 79 ASSIGNMENT = seq( VARIABLE_NAME, char('=').ignore, lazy { EXPRESSION },

    ).compact.map do |data| AssignNode.new(data[0], data[1]) end
  80. 80 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, )


  81. 81 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, ASSIGNMENT, )

  82. 82 def eval_expr(expr) case expr when … … when AssignNode

    eval_assign(expr) end end
  83. 83 def eval_assign(expr) $env[expr.name] = eval_expr(expr.rhs) end

  84. Cleanup 3 84 Print function

  85. 85 def handle_success(success) success.data.each do |expr| p eval_expr(expr) end end

  86. 86 def handle_success(success) success.data.each do |expr| eval_expr(expr) end end

  87. 87 $env['print'] = FunDef.new( ['a'], ->(a) { p a },

    )
  88. % 88 cat stuff.cke 4173 amount=5 print(add(2,amount))

  89. % 89 cat stuff.cke | ./clarke 7

  90. Cleanup 4 90 Immutable env

  91. 91 def eval_expr(expr) case expr when … … when AssignNode

    eval_assign(expr) end end
  92. 92 def eval_expr(expr, env) case expr when … … when

    AssignNode eval_assign(expr, env) end end
  93. 93 def eval_assign(expr) $env[expr.name] = eval_expr(expr.rhs) end
 


  94. 94 def eval_assign(expr, env) $env[expr.name] = eval_expr(expr.rhs) end
 


  95. 95 def eval_assign(expr, env) val = eval_expr(expr.rhs) new_env = env.merge(expr.name

    => val) ValAndEnv.new(val, new_env) end
  96. 96 class ValAndEnv < Struct.new(:val, :env) end

  97. 97 def handle_success(success) success.data.each do |expr| eval_expr(expr) end end

  98. 98 def handle_success(success) init = ValAndEv.new(0, INITIAL_ENV) 
 
 


  99. 99 def handle_success(success) init = ValAndEv.new(0, INITIAL_ENV) success.data.reduce(init) do |result,

    e| eval_expr(e, result.env) end end
  100. 100 INITIAL_ENV = { 'add' => FunDef.new( ['a', 'b'], ->(a,

    b) { a + b }, ) }
  101. Language 4 101 Scopes

  102. % 102 cat stuff.cke 4173 amount = { a =

    2 b = 3 add(a, b) } print(add(2, amount))
  103. % 103 cat stuff.cke | ./clarke 7

  104. 104 class ScopeNode < Struct.new(:exprs) end

  105. 105 SCOPE = seq( char('{'), repeat1(lazy { EXPRESSION }), char('}'),

    )
  106. 106 SCOPE = seq( char('{').ignore, repeat1(lazy { EXPRESSION }), char('}').ignore,

    ).compact
  107. 107 SCOPE = seq( char('{').ignore, repeat1(lazy { EXPRESSION }), char('}').ignore,

    ).compact.map do |data| ScopeNode.new(data[0]) end
  108. 108 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, ASSIGNMENT, )


  109. 109 EXPRESSION = alt( NUMBER, FUNCTION_CALL, VARIABLE_REFERENCE, ASSIGNMENT, SCOPE, )

  110. 110 def eval_expr(expr, env) case expr when … … when

    ScopeNode eval_scope(expr, env) end end
  111. 111 def eval_scope(expr, env) init = ValAndEnv.new(0, env) result =

    expr.exprs.reduce(init) do |result, e| eval_expr(e, result.env) end ValAndEnv.new(result.val, env) end
  112. 112 def eval_scope(expr, env) init = ValAndEnv.new(0, env) result =

    expr.exprs.reduce(init) do |result, e| eval_expr(e, result.env) end ValAndEnv.new(result.val, env) end
  113. % 113 cat stuff.cke 4173 amount = { a =

    2 b = 3 add(a, b) } print(add(2, amount))
  114. Language 5 114 Conditionals

  115. % 115 cat stuff.cke if (0) { print(100) } else

    { print(200) }
  116. % 116 cat stuff.cke | ./clarke 200

  117. SKIP 117

  118. Language 6 118 Function definitions

  119. % 119 cat stuff.cke fun multiply(a, b) { if (b)

    { add(a, multiply(a, sub(b, 1))) } else { 0 } } print(multiply(3, 5))
  120. 120 class FunDefNode < Struct.new( :name, :argument_names, :body) end

  121. 121 FUNCTION_DEF = seq( string('fun'), FUNCTION_NAME, char('('), intersperse( VARIABLE_NAME, char(','),

    ), char(')'), SCOPE, )
 

  122. 122 FUNCTION_DEF = seq( string('fun').ignore, FUNCTION_NAME, char('(').ignore, intersperse( VARIABLE_NAME, char(',').ignore,

    ).compact, char(')').ignore, SCOPE, ).compact
 

  123. 123 FUNCTION_DEF = seq( string('fun').ignore, FUNCTION_NAME, char('(').ignore, intersperse( VARIABLE_NAME, char(',').ignore,

    ).compact, char(')').ignore, SCOPE, ).compact.map do |data| FunDefNode.new(data[0], data[1], data[2]) end
  124. 124 class FunDef < Struct.new(:argument_names, :body) end

  125. 125

  126. 125 def eval_function_def(expr, env)

  127. 125 def eval_function_def(expr, env) fun_def = FunDef.new(
 expr.argument_names, expr.body)


  128. 125 def eval_function_def(expr, env) fun_def = FunDef.new(
 expr.argument_names, expr.body)
 ValAndEnv.new(


    fun_def,
 env.merge(expr.name => fun_def),
 )
 end
  129. 126 def eval_function_call(expr, env) fun = env[expr.name] values = expr.args.map

    { |e| eval_expr(e) } 
 
 
 
 fun.body.call(*values)
  130. 127 def eval_function_call(expr, env) fun = env[expr.name] values = expr.args.map

    { |e| eval_expr(e) } case fun.body when Proc when Scope … fun.body.call(*values)
  131. 127 def eval_function_call(expr, env) fun = env[expr.name] values = expr.args.map

    { |e| eval_expr(e) } case fun.body when Proc when Scope … fun.body.call(*values)
  132. 128

  133. 128 when Scope
 new_env = env.merge(
 Hash[fun.argument_names.zip(values)])


  134. 128 when Scope
 new_env = env.merge(
 Hash[fun.argument_names.zip(values)])
 eval_scope(fun.body, new_env)
 end

  135. % 129 cat stuff.cke fun multiply(a, b) { if (b)

    { add(a, multiply(a, sub(b, 1))) } else { 0 } } print(multiply(3, 5))
  136. Language 7 130 Operators

  137. % 131 cat stuff.cke fun multiply(a, b) { if (b)

    { a + multiply(a, b - 1) } else { 0 } } print(multiply(3, 5))
  138. 132 10 - 2 - 5 * 2 + 2

    ^ 3 ^ 4
  139. SKIP 133

  140. etc.

  141. Types! Loops! Closures! Objects!
 Classes! Inheritance! Tuples! Records! Typechecking! Multimethods!

    Enumerations! Pattern matching! Currying! Modules! 135
  142. 136

  143. 136 L1 Numbers

  144. 136 L1 Numbers L2 Functions

  145. 136 L1 Numbers L2 Functions C1 Abstract syntax

  146. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

  147. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

    L3 Variables
  148. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

    L3 Variables C3 Print function
  149. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

    L3 Variables C3 Print function C4 Immutable environment
  150. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

    L3 Variables C3 Print function C4 Immutable environment L4 Scopes
  151. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

    L3 Variables C3 Print function C4 Immutable environment L4 Scopes L5 Conditionals
  152. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

    L3 Variables C3 Print function C4 Immutable environment L4 Scopes L5 Conditionals L6 Function definitions
  153. 136 L1 Numbers L2 Functions C1 Abstract syntax C2 Environment

    L3 Variables C3 Print function C4 Immutable environment L4 Scopes L5 Conditionals L6 Function definitions L7 Operators
  154. 137 github.com/ddfreyne/clarke

  155. 137 github.com/ddfreyne/clarke SOON

  156. 138 Ready to interpret your questions. DENIS@STONESHIP.ORG @DDFREYNE