Self-hosting Whitespace

Self-hosting Whitespace

7bc6612fa20296bf652f6b0357db81c1?s=128

pocke

May 24, 2019
Tweet

Transcript

  1. Self-hosting Whitespace 24th, May. 2019 Roppongi.rb#10 https://roppongirb.connpass.com/event/129603/

  2. ⚠ IT NEEDS RUBY TRUNK (2.7) ⚠

  3. Agenda I implemented a self-hosted Whitespace interpreter. I’ll talk about

    the interpreter.
  4. Agenda • Prerequisite knowledge ◦ What is self-hosting? ◦ What

    is Whitespace? • Self-hosted interpreter ◦ How implement the interpreter ◦ Implementation Tips
  5. What is self-hosting?

  6. Self-hosting is... • Implementing a programming language by itself •

    For instance ◦ Go compiler is written in Go. (since v1.5) ◦ minruby ▪ 遠藤侑介 (2017) 『RubyでつくるRuby』ラムダノート.
  7. Example: ruby interpreter on ruby interpreter $ ruby a.rb $

    ruby interp.rb a.rb $ ruby interp.rb interp.rb a.rb $ ruby interp.rb interp.rb interp.rb interp.rb interp.rb a.rb
  8. What is Whitespace?

  9. Do you know Whitespace language? • Never mind if you

    don’t know! https://twitter.com/p_ck_/status/1123410375804768256 https://twitter.com/p_ck_/status/1123414869737263104
  10. Whitespace • Whitespace is an esoteric programming language (esolang) ◦

    Esolang example: Brainf**k, Piet, etc • It consists only of space, tab and newline
  11. hello.ws (Space → S, Tab → T, NewLine → N)

    SSSTSSTSSSNTNSSSSSTTSSTSTNTNSSSSSTTSTTSSNTNSSSS STTSTTSSNTNSSSSSTTSTTTTNTNSSSSSTSSSSSNTNSSSSSTS TSTTTNTNSSSSSTTSTTTTNTNSSSSSTTTSSTSNTNSSSSSTTS TTSSNTNSSSSSTTSSTSSNTNSSSSSSSSTSTSNTNSSNNN Space -> S, Tab -> T Ref: https://magazine.rubyist.net/articles/0022/0022-Le gwork.html
  12. Inside of Whitespace • It is a stack based language

    • It has stack and heap for data store • All value is a signed Integer • It has goto and subroutine for flow control • It has 22 commands to manipulate stack and so on ◦ All commands are encoded only with whitespace chars
  13. Whitespace Commands (1) Stack commands • stack.push(val) SS • stack.pop()

    SNN • stack.dup() SNS ◦ stack.push(stack.last) • stack.swap() SNT ◦ stack[-1], stack[-2] = stack[-2], stack[-1] Heap commands • heap.save(key, val) TTS ◦ val = stack.pop; key = stack.pop heap[key] = val • heap.load(key) TTT ◦ stack.push heap[key.pop()]
  14. Whitespace Commands (2) Calculations a = stack.pop; b = stack.pop

    stack.push(op(b, a)) • add() TSSS • sub() TSST • multi() TSSN • div() TSTS • mod() TSTT IO • write_char() TNSS • write_number() TNST ◦ print stack.pop() • read_char() TNTS • read_number() TNTT ◦ heap[stack.pop] = read
  15. Whitespace Commands (3) Flow • define(label) NSS ◦ define_method(label) •

    call(label) NST ◦ __send__(label) • end NTN • exit() NNN • jump(label) NSN ◦ goto label • jump_if_zero(label) NTS ◦ jump(label) if stack.pop == 0 • jump_if_negative(label) NTT ◦ jump(label) if stack.pop < 0
  16. Command Example: Stack and IO SSSTSTSTSNTNST stack.push(42) write_number # =>

    42
  17. Command Example: Heap SSSTTNSSSTSTSTSNTTSSSSTTNTTTTNST stack.push(3) stack.push(42) heap.save stack.push(3) heap.load write_number

    # 3 is address # 42 is a value # Save 42 to addr 3 # Load 42 from addr 3 # => 42
  18. Self-hosted Interpreter

  19. The interpreter is here • https://gist.github.com/pocke/2847214a8713 9a0d1babd9338159d667 • 72,367 lines

    • 565,639 bytes
  20. DEMO $ gows fizzbuzz.ws $ cat fizzbuzz.ws sep | gows

    whitespace.ws $ cat whitespace.ws sep fizzbuzz.ws sep | gows whitespace.ws gows is a Whitespace interpreter written in Go. https://github.com/pocke/gows
  21. How implement the interpreter

  22. Did I handwrote the 70,000+ lines Whitespace code? Definitely Not.

  23. How implement Whitespace in Whitespace Handwriting Whitespace is difficult, so

    1. Write a compiler that compiles Ruby to Whitespace 2. Write Whitespace interpreter in Ruby 3. Generate Whitespace from Ruby!
  24. 1. Write a compiler that compiles Ruby to Whitespace

  25. Akaza: A toolchain of Whitespace, written in Ruby • https://github.com/pocke/akaza

    • It has 3 features ◦ Whitespace interpreter ◦ Whitespace compiler: wsrb ← Today’s theme! ◦ Embedded Whitespace in Ruby ▪ https://pocke.hatenablog.com/entry/2019/04/28/183509
  26. wsrb: Whitespace compiler and the language • wsrb language is

    a subset of Ruby • It supports ◦ Local vars, method call and def, Integer, Array, Hash, true, false, nil, if, case, while • It does not support ◦ Most built-in methods, String, class, module, meta programming, and so on
  27. wsrb implementation • https://github.com/pocke/akaza/blob/master/li b/akaza/ruby2ws.rb • It uses RubyVM::AST (since

    Ruby 2.6) and pattern matching (since Ruby 2.7) ◦ Parse Ruby code to AST with RubyVM::AST ◦ Translate the AST to Whitespace with pattern matching
  28. 2. Write Whitespace interpreter in Ruby

  29. Interpreter implementation • Simple parser and VM • https://github.com/pocke/self-hosting-whitesp ace/blob/master/src/whitespace.ws.rb

  30. 3. Generate Whitespace from Ruby

  31. Compile $ akaza wsrb src/whitespace.ws.rb DEMO

  32. Implementation Tips

  33. Value and Class

  34. Value with Class • Wsrb has Integer, Array and Hash

    classes • Wsrb needs to distinguish classes ◦ e.g. `hash[1]` and `array[1]` should call different method • So, each value should have class information
  35. Value with Class • Wsrb uses lower 2 bits for

    types • For example ◦ Value is 42. (42.to_s(2) #=> 101010) ◦ Integer class is 0b01 ◦ In wsrb, the value is 10101001 see: http://i.loveruby.net/ja/rhg/book/object.html
  36. Array

  37. Array • 3 meta data ◦ Address of the first

    item ◦ Size: Same as Array#size ◦ Capacity: Allocated items count (default: 10) ▪ size <= capacity ▪ If cap is too small, it’s re-allocated • Items Addr of First Item Size Capacity Item 1 Item 2 Item 3 ...
  38. Hash

  39. Hash • Use Hash table and linked list • Hash

    table key is `key mod HASH_SIZE` ◦ Default HASH_SIZE: 11 • When conflict, it uses linked list
  40. Hash Addr of FIrst Item Key1 Value1 Addr to next

    key Key2 Value2 Addr to next item ...(3*HASH _SIZE) Key1’ Value1’ Addr to next key Key1’’ Value1’’ Addr to next key
  41. Differences between Go and Ruby

  42. Differences between Go and Ruby • I found the differences

    when I was implementing gows • Division and modulo have differences https://play.golang.org/p/_LGKTmlMAGJ
  43. Division difference • -5 / 3 ◦ Ruby: -2 ◦

    Go: -1 • Absorb the difference by ◦ return int(math.Floor(float64(left) / float64(right)))
  44. Modulo differene • -5 % 3 ◦ Ruby: 1 ◦

    Go: -2 • Absorb the difference by ◦ res := left % right if res < 0 { return j + res } else { return res }
  45. Conclusion

  46. Conclusion • We learned how to implement a programing language

    • Try implementing Whitespace, or other esolangs! • Try self-hosting! • It is not productive, but it is fun! I recommend it
  47. pp self • Masataka “pocke” Kuwabara • Work for Bit

    Journey, Inc. / Kibela • Language: Ruby, Go, TypeScript Whitespace ← New! • github.com/pocke, twitter.com/p_ck_ Thank you for listening!