Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Self-hosting Whitespace

Self-hosting Whitespace

pocke

May 24, 2019
Tweet

More Decks by pocke

Other Decks in Programming

Transcript

  1. Self-hosting Whitespace
    24th, May. 2019
    Roppongi.rb#10
    https://roppongirb.connpass.com/event/129603/

    View Slide

  2. ⚠ IT NEEDS RUBY TRUNK (2.7) ⚠

    View Slide

  3. Agenda
    I implemented a self-hosted Whitespace
    interpreter.
    I’ll talk about the interpreter.

    View Slide

  4. Agenda
    ● Prerequisite knowledge
    ○ What is self-hosting?
    ○ What is Whitespace?
    ● Self-hosted interpreter
    ○ How implement the interpreter
    ○ Implementation Tips

    View Slide

  5. What is self-hosting?

    View Slide

  6. Self-hosting is...
    ● Implementing a programming language by itself
    ● For instance
    ○ Go compiler is written in Go. (since v1.5)
    ○ minruby
    ■ 遠藤侑介 (2017) 『RubyでつくるRuby』ラムダノート.

    View Slide

  7. Example: ruby interpreter on ruby interpreter
    $ ruby a.rb
    $ ruby interp.rb a.rb
    $ ruby interp.rb interp.rb a.rb
    $ ruby interp.rb interp.rb interp.rb interp.rb
    interp.rb a.rb

    View Slide

  8. What is Whitespace?

    View Slide

  9. Do you know Whitespace language?
    ● Never mind if you don’t know!
    https://twitter.com/p_ck_/status/1123410375804768256
    https://twitter.com/p_ck_/status/1123414869737263104

    View Slide

  10. Whitespace
    ● Whitespace is an esoteric programming
    language (esolang)
    ○ Esolang example: Brainf**k, Piet, etc
    ● It consists only of space, tab and newline

    View Slide

  11. hello.ws (Space → S, Tab → T, NewLine → N)
    SSSTSSTSSSNTNSSSSSTTSSTSTNTNSSSSSTTSTTSSNTNSSSS
    STTSTTSSNTNSSSSSTTSTTTTNTNSSSSSTSSSSSNTNSSSSSTS
    TSTTTNTNSSSSSTTSTTTTNTNSSSSSTTTSSTSNTNSSSSSTTS
    TTSSNTNSSSSSTTSSTSSNTNSSSSSSSSTSTSNTNSSNNN
    Space -> S, Tab -> T
    Ref:
    https://magazine.rubyist.net/articles/0022/0022-Le
    gwork.html

    View Slide

  12. Inside of Whitespace
    ● It is a stack based language
    ● It has stack and heap for data store
    ● All value is a signed Integer
    ● It has goto and subroutine for flow control
    ● It has 22 commands to manipulate stack and so
    on
    ○ All commands are encoded only with whitespace chars

    View Slide

  13. Whitespace Commands (1)
    Stack commands
    ● stack.push(val) SS
    ● stack.pop() SNN
    ● stack.dup() SNS
    ○ stack.push(stack.last)
    ● stack.swap() SNT
    ○ stack[-1], stack[-2] =
    stack[-2], stack[-1]
    Heap commands
    ● heap.save(key, val) TTS
    ○ val = stack.pop; key =
    stack.pop
    heap[key] = val
    ● heap.load(key) TTT
    ○ stack.push heap[key.pop()]

    View Slide

  14. Whitespace Commands (2)
    Calculations
    a = stack.pop; b = stack.pop
    stack.push(op(b, a))
    ● add() TSSS
    ● sub() TSST
    ● multi() TSSN
    ● div() TSTS
    ● mod() TSTT
    IO
    ● write_char() TNSS
    ● write_number() TNST
    ○ print stack.pop()
    ● read_char() TNTS
    ● read_number() TNTT
    ○ heap[stack.pop] = read

    View Slide

  15. Whitespace Commands (3)
    Flow
    ● define(label) NSS
    ○ define_method(label)
    ● call(label) NST
    ○ __send__(label)
    ● end NTN
    ● exit() NNN
    ● jump(label) NSN
    ○ goto label
    ● jump_if_zero(label) NTS
    ○ jump(label) if stack.pop == 0
    ● jump_if_negative(label) NTT
    ○ jump(label) if stack.pop < 0

    View Slide

  16. Command Example: Stack and IO
    SSSTSTSTSNTNST
    stack.push(42)
    write_number # => 42

    View Slide

  17. Command Example: Heap
    SSSTTNSSSTSTSTSNTTSSSSTTNTTTTNST
    stack.push(3)
    stack.push(42)
    heap.save
    stack.push(3)
    heap.load
    write_number
    # 3 is address
    # 42 is a value
    # Save 42 to addr 3
    # Load 42 from addr 3
    # => 42

    View Slide

  18. Self-hosted Interpreter

    View Slide

  19. The interpreter is here
    ● https://gist.github.com/pocke/2847214a8713
    9a0d1babd9338159d667
    ● 72,367 lines
    ● 565,639 bytes

    View Slide

  20. DEMO
    $ gows fizzbuzz.ws
    $ cat fizzbuzz.ws sep | gows whitespace.ws
    $ cat whitespace.ws sep fizzbuzz.ws sep | gows
    whitespace.ws
    gows is a Whitespace interpreter written in Go.
    https://github.com/pocke/gows

    View Slide

  21. How implement the interpreter

    View Slide

  22. Did I handwrote the
    70,000+ lines
    Whitespace code?
    Definitely Not.

    View Slide

  23. How implement Whitespace in Whitespace
    Handwriting Whitespace is difficult, so
    1. Write a compiler that compiles Ruby to
    Whitespace
    2. Write Whitespace interpreter in Ruby
    3. Generate Whitespace from Ruby!

    View Slide

  24. 1. Write a compiler that compiles Ruby to
    Whitespace

    View Slide

  25. Akaza: A toolchain of Whitespace, written in Ruby
    ● https://github.com/pocke/akaza
    ● It has 3 features
    ○ Whitespace interpreter
    ○ Whitespace compiler: wsrb ← Today’s theme!
    ○ Embedded Whitespace in Ruby
    ■ https://pocke.hatenablog.com/entry/2019/04/28/183509

    View Slide

  26. wsrb: Whitespace compiler and the language
    ● wsrb language is a subset of Ruby
    ● It supports
    ○ Local vars, method call and def, Integer, Array, Hash,
    true, false, nil, if, case, while
    ● It does not support
    ○ Most built-in methods, String, class, module, meta
    programming, and so on

    View Slide

  27. wsrb implementation
    ● https://github.com/pocke/akaza/blob/master/li
    b/akaza/ruby2ws.rb
    ● It uses RubyVM::AST (since Ruby 2.6) and
    pattern matching (since Ruby 2.7)
    ○ Parse Ruby code to AST with RubyVM::AST
    ○ Translate the AST to Whitespace with pattern matching

    View Slide

  28. 2. Write Whitespace interpreter in Ruby

    View Slide

  29. Interpreter implementation
    ● Simple parser and VM
    ● https://github.com/pocke/self-hosting-whitesp
    ace/blob/master/src/whitespace.ws.rb

    View Slide

  30. 3. Generate Whitespace from Ruby

    View Slide

  31. Compile
    $ akaza wsrb src/whitespace.ws.rb
    DEMO

    View Slide

  32. Implementation Tips

    View Slide

  33. Value and Class

    View Slide

  34. Value with Class
    ● Wsrb has Integer, Array and Hash classes
    ● Wsrb needs to distinguish classes
    ○ e.g. `hash[1]` and `array[1]` should call different method
    ● So, each value should have class information

    View Slide

  35. Value with Class
    ● Wsrb uses lower 2 bits for types
    ● For example
    ○ Value is 42. (42.to_s(2) #=> 101010)
    ○ Integer class is 0b01
    ○ In wsrb, the value is 10101001
    see: http://i.loveruby.net/ja/rhg/book/object.html

    View Slide

  36. Array

    View Slide

  37. Array
    ● 3 meta data
    ○ Address of the first item
    ○ Size: Same as Array#size
    ○ Capacity: Allocated items count (default: 10)
    ■ size <= capacity
    ■ If cap is too small, it’s re-allocated
    ● Items
    Addr of
    First Item
    Size Capacity Item 1 Item 2 Item 3 ...

    View Slide

  38. Hash

    View Slide

  39. Hash
    ● Use Hash table and linked list
    ● Hash table key is `key mod HASH_SIZE`
    ○ Default HASH_SIZE: 11
    ● When conflict, it uses linked list

    View Slide

  40. Hash
    Addr of
    FIrst Item
    Key1 Value1 Addr to
    next key
    Key2 Value2 Addr to
    next item
    ...(3*HASH
    _SIZE)
    Key1’ Value1’ Addr to
    next key
    Key1’’ Value1’’ Addr to
    next key

    View Slide

  41. Differences
    between
    Go and Ruby

    View Slide

  42. Differences between Go and Ruby
    ● I found the differences when I was
    implementing gows
    ● Division and modulo have differences
    https://play.golang.org/p/_LGKTmlMAGJ

    View Slide

  43. Division difference
    ● -5 / 3
    ○ Ruby: -2
    ○ Go: -1
    ● Absorb the difference by
    ○ return int(math.Floor(float64(left) /
    float64(right)))

    View Slide

  44. Modulo differene
    ● -5 % 3
    ○ Ruby: 1
    ○ Go: -2
    ● Absorb the difference by
    ○ res := left % right
    if res < 0 {
    return j + res
    } else { return res }

    View Slide

  45. Conclusion

    View Slide

  46. Conclusion
    ● We learned how to implement a programing
    language
    ● Try implementing Whitespace, or other
    esolangs!
    ● Try self-hosting!
    ● It is not productive, but it is fun! I recommend it

    View Slide

  47. pp self
    ● Masataka “pocke” Kuwabara
    ● Work for Bit Journey, Inc. / Kibela
    ● Language: Ruby, Go, TypeScript
    Whitespace ← New!
    ● github.com/pocke, twitter.com/p_ck_
    Thank you for listening!

    View Slide