Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ErRuby - Ruby on Erlang

3f7d9611fc919c98512b779cde637dfc?s=47 Johnlin
September 08, 2016

ErRuby - Ruby on Erlang

Talk at Rubykaigi 2016.

3f7d9611fc919c98512b779cde637dfc?s=128

Johnlin

September 08, 2016
Tweet

More Decks by Johnlin

Other Decks in Programming

Transcript

  1. ErRuby - Ruby on Erlang/OTP +PIO-JO!KPIOMJOWD

  2. John Lin • @johnlinvc • From Taiwan • Senior Engineer

    At iCHEF
  3. I ❤ programming languages

  4. None
  5. • It’s Ruby-Like Language running on Erlang VM(BEAM). • It’s

    good at concurrency, just like Erlang.
  6. So I wonder • Do we have a Real® Ruby

    on the Erlang VM? Just Like JRuby. • It’ll have all the nice thing of Ruby and great concurrency features of Erlang. • Best of both world.
  7. Was there a Ruby on Erlang VM?

  8. No

  9. None
  10. ErRuby • A Ruby interpreter written in Erlang, running on

    the BEAM VM. • https://github.com/johnlinvc/erruby • Goals are: • Implement major features of Ruby on Erlang VM. • experimenting builtin concurrent features. • exploring issues we’ll encounter when we introduce first class concurrency to Ruby.
  11. Erlang features • Immutable data • Actor Model • Selective

    message receive • Lightweight process
  12. Immutable data • Everything is a constant. • Things like

    “erlang”.capitalize! is impossible in Erlang • Everything will be the same after you assign value to it.
  13. Immutable variable • Each Variable can only get assigned once.

    • Erlang will raise error if an variable get assigned more than once. • The value of variable is defined at one unique place. Which make debugging much easier.
  14. Actor model • A concurrent computation model based on Actor

    and message passing. • Actors shares nothing with each others, only communicate with each other by sending and receiving messages.
  15. Actor • Actor is the minimum computation unit. • Each

    of them has it’s own brain and storage. • Actors can only communicate with each others by sending and receiving messages. • Each actor have one or more addresses to receive messages.
  16. Email system a@foo.com b@baz.com m@maz.com

  17. When receiving message • Actor can send messages to some

    Actors. • Actor can create some more Actors. • Actor can change it’s internal state.
  18. Actor can send messages ruby-core@ruby-lang.org RubyKaigi is
 Awesome

  19. Actor can create actors a@foo.com b@foo.com c@foo.com d@foo.com

  20. Actor can change its state ruby-core-join@ruby-lang.org Join List

  21. Selective receive • Actor can choose what they want to

    process when waiting to receive. • Like me waiting for the CFP result mail from RubyKaigi, I ignore every other mail.
  22. Selective receive ruby-core@ruby-lang.org junkmail junk@non.member

  23. Light weight process • In Erlang each Actor is an

    independent Process. • Actors are the fundamental of Erlang. Just like Object in Ruby. • So spawning new processes need to be effective. • Erlang process are actually M to N green threads that shares nothing with each others.
  24. Erlang is good at concurrency • Immutable data prevents data

    racing problems • Actor model provides great horizontal scaling ability • Lightweight Processes supports the whole concurrent model.
  25. Implementing ErRuby
 in Erlang

  26. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  27. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  28. Parser • Ruby is a complex language. It’s very hard

    to parse it correctly. • Build upon the Parser Gem. • made some minor tweaks for new concurrent features.
  29. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  30. define local var greeting = "hello world" greeting

  31. Abstract syntax tree begin lvasgn :greeting str “hello world” lvar

    :greeting
  32. Conventional stack begin lvasgn :greeting str “hello world” lvar :greeting

    stack_ptr
  33. Conventional stack begin lvasgn :greeting str “hello world” lvar :greeting

    stack_ptr greeting
  34. Creating mutable realm in Erlang • Erlang is an immutable

    language • We couldn’t have a mutable variable for the stack • How can we make an mutable world in an immutable universe?
  35. Continuation passing • The whole ruby running stack is represented

    by a single environment dictionary. • The dictionary is passed as an argument when evaluating an AST Node. • Evaluating an AST Node will return a new dictionary, the new dictionary will be an argument of next evaluation.
  36. Continuation passing AST1 AST2 AST3 Env1 = Eval(AST1, Env0)

  37. Continuation passing AST1 AST2 AST3 Env1 = Eval(AST1, Env0) Env2

    = Eval(AST2, Env1)
  38. Continuation passing AST1 AST2 AST3 Env1 = Eval(AST1, Env0) Env2

    = Eval(AST2, Env1) Env3 = Eval(AST3, Env2)
  39. Contents of environment dictionary • Dictionary of local variable’s name

    to reference • Reference to the self object of current scope • Last expression’s return value • Content of last frame.
  40. ErRuby stack begin lvasgn :greeting str “hello world” lvar :greeting

    eval_ast(begin,
 {
 “lvars” => {},
 “self” => <ref>,
 “ret_val” => nil
 }
 )
  41. begin lvasgn :greeting str “hello world” lvar :greeting eval_ast(lvasgn,
 {


    “lvars” => {},
 “self” => <ref>,
 “ret_val” => nil
 }
 ) ErRuby stack
  42. begin lvasgn :greeting str “hello world” lvar :greeting eval_ast(lvar,
 {


    “lvars” => {
 “greeting” =>
 “hello world”
 },
 “self” => <ref>,
 “ret_val” => nil
 }
 ) ErRuby stack
  43. begin lvasgn :greeting str “hello world” lvar :greeting ErRuby stack

    {
 “lvars” => {
 “greeting” =>
 “hello world”
 },
 “self” => <ref>,
 “ret_val” => 
 “hello world”
 }
  44. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  45. Ruby object • Also influenced by Smalltalk • In Ruby,

    method call actually calls send under the hood. • The major difference from Actor is that it must get response before proceeding.
  46. Making Ruby objects with Erlang process • Every Ruby Object

    is a distinct Erlang actor. • All object operations are done by message passing instead of direct memory access. • Update object state after receive message
  47. Object process Waiting for message Update State {Sender, Type, Message}

    Handle message Call Stack Object Handle object related AST Waiting for response {Message, Result} Handle next entry
  48. Waiting for message Update State {Sender, sync, Message} Handle message

    Call Stack Object Handle object related AST Waiting for response {Message, Result} Handle next entry Object process
  49. Waiting for message Update State {Sender, async, Message} Handle message

    Call Stack Object Handle object related AST Waiting for response Handle next entry Object process
  50. What’s inside object state? • Instance variables’s name and reference.

    • Constants’s name and reference • Reference to it’s class • Reference to self
  51. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  52. Setting/Getting instance variable @conf = "rubykaigi" @conf

  53. @conf = "rubykaigi" Waiting for message Update State Handle message

    Call Stack Object Handle object related AST Waiting for response Handle next entry {ivar => {}}
  54. Waiting for message Update State {CallStackID, async, {def_ivar, “conf”, “rubykaigi”}}

    Handle message Call Stack Object Handle object related AST Waiting for response Handle next entry @conf = "rubykaigi" {ivar => {}}
  55. Waiting for message Update State {CallStackID, async, {def_ivar, “conf”, “rubykaigi”}}

    Handle message Call Stack Object Handle object related AST Waiting for response Handle next entry {ivars => {“conf”=>
 “rubykaigi”}} @conf = "rubykaigi"
  56. @conf Waiting for message Update State Handle message Call Stack

    Object Handle object related AST Waiting for response Handle next entry {ivars => {“conf”=>
 “rubykaigi”}}
  57. Waiting for message Update State {CallStackID, sync, {find_ivar, “conf”}} Handle

    message Call Stack Object Handle object related AST Waiting for response Handle next entry {ivars => {“conf”=>
 “rubykaigi”}} @conf
  58. Waiting for message Update State {CallStackID, sync, {find_ivar, “conf”}} Handle

    message Call Stack Object Handle object related AST Waiting for response Handle next entry {{find_ivar, “conf”}, “rubykaigi”} {ivars => {“conf”=>
 “rubykaigi”}} @conf
  59. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  60. Define Method class Conf def attend(rubyist) "hello #{rubyist}" end end

  61. Define Method Waiting for message Update State Handle message Call

    Stack Conf Handle object related AST Waiting for response Handle next entry {methods => {}}
  62. Define Method Waiting for message Update State Handle message Call

    Stack Conf Handle object related AST Waiting for response Handle next entry {CallStackID, async, {def_method, “attend”,
 [“rubyist”], BodyAST
 }} {methods => {}}
  63. Define Method Waiting for message Update State Handle message Call

    Stack Conf Handle object related AST Waiting for response Handle next entry {CallStackID, async, {def_method, “attend”,
 [“rubyist”], BodyAST
 }} {methods => {“attend” => {[“rubyist”], BodyAST, 1}}}
  64. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  65. Execute Method rubykaigi_2016 = Conf.new rubykaigi_2016.attend("john lin")

  66. Waiting for message Update State Handle message Call Stack rubykaigi_2016

    Handle send Waiting for response Handle next entry Find method definition {class => Conf}
  67. Waiting for message Update State {CallStackID, sync, {find_instance_ method, “attend”}}

    Handle message Call Stack rubykaigi_2016 Handle send Waiting for response Handle next entry Find method definition {class => Conf}
  68. Waiting for message Update State {CallStackID, sync, {find_instance_ method, “attend”}}

    Handle message rubykaigi_2016 {class => Conf} Waiting for message Update State Handle message Conf {methods => {attend => Definition}} Find method definition
  69. Waiting for message Update State {CallStackID, sync, {find_instance_ method, “attend”}}

    Handle message rubykaigi_2016 {class => Conf} Waiting for message Update State {ObjectID, sync, {find_method, “attend”}} Handle message Conf {methods => {attend => Definition}} Find method definition
  70. Waiting for message Update State {CallStackID, sync, {find_instance_ method, “attend”}}

    Handle message rubykaigi_2016 {class => Conf} Waiting for message Update State {ObjectID, sync, {find_method, “attend”}} Handle message Conf {methods => {attend => Definition}} {{find_method, “attend”}, Definition} Find method definition
  71. Waiting for message Update State {CallStackID, sync, {find_instance_ method, “attend”}}

    Handle message Call Stack rubykaigi_2016 Handle send Waiting for response Eval method {class => Conf} {ObjectID, sync, {find_method, “attend”}} {{find_method, “attend”}, Definition} Find method definition {{find_instance _method, “attend”}, Definition}
  72. Eval argument Push a new frame Bind argument Eval Body

    Pop frame {lvars => {}, self => Toplevel} rubykaigi_2016.attend("john lin")
  73. Eval argument Push a new frame Bind argument Eval Body

    Pop frame {lvars => {}, self => Toplevel} rubykaigi_2016.attend("john lin")
  74. rubykaigi_2016.attend("john lin") Eval argument Push a new frame Bind argument

    Eval Body Pop frame {lvars => {}, self => rubykaigi_2016, prev_frame => {lvars => {},self => Toplevel} }
  75. rubykaigi_2016.attend("john lin") Eval argument Push a new frame Bind argument

    Eval Body Pop frame {lvars => {rubyist => “john lin”}, self => rubykaigi_2016, prev_frame => 
 {lvars => {},self => Toplevel} }
  76. rubykaigi_2016.attend("john lin") Eval argument Push a new frame Bind argument

    Eval Body Pop frame {lvars => {rubyist => “john lin”}, self => rubykaigi_2016,
 ret_val => 
 “hello john lin” prev_frame => 
 {lvars => {},self => Toplevel} }
  77. rubykaigi_2016.attend("john lin") Eval argument Push a new frame Bind argument

    Eval Body Pop frame {lvars => {}, self => Toplevel, ret_val => 
 “hello john lin”}
  78. What should a Ruby interpreter do? • Parse source code

    • Manage local variables • Create Ruby objects • Manage instance variable • Define method • Invoke method
  79. Current features • local/instance variables • method definition and calling

    • class and inheritance • block & yield • Boolean, Integer, String, Array with limited methods.
  80. Experimental features
 ⚗

  81. First class concurrency • One of the main goals of

    ErRuby. • Which should make programmers happy, not the machine. • Principle of least astonishment
  82. foo|.bar • No impact on existing code. • Inspired by

    actor model & gorouting • The symbol itself feels pretty concurrent. • | (or) is a good friend of & (and)
  83. Example def upload_file(payload) #upload file to a remote server end

    def send_heartbeat #send heartbeat to client every minute end response = self|.upload_file(kyoto_photos) self|.send_heartbeat #do other stuffs puts(response.code)
  84. Features • Asynchronous method calling. • Returns a Future object,

    which wraps the return value of the method call. • Only blocks when we actually need the result.
  85. Under the hood • Creates a new process for the

    method invocation. • Returns a Future object immediately . • Sends back the return value after it’s done. • Blocks and waits for the result when getting data from the Future object.
  86. Future • A proxy object represent the result of a

    computation event. • Can be passed around just like normal objects. • Blocks and wait for the result when accessing the value.
  87. Blocks if the result isn’t ready future = foo|.bar access

    actual
 value of future Running bar concurrently blocks
  88. Doesn’t block if the result is ready future = foo|.bar

    access actual
 value of future Running bar concurrently
  89. When do we need the value? • The later the

    better. • If it blocks right away, we’re not having any concurrency at all. • Some operations don’t need the actual value.
  90. foo = future • We don’t actually need the value

    when assign it to a variable. • A variable is just a reference of the object. We can just reference the future instead of the object
  91. future.to_s • We need the actual value to find out

    it’s class • we should blocks when calling method on Future
  92. puts “hello” if future • We’ll need the actual value

    of the Future when use it in language constructs • We’ll block in this case
  93. Passing Future as argument • There’s 2 different type of

    methods to consider • User defined method • Language builtin method
  94. kaigi.attend(future) • Inside user method, we can do one of

    the following things • assign a it to a variable • call method on it • pass it as argument of system method. • there’s no need to block in this case.
  95. puts(future) • If it’s a language method, we’ll block. •

    We need the actual value to do actual works.
  96. What foo|.bar doesn’t solve • Data sharing problem • Racing

    conditions
  97. Future Challenges • Find a way to handle data sharing

    problem between processes. • Garbage collection • Exceptions
  98. Takeaways • ErRuby is an experimental Ruby interpreter on Erlang.

    • It’s possible to implement Ruby in a Functional language • foo|.bar and call by future fits into Ruby.
  99. Thank You For Listening

  100. Q&A • Erruby is at https://github.com/johnlinvc/erruby • My Twitter handle

    is @johnlinvc