Minx - Communicating Sequential Processes in Ruby

A Ruby implementation of the CSP language defined by Tony Hoare.

Transcript

2. Daniel Schierbeck @dasch
3. Worked at Zendesk since September. Gettin’ my Zen on.
4. Minx Communicating Sequential Processes in Ruby
5. Minx Topics Covered • Concurrency models • threads • call/cc • fibers • My awesome concurrency library, Minx!
6. Minx What is concurrency? • Computations happen simultaneously • Not necessarily at the same exact time • Allows us to program non-sequentially
7. Minx What we’re used to • Green threads • Native threads • Reactor pattern (event driven code)
8. Minx Green threads • Allows us to keep doing stuff while waiting on I/O • Doesn’t use more than one core, so no performance benefit besides the I/O stuff • Shared state is a problem for some uses
9. Minx Native threads • Utilizes all your cores • Shared state is a bitch • Synchronization around shared state is really hard
10. Example: shared state 1 class Account 2 def transfer(recipient, amount) 3 lock(self) do 4 lock(recipient) do 5 withdraw(amount) 6 recipient.deposit(amount) 7 end 8 end 9 end 10 end
11. Minx Deadlock • What happens when a.transfer(b, ...) and b.transfer(a, ...) happen concurrently? • Deadlock
12. Communicating Sequential Processes Describing interaction in concurrent systems
13. Minx A formal language • Consists of two primitives: • Processes are independent pieces of behavior • Events represent communication or interaction
14. Minx Events drive change • Processes react to changes: a -> P describes a process that waits for a and then behaves like P • Processes can wait for different events simultaneously: (a -> P ⬛ b -> Q) • Lots of interesting stuff, but really geeky.
15. Minx CSP for human beings. Well, Ruby developers.
16. Minx An adaptation of CSP • Processes are still the central primitives used to describe behavior • Channels are used for communications and synchronization between processes • Channels are basically parameterized events
17. Spawning a process Minx.spawn { puts “Hello, World!” }
18. Communicating between processes chan = Minx.channel a = Minx.spawn { chan << “hello” } b = Minx.spawn { puts “received: “ + chan.read } Minx.join(a, b) # => “received: hello”
19. Reading from multiple channels msg = Minx.select(chan1, chan2) Order of channels does not matter, so deadlocks are much easier to avoid.
20. Writing to either of multiple channels Minx.push(“hello”, chan1, chan2)
21. Minx Composite processes • Use several processes behind the scene • Still act sequentially to the outside world
22. Minx Solving problems with Minx • Protect critical data with channels and processes • Use channels as synchronization primitives
23. The Account example 1 class Account 2 attr_reader :balance 3 4 def initialize(options = {}) 5 @deposits = Minx.channel 6 7 @balance = options.fetch(:balance, 0.0) 8 9 Minx.spawn do 10 @deposits.each {|amount| @balance += amount } 11 end 12 end 13 end
24. 1 class Account 2 def deposit(amount) 3 @deposits << amount 4 end 5 6 def withdraw(amount) 7 @deposits << -amount 8 end 9 10 def transfer(recipient, amount) 11 Minx.join( 12 Minx.spawn { withdraw(amount) }, 13 Minx.spawn { recipient.deposit(amount) }) 14 end 15 end
25. 1 acc1 = Account.new(:balance => 100.0) 2 acc2 = Account.new(:balance => 50.0) 3 4 acc1.transfer(acc2, 60.0) 5 6 acc1.balance #=> 40.0 7 acc2.balance #=> 110.0
26. Deadlock: solved
27. Implementation The foundation of Minx
28. Minx Several options • Green threads • Native threads •call/cc • Fibers
29. Minx Threads • Too much overhead • Too little control: pre-emptive scheduling
30. Minx call/cc • Pretty cool • Very complex
31. Implementing GOTO 1 require 'continuation' 2 3 names = %w[John James Max David] 4 5 callcc {|cc| $cc = cc } 6 7 name = names.shift 8 puts name 9 10 $cc.call unless name == "Max" 11 12 # Outputs: 13 # 14 # John 15 # James 16 # Max
32. Simple exceptions 1 require 'continuation' 2 3 $cc_stack = [] 4 5 def try(&block) 6 callcc do |cc| 7 begin 8 $cc_stack.push(cc) 9 block.call 10 ensure 11 $cc_stack.pop 12 end 13 end 14 end 15 16 def error! 17 $cc_stack.pop.call 18 end
33. Simple exceptions 1 try do 2 puts "a" 3 error! 4 puts "b" 5 end 6 7 # Outputs: 8 # 9 # a Multiple exits from method calls
34. Minx Issues with call/cc • Very difficult to manage continuations • Poor exception handling • Stuff would easily get lost
35. Fibers An abstraction on top of continuations
36. Minx Fibers • Provides a simple API for defining “fibers” of execution • Like threads, but non-preemptive • Allows granular control of execution
37. Python-like generators 1 require 'fiber' 2 3 fiber = Fiber.new do 4 Fiber.yield :uno 5 Fiber.yield :dos 6 :tres 7 end 8 9 fiber.resume #=> :uno 10 fiber.resume #=> :dos 11 fiber.resume #=> :tres 12 fiber.resume #=> FiberError: dead fiber called Multiple method exits!
38. Ping-Pong! 1 ping = Fiber.new do |pong| 2 while true 3 puts "ping!" 4 sleep 0.2 5 pong.transfer 6 end 7 end 8 9 pong = Fiber.new do 10 while true 11 puts "pong!" 12 sleep 0.2 13 ping.transfer 14 end 15 end 16 17 ping.resume(pong) Never overflows the stack!
39. Non-blocking I/O 1 def read(io) 2 Fiber.yield until io.ready? 3 io.read 4 end No more callbacks! 1 data = {} 2 fibers = files.map {|f| Fiber.new { data[f] = read(file) } } 3 4 until fibers.none? {|fiber| fiber.alive? } 5 fibers.each {|fiber| fiber.resume if fiber.alive? } 6 end
40. Minx Control Flow • Fibers can pass on execution to each other • #resume is best used for caller/callee scenarios • #transfer is used for peer-to-peer communication
41. Minx #resume 1 fib1 = Fiber.new do 2 puts "foo" 3 Fiber.yield 4 puts "bar" 5 end 6 7 fib2 = Fiber.new do 8 fib1.resume 9 puts "baz" 10 end 11 12 fib2.resume 13 # foo 14 # baz
42. Minx #transfer 1 fib1 = Fiber.new do 2 puts "foo" 3 Fiber.yield 4 puts "bar" 5 end 6 7 fib2 = Fiber.new do 8 fib1.transfer 9 puts "baz" 10 end 11 12 fib2.resume 13 # foo
43. Minx Passing Messages 1 def receive 2 message = Fiber.yield 3 end 4 5 def send(recipient, message) 6 recipient.transfer(message) 7 end 8 9 fib1 = Fiber.new { puts receive } 10 fib2 = Fiber.new { send(fib1, "hello!") } 11 12 fib1.resume 13 fib2.resume
44. Questions? Don’t be afraid.