Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A tale of three chat rooms

A tale of three chat rooms

How three types of concurrency handle an audience full of excited developers. At Railsconf 2016. Code example are here: https://github.com/thijsc/three-chat-servers

Thijs Cadier

May 05, 2016
Tweet

More Decks by Thijs Cadier

Other Decks in Programming

Transcript

  1. A TALE OF THREE CHAT ROOMS How three types of

    concurrency handle an audience full of excited developers. A tale of three chat rooms
  2. A TALE OF THREE CHAT ROOMS A developer (me) was

    working on a monitoring tool called AppSignal. SOMEWHERE… IN A COUNTRY FAR FAR AWAY…
  3. A TALE OF THREE CHAT ROOMS code Our chat client

    1 require 'socket' 2 client = TCPSocket.open(ARGV[0], 2000) 3 4 Thread.new do 5 while line = client.gets 6 puts line.chop 7 end 8 end 9 10 while input = STDIN.gets.chomp 11 client.puts input 12 end
  4. A TALE OF THREE CHAT ROOMS • A master process

    starts multiple worker processes • The worker process handles the actual requests • The master manages the workers Multi-process
  5. A TALE OF THREE CHAT ROOMS root 132 unicorn master

    root 133 unicorn worker_1 root 134 unicorn worker_2 root 135 unicorn worker_3 root 136 unicorn worker_4 root 137 unicorn worker_5 root 82 crypto root 81 ecrptfs-krthrea
  6. A TALE OF THREE CHAT ROOMS code Start a TCP

    server 1 require 'socket' 3 4 puts "Starting server on port 2000" 5 6 server = TCPServer.open(2000)
  7. A TALE OF THREE CHAT ROOMS code Communicate between processes

    8 client_writers = [] 9 master_reader, master_writer = IO.pipe 10 11 write_incoming_messages_to_child_processes
  8. A TALE OF THREE CHAT ROOMS code Start a worker

    process 15 loop do 16 while socket = server.accept 20 client_reader, client_writer = IO.pipe 23 client_writers.push(client_writer) .. 26 fork do 39 end 41 end 42 end
  9. A TALE OF THREE CHAT ROOMS code What’s running in

    the worker process 27 nickname = read_line_from(socket) 29 client.puts 'Connected to chat server' 30 31 write_incoming_messages_to_client 32 33 # Read incoming messages from the client. 34 while incoming = read_line_from(socket) 35 master_writer.puts "#{nickname}: #{incoming}" 36 end
  10. A TALE OF THREE CHAT ROOMS Multi-process pros :) •

    Write code without thinking about concurrency at all! • Each worker can crash without damaging the rest of the system
  11. A TALE OF THREE CHAT ROOMS Multi-process cons :( •

    Each process loads the full codebase in memory • This makes it memory-intensive • Hence, it does not scale to large amounts of concurrent connections
  12. A TALE OF THREE CHAT ROOMS • Allows one process

    to handle multiple requests at the same time • It does so by running multiple threads within a single process Multi-threading
  13. A TALE OF THREE CHAT ROOMS code Start a TCP

    server (exactly the same) 1 require 'socket' 3 4 puts "Starting server on port 2000" 5 6 server = TCPServer.open(2000)
  14. A TALE OF THREE CHAT ROOMS code Store incoming messages

    8 mutex = Mutex.new 9 messages = []
  15. A TALE OF THREE CHAT ROOMS code Run a thread

    per connection 11 loop do 14 Thread.new(server.accept) do |socket| 15 nickname = read_line_from(socket) 17 client.puts 'Connected to chat server' 19 send_incoming_messages_thread 38 read_incoming_messages 47 end 49 end
  16. A TALE OF THREE CHAT ROOMS code Store a message

    38 while incoming = read_line_from(socket) 39 mutex.synchronize do 40 messages.push( 41 :time => Time.now, 42 :nickname => nickname, 43 :text => incoming 44 ) 45 end 46 end
  17. A TALE OF THREE CHAT ROOMS code Send messages to

    the client 22 loop do 23 messages_to_send = mutex.synchronize do 24 get_messages_to_send().tap do 25 sent_until = Time.now 26 end 27 end 28 messages_to_send.each do |message| 29 socket.puts message[:text] 30 end 31 sleep 0.2 32 end
  18. A TALE OF THREE CHAT ROOMS root 132 puma |

    ——— thread #1 | ——— thread #2 | ——— thread #3 | ——— thread #4 root 82 crypto root 81 ecrptfs-krthrea
  19. A TALE OF THREE CHAT ROOMS • A lock around

    the execution of all Ruby code • Even though our threads appear to run in parallel, only one thread is active at a time • IO operates outside of the GIL Global Interpreter Lock
  20. A TALE OF THREE CHAT ROOMS If you use multiple

    threads you have to be careful to write all code that manipulates shared data in a thread safe way Thread safety
  21. A TALE OF THREE CHAT ROOMS Multi-threading pros :) •

    Uses less memory than multi-process • You can share data easily because everything runs in the same process
  22. A TALE OF THREE CHAT ROOMS Multi-threading cons :( •

    You have to make sure your code is thread safe • If a thread causes a a crash, it could take down your process • The GIL locks all operations except I/O
  23. A TALE OF THREE CHAT ROOMS • For large number

    of concurrent I/O operations • Not actually executed at the same time • It is an efficient way to handle concurrent users Event-loop
  24. A TALE OF THREE CHAT ROOMS OS [network / disk]

    Event queue Oldest event Event Event Event Event Event Newest event Storage in memory Event loop Add IO ready event Add event Register interest The mental model…
  25. A TALE OF THREE CHAT ROOMS • We’ll use IO.select

    instead of registering interest with the OS • The client is implemented as a Fiber • The event loop will inform the fiber when its connection is ready Our example: Poor man’s event loop
  26. A TALE OF THREE CHAT ROOMS code Fibers 1 fiber

    = Fiber.new do 2 loop do 3 puts Fiber.yield 4 end 5 end 6 7 fiber.resume 1 8 fiber.resume 2 9 fiber.resume 3 10 fiber.resume 4
  27. A TALE OF THREE CHAT ROOMS code You’ve seen this

    before 1 require 'socket' 3 4 puts "Starting server on port 2000" 5 6 server = TCPServer.open(2000)
  28. A TALE OF THREE CHAT ROOMS code The client representation

    12 Fiber.new do 13 last_write = Time.now 15 loop do 16 state = Fiber.yield 17 18 if state == :readable 19 # Read a message from the socket 37 elsif state == :writable 38 # Write messages to the socket 44 end 45 end 46 end
  29. A TALE OF THREE CHAT ROOMS code Read a message

    18 if state == :readable 20 incoming = read_line_from(socket) 32 $messages.push( 33 :time => Time.now, 34 :nickname => nickname, 35 :text => incoming 36 )
  30. A TALE OF THREE CHAT ROOMS code Write a message

    37 elsif state == :writable 38 # Write messages to the socket 39 get_messages_to_send().each do |message| 40 socket.puts message 41 end 42 43 last_write = Time.now 44 end
  31. A TALE OF THREE CHAT ROOMS code Structure of the

    event loop 12 loop do 13 accept_incoming_connections 14 15 get_ready_connections 16 17 read_from_readable_connections 18 19 write_to_writable_connections 20 end
  32. A TALE OF THREE CHAT ROOMS code Accept incoming connections

    15 begin 16 socket = server.accept_nonblock 17 nickname = socket.gets.chomp 18 client = { 19 :last_write => Time.now, 20 :nickname => nickname, 21 :ip => socket.addr.last 22 } 23 clients[socket] = client 25 rescue IO::WaitReadable, Errno::EINTR 26 # No new incoming connections at the moment 27 end
  33. A TALE OF THREE CHAT ROOMS code See which clients

    are ready 30 readable, writable = IO.select( 31 clients.keys, 32 clients.keys, 33 clients.keys, 34 0.01 35 )
  34. A TALE OF THREE CHAT ROOMS code Read from readable

    connections 38 if readable 39 readable.each do |ready_socket| 41 incoming = read_line_from(ready_socket) 44 client = clients[ready_socket] 55 messages.push( 56 :time => Time.now, 57 :nickname => client[:nickname], 58 :text => incoming 59 ) 60 end 61 end
  35. A TALE OF THREE CHAT ROOMS code Write to writable

    connections 64 if writable 65 writable.each do |ready_socket| 67 client = clients[ready_socket] 69 70 get_messages_to_send.each do |message| 71 ready_socket.puts message[:text] 72 end 73 74 client[:last_write] = Time.now 75 end 76 end
  36. A TALE OF THREE CHAT ROOMS Event-loop pros :) •

    Almost no memory overhead per connection • Scales to a huge number of parallel connections
  37. A TALE OF THREE CHAT ROOMS Event-loop cons :( •

    It can be hard to debug if your system gets more complex (callbacks) • The work done per tick of the loop must be small and predictable to avoid high latency
  38. A TALE OF THREE CHAT ROOMS Who fancies a chat?

    How many developers will it take to break three different ways I built a chat? Let’s see…..
  39. A TALE OF THREE CHAT ROOMS Thijs Cadier Co-founder AppSignal,

    Email Twitter Github [email protected] @thijsc thijsc We’re hiring! If you want to help other developers build better software, contact us! Use the coupon code concurrency and get $50 credit! Thank you!