Share
Embed
A tale of three chat rooms
How three types of concurrency handle an audience full of excited developers. At Railsconf 2016. Code example are here: https://github.com/thijsc/three-chat-servers
Thijs Cadier
May 05, 2016
Transcript
46 slides
-
A TALE OF THREE CHAT ROOMS git clone [email protected]:thijsc/three-chat-servers.git To
follow along and join the demo, clone the examples: -
A TALE OF THREE CHAT ROOMS How three types of
concurrency handle an audience full of excited developers. A tale of three chat rooms -
A TALE OF THREE CHAT ROOMS A developer (me) was
working on a monitoring tool called AppSignal. SOMEWHERE… IN A COUNTRY FAR FAR AWAY… -
A TALE OF THREE CHAT ROOMS Multi-process Multi-threading Event-loop THREE
TYPES OF CONCURRENCY -
A TALE OF THREE CHAT ROOMS Multi-process Multi-threading Event-loop THREE
TYPES OF CONCURRENCY Unicorn Puma Thin -
A TALE OF THREE CHAT ROOMS Let’s try to build
a (very) minimalistic Slack -
A TALE OF THREE CHAT ROOMS code Our chat client
1 require 'socket' 2 client = TCPSocket.open(ARGV[0], 2000) 3 4 Thread.new do 5 while line = client.gets 6 puts line.chop 7 end 8 end 9 10 while input = STDIN.gets.chomp 11 client.puts input 12 end -
A TALE OF THREE CHAT ROOMS • A master process
starts multiple worker processes • The worker process handles the actual requests • The master manages the workers Multi-process -
A TALE OF THREE CHAT ROOMS root 132 unicorn master
root 133 unicorn worker_1 root 134 unicorn worker_2 root 135 unicorn worker_3 root 136 unicorn worker_4 root 137 unicorn worker_5 root 82 crypto root 81 ecrptfs-krthrea -
A TALE OF THREE CHAT ROOMS code Start a TCP
server 1 require 'socket' 3 4 puts "Starting server on port 2000" 5 6 server = TCPServer.open(2000) -
A TALE OF THREE CHAT ROOMS code Communicate between processes
8 client_writers = [] 9 master_reader, master_writer = IO.pipe 10 11 write_incoming_messages_to_child_processes -
A TALE OF THREE CHAT ROOMS code Start a worker
process 15 loop do 16 while socket = server.accept 20 client_reader, client_writer = IO.pipe 23 client_writers.push(client_writer) .. 26 fork do 39 end 41 end 42 end -
A TALE OF THREE CHAT ROOMS code What’s running in
the worker process 27 nickname = read_line_from(socket) 29 client.puts 'Connected to chat server' 30 31 write_incoming_messages_to_client 32 33 # Read incoming messages from the client. 34 while incoming = read_line_from(socket) 35 master_writer.puts "#{nickname}: #{incoming}" 36 end -
A TALE OF THREE CHAT ROOMS Multi-process pros :) •
Write code without thinking about concurrency at all! • Each worker can crash without damaging the rest of the system -
A TALE OF THREE CHAT ROOMS Multi-process cons :( •
Each process loads the full codebase in memory • This makes it memory-intensive • Hence, it does not scale to large amounts of concurrent connections -
A TALE OF THREE CHAT ROOMS • Allows one process
to handle multiple requests at the same time • It does so by running multiple threads within a single process Multi-threading -
A TALE OF THREE CHAT ROOMS code Start a TCP
server (exactly the same) 1 require 'socket' 3 4 puts "Starting server on port 2000" 5 6 server = TCPServer.open(2000) -
A TALE OF THREE CHAT ROOMS code Store incoming messages
8 mutex = Mutex.new 9 messages = [] -
A TALE OF THREE CHAT ROOMS code Run a thread
per connection 11 loop do 14 Thread.new(server.accept) do |socket| 15 nickname = read_line_from(socket) 17 client.puts 'Connected to chat server' 19 send_incoming_messages_thread 38 read_incoming_messages 47 end 49 end -
A TALE OF THREE CHAT ROOMS code Store a message
38 while incoming = read_line_from(socket) 39 mutex.synchronize do 40 messages.push( 41 :time => Time.now, 42 :nickname => nickname, 43 :text => incoming 44 ) 45 end 46 end -
A TALE OF THREE CHAT ROOMS code Send messages to
the client 22 loop do 23 messages_to_send = mutex.synchronize do 24 get_messages_to_send().tap do 25 sent_until = Time.now 26 end 27 end 28 messages_to_send.each do |message| 29 socket.puts message[:text] 30 end 31 sleep 0.2 32 end -
A TALE OF THREE CHAT ROOMS root 132 puma |
——— thread #1 | ——— thread #2 | ——— thread #3 | ——— thread #4 root 82 crypto root 81 ecrptfs-krthrea -
A TALE OF THREE CHAT ROOMS • A lock around
the execution of all Ruby code • Even though our threads appear to run in parallel, only one thread is active at a time • IO operates outside of the GIL Global Interpreter Lock -
A TALE OF THREE CHAT ROOMS If you use multiple
threads you have to be careful to write all code that manipulates shared data in a thread safe way Thread safety -
A TALE OF THREE CHAT ROOMS Multi-threading pros :) •
Uses less memory than multi-process • You can share data easily because everything runs in the same process -
A TALE OF THREE CHAT ROOMS Multi-threading cons :( •
You have to make sure your code is thread safe • If a thread causes a a crash, it could take down your process • The GIL locks all operations except I/O -
A TALE OF THREE CHAT ROOMS • For large number
of concurrent I/O operations • Not actually executed at the same time • It is an efficient way to handle concurrent users Event-loop -
A TALE OF THREE CHAT ROOMS OS [network / disk]
Event queue Oldest event Event Event Event Event Event Newest event Storage in memory Event loop Add IO ready event Add event Register interest The mental model… -
A TALE OF THREE CHAT ROOMS • We’ll use IO.select
instead of registering interest with the OS • The client is implemented as a Fiber • The event loop will inform the fiber when its connection is ready Our example: Poor man’s event loop -
A TALE OF THREE CHAT ROOMS code Fibers 1 fiber
= Fiber.new do 2 loop do 3 puts Fiber.yield 4 end 5 end 6 7 fiber.resume 1 8 fiber.resume 2 9 fiber.resume 3 10 fiber.resume 4 -
A TALE OF THREE CHAT ROOMS code You’ve seen this
before 1 require 'socket' 3 4 puts "Starting server on port 2000" 5 6 server = TCPServer.open(2000) -
A TALE OF THREE CHAT ROOMS code Clients & messages
1 clients = {} 2 messages = [] -
A TALE OF THREE CHAT ROOMS code The client representation
12 Fiber.new do 13 last_write = Time.now 15 loop do 16 state = Fiber.yield 17 18 if state == :readable 19 # Read a message from the socket 37 elsif state == :writable 38 # Write messages to the socket 44 end 45 end 46 end -
A TALE OF THREE CHAT ROOMS code Read a message
18 if state == :readable 20 incoming = read_line_from(socket) 32 $messages.push( 33 :time => Time.now, 34 :nickname => nickname, 35 :text => incoming 36 ) -
A TALE OF THREE CHAT ROOMS code Write a message
37 elsif state == :writable 38 # Write messages to the socket 39 get_messages_to_send().each do |message| 40 socket.puts message 41 end 42 43 last_write = Time.now 44 end -
A TALE OF THREE CHAT ROOMS code Structure of the
event loop 12 loop do 13 accept_incoming_connections 14 15 get_ready_connections 16 17 read_from_readable_connections 18 19 write_to_writable_connections 20 end -
A TALE OF THREE CHAT ROOMS code Accept incoming connections
15 begin 16 socket = server.accept_nonblock 17 nickname = socket.gets.chomp 18 client = { 19 :last_write => Time.now, 20 :nickname => nickname, 21 :ip => socket.addr.last 22 } 23 clients[socket] = client 25 rescue IO::WaitReadable, Errno::EINTR 26 # No new incoming connections at the moment 27 end -
A TALE OF THREE CHAT ROOMS code See which clients
are ready 30 readable, writable = IO.select( 31 clients.keys, 32 clients.keys, 33 clients.keys, 34 0.01 35 ) -
A TALE OF THREE CHAT ROOMS code Read from readable
connections 38 if readable 39 readable.each do |ready_socket| 41 incoming = read_line_from(ready_socket) 44 client = clients[ready_socket] 55 messages.push( 56 :time => Time.now, 57 :nickname => client[:nickname], 58 :text => incoming 59 ) 60 end 61 end -
A TALE OF THREE CHAT ROOMS code Write to writable
connections 64 if writable 65 writable.each do |ready_socket| 67 client = clients[ready_socket] 69 70 get_messages_to_send.each do |message| 71 ready_socket.puts message[:text] 72 end 73 74 client[:last_write] = Time.now 75 end 76 end -
A TALE OF THREE CHAT ROOMS Event-loop pros :) •
Almost no memory overhead per connection • Scales to a huge number of parallel connections -
A TALE OF THREE CHAT ROOMS Event-loop cons :( •
It can be hard to debug if your system gets more complex (callbacks) • The work done per tick of the loop must be small and predictable to avoid high latency -
A TALE OF THREE CHAT ROOMS Who fancies a chat?
How many developers will it take to break three different ways I built a chat? Let’s see….. -
A TALE OF THREE CHAT ROOMS git clone [email protected]:thijsc/three-chat-servers.git Clone
from here: Then run: ruby src/client.rb <ip> YourNick -
A TALE OF THREE CHAT ROOMS Thijs Cadier Co-founder AppSignal,
Email Twitter Github [email protected] @thijsc thijsc We’re hiring! If you want to help other developers build better software, contact us! Use the coupon code concurrency and get $50 credit! Thank you!