A tale of three web servers - Code & Coffee

Transcript

2. A Tale Of Three Ruby Web Servers
3. I’m using Ruby in this presentation, but the concepts are the same in other languages
4. Why do different Ruby web servers even exist?
5. All three use a similar way of receiving incoming requests and sending responses
6. A web server has to do more than one thing at a time
7. Wikipedia Concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other.
8. Three main ways of doing concurrency • Running multiple processes • Threading • Event-driven
10. Running multiple processes
11. Running multiple processes • When you start the server you only start a master process • The master process does not handle requests, but controls one or more child processes that do • It starts these processes by forking itself
12. Fork is a Unix command that makes a copy of a process, with the exact state it has at the time of forking.
13. Code Console output @winner = ‘The Netherlands’
 
 puts "#{Process.pid}: I'm the original process"
 
 if fork
 puts "#{Process.pid}: I’m the master"
 else
 puts "#{Process.pid}: I'm the child” @winner = ‘Argentina’
 end
 
 puts "#{Process.pid}: The winner is #{@winner}" 351: I'm the original process 351: I’m the master 351: The winner is The Netherlands 372: I'm the child 372: The winner is Argentina
14. Ruby has one to one mappings to various Unix commands, such as fork.
15. How can a web server use this?
16. def spawn_missing_workers
 worker_nr = -1
 until all_workers_started
 worker = Worker.new(worker_nr)
 if pid = fork
 # Run in the master
 WORKERS[pid] = worker
 else
 # Start the loop that handles incoming requests
 worker_loop(worker)
 end
 end
 end
17. WORKERS.each_key do |pid|
 Process.kill(pid)
 end
18. socket = Socket.new
 loop do
 # Wait for incoming connection
 socket, addr = socket.accept
 # Process incoming request with some backend
 process_request(socket.gets)
 end
20. Recap: Running multiple processes • Handle concurrency by running separate worker processes that handle requests • If you expect your workers to break it’s easy to kill them without affecting other workers • Concurrency is limited by the number of processes • Every process uses the full amount of memory.
21. Threading
22. Threading • One process handles multiple requests by running multiple threads • Threads live in the same process and therefore share the same global state
23. You have to take the shared global state into account when using threaded code, let’s look at some examples.
24. Code Console output 5.times do |i|
 Thread.new do
 sleep rand(5)
 puts "I'm thread #{i}"
 end
 end
 
 sleep 10 I'm thread 1 I'm thread 3 I'm thread 4 I'm thread 0 I'm thread 2
25. Code Console output @winner = ‘The Netherlands’
 
 5.times do |i|
 Thread.new do
 sleep rand(5)
 puts "I'm thread #{i} and the winner is #{@winner}"
 end
 end
 
 sleep 2
 @winner = ‘Argentina’
 
 sleep 30 I'm thread 2 and the winner is The Netherlands I'm thread 4 and the winner is The Netherlands I'm thread 0 and the winner is The Netherlands I'm thread 1 and the winner is Argentina I'm thread 3 and the winner is Argentina
26. Code Console output @total = 0
 
 100.times do |i|
 Thread.new do
 sleep rand(0.5)
 snapshot_of_total = @total
 sleep rand(0.5)
 @total = snapshot_of_total + 1
 end
 end
 
 sleep 5
 puts @total 6
27. Code Console output @total = 0
 @lock = Mutex.new
 
 100.times do |i|
 Thread.new do
 sleep rand(0.5)
 @lock.synchronize do
 snapshot_of_total = @total
 sleep rand(0.5)
 @total = snapshot_of_total + 1
 end
 end
 end
 
 sleep 60
 puts @total 100
28. Code Console output @total = 0
 
 100.times do |i|
 snapshot_of_total = @total
 sleep rand(0.5)
 @total = snapshot_of_total + 1
 end
 
 puts @total 100
29. Threads can work well to achieve concurrency, but it can be really hard to make them independent of each other.
30. How can a web server use this?
31. class Server
 def start
 @cond = ConditionVariable.new
 @mutex = Mutex.new
 @workers = []
 @mutex.synchronize do
 @min_workers.times { spawn_thread }
 end
 end
 end
32. while true
 mutex.synchronize do
 while @todo.empty?
 @cond.wait(mutex)
 end
 
 work = @todo.pop
 end
 
 worker.call(work)
 end
33. def <<(work)
 @mutex.synchronize do
 @todo << work
 
 if too_little_workers?
 spawn_thread
 end
 
 @cond.signal
 end
 end
34. loop do
 IO.select(sockets).each do |sock|
 pool << Client.new io
 end
 end
35. Recap: Threading • Server keeps a pool of worker threads • They wait for work to come in and process it outside of the server’s main lock • Concurrency is limited by size of the thread pool • Worker threads use little memory compared to processes
36. Event driven
37. Event driven • One process handles multiple requests by running an event loop that schedules work • All code that gets run in this loop has to split itself up in the smallest feasible units of work
38. loop do
 events = select(operations_being_watched)
 
 events.each do |event|
 find_callback_for_event(event).call
 end
 end
39. EventMachine
40. Code Console output require 'eventmachine'
 
 EM.run do
 5.times do |i|
 EM.add_timer(rand(5)) do
 puts "I'm callback #{i}"
 end
 end
 end I'm callback 1 I'm callback 2 I'm callback 0 I'm callback 3 I'm callback 4
41. Code Console output require 'eventmachine'
 
 @winner = ‘The Netherlands’
 
 EM.run do
 5.times do |i|
 EM.add_timer(rand(5)) do
 puts "I'm callback #{i} and” the winner is #{@winner}"
 end
 end
 
 EM.add_timer(2) do
 @winner = ‘Argentina’
 end
 end I'm callback 1 and the winner is The Netherlands I'm callback 3 and the winner is The Netherlands I'm callback 0 and the winner is Argentina I'm callback 2 and the winner is Argentina I'm callback 4 and the winner is Argentina
42. Code Console output require 'eventmachine'
 
 @total = 0
 
 EM.run do
 100.times do |i|
 EM.add_timer(rand(0.5)) do
 snapshot_of_total = @total
 EM.add_timer(rand(0.5)) do
 @total = snapshot_of_total + 1
 end
 end
 end
 
 EM.add_timer(5) do
 puts @total
 end
 end 4
43. We can’t fix this
44. You can only use code that works in an event-driven way, or you’ll end up with a program that’s not concurrent
45. How can a web server use this?
46. EventMachine.start_server(
 @host,
 @port,
 Connection
 ) @port,
 Connection
 )
47. EventMachine.defer(
 operation,
 callback
 )
48. module Thin
 module Connection
 def receive_data(data)
 process if @request.parse(data)
 end
 
 def process
 EventMachine.defer(
 method(:pre_process),
 method(:post_process)
 )
 end
 
 def pre_process @app.call(@request.env)
 end
 
 def post_process send_data(@response)
 end
 end
 end @port,
 Connection
 )
49. Recap: Event-driven • Server runs a loop that schedules execution of operations and callbacks • Whenever an operation has to wait for something it stops, a callback gets called when the wait is over • Hardly any memory is used by the callbacks, they’re just Ruby blocks • Concurrency is an order of magnitude bigger than the other two models • All code running in the loop has to be event-driven
50. So which concurrency model should I use in my own code?
51. It depends
52. • For most apps threading is a good tradeoff • If you run highly concurrent apps with long-running streams event-driven allows you to scale • If you don’t have a high-volume problem or you expect your processes to get out of control go for multi-process
53. If you want to build the next Whatsapp • Actor model, used in Erlang • Coroutines, used in Go
54. Thank you
55. More information • http://tomayko.com/writings/unicorn-is-unix • http://www.jstorimer.com/products/working-with-unix-processes • http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock/ • http://jordanhollinger.com/2011/05/15/writing-an-ajax-long-polling-server-in-ruby-part-1 • http://dailyjs.com/2012/09/27/truth-about-event-loops/