From Rails to the Webserver to the Browser

From Rails to the Webserver to the Browser

Most of us know how to build beautiful web applications with Rails. With the help of templating tools like ERB and HAML our web apps create HTML documents, but, do you know exactly how those HTML documents end up in a browser?

During this talk I will show you the bits that make it all happen. We will dissect the relevant code within Rails, Rack and the thin web server to discover exactly how the web server starts and listens to a TCP port, communicates with Rails and returns the HTML document that your browser parses.

Why? Because we're curious about it, that's why.

0f9c9bbecc4067b9bce445cb11ed5d53?s=128

David Padilla

April 30, 2013
Tweet

Transcript

  1. From Rails to the web server to the browser David

    Padilla @dabit
  2. David Padilla @dabit

  3. Hire us!

  4. Training not poaching

  5. MagmaConf

  6. None
  7. Manzanillo, Mexico

  8. June 5 - 7

  9. www.magmaconf.com

  10. None
  11. </me>

  12. From Rails to the web server to the browser

  13. Life of a Request

  14. www.app.com/hello HTTP Request

  15. HTTP Request

  16. HTTP Request

  17. <html> <head> <title>Railsconf</title> </head> <body>LOL</body> </html>

  18. <html> <head> <title>Railsconf</title> </head> <body>LOL</body> </html>

  19. None
  20. Rack

  21. Rack

  22. Rack

  23. class App def call(env) # ... return [http_code, headers, body]

    end end
  24. [http_code, headers, body] Integer Hash Responds to :each

  25. Somewhere on the web server...

  26. # ... # ... # ... @app.call(env) # ... #

    ... # ...
  27. Somewhere on the Application...

  28. class App def call(env) # ... return [http_code, headers, body]

    end end
  29. config.ru

  30. require 'rack/app' run Rack::App.new config.ru

  31. None
  32. None
  33. rackup

  34. None
  35. None
  36. None
  37. None
  38. None
  39. All Rails apps are Rack apps

  40. None
  41. None
  42. class HelloController def show render text: "Hello World" end end

    app/controllers/hello_controller.rb
  43. None
  44. App::Application.routes.draw do root to: "hello#show" end config/routes.rb

  45. None
  46. None
  47. require 'test/unit' require 'turn' require 'net/http' class TestServer < Test::Unit::TestCase

    def test_the_request uri = URI('http://127.0.0.1:9292') assert "Hello World", Net::HTTP.get(uri) end end test/test.rb
  48. None
  49. None
  50. group :development do gem 'pry-rails' gem 'pry-debugger' end Gemfile

  51. None
  52. None
  53. None
  54. None
  55. None
  56. None
  57. None
  58. @app.call({ "REQUEST_METHOD" => "GET", "PATH_INFO" => "/", "rack.input" => ""

    })
  59. [status, headers, body] Rack

  60. None
  61. None
  62. None
  63. None
  64. None
  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. None
  72. None
  73. StringIO.new(“Hello World”)

  74. None
  75. class App def call(env) content = "Hello World" status_code =

    200 headers = { "Content-Length" => content.length.to_s } [status_code, headers, [content]] end end rack/app.rb
  76. None
  77. Webservers

  78. Down the Rabbit Hole

  79. # ... # ... # ... @app.call(env) # ... #

    ... # ...
  80. Thin

  81. thin start

  82. module Thin # The uterly famous Thin HTTP server. #

    It listen for incoming request through # a given +backend+ # and forward all request to +app+. # # == TCP server # Create a new TCP server on bound # to <tt>host:port</tt> by specifiying +host+ # and +port+ as the first 2 arguments. # # Thin::Server.start('0.0.0.0', 3000, app) lib/thin/server.rb
  83. Thin::Server.start('0.0.0.0', 3000, app) thin start

  84. # Start the server and listen for connections. def start

    raise ArgumentError, 'app required' unless @app log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C debug ">> Debugging ON" trace ">> Tracing ON" log ">> Maximum connections set to #{@backend.maximum_connections}" log ">> Listening on #{@backend}, CTRL+C to stop" @backend.start end lib/thin/server.rb
  85. # Start the server and listen for connections. def start

    raise ArgumentError, 'app required' unless @app log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C debug ">> Debugging ON" trace ">> Tracing ON" log ">> Maximum connections set to #{@backend.maximum_connections}" log ">> Listening on #{@backend}, CTRL+C to stop" @backend.start end lib/thin/server.rb
  86. @backend.start lib/thin/server.rb

  87. def select_backend(host, port, options) case when options.has_key?(:backend) raise ArgumentError, ":backend

    must be options[:backend].new(host, port, optio when options.has_key?(:swiftiply) Backends::SwiftiplyClient.new(host, por when host.include?('/') Backends::UnixServer.new(host) else Backends::TcpServer.new(host, port) end end lib/thin/server.rb
  88. def select_backend(host, port, options) case when options.has_key?(:backend) raise ArgumentError, ":backend

    must be options[:backend].new(host, port, optio when options.has_key?(:swiftiply) Backends::SwiftiplyClient.new(host, por when host.include?('/') Backends::UnixServer.new(host) else Backends::TcpServer.new(host, port) end end lib/thin/server.rb
  89. def select_backend(host, port, options) case when options.has_key?(:backend) raise ArgumentError, ":backend

    must be options[:backend].new(host, port, optio when options.has_key?(:swiftiply) Backends::SwiftiplyClient.new(host, por when host.include?('/') Backends::UnixServer.new(host) else Backends::TcpServer.new(host, port) end end lib/thin/server.rb
  90. lib/thin/backends/tcp_server.rb

  91. module Thin module Backends # Backend to act as a

    TCP socket server. class TcpServer < Base # Address and port on which the server is listening for connections. attr_accessor :host, :port def initialize(host, port) @host = host @port = port super() end # Connect the server def connect @signature = EventMachine.start_server(@host, @port, Connection, &method(:initializ end # Stops the server def disconnect EventMachine.stop_server(@signature) end def to_s "#{@host}:#{@port}" end end end end lib/thin/backends/tcp_server.rb
  92. # Connect the server def connect @signature = EventMachine.start_server(@ end

    lib/thin/backends/tcp_server.rb
  93. EventMachine

  94. module Connection def post_init # A client connected end def

    receive_data data # Data received end def unbind # Client disconnected end end EventMachine.start_server("0.0.0.0", 8081, Connection) EventMachine
  95. # Connect the server def connect @signature = EventMachine.start_server(@ end

    lib/thin/backends/tcp_server.rb
  96. ne.start_server(@host, @port, Connection, &metho lib/thin/backends/tcp_server.rb

  97. ne.start_server(@host, @port, Connection, &metho lib/thin/backends/tcp_server.rb

  98. lib/thin/connection.rb

  99. None
  100. lib/thin/connection.rb

  101. # Called when data is received from the client. def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  102. # Called when data is received from the client. def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  103. Thin::Request

  104. lib/thin/request.rb

  105. # Parse a chunk of data into the request environment

    # Raises a +InvalidRequest+ if invalid. # Returns +true+ if the parsing is complete. def parse(data) if @parser.finished? # Header finished, can only be some more body @body << data else # Parse more header using the super parser @data << data raise InvalidRequest, 'Header longer than allowed' if @data.size > MAX_HEADER @nparsed = @parser.execute(@env, @data, @nparsed) # Transfert to a tempfile if body is very big move_body_to_tempfile if @parser.finished? && content_length > MAX_BODY end if finished? # Check if header and body are complete @data = nil @body.rewind true # Request is fully parsed else false # Not finished, need more data end end lib/thin/request.rb
  106. Mongrel Parser

  107. /** * Copyright (c) 2005 Zed A. Shaw * You

    can redistribute it and/or modify * it under the same terms as Ruby. */ #ifndef http11_parser_h #define http11_parser_h #include <sys/types.h> #if defined(_WIN32) #include <stddef.h> #endif
  108. GET /index.html HTTP/1.1 Host:localhost HTTP Request

  109. None
  110. None
  111. None
  112. None
  113. None
  114. # Called when data is received from the client def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  115. # Called when data is received from the client def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  116. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), else @request.threaded = false post_process(pre_process) end end lib/thin/connection.rb
  117. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), else @request.threaded = false post_process(pre_process) end end lib/thin/connection.rb
  118. def pre_process # Add client info to the request env

    @request.remote_address = remote_address # Connection may be closed unless the App# # It should be noted that connection objec # callback is no longer referenced, so be @request.async_callback = method(:post_pro if @backend.ssl? @request.env["rack.url_scheme"] = "https if cert = get_peer_cert @request.env['rack.peer_cert'] = cert end end
  119. end # When we're under a non-async framework l #

    off async responses using the callback i # in removing this. response = AsyncResponse catch(:async) do # Process the request calling the Rack a response = @app.call(@request.env) end response rescue Exception handle_error # Pass through error response can_persist? && @request.persistent? ? Res end
  120. end # When we're under a non-async framework l #

    off async responses using the callback i # in removing this. response = AsyncResponse catch(:async) do # Process the request calling the Rack a response = @app.call(@request.env) end response rescue Exception handle_error # Pass through error response can_persist? && @request.persistent? ? Res end lib/thin/connection.rb
  121. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), else @request.threaded = false post_process(pre_process) end end lib/thin/connection.rb
  122. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), else @request.threaded = false post_process(pre_process) end end
  123. HTTP/1.1 200 OK Content-Length: 11 Hello World HTTP Response

  124. Unicorn

  125. EventMachine No

  126. @app.call(env) YES

  127. lib/unicorn/http_server.rb def process_client(client)

  128. Uses the mongrel parser Kinda

  129. PUMA

  130. What we’ve learned so far

  131. EventMachine.start_server("0.0.0.0", 8081, Connection)

  132. GET /index.html HTTP/1.1 Host:localhost

  133. GET /index.html HTTP/1.1 Host:localhost { "REQUEST_METHOD" => "GET", "PATH_INFO" =>

    "/", "rack.input" => "" }
  134. @app.call({ "REQUEST_METHOD" => "GET", "PATH_INFO" => "/", "rack.input" => ""

    })
  135. [http_code, headers, body]

  136. [http_code, headers, body] HTTP/1.1 200 OK Content-Length: 11 Hello World

  137. HTTP/1.1 200 OK Content-Length: 11 Hello World

  138. Our own Webserver

  139. None
  140. module Server def receive_data(data) puts data end end rack/server.rb

  141. None
  142. None
  143. module Server def receive_data(data) request = Thin::Request.new request.parse(data) puts request.env

    end end rack/server.rb
  144. None
  145. None
  146. def receive_data(data) request = Thin::Request.new request.parse(data) app = App.new response

    = app.call(request.env) end rack/server.rb
  147. None
  148. HTTP/1.1 200 OK Content-Length: 11 Hello World HTTP Response

  149. None
  150. None
  151. None
  152. None
  153. None
  154. None
  155. None
  156. Do not try this at home

  157. Do not try this in production

  158. Be curious

  159. dabit/rails-server-browser

  160. El fin

  161. Thank you! David Padilla @dabit david@crowdint.com