From Rails to the Webserver to the Browser (Ruby Kaigi)

From Rails to the Webserver to the Browser (Ruby Kaigi)

Most of us know how to build beautiful web applications with Rails. With the help of templating tools like ERB and HAML our web apps create HTML documents, but, do you know exactly how those HTML documents end up in a browser?
During this talk I will show you the bits that make it all happen. We will dissect the relevant code within Rails, Rack and the thin web server to discover exactly how the web server starts and listens to a TCP port, communicates with Rails and returns the HTML document that your browser parses.
Why? Because we're curious about it, that's why.

0f9c9bbecc4067b9bce445cb11ed5d53?s=128

David Padilla

June 01, 2013
Tweet

Transcript

  1. From Rails to the web server to the browser David

    Padilla @dabit
  2. David Padilla - @dabit From Rails to the webserver to

    the browser
  3. David Padilla @dabit

  4. www.crowdint.com

  5. Image from: ezilon.com

  6. Image from: ezilon.com

  7. Image from: ezilon.com

  8. None
  9. None
  10. Population Tokyo Colima 0 3,750,000.00 7,500,000.00 11,250,000.00 15,000,000.00

  11. Population Tokyo Colima 0 3,750,000.00 7,500,000.00 11,250,000.00 15,000,000.00

  12. Population Tokyo Colima 0 3,750,000.00 7,500,000.00 11,250,000.00 15,000,000.00

  13. MagmaConf

  14. Manzanillo, Mexico

  15. June 5 - 7

  16. MagmaConf

  17. None
  18. From Rails to the web server to the browser

  19. Life of a Request

  20. None
  21. www.app.com/hello

  22. www.app.com/hello HTTP Request

  23. None
  24. HTTP Request

  25. None
  26. HTTP Request

  27. <html> <head> <title>Railsconf</title> </head> <body>LOL</body> </html>

  28. None
  29. None
  30. <html> <head> <title>Railsconf</title> </head> <body>LOL</body> </html>

  31. None
  32. None
  33. None
  34. Rack

  35. Rack

  36. Rack

  37. Rack

  38. None
  39. class App def call(env) # ... return [http_code, headers, body]

    end end
  40. [http_code, headers, body]

  41. [http_code, headers, body] Integer

  42. [http_code, headers, body] Integer Hash

  43. [http_code, headers, body] Integer Hash Responds to :each

  44. Somewhere on the web server...

  45. # ... # ... # ... @app.call(env) # ... #

    ... # ...
  46. Somewhere on the Application...

  47. class App def call(env) # ... return [http_code, headers, body]

    end end
  48. config.ru

  49. require 'rack/app' run Rack::App.new config.ru

  50. None
  51. None
  52. rackup

  53. None
  54. None
  55. None
  56. None
  57. None
  58. None
  59. All Rails apps are Rack apps

  60. None
  61. None
  62. class HelloController def show render text: "Hello World" end end

    app/controllers/hello_controller.rb
  63. None
  64. App::Application.routes.draw do root to: "hello#show" end config/routes.rb

  65. None
  66. None
  67. class TestServer < Test::Unit::TestCase def test_the_request uri = URI('http://127.0.0.1:9292') assert

    "Hello World", Net::HTTP.get(uri) end end test/test.rb
  68. None
  69. None
  70. None
  71. None
  72. None
  73. None
  74. None
  75. None
  76. None
  77. @app.call({ "REQUEST_METHOD" => "GET", "PATH_INFO" => "/", "rack.input" => ""

    })
  78. [status, headers, body] Rack

  79. None
  80. None
  81. None
  82. None
  83. None
  84. None
  85. None
  86. None
  87. None
  88. None
  89. None
  90. None
  91. None
  92. None
  93. None
  94. StringIO.new(“Hello World”)

  95. None
  96. class App def call(env) content = "Hello World" status_code =

    200 headers = { "Content-Length" => content.length.to_s } [status_code, headers, [content]] end end rack/app.rb
  97. None
  98. Webservers

  99. # ... # ... # ... @app.call(env) # ... #

    ... # ...
  100. Thin

  101. thin start

  102. module Thin # The uterly famous Thin HTTP server. #

    It listen for incoming request through # a given +backend+ # and forward all request to +app+. # # == TCP server # Create a new TCP server on bound # to <tt>host:port</tt> by specifiying +host+ # and +port+ as the first 2 arguments. # # Thin::Server.start('0.0.0.0', 3000, app) lib/thin/server.rb
  103. Thin::Server.start('0.0.0.0', 3000, app) thin start

  104. # Start the server and listen for connections. def start

    raise ArgumentError, 'app required' unless @app log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C debug ">> Debugging ON" trace ">> Tracing ON" log ">> Maximum connections set to #{@backend.maximum_connections}" log ">> Listening on #{@backend}, CTRL+C to stop" @backend.start end lib/thin/server.rb
  105. # Start the server and listen for connections. def start

    raise ArgumentError, 'app required' unless @app log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C debug ">> Debugging ON" trace ">> Tracing ON" log ">> Maximum connections set to #{@backend.maximum_connections}" log ">> Listening on #{@backend}, CTRL+C to stop" @backend.start end lib/thin/server.rb
  106. @backend.start lib/thin/server.rb

  107. def select_backend(host, port, options) case when options.has_key?(:backend) raise ArgumentError, ":backend

    must be options[:backend].new(host, port, optio when options.has_key?(:swiftiply) Backends::SwiftiplyClient.new(host, por when host.include?('/') Backends::UnixServer.new(host) else Backends::TcpServer.new(host, port) end end lib/thin/server.rb
  108. def select_backend(host, port, options) case when options.has_key?(:backend) raise ArgumentError, ":backend

    must be options[:backend].new(host, port, optio when options.has_key?(:swiftiply) Backends::SwiftiplyClient.new(host, por when host.include?('/') Backends::UnixServer.new(host) else Backends::TcpServer.new(host, port) end end lib/thin/server.rb
  109. def select_backend(host, port, options) case when options.has_key?(:backend) raise ArgumentError, ":backend

    must be options[:backend].new(host, port, optio when options.has_key?(:swiftiply) Backends::SwiftiplyClient.new(host, por when host.include?('/') Backends::UnixServer.new(host) else Backends::TcpServer.new(host, port) end end lib/thin/server.rb
  110. lib/thin/backends/tcp_server.rb

  111. module Thin module Backends # Backend to act as a

    TCP socket server. class TcpServer < Base # Address and port on which the server is listening for connections. attr_accessor :host, :port def initialize(host, port) @host = host @port = port super() end # Connect the server def connect @signature = EventMachine.start_server(@host, @port, Connection, &method(:initializ end # Stops the server def disconnect EventMachine.stop_server(@signature) end def to_s "#{@host}:#{@port}" end end end end lib/thin/backends/tcp_server.rb
  112. # Connect the server def connect @signature = EventMachine.start_server(@ end

    lib/thin/backends/tcp_server.rb
  113. EventMachine

  114. module Connection def post_init # A client connected end def

    receive_data data # Data received end def unbind # Client disconnected end end EventMachine.start_server("0.0.0.0", 8081, Connection) EventMachine
  115. # Connect the server def connect @signature = EventMachine.start_server(@ end

    lib/thin/backends/tcp_server.rb
  116. ne.start_server(@host, @port, Connection, &metho lib/thin/backends/tcp_server.rb

  117. ne.start_server(@host, @port, Connection, &metho lib/thin/backends/tcp_server.rb

  118. ne.start_server(@host, @port, Connection, &metho lib/thin/backends/tcp_server.rb

  119. lib/thin/connection.rb

  120. lib/thin/connection.rb

  121. # Called when data is received from the client. def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  122. # Called when data is received from the client. def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  123. Thin::Request

  124. lib/thin/request.rb

  125. # Parse a chunk of data into the request environment

    # Raises a +InvalidRequest+ if invalid. # Returns +true+ if the parsing is complete. def parse(data) if @parser.finished? # Header finished, can only be some more body @body << data else # Parse more header using the super parser @data << data raise InvalidRequest, 'Header longer than allowed' if @data.size > MAX_HEADER @nparsed = @parser.execute(@env, @data, @nparsed) # Transfert to a tempfile if body is very big move_body_to_tempfile if @parser.finished? && content_length > MAX_BODY end if finished? # Check if header and body are complete @data = nil @body.rewind true # Request is fully parsed else false # Not finished, need more data end end lib/thin/request.rb
  126. Mongrel Parser

  127. /** * Copyright (c) 2005 Zed A. Shaw * You

    can redistribute it and/or modify * it under the same terms as Ruby. */ #ifndef http11_parser_h #define http11_parser_h #include <sys/types.h> #if defined(_WIN32) #include <stddef.h> #endif
  128. GET /index.html HTTP/1.1 Host:localhost HTTP Request

  129. None
  130. GET /index.html HTTP/1.1\r\n Host:localhost\r\n\r\n HTTP Request

  131. None
  132. None
  133. None
  134. None
  135. None
  136. # Called when data is received from the client def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  137. # Called when data is received from the client def

    receive_data(data) @idle = false trace { data } process if @request.parse(data) rescue InvalidRequest => e log "!! Invalid request" log_error e post_process Response::BAD_REQUEST end lib/thin/connection.rb
  138. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), me else @request.threaded = false post_process(pre_process) end end lib/thin/connection.rb
  139. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), else @request.threaded = false post_process(pre_process) end end lib/thin/connection.rb
  140. def pre_process # Add client info to the request env

    @request.remote_address = remote_address # Connection may be closed unless the App# # It should be noted that connection objec # callback is no longer referenced, so be @request.async_callback = method(:post_pro if @backend.ssl? @request.env["rack.url_scheme"] = "https if cert = get_peer_cert @request.env['rack.peer_cert'] = cert end end
  141. end # When we're under a non-async framework l #

    off async responses using the callback i # in removing this. response = AsyncResponse catch(:async) do # Process the request calling the Rack a response = @app.call(@request.env) end response rescue Exception handle_error # Pass through error response can_persist? && @request.persistent? ? Res end
  142. end # When we're under a non-async framework l #

    off async responses using the callback i # in removing this. response = AsyncResponse catch(:async) do # Process the request calling the Rack a response = @app.call(@request.env) end response rescue Exception handle_error # Pass through error response can_persist? && @request.persistent? ? Res end lib/thin/connection.rb
  143. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), else @request.threaded = false post_process(pre_process) end end lib/thin/connection.rb
  144. # Called when all data was received and # the

    request is ready to be processed. def process if threaded? @request.threaded = true EventMachine.defer(method(:pre_process), else @request.threaded = false post_process(pre_process) end end
  145. HTTP/1.1 200 OK Content-Length: 11 Hello World HTTP Response

  146. Unicorn

  147. EventMachine

  148. EventMachine No

  149. @app.call(env)

  150. @app.call(env) YES

  151. lib/unicorn/http_server.rb def process_client(client)

  152. Uses the mongrel parser

  153. Uses the mongrel parser Kinda

  154. PUMA

  155. What we’ve learned so far

  156. None
  157. EventMachine.start_server("0.0.0.0", 8081, Connection)

  158. None
  159. GET /index.html HTTP/1.1 Host:localhost

  160. GET /index.html HTTP/1.1 Host:localhost { "REQUEST_METHOD" => "GET", "PATH_INFO" =>

    "/", "rack.input" => "" }
  161. @app.call({ "REQUEST_METHOD" => "GET", "PATH_INFO" => "/", "rack.input" => ""

    })
  162. [http_code, headers, body]

  163. [http_code, headers, body] HTTP/1.1 200 OK Content-Length: 11 Hello World

  164. HTTP/1.1 200 OK Content-Length: 11 Hello World

  165. Our own Webserver

  166. None
  167. module Server def receive_data(data) puts data end end rack/server.rb

  168. None
  169. None
  170. module Server def receive_data(data) request = Thin::Request.new request.parse(data) puts request.env

    end end rack/server.rb
  171. None
  172. None
  173. def receive_data(data) request = Thin::Request.new request.parse(data) app = App.new response

    = app.call(request.env) end rack/server.rb
  174. None
  175. HTTP/1.1 200 OK Content-Length: 11 Hello World HTTP Response

  176. None
  177. None
  178. None
  179. None
  180. None
  181. None
  182. None
  183. Do not try this at home

  184. Do not try this in production

  185. Be curious

  186. dabit/rails-server-browser

  187. El fin

  188. Thank you! David Padilla @dabit david@crowdint.com