Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Rails to the Webserver to the Browser (Ruby Kaigi)

From Rails to the Webserver to the Browser (Ruby Kaigi)

Most of us know how to build beautiful web applications with Rails. With the help of templating tools like ERB and HAML our web apps create HTML documents, but, do you know exactly how those HTML documents end up in a browser?
During this talk I will show you the bits that make it all happen. We will dissect the relevant code within Rails, Rack and the thin web server to discover exactly how the web server starts and listens to a TCP port, communicates with Rails and returns the HTML document that your browser parses.
Why? Because we're curious about it, that's why.

David Padilla

June 01, 2013
Tweet

More Decks by David Padilla

Other Decks in Technology

Transcript

  1. From Rails to the web
    server to the browser
    David Padilla
    @dabit

    View Slide

  2. David Padilla - @dabit
    From Rails to the webserver to the browser

    View Slide

  3. David Padilla
    @dabit

    View Slide

  4. www.crowdint.com

    View Slide

  5. Image from: ezilon.com

    View Slide

  6. Image from: ezilon.com

    View Slide

  7. Image from: ezilon.com

    View Slide

  8. View Slide

  9. View Slide

  10. Population
    Tokyo Colima
    0
    3,750,000.00
    7,500,000.00
    11,250,000.00
    15,000,000.00

    View Slide

  11. Population
    Tokyo Colima
    0
    3,750,000.00
    7,500,000.00
    11,250,000.00
    15,000,000.00

    View Slide

  12. Population
    Tokyo Colima
    0
    3,750,000.00
    7,500,000.00
    11,250,000.00
    15,000,000.00

    View Slide

  13. MagmaConf

    View Slide

  14. Manzanillo, Mexico

    View Slide

  15. June 5 - 7

    View Slide

  16. MagmaConf

    View Slide

  17. View Slide

  18. From Rails to
    the web server
    to the browser

    View Slide

  19. Life of a
    Request

    View Slide

  20. View Slide

  21. www.app.com/hello

    View Slide

  22. www.app.com/hello
    HTTP Request

    View Slide

  23. View Slide

  24. HTTP Request

    View Slide

  25. View Slide

  26. HTTP Request

    View Slide



  27. Railsconf

    LOL

    View Slide

  28. View Slide

  29. View Slide



  30. Railsconf

    LOL

    View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. Rack

    View Slide

  35. Rack

    View Slide

  36. Rack

    View Slide

  37. Rack

    View Slide

  38. View Slide

  39. class App
    def call(env)
    # ...
    return [http_code, headers, body]
    end
    end

    View Slide

  40. [http_code, headers, body]

    View Slide

  41. [http_code, headers, body]
    Integer

    View Slide

  42. [http_code, headers, body]
    Integer
    Hash

    View Slide

  43. [http_code, headers, body]
    Integer
    Hash
    Responds to
    :each

    View Slide

  44. Somewhere on
    the web
    server...

    View Slide

  45. # ...
    # ...
    # ...
    @app.call(env)
    # ...
    # ...
    # ...

    View Slide

  46. Somewhere on
    the
    Application...

    View Slide

  47. class App
    def call(env)
    # ...
    return [http_code, headers, body]
    end
    end

    View Slide

  48. config.ru

    View Slide

  49. require 'rack/app'
    run Rack::App.new
    config.ru

    View Slide

  50. View Slide

  51. View Slide

  52. rackup

    View Slide

  53. View Slide

  54. View Slide

  55. View Slide

  56. View Slide

  57. View Slide

  58. View Slide

  59. All Rails apps
    are Rack apps

    View Slide

  60. View Slide

  61. View Slide

  62. class HelloController
    def show
    render text: "Hello World"
    end
    end
    app/controllers/hello_controller.rb

    View Slide

  63. View Slide

  64. App::Application.routes.draw do
    root to: "hello#show"
    end
    config/routes.rb

    View Slide

  65. View Slide

  66. View Slide

  67. class TestServer < Test::Unit::TestCase
    def test_the_request
    uri = URI('http://127.0.0.1:9292')
    assert "Hello World", Net::HTTP.get(uri)
    end
    end
    test/test.rb

    View Slide

  68. View Slide

  69. View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. View Slide

  74. View Slide

  75. View Slide

  76. View Slide

  77. @app.call({
    "REQUEST_METHOD" => "GET",
    "PATH_INFO" => "/",
    "rack.input" => ""
    })

    View Slide

  78. [status, headers, body]
    Rack

    View Slide

  79. View Slide

  80. View Slide

  81. View Slide

  82. View Slide

  83. View Slide

  84. View Slide

  85. View Slide

  86. View Slide

  87. View Slide

  88. View Slide

  89. View Slide

  90. View Slide

  91. View Slide

  92. View Slide

  93. View Slide

  94. StringIO.new(“Hello World”)

    View Slide

  95. View Slide

  96. class App
    def call(env)
    content = "Hello World"
    status_code = 200
    headers = {
    "Content-Length" => content.length.to_s
    }
    [status_code, headers, [content]]
    end
    end
    rack/app.rb

    View Slide

  97. View Slide

  98. Webservers

    View Slide

  99. # ...
    # ...
    # ...
    @app.call(env)
    # ...
    # ...
    # ...

    View Slide

  100. Thin

    View Slide

  101. thin start

    View Slide

  102. module Thin
    # The uterly famous Thin HTTP server.
    # It listen for incoming request through
    # a given +backend+
    # and forward all request to +app+.
    #
    # == TCP server
    # Create a new TCP server on bound
    # to host:port by specifiying +host+
    # and +port+ as the first 2 arguments.
    #
    # Thin::Server.start('0.0.0.0', 3000, app)
    lib/thin/server.rb

    View Slide

  103. Thin::Server.start('0.0.0.0', 3000, app)
    thin start

    View Slide

  104. # Start the server and listen for connections.
    def start
    raise ArgumentError, 'app required' unless @app
    log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C
    debug ">> Debugging ON"
    trace ">> Tracing ON"
    log ">> Maximum connections set to #{@backend.maximum_connections}"
    log ">> Listening on #{@backend}, CTRL+C to stop"
    @backend.start
    end
    lib/thin/server.rb

    View Slide

  105. # Start the server and listen for connections.
    def start
    raise ArgumentError, 'app required' unless @app
    log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C
    debug ">> Debugging ON"
    trace ">> Tracing ON"
    log ">> Maximum connections set to #{@backend.maximum_connections}"
    log ">> Listening on #{@backend}, CTRL+C to stop"
    @backend.start
    end
    lib/thin/server.rb

    View Slide

  106. @backend.start
    lib/thin/server.rb

    View Slide

  107. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb

    View Slide

  108. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb

    View Slide

  109. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb

    View Slide

  110. lib/thin/backends/tcp_server.rb

    View Slide

  111. module Thin
    module Backends
    # Backend to act as a TCP socket server.
    class TcpServer < Base
    # Address and port on which the server is listening for connections.
    attr_accessor :host, :port
    def initialize(host, port)
    @host = host
    @port = port
    super()
    end
    # Connect the server
    def connect
    @signature = EventMachine.start_server(@host, @port, Connection, &method(:initializ
    end
    # Stops the server
    def disconnect
    EventMachine.stop_server(@signature)
    end
    def to_s
    "#{@host}:#{@port}"
    end
    end
    end
    end
    lib/thin/backends/tcp_server.rb

    View Slide

  112. # Connect the server
    def connect
    @signature = EventMachine.start_server(@
    end
    lib/thin/backends/tcp_server.rb

    View Slide

  113. EventMachine

    View Slide

  114. module Connection
    def post_init
    # A client connected
    end
    def receive_data data
    # Data received
    end
    def unbind
    # Client disconnected
    end
    end
    EventMachine.start_server("0.0.0.0", 8081, Connection)
    EventMachine

    View Slide

  115. # Connect the server
    def connect
    @signature = EventMachine.start_server(@
    end
    lib/thin/backends/tcp_server.rb

    View Slide

  116. ne.start_server(@host, @port, Connection, &metho
    lib/thin/backends/tcp_server.rb

    View Slide

  117. ne.start_server(@host, @port, Connection, &metho
    lib/thin/backends/tcp_server.rb

    View Slide

  118. ne.start_server(@host, @port, Connection, &metho
    lib/thin/backends/tcp_server.rb

    View Slide

  119. lib/thin/connection.rb

    View Slide

  120. lib/thin/connection.rb

    View Slide

  121. # Called when data is received from the client.
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  122. # Called when data is received from the client.
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  123. Thin::Request

    View Slide

  124. lib/thin/request.rb

    View Slide

  125. # Parse a chunk of data into the request environment
    # Raises a +InvalidRequest+ if invalid.
    # Returns +true+ if the parsing is complete.
    def parse(data)
    if @parser.finished? # Header finished, can only be some more body
    @body << data
    else # Parse more header using the super parser
    @data << data
    raise InvalidRequest, 'Header longer than allowed' if @data.size > MAX_HEADER
    @nparsed = @parser.execute(@env, @data, @nparsed)
    # Transfert to a tempfile if body is very big
    move_body_to_tempfile if @parser.finished? && content_length > MAX_BODY
    end
    if finished? # Check if header and body are complete
    @data = nil
    @body.rewind
    true # Request is fully parsed
    else
    false # Not finished, need more data
    end
    end
    lib/thin/request.rb

    View Slide

  126. Mongrel Parser

    View Slide

  127. /**
    * Copyright (c) 2005 Zed A. Shaw
    * You can redistribute it and/or modify
    * it under the same terms as Ruby.
    */
    #ifndef http11_parser_h
    #define http11_parser_h
    #include
    #if defined(_WIN32)
    #include
    #endif

    View Slide

  128. GET /index.html HTTP/1.1
    Host:localhost
    HTTP Request

    View Slide

  129. View Slide

  130. GET /index.html HTTP/1.1\r\n
    Host:localhost\r\n\r\n
    HTTP Request

    View Slide

  131. View Slide

  132. View Slide

  133. View Slide

  134. View Slide

  135. View Slide

  136. # Called when data is received from the client
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  137. # Called when data is received from the client
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  138. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process), me
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end
    lib/thin/connection.rb

    View Slide

  139. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process),
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end
    lib/thin/connection.rb

    View Slide

  140. def pre_process
    # Add client info to the request env
    @request.remote_address = remote_address
    # Connection may be closed unless the App#
    # It should be noted that connection objec
    # callback is no longer referenced, so be
    @request.async_callback = method(:post_pro
    if @backend.ssl?
    @request.env["rack.url_scheme"] = "https
    if cert = get_peer_cert
    @request.env['rack.peer_cert'] = cert
    end
    end

    View Slide

  141. end
    # When we're under a non-async framework l
    # off async responses using the callback i
    # in removing this.
    response = AsyncResponse
    catch(:async) do
    # Process the request calling the Rack a
    response = @app.call(@request.env)
    end
    response
    rescue Exception
    handle_error
    # Pass through error response
    can_persist? && @request.persistent? ? Res
    end

    View Slide

  142. end
    # When we're under a non-async framework l
    # off async responses using the callback i
    # in removing this.
    response = AsyncResponse
    catch(:async) do
    # Process the request calling the Rack a
    response = @app.call(@request.env)
    end
    response
    rescue Exception
    handle_error
    # Pass through error response
    can_persist? && @request.persistent? ? Res
    end
    lib/thin/connection.rb

    View Slide

  143. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process),
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end
    lib/thin/connection.rb

    View Slide

  144. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process),
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end

    View Slide

  145. HTTP/1.1 200 OK
    Content-Length: 11
    Hello World
    HTTP Response

    View Slide

  146. Unicorn

    View Slide

  147. EventMachine

    View Slide

  148. EventMachine
    No

    View Slide

  149. @app.call(env)

    View Slide

  150. @app.call(env)
    YES

    View Slide

  151. lib/unicorn/http_server.rb
    def process_client(client)

    View Slide

  152. Uses the
    mongrel parser

    View Slide

  153. Uses the
    mongrel parser
    Kinda

    View Slide

  154. PUMA

    View Slide

  155. What we’ve
    learned so far

    View Slide

  156. View Slide

  157. EventMachine.start_server("0.0.0.0", 8081, Connection)

    View Slide

  158. View Slide

  159. GET /index.html HTTP/1.1
    Host:localhost

    View Slide

  160. GET /index.html HTTP/1.1
    Host:localhost
    {
    "REQUEST_METHOD" => "GET",
    "PATH_INFO" => "/",
    "rack.input" => ""
    }

    View Slide

  161. @app.call({
    "REQUEST_METHOD" => "GET",
    "PATH_INFO" => "/",
    "rack.input" => ""
    })

    View Slide

  162. [http_code, headers, body]

    View Slide

  163. [http_code, headers, body]
    HTTP/1.1 200 OK
    Content-Length: 11
    Hello World

    View Slide

  164. HTTP/1.1 200 OK
    Content-Length: 11
    Hello World

    View Slide

  165. Our own
    Webserver

    View Slide

  166. View Slide

  167. module Server
    def receive_data(data)
    puts data
    end
    end
    rack/server.rb

    View Slide

  168. View Slide

  169. View Slide

  170. module Server
    def receive_data(data)
    request = Thin::Request.new
    request.parse(data)
    puts request.env
    end
    end
    rack/server.rb

    View Slide

  171. View Slide

  172. View Slide

  173. def receive_data(data)
    request = Thin::Request.new
    request.parse(data)
    app = App.new
    response = app.call(request.env)
    end
    rack/server.rb

    View Slide

  174. View Slide

  175. HTTP/1.1 200 OK
    Content-Length: 11
    Hello World
    HTTP Response

    View Slide

  176. View Slide

  177. View Slide

  178. View Slide

  179. View Slide

  180. View Slide

  181. View Slide

  182. View Slide

  183. Do not try this
    at home

    View Slide

  184. Do not try this
    in production

    View Slide

  185. Be curious

    View Slide

  186. dabit/rails-server-browser

    View Slide

  187. El fin

    View Slide

  188. Thank you!
    David Padilla
    @dabit
    [email protected]

    View Slide