Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Rails to the Webserver to the Browser

From Rails to the Webserver to the Browser

Most of us know how to build beautiful web applications with Rails. With the help of templating tools like ERB and HAML our web apps create HTML documents, but, do you know exactly how those HTML documents end up in a browser?

During this talk I will show you the bits that make it all happen. We will dissect the relevant code within Rails, Rack and the thin web server to discover exactly how the web server starts and listens to a TCP port, communicates with Rails and returns the HTML document that your browser parses.

Why? Because we're curious about it, that's why.

David Padilla

April 30, 2013
Tweet

More Decks by David Padilla

Other Decks in Programming

Transcript

  1. From Rails to the web
    server to the browser
    David Padilla
    @dabit

    View Slide

  2. David Padilla
    @dabit

    View Slide

  3. Hire us!

    View Slide

  4. Training not
    poaching

    View Slide

  5. MagmaConf

    View Slide

  6. View Slide

  7. Manzanillo, Mexico

    View Slide

  8. June 5 - 7

    View Slide

  9. www.magmaconf.com

    View Slide

  10. View Slide


  11. View Slide

  12. From Rails to
    the web server
    to the browser

    View Slide

  13. Life of a
    Request

    View Slide

  14. www.app.com/hello
    HTTP Request

    View Slide

  15. HTTP Request

    View Slide

  16. HTTP Request

    View Slide



  17. Railsconf

    LOL

    View Slide



  18. Railsconf

    LOL

    View Slide

  19. View Slide

  20. Rack

    View Slide

  21. Rack

    View Slide

  22. Rack

    View Slide

  23. class App
    def call(env)
    # ...
    return [http_code, headers, body]
    end
    end

    View Slide

  24. [http_code, headers, body]
    Integer
    Hash
    Responds to
    :each

    View Slide

  25. Somewhere on
    the web
    server...

    View Slide

  26. # ...
    # ...
    # ...
    @app.call(env)
    # ...
    # ...
    # ...

    View Slide

  27. Somewhere on
    the
    Application...

    View Slide

  28. class App
    def call(env)
    # ...
    return [http_code, headers, body]
    end
    end

    View Slide

  29. config.ru

    View Slide

  30. require 'rack/app'
    run Rack::App.new
    config.ru

    View Slide

  31. View Slide

  32. View Slide

  33. rackup

    View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. All Rails apps
    are Rack apps

    View Slide

  40. View Slide

  41. View Slide

  42. class HelloController
    def show
    render text: "Hello World"
    end
    end
    app/controllers/hello_controller.rb

    View Slide

  43. View Slide

  44. App::Application.routes.draw do
    root to: "hello#show"
    end
    config/routes.rb

    View Slide

  45. View Slide

  46. View Slide

  47. require 'test/unit'
    require 'turn'
    require 'net/http'
    class TestServer < Test::Unit::TestCase
    def test_the_request
    uri = URI('http://127.0.0.1:9292')
    assert "Hello World", Net::HTTP.get(uri)
    end
    end
    test/test.rb

    View Slide

  48. View Slide

  49. View Slide

  50. group :development do
    gem 'pry-rails'
    gem 'pry-debugger'
    end
    Gemfile

    View Slide

  51. View Slide

  52. View Slide

  53. View Slide

  54. View Slide

  55. View Slide

  56. View Slide

  57. View Slide

  58. @app.call({
    "REQUEST_METHOD" => "GET",
    "PATH_INFO" => "/",
    "rack.input" => ""
    })

    View Slide

  59. [status, headers, body]
    Rack

    View Slide

  60. View Slide

  61. View Slide

  62. View Slide

  63. View Slide

  64. View Slide

  65. View Slide

  66. View Slide

  67. View Slide

  68. View Slide

  69. View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. StringIO.new(“Hello World”)

    View Slide

  74. View Slide

  75. class App
    def call(env)
    content = "Hello World"
    status_code = 200
    headers = {
    "Content-Length" => content.length.to_s
    }
    [status_code, headers, [content]]
    end
    end
    rack/app.rb

    View Slide

  76. View Slide

  77. Webservers

    View Slide

  78. Down the Rabbit
    Hole

    View Slide

  79. # ...
    # ...
    # ...
    @app.call(env)
    # ...
    # ...
    # ...

    View Slide

  80. Thin

    View Slide

  81. thin start

    View Slide

  82. module Thin
    # The uterly famous Thin HTTP server.
    # It listen for incoming request through
    # a given +backend+
    # and forward all request to +app+.
    #
    # == TCP server
    # Create a new TCP server on bound
    # to host:port by specifiying +host+
    # and +port+ as the first 2 arguments.
    #
    # Thin::Server.start('0.0.0.0', 3000, app)
    lib/thin/server.rb

    View Slide

  83. Thin::Server.start('0.0.0.0', 3000, app)
    thin start

    View Slide

  84. # Start the server and listen for connections.
    def start
    raise ArgumentError, 'app required' unless @app
    log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C
    debug ">> Debugging ON"
    trace ">> Tracing ON"
    log ">> Maximum connections set to #{@backend.maximum_connections}"
    log ">> Listening on #{@backend}, CTRL+C to stop"
    @backend.start
    end
    lib/thin/server.rb

    View Slide

  85. # Start the server and listen for connections.
    def start
    raise ArgumentError, 'app required' unless @app
    log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C
    debug ">> Debugging ON"
    trace ">> Tracing ON"
    log ">> Maximum connections set to #{@backend.maximum_connections}"
    log ">> Listening on #{@backend}, CTRL+C to stop"
    @backend.start
    end
    lib/thin/server.rb

    View Slide

  86. @backend.start
    lib/thin/server.rb

    View Slide

  87. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb

    View Slide

  88. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb

    View Slide

  89. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb

    View Slide

  90. lib/thin/backends/tcp_server.rb

    View Slide

  91. module Thin
    module Backends
    # Backend to act as a TCP socket server.
    class TcpServer < Base
    # Address and port on which the server is listening for connections.
    attr_accessor :host, :port
    def initialize(host, port)
    @host = host
    @port = port
    super()
    end
    # Connect the server
    def connect
    @signature = EventMachine.start_server(@host, @port, Connection, &method(:initializ
    end
    # Stops the server
    def disconnect
    EventMachine.stop_server(@signature)
    end
    def to_s
    "#{@host}:#{@port}"
    end
    end
    end
    end
    lib/thin/backends/tcp_server.rb

    View Slide

  92. # Connect the server
    def connect
    @signature = EventMachine.start_server(@
    end
    lib/thin/backends/tcp_server.rb

    View Slide

  93. EventMachine

    View Slide

  94. module Connection
    def post_init
    # A client connected
    end
    def receive_data data
    # Data received
    end
    def unbind
    # Client disconnected
    end
    end
    EventMachine.start_server("0.0.0.0", 8081, Connection)
    EventMachine

    View Slide

  95. # Connect the server
    def connect
    @signature = EventMachine.start_server(@
    end
    lib/thin/backends/tcp_server.rb

    View Slide

  96. ne.start_server(@host, @port, Connection, &metho
    lib/thin/backends/tcp_server.rb

    View Slide

  97. ne.start_server(@host, @port, Connection, &metho
    lib/thin/backends/tcp_server.rb

    View Slide

  98. lib/thin/connection.rb

    View Slide

  99. View Slide

  100. lib/thin/connection.rb

    View Slide

  101. # Called when data is received from the client.
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  102. # Called when data is received from the client.
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  103. Thin::Request

    View Slide

  104. lib/thin/request.rb

    View Slide

  105. # Parse a chunk of data into the request environment
    # Raises a +InvalidRequest+ if invalid.
    # Returns +true+ if the parsing is complete.
    def parse(data)
    if @parser.finished? # Header finished, can only be some more body
    @body << data
    else # Parse more header using the super parser
    @data << data
    raise InvalidRequest, 'Header longer than allowed' if @data.size > MAX_HEADER
    @nparsed = @parser.execute(@env, @data, @nparsed)
    # Transfert to a tempfile if body is very big
    move_body_to_tempfile if @parser.finished? && content_length > MAX_BODY
    end
    if finished? # Check if header and body are complete
    @data = nil
    @body.rewind
    true # Request is fully parsed
    else
    false # Not finished, need more data
    end
    end
    lib/thin/request.rb

    View Slide

  106. Mongrel Parser

    View Slide

  107. /**
    * Copyright (c) 2005 Zed A. Shaw
    * You can redistribute it and/or modify
    * it under the same terms as Ruby.
    */
    #ifndef http11_parser_h
    #define http11_parser_h
    #include
    #if defined(_WIN32)
    #include
    #endif

    View Slide

  108. GET /index.html HTTP/1.1
    Host:localhost
    HTTP Request

    View Slide

  109. View Slide

  110. View Slide

  111. View Slide

  112. View Slide

  113. View Slide

  114. # Called when data is received from the client
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  115. # Called when data is received from the client
    def receive_data(data)
    @idle = false
    trace { data }
    process if @request.parse(data)
    rescue InvalidRequest => e
    log "!! Invalid request"
    log_error e
    post_process Response::BAD_REQUEST
    end
    lib/thin/connection.rb

    View Slide

  116. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process),
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end
    lib/thin/connection.rb

    View Slide

  117. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process),
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end
    lib/thin/connection.rb

    View Slide

  118. def pre_process
    # Add client info to the request env
    @request.remote_address = remote_address
    # Connection may be closed unless the App#
    # It should be noted that connection objec
    # callback is no longer referenced, so be
    @request.async_callback = method(:post_pro
    if @backend.ssl?
    @request.env["rack.url_scheme"] = "https
    if cert = get_peer_cert
    @request.env['rack.peer_cert'] = cert
    end
    end

    View Slide

  119. end
    # When we're under a non-async framework l
    # off async responses using the callback i
    # in removing this.
    response = AsyncResponse
    catch(:async) do
    # Process the request calling the Rack a
    response = @app.call(@request.env)
    end
    response
    rescue Exception
    handle_error
    # Pass through error response
    can_persist? && @request.persistent? ? Res
    end

    View Slide

  120. end
    # When we're under a non-async framework l
    # off async responses using the callback i
    # in removing this.
    response = AsyncResponse
    catch(:async) do
    # Process the request calling the Rack a
    response = @app.call(@request.env)
    end
    response
    rescue Exception
    handle_error
    # Pass through error response
    can_persist? && @request.persistent? ? Res
    end
    lib/thin/connection.rb

    View Slide

  121. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process),
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end
    lib/thin/connection.rb

    View Slide

  122. # Called when all data was received and
    # the request is ready to be processed.
    def process
    if threaded?
    @request.threaded = true
    EventMachine.defer(method(:pre_process),
    else
    @request.threaded = false
    post_process(pre_process)
    end
    end

    View Slide

  123. HTTP/1.1 200 OK
    Content-Length: 11
    Hello World
    HTTP Response

    View Slide

  124. Unicorn

    View Slide

  125. EventMachine
    No

    View Slide

  126. @app.call(env)
    YES

    View Slide

  127. lib/unicorn/http_server.rb
    def process_client(client)

    View Slide

  128. Uses the
    mongrel parser
    Kinda

    View Slide

  129. PUMA

    View Slide

  130. What we’ve
    learned so far

    View Slide

  131. EventMachine.start_server("0.0.0.0", 8081, Connection)

    View Slide

  132. GET /index.html HTTP/1.1
    Host:localhost

    View Slide

  133. GET /index.html HTTP/1.1
    Host:localhost
    {
    "REQUEST_METHOD" => "GET",
    "PATH_INFO" => "/",
    "rack.input" => ""
    }

    View Slide

  134. @app.call({
    "REQUEST_METHOD" => "GET",
    "PATH_INFO" => "/",
    "rack.input" => ""
    })

    View Slide

  135. [http_code, headers, body]

    View Slide

  136. [http_code, headers, body]
    HTTP/1.1 200 OK
    Content-Length: 11
    Hello World

    View Slide

  137. HTTP/1.1 200 OK
    Content-Length: 11
    Hello World

    View Slide

  138. Our own
    Webserver

    View Slide

  139. View Slide

  140. module Server
    def receive_data(data)
    puts data
    end
    end
    rack/server.rb

    View Slide

  141. View Slide

  142. View Slide

  143. module Server
    def receive_data(data)
    request = Thin::Request.new
    request.parse(data)
    puts request.env
    end
    end
    rack/server.rb

    View Slide

  144. View Slide

  145. View Slide

  146. def receive_data(data)
    request = Thin::Request.new
    request.parse(data)
    app = App.new
    response = app.call(request.env)
    end
    rack/server.rb

    View Slide

  147. View Slide

  148. HTTP/1.1 200 OK
    Content-Length: 11
    Hello World
    HTTP Response

    View Slide

  149. View Slide

  150. View Slide

  151. View Slide

  152. View Slide

  153. View Slide

  154. View Slide

  155. View Slide

  156. Do not try this
    at home

    View Slide

  157. Do not try this
    in production

    View Slide

  158. Be curious

    View Slide

  159. dabit/rails-server-browser

    View Slide

  160. El fin

    View Slide

  161. Thank you!
    David Padilla
    @dabit
    [email protected]

    View Slide