From Rails to the Webserver to the Browser

Most of us know how to build beautiful web applications with Rails. With the help of templating tools like ERB and HAML our web apps create HTML documents, but, do you know exactly how those HTML documents end up in a browser?

During this talk I will show you the bits that make it all happen. We will dissect the relevant code within Rails, Rack and the thin web server to discover exactly how the web server starts and listens to a TCP port, communicates with Rails and returns the HTML document that your browser parses.

Why? Because we're curious about it, that's why.

Transcript

1. From Rails to the web
   server to the browser
   David Padilla
   @dabit
   

   View Slide

2. David Padilla
   @dabit
   

   View Slide

3. Hire us!
   

   View Slide

4. Training not
   poaching
   

   View Slide

5. MagmaConf
   

   View Slide

6. View Slide

7. Manzanillo, Mexico
   

   View Slide

8. June 5 - 7
   

   View Slide

9. www.magmaconf.com
   

   View Slide

10. View Slide

11. 
    

    View Slide

12. From Rails to
    the web server
    to the browser
    

    View Slide

13. Life of a
    Request
    

    View Slide

14. www.app.com/hello
    HTTP Request
    

    View Slide

15. HTTP Request
    

    View Slide

16. HTTP Request
    

    View Slide

17. 
    
    Railsconf
    
    LOL
    
    

    View Slide

18. 
    
    Railsconf
    
    LOL
    
    

    View Slide

19. View Slide

20. Rack
    

    View Slide

21. Rack
    

    View Slide

22. Rack
    

    View Slide

23. class App
    def call(env)
    # ...
    return [http_code, headers, body]
    end
    end
    

    View Slide

24. [http_code, headers, body]
    Integer
    Hash
    Responds to
    :each
    

    View Slide

25. Somewhere on
    the web
    server...
    

    View Slide

26. # ...
    # ...
    # ...
    @app.call(env)
    # ...
    # ...
    # ...
    

    View Slide

27. Somewhere on
    the
    Application...
    

    View Slide

28. class App
    def call(env)
    # ...
    return [http_code, headers, body]
    end
    end
    

    View Slide

29. config.ru
    

    View Slide

30. require 'rack/app'
    run Rack::App.new
    config.ru
    

    View Slide

31. View Slide

32. View Slide

33. rackup
    

    View Slide

34. View Slide

35. View Slide

36. View Slide

37. View Slide

38. View Slide

39. All Rails apps
    are Rack apps
    

    View Slide

40. View Slide

41. View Slide

42. class HelloController
    def show
    render text: "Hello World"
    end
    end
    app/controllers/hello_controller.rb
    

    View Slide

43. View Slide

44. App::Application.routes.draw do
    root to: "hello#show"
    end
    config/routes.rb
    

    View Slide

45. View Slide

46. View Slide

47. require 'test/unit'
    require 'turn'
    require 'net/http'
    class TestServer < Test::Unit::TestCase
    def test_the_request
    uri = URI('http://127.0.0.1:9292')
    assert "Hello World", Net::HTTP.get(uri)
    end
    end
    test/test.rb
    

    View Slide

48. View Slide

49. View Slide

50. group :development do
    gem 'pry-rails'
    gem 'pry-debugger'
    end
    Gemfile
    

    View Slide

51. View Slide

52. View Slide

53. View Slide

54. View Slide

55. View Slide

56. View Slide

57. View Slide

58. @app.call({
    "REQUEST_METHOD" => "GET",
    "PATH_INFO" => "/",
    "rack.input" => ""
    })
    

    View Slide

59. [status, headers, body]
    Rack
    

    View Slide

60. View Slide

61. View Slide

62. View Slide

63. View Slide

64. View Slide

65. View Slide

66. View Slide

67. View Slide

68. View Slide

69. View Slide

70. View Slide

71. View Slide

72. View Slide

73. StringIO.new(“Hello World”)
    

    View Slide

74. View Slide

75. class App
    def call(env)
    content = "Hello World"
    status_code = 200
    headers = {
    "Content-Length" => content.length.to_s
    }
    [status_code, headers, [content]]
    end
    end
    rack/app.rb
    

    View Slide

76. View Slide

77. Webservers
    

    View Slide

78. Down the Rabbit
    Hole
    

    View Slide

79. # ...
    # ...
    # ...
    @app.call(env)
    # ...
    # ...
    # ...
    

    View Slide

80. Thin
    

    View Slide

81. thin start
    

    View Slide

82. module Thin
    # The uterly famous Thin HTTP server.
    # It listen for incoming request through
    # a given +backend+
    # and forward all request to +app+.
    #
    # == TCP server
    # Create a new TCP server on bound
    # to host:port by specifiying +host+
    # and +port+ as the first 2 arguments.
    #
    # Thin::Server.start('0.0.0.0', 3000, app)
    lib/thin/server.rb
    

    View Slide

83. Thin::Server.start('0.0.0.0', 3000, app)
    thin start
    

    View Slide

84. # Start the server and listen for connections.
    def start
    raise ArgumentError, 'app required' unless @app
    log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C
    debug ">> Debugging ON"
    trace ">> Tracing ON"
    log ">> Maximum connections set to #{@backend.maximum_connections}"
    log ">> Listening on #{@backend}, CTRL+C to stop"
    @backend.start
    end
    lib/thin/server.rb
    

    View Slide

85. # Start the server and listen for connections.
    def start
    raise ArgumentError, 'app required' unless @app
    log ">> Thin web server (v#{VERSION::STRING} codename #{VERSION::C
    debug ">> Debugging ON"
    trace ">> Tracing ON"
    log ">> Maximum connections set to #{@backend.maximum_connections}"
    log ">> Listening on #{@backend}, CTRL+C to stop"
    @backend.start
    end
    lib/thin/server.rb
    

    View Slide

86. @backend.start
    lib/thin/server.rb
    

    View Slide

87. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb
    

    View Slide

88. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb
    

    View Slide

89. def select_backend(host, port, options)
    case
    when options.has_key?(:backend)
    raise ArgumentError, ":backend must be
    options[:backend].new(host, port, optio
    when options.has_key?(:swiftiply)
    Backends::SwiftiplyClient.new(host, por
    when host.include?('/')
    Backends::UnixServer.new(host)
    else
    Backends::TcpServer.new(host, port)
    end
    end
    lib/thin/server.rb
    

    View Slide

90. lib/thin/backends/tcp_server.rb
    

    View Slide

91. module Thin
    module Backends
    # Backend to act as a TCP socket server.
    class TcpServer < Base
    # Address and port on which the server is listening for connections.
    attr_accessor :host, :port
    def initialize(host, port)
    @host = host
    @port = port
    super()
    end
    # Connect the server
    def connect
    @signature = EventMachine.start_server(@host, @port, Connection, &method(:initializ
    end
    # Stops the server
    def disconnect
    EventMachine.stop_server(@signature)
    end
    def to_s
    "#{@host}:#{@port}"
    end
    end
    end
    end
    lib/thin/backends/tcp_server.rb
    

    View Slide

92. # Connect the server
    def connect
    @signature = EventMachine.start_server(@
    end
    lib/thin/backends/tcp_server.rb
    

    View Slide

93. EventMachine
    

    View Slide

94. module Connection
    def post_init
    # A client connected
    end
    def receive_data data
    # Data received
    end
    def unbind
    # Client disconnected
    end
    end
    EventMachine.start_server("0.0.0.0", 8081, Connection)
    EventMachine
    

    View Slide

95. # Connect the server
    def connect
    @signature = EventMachine.start_server(@
    end
    lib/thin/backends/tcp_server.rb
    

    View Slide

96. ne.start_server(@host, @port, Connection, &metho
    lib/thin/backends/tcp_server.rb
    

    View Slide

97. ne.start_server(@host, @port, Connection, &metho
    lib/thin/backends/tcp_server.rb
    

    View Slide

98. lib/thin/connection.rb
    

    View Slide

99. View Slide

100. lib/thin/connection.rb
     

     View Slide

101. # Called when data is received from the client.
     def receive_data(data)
     @idle = false
     trace { data }
     process if @request.parse(data)
     rescue InvalidRequest => e
     log "!! Invalid request"
     log_error e
     post_process Response::BAD_REQUEST
     end
     lib/thin/connection.rb
     

     View Slide

102. # Called when data is received from the client.
     def receive_data(data)
     @idle = false
     trace { data }
     process if @request.parse(data)
     rescue InvalidRequest => e
     log "!! Invalid request"
     log_error e
     post_process Response::BAD_REQUEST
     end
     lib/thin/connection.rb
     

     View Slide

103. Thin::Request
     

     View Slide

104. lib/thin/request.rb
     

     View Slide

105. # Parse a chunk of data into the request environment
     # Raises a +InvalidRequest+ if invalid.
     # Returns +true+ if the parsing is complete.
     def parse(data)
     if @parser.finished? # Header finished, can only be some more body
     @body << data
     else # Parse more header using the super parser
     @data << data
     raise InvalidRequest, 'Header longer than allowed' if @data.size > MAX_HEADER
     @nparsed = @parser.execute(@env, @data, @nparsed)
     # Transfert to a tempfile if body is very big
     move_body_to_tempfile if @parser.finished? && content_length > MAX_BODY
     end
     if finished? # Check if header and body are complete
     @data = nil
     @body.rewind
     true # Request is fully parsed
     else
     false # Not finished, need more data
     end
     end
     lib/thin/request.rb
     

     View Slide

106. Mongrel Parser
     

     View Slide

107. /**
     * Copyright (c) 2005 Zed A. Shaw
     * You can redistribute it and/or modify
     * it under the same terms as Ruby.
     */
     #ifndef http11_parser_h
     #define http11_parser_h
     #include
     #if defined(_WIN32)
     #include
     #endif
     

     View Slide

108. GET /index.html HTTP/1.1
     Host:localhost
     HTTP Request
     

     View Slide

109. View Slide

110. View Slide

111. View Slide

112. View Slide

113. View Slide

114. # Called when data is received from the client
     def receive_data(data)
     @idle = false
     trace { data }
     process if @request.parse(data)
     rescue InvalidRequest => e
     log "!! Invalid request"
     log_error e
     post_process Response::BAD_REQUEST
     end
     lib/thin/connection.rb
     

     View Slide

115. # Called when data is received from the client
     def receive_data(data)
     @idle = false
     trace { data }
     process if @request.parse(data)
     rescue InvalidRequest => e
     log "!! Invalid request"
     log_error e
     post_process Response::BAD_REQUEST
     end
     lib/thin/connection.rb
     

     View Slide

116. # Called when all data was received and
     # the request is ready to be processed.
     def process
     if threaded?
     @request.threaded = true
     EventMachine.defer(method(:pre_process),
     else
     @request.threaded = false
     post_process(pre_process)
     end
     end
     lib/thin/connection.rb
     

     View Slide

117. # Called when all data was received and
     # the request is ready to be processed.
     def process
     if threaded?
     @request.threaded = true
     EventMachine.defer(method(:pre_process),
     else
     @request.threaded = false
     post_process(pre_process)
     end
     end
     lib/thin/connection.rb
     

     View Slide

118. def pre_process
     # Add client info to the request env
     @request.remote_address = remote_address
     # Connection may be closed unless the App#
     # It should be noted that connection objec
     # callback is no longer referenced, so be
     @request.async_callback = method(:post_pro
     if @backend.ssl?
     @request.env["rack.url_scheme"] = "https
     if cert = get_peer_cert
     @request.env['rack.peer_cert'] = cert
     end
     end
     

     View Slide

119. end
     # When we're under a non-async framework l
     # off async responses using the callback i
     # in removing this.
     response = AsyncResponse
     catch(:async) do
     # Process the request calling the Rack a
     response = @app.call(@request.env)
     end
     response
     rescue Exception
     handle_error
     # Pass through error response
     can_persist? && @request.persistent? ? Res
     end
     

     View Slide

120. end
     # When we're under a non-async framework l
     # off async responses using the callback i
     # in removing this.
     response = AsyncResponse
     catch(:async) do
     # Process the request calling the Rack a
     response = @app.call(@request.env)
     end
     response
     rescue Exception
     handle_error
     # Pass through error response
     can_persist? && @request.persistent? ? Res
     end
     lib/thin/connection.rb
     

     View Slide

121. # Called when all data was received and
     # the request is ready to be processed.
     def process
     if threaded?
     @request.threaded = true
     EventMachine.defer(method(:pre_process),
     else
     @request.threaded = false
     post_process(pre_process)
     end
     end
     lib/thin/connection.rb
     

     View Slide

122. # Called when all data was received and
     # the request is ready to be processed.
     def process
     if threaded?
     @request.threaded = true
     EventMachine.defer(method(:pre_process),
     else
     @request.threaded = false
     post_process(pre_process)
     end
     end
     

     View Slide

123. HTTP/1.1 200 OK
     Content-Length: 11
     Hello World
     HTTP Response
     

     View Slide

124. Unicorn
     

     View Slide

125. EventMachine
     No
     

     View Slide

126. @app.call(env)
     YES
     

     View Slide

127. lib/unicorn/http_server.rb
     def process_client(client)
     

     View Slide

128. Uses the
     mongrel parser
     Kinda
     

     View Slide

129. PUMA
     

     View Slide

130. What we’ve
     learned so far
     

     View Slide

131. EventMachine.start_server("0.0.0.0", 8081, Connection)
     

     View Slide

132. GET /index.html HTTP/1.1
     Host:localhost
     

     View Slide

133. GET /index.html HTTP/1.1
     Host:localhost
     {
     "REQUEST_METHOD" => "GET",
     "PATH_INFO" => "/",
     "rack.input" => ""
     }
     

     View Slide

134. @app.call({
     "REQUEST_METHOD" => "GET",
     "PATH_INFO" => "/",
     "rack.input" => ""
     })
     

     View Slide

135. [http_code, headers, body]
     

     View Slide

136. [http_code, headers, body]
     HTTP/1.1 200 OK
     Content-Length: 11
     Hello World
     

     View Slide

137. HTTP/1.1 200 OK
     Content-Length: 11
     Hello World
     

     View Slide

138. Our own
     Webserver
     

     View Slide

139. View Slide

140. module Server
     def receive_data(data)
     puts data
     end
     end
     rack/server.rb
     

     View Slide

141. View Slide

142. View Slide

143. module Server
     def receive_data(data)
     request = Thin::Request.new
     request.parse(data)
     puts request.env
     end
     end
     rack/server.rb
     

     View Slide

144. View Slide

145. View Slide

146. def receive_data(data)
     request = Thin::Request.new
     request.parse(data)
     app = App.new
     response = app.call(request.env)
     end
     rack/server.rb
     

     View Slide

147. View Slide

148. HTTP/1.1 200 OK
     Content-Length: 11
     Hello World
     HTTP Response
     

     View Slide

149. View Slide

150. View Slide

151. View Slide

152. View Slide

153. View Slide

154. View Slide

155. View Slide

156. Do not try this
     at home
     

     View Slide

157. Do not try this
     in production
     

     View Slide

158. Be curious
     

     View Slide

159. dabit/rails-server-browser
     

     View Slide

160. El fin
     

     View Slide

161. Thank you!
     David Padilla
     @dabit
     david@crowdint.com
     

     View Slide