Web servers Igor Afonov @iafonov (And how I created my own one) 

Basic principles Web server 

Basic principles Web server HTTP Request HTTP Response 

HTTP • Text based • RFC 2616 

HTTP POST /files/new/isubjkzdaeqtfhmycrnwlpvgxo HTTP/1.1 Host: 127.0.0.1:31337 Connection: keep-alive Content-Length: 219 Origin: http://127.0.0.1:31337 Accept-Encoding: gzip,deflate Accept-Language: en-US Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 ------WebKitFormBoundaryVreBnvR3qKu3B5Iv Content-Disposition: form-data; name="file"; filename="smally" Content-Type: application/octet-stream Very small file. ------WebKitFormBoundaryVreBnvR3qKu3B5Iv-- 

Lifecycle • Setup networking & signal handlers • Accept connection • Parse & process request • Serialize response • Send it back 

Networking • socket • bind • listen • accept • send/recv/sendfile* • close do the job POSIX 

Blocking IO • select • Forking (Pre-forking) • Threading • Bad idea for web sockets (Apache) 

Non-blocking IO • kqueue/poll/epoll • libevent/libev/libuv • Reactor/Notifier/Dispatcher pattern • Works good for web sockets & persistent connections (Nginx, node.js, tornado, event machine...) 

The Big Picture Your Application Rack Web server Operating System Rails Request Response 

Cosmonaut • Blocking (At least for now) • Simple & straightforward • Made from tiny isolated modules • Fast, not so slow (Implemented in C) • 3 weeks old 

Features • Serves static content (via sendfile) • Parses multipart data • Basic routing • Provides simple API • Tiny memory footprint 

Architecture • Main process forks workers for each request • Forking is fast & reliable with copy-on-write • ~10K ram per request • Context switching is expensive :( • ~600 concurrent processes max on Xeon 

C • Surprisingly simple • Dangerous by default, safe by demand • Still modern 

C & OOP module.h typedef struct module module; typedef struct module_state module_state; struct module { void *public_data; polymorfic *polymorfic; module_state *_s; }; module *module_init(...); void module_free(module *m); int module_do_something(module *m); 

Memory Management • Allocate & free at the same level • Allocate & free at the same module • Use Valgrind • Be careful 

Parsing HTTP • Server is set of parsers • Callback based • Must work with chunks of data • Stateful • No buffering • No bugs 

API #include void action_index(http_request *request, http_response *response) { render_text(response, "Hello world"); } void configure() { mount("/", action_index); } int main(int argc, char *argv[]) { return cosmonaut_start(argc, argv, configure); } Hello World 

API struct http_request { url *url; headers_map *headers; params_map *params; configuration *configuration; route *route; void *data; char *uid; }; char *c_type = headers_map_get(request->headers, "Content-Type"); Request 

API typedef struct http_response { int code; char *header_summary; char *file_path; char *content_type; int content_length; headers_map* headers; char* raw_response; } http_response; render_file(http_response *response, const char *path); render_text(http_response *response, const char *text); render_json(http_response *response, const char *json); Response 

API mount("/", action_index); mount("/upload_file", action_upload); Routing pt.1 

API mount("/photos/(:id)", action); void action(http_request *request, http_response *response) { int photo_id = params_map_get(request->params, "id")->val; ... } Routing pt.2 

API mount("/patients/(:id)/(:action)", rest_application); void rest_application(http_request *request, http_response *response) { char *action = params_map_get(request->params, "action")->val; int patient_id = params_map_get(request->params, "id")->val; process_data(patient, action, response); } /patients/1/new // params: {id => 1, action => "new"} /patients/1/show // params: {id => 1, action => "show"} /patients/1/edit // params: {id => 1, action => "edit"} /patients/1/delete // params: {id => 1, action => "delete"} Routing pt.3 

API route *rt = mount("/files/new/(:upload_id)", action_upload); rt->before_filter = check_upload_size; rt->after_filter = clear_db_connections; Filters 

Testing • Cucumber • Rest-client • Travis CI 

Testing Feature: Routing and params processing Scenario Outline: Requesting action mounted to '/photos/(:id)' When I request "" Then I should get "" Examples: | path | expected_response | | /photos/1 | [id:1] | | /photos/word | [id:word] | | /photos/1/somth | Not Found | | /photos/1/ | Not Found | | /photos1/1 | Not Found | 

Testing When /^I request "([^"]*)"$/ do |path| begin @response = RestClient.get("http://127.0.0.1:31337#{path}") rescue Exception => e @client_exception = e end end Then /^I should get "([^"]*)"$/ do |body| response_body = @client_exception.nil? ? @response.to_s : @client_exception.http_body response_body.should == body end 

Sample Application • Supports file uploads/downloads • Supports tracking of upload progress • Uses redis for persistence & IPC https://github.com/iafonov/uploader_app 

Don't reinvent the wheel Do it! 

https://github.com/iafonov/cosmonaut https://github.com/iafonov/cosmonaut/wiki/Links http://iafonov.github.com/ @iafonov