Web servers - and how I created my own one

Web servers - and how I created my own one

Coffe and Code Donetsk Jan 2012

Dd13a61bab3fe4243f3ebc683f9219eb?s=128

Igor Afonov

January 21, 2012
Tweet

Transcript

  1. Web servers Igor Afonov @iafonov (And how I created my

    own one)
  2. Basic principles Web server

  3. Basic principles Web server HTTP Request HTTP Response

  4. HTTP • Text based • RFC 2616

  5. HTTP POST /files/new/isubjkzdaeqtfhmycrnwlpvgxo HTTP/1.1 Host: 127.0.0.1:31337 Connection: keep-alive Content-Length: 219

    Origin: http://127.0.0.1:31337 Accept-Encoding: gzip,deflate Accept-Language: en-US Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 ------WebKitFormBoundaryVreBnvR3qKu3B5Iv Content-Disposition: form-data; name="file"; filename="smally" Content-Type: application/octet-stream Very small file. ------WebKitFormBoundaryVreBnvR3qKu3B5Iv--
  6. Lifecycle • Setup networking & signal handlers • Accept connection

    • Parse & process request • Serialize response • Send it back
  7. Networking • socket • bind • listen • accept •

    send/recv/sendfile* • close do the job POSIX
  8. Blocking IO • select • Forking (Pre-forking) • Threading •

    Bad idea for web sockets (Apache)
  9. Non-blocking IO • kqueue/poll/epoll • libevent/libev/libuv • Reactor/Notifier/Dispatcher pattern •

    Works good for web sockets & persistent connections (Nginx, node.js, tornado, event machine...)
  10. The Big Picture Your Application Rack Web server Operating System

    Rails Request Response
  11. Cosmonaut • Blocking (At least for now) • Simple &

    straightforward • Made from tiny isolated modules • Fast, not so slow (Implemented in C) • 3 weeks old
  12. Features • Serves static content (via sendfile) • Parses multipart

    data • Basic routing • Provides simple API • Tiny memory footprint
  13. Architecture • Main process forks workers for each request •

    Forking is fast & reliable with copy-on-write • ~10K ram per request • Context switching is expensive :( • ~600 concurrent processes max on Xeon
  14. C • Surprisingly simple • Dangerous by default, safe by

    demand • Still modern
  15. C & OOP module.h typedef struct module module; typedef struct

    module_state module_state; struct module { void *public_data; polymorfic *polymorfic; module_state *_s; }; module *module_init(...); void module_free(module *m); int module_do_something(module *m);
  16. Memory Management • Allocate & free at the same level

    • Allocate & free at the same module • Use Valgrind • Be careful
  17. Parsing HTTP • Server is set of parsers • Callback

    based • Must work with chunks of data • Stateful • No buffering • No bugs
  18. API #include <cosmonaut/cosmonaut.h> void action_index(http_request *request, http_response *response) { render_text(response,

    "Hello world"); } void configure() { mount("/", action_index); } int main(int argc, char *argv[]) { return cosmonaut_start(argc, argv, configure); } Hello World
  19. API struct http_request { url *url; headers_map *headers; params_map *params;

    configuration *configuration; route *route; void *data; char *uid; }; char *c_type = headers_map_get(request->headers, "Content-Type"); Request
  20. API typedef struct http_response { int code; char *header_summary; char

    *file_path; char *content_type; int content_length; headers_map* headers; char* raw_response; } http_response; render_file(http_response *response, const char *path); render_text(http_response *response, const char *text); render_json(http_response *response, const char *json); Response
  21. API mount("/", action_index); mount("/upload_file", action_upload); Routing pt.1

  22. API mount("/photos/(:id)", action); void action(http_request *request, http_response *response) { int

    photo_id = params_map_get(request->params, "id")->val; ... } Routing pt.2
  23. API mount("/patients/(:id)/(:action)", rest_application); void rest_application(http_request *request, http_response *response) { char

    *action = params_map_get(request->params, "action")->val; int patient_id = params_map_get(request->params, "id")->val; process_data(patient, action, response); } /patients/1/new // params: {id => 1, action => "new"} /patients/1/show // params: {id => 1, action => "show"} /patients/1/edit // params: {id => 1, action => "edit"} /patients/1/delete // params: {id => 1, action => "delete"} Routing pt.3
  24. API route *rt = mount("/files/new/(:upload_id)", action_upload); rt->before_filter = check_upload_size; rt->after_filter

    = clear_db_connections; Filters
  25. Testing • Cucumber • Rest-client • Travis CI

  26. Testing Feature: Routing and params processing Scenario Outline: Requesting action

    mounted to '/photos/(:id)' When I request "<path>" Then I should get "<expected_response>" Examples: | path | expected_response | | /photos/1 | [id:1] | | /photos/word | [id:word] | | /photos/1/somth | Not Found | | /photos/1/ | Not Found | | /photos1/1 | Not Found |
  27. Testing When /^I request "([^"]*)"$/ do |path| begin @response =

    RestClient.get("http://127.0.0.1:31337#{path}") rescue Exception => e @client_exception = e end end Then /^I should get "([^"]*)"$/ do |body| response_body = @client_exception.nil? ? @response.to_s : @client_exception.http_body response_body.should == body end
  28. Sample Application • Supports file uploads/downloads • Supports tracking of

    upload progress • Uses redis for persistence & IPC https://github.com/iafonov/uploader_app
  29. Don't reinvent the wheel Do it!

  30. https://github.com/iafonov/cosmonaut https://github.com/iafonov/cosmonaut/wiki/Links http://iafonov.github.com/ @iafonov