Upgrade to Pro — share decks privately, control downloads, hide ads and more …

There's this thing called HTTP

Thijs Feryn
October 25, 2012

There's this thing called HTTP

First time I've done this one. Presented at a Fronteersbe local user group meeting at Combell.

HTTP is out there and you should know how to use it!

Thijs Feryn

October 25, 2012
Tweet

More Decks by Thijs Feryn

Other Decks in Technology

Transcript

  1. There’s  
    this  thing  
    called  
    HTTP
    By  Thijs  Feryn

    View Slide

  2. Hi  
    my  name  
    is  Thijs

    View Slide

  3. I’m  
    an  evangelist  at

    View Slide

  4. I’m  
    a  board  member  
    at

    View Slide

  5. View Slide

  6. h"p://joind.in/7581
    Please  
    give  me  
    feedback

    View Slide

  7. View Slide

  8. Name  5  
    things  you  
    like  about  
    HTTP

    View Slide

  9. Name  5  
    things  you  
    hate  about  
    HTTP

    View Slide

  10. What  the  
    heck  is  
    HTTP  
    actually?

    View Slide

  11. It  might  get  a  bit  boring  ...  just  a  bit

    View Slide

  12. Application  
    level  protocol  
    in  the  TCP/IP  
    network  stack

    View Slide

  13. View Slide

  14. Request
    &
    Response

    View Slide

  15. Methods
    &
    Status  
    codes

    View Slide

  16. Headers
    &
    Body

    View Slide

  17. GET  /  HTTP/1.1
    Host:  fronteers.nl
    User-­‐Agent:  Mozilla/5.0  (Macintosh;  Intel  
    Mac  OS  X  10.8;  rv:16.0)  Gecko/20100101  
    Firefox/16.0
    Accept:  text/html,application/xhtml
    +xml,application/xml;q=0.9,*/*;q=0.8
    Accept-­‐Language:  en,nl;q=0.7,fr-­‐be;q=0.3
    Accept-­‐Encoding:  gzip,  deflate
    Connection:  keep-­‐alive
    Cookie:  first_pv_60255=1;  _eventqueue=%7B
    %22heatmap%22%3A%5B%5D%2C%22events%22%3A%5B
    %5D%7D;  _jsuid=608348910;  unpoco_60255=1

    View Slide

  18. HTTP/1.1  200  OK
    Date:  Wed,  17  Oct  2012  13:19:58  GMT
    Server:  Apache
    Content-­‐Encoding:  gzip
    Vary:  Accept-­‐Encoding
    X-­‐UA-­‐Compatible:  IE=Edge,chrome=1
    Content-­‐Length:  5566
    Keep-­‐Alive:  timeout=11,  max=97
    Connection:  Keep-­‐Alive
    Content-­‐Type:  text/html;  charset=UTF-­‐8

    View Slide

  19. Status  
    codes
    h"p://www.w3.org/Protocols/rfc2616/rfc2616-­‐sec10.html
    h"p://en.wikipedia.org/wiki/List_of_HTTP_status_codes

    View Slide

  20. 100  range
    Magic,  
    fuggedaboudid!

    View Slide

  21. 200  range
    Check!

    View Slide

  22. 200
    201
    202
    203
    OK
    Created
    Accepted
    Non-­‐Authorita7ve  
    Informa7on

    View Slide

  23. 204
    205
    206
    No  Content
    Reset  Content
    Par7al  Content

    View Slide

  24. 300  range
    Talk  to  that  guy  over  
    there

    View Slide

  25. 301
    302
    303
    304
    Moved  permanently
    Found
    See  other
    Not  modified

    View Slide

  26. 305
    306
    307
    Use  Proxy
    Unused
    Temporary  Redirect

    View Slide

  27. 400  range
    Dude,  you  f’d  up!

    View Slide

  28. 400
    403
    404
    405
    408
    Bad  request
    Forbidden
    Not  found
    Method  not  allowed
    Request  7meout

    View Slide

  29. 400
    401
    402
    403
    404
    Bad  Request
    Unauthorized
    Payment  Required
    Forbidden
    Not  Found

    View Slide

  30. 405
    406
    407
    408
    409
    Method  Not  Allowed
    Not  Acceptable
    Proxy  Authen7ca7on  
    Required
    Request  Timeout
    Conflict

    View Slide

  31. 410
    411
    412
    413
    414
    Gone
    Length  Required
    Precondi7on  Failed
    Request  En7ty  Too  Large
    Request-­‐URI  Too  Long

    View Slide

  32. 415
    416
    417
    Unsupported  Media  Type
    Requested  Range  Not  
    Sa7sfiable
    Expecta7on  Failed

    View Slide

  33. 500  range
    Sorry  man,  we  f’d  
    up!

    View Slide

  34. 500
    501
    502
    503
    504
    505
    Internal  server  error
    Not  implemented
    Bad  gateway
    Service  unavailable
    Gateway  7meout
    HTTP  Version  Not  
    Supported

    View Slide

  35. Request
    Headers

    View Slide

  36. Accept
    Accept-­‐Charset
    Accept-­‐Encoding
    Accept-­‐language
    Authorization
    Cache-­‐control
    Connection
    Cookie
    Content-­‐length
    Date
    text/plain
    u^8
    gzip,  deflate
    en,nl;q=0.7,fr-­‐be;q=0.3
    Basic  QWxhZGRpbjpvcGVuIHNlc2FtZQ==
    no-­‐cache
    close
    bla=ja;  foo=bar
    1024
    Tue,  15  Nov  1994  08:12:31  GMT

    View Slide

  37. Host
    If-­‐None-­‐Match
    Pragma
    Proxy-­‐Authorization
    Referer
    Upgrade
    User-­‐Agent
    Via
    www.fronteers.nl
    "737060cd8c284d8af7ad3082f209582d"
    no-­‐cache
    Basic  
    QWxhZGRpbjpvcGVuIHNl
    c2FtZQ==
    hQp://www.combell.com
    Mozilla/5.0  
    websocket
    1.0  fred,  1.1  example.com  (Apache/1.1)

    View Slide

  38. Response
    Headers

    View Slide

  39. Age
    Allow
    Cache-­‐control
    Connection
    Content-­‐Encoding
    Content-­‐Language
    Content-­‐Length
    Content-­‐Type
    Date
    12
    GET,HEAD
    max-­‐age=3600
    close
    gzip
    nl
    1024
    text/html;  charset=u^-­‐8
    Tue,  15  Nov  1994  08:12:31  GMT

    View Slide

  40. ETag
    Expires
    Last-­‐Modified
    Location
    Pragma
    Server
    Set-­‐Cookie
    Transfer-­‐Encoding
    Vary
    737060cd8c284d8af7ad3082f209582d
    Thu,  01  Dec  1994  16:00:00  GMT
    Thu,  01  Dec  1994  16:00:00  GMT
    hkp://www.combell.com
    no-­‐cache
    My  magic  server
    bla=abc;  session=12345
    chuncked
    Accept-­‐encoding

    View Slide

  41. GET  /  HTTP/1.1
    Host:  fronteers.nl
    User-­‐Agent:  Mozilla/5.0  (Macintosh;  Intel  
    Mac  OS  X  10.8;  rv:16.0)  Gecko/20100101  
    Firefox/16.0
    Accept:  text/html,application/xhtml
    +xml,application/xml;q=0.9,*/*;q=0.8
    Accept-­‐Language:  en,nl;q=0.7,fr-­‐be;q=0.3
    Accept-­‐Encoding:  gzip,  deflate
    Connection:  keep-­‐alive
    Cookie:  first_pv_60255=1;  _eventqueue=%7B
    %22heatmap%22%3A%5B%5D%2C%22events%22%3A%5B
    %5D%7D;  _jsuid=608348910;  unpoco_60255=1

    View Slide

  42. HTTP/1.1  200  OK
    Date:  Wed,  17  Oct  2012  13:19:58  GMT
    Server:  Apache
    Content-­‐Encoding:  gzip
    Vary:  Accept-­‐Encoding
    X-­‐UA-­‐Compatible:  IE=Edge,chrome=1
    Content-­‐Length:  5566
    Keep-­‐Alive:  timeout=11,  max=97
    Connection:  Keep-­‐Alive
    Content-­‐Type:  text/html;  charset=UTF-­‐8

    View Slide

  43. Implementations

    View Slide

  44. Clients +  
    cURL  &  
    Lynx

    View Slide

  45. Servers
    The  
    classics

    View Slide

  46. The  
    popular  
    new  kid
    Hype  
    factor  1

    View Slide

  47. NoSQL
    Hype  
    factor  2

    View Slide

  48. Varnish
    Hype  
    factor  
    2.5

    View Slide

  49. Hype  
    factor  3
    A  hipster’s  wet  dream

    View Slide

  50. PHP’s  reac2on  to  node.js
    Pixels  
    on  
    purpose

    View Slide

  51. What
    do
    we
    do
    with
    HTTP
    ?

    View Slide

  52. Serve  
    files

    View Slide

  53. Display  pages

    View Slide

  54. Interact  
    with  services

    View Slide

  55. Search  &  query

    View Slide

  56. Invalidate  cache

    View Slide

  57. Obstacles

    View Slide

  58. Caching

    View Slide

  59. ETag
    If-­‐Non-­‐Match:  "3e86-­‐410-­‐3596?bc"
    ETag:  "3e86-­‐410-­‐3596?bc"

    View Slide

  60. Expires
    Expires  "Wed,  1  Jan  2014  20:00:00  GMT"

    View Slide

  61. Cache-­‐control
    Cache-­‐Control  “max-­‐age=3600,  s-­‐
    maxage=1000,  public,  must-­‐revalidate”

    View Slide

  62. Max-­‐Age
    S-­‐maxage
    Public
    Private
    No-­‐cache
    No-­‐store
    Must-­‐revalidate
    Proxy-­‐revalidate
    TTL  for  browsers  in  seconds
    TTL  for  proxies  in  seconds
    Proxies  &  browsers  can  cache
    Only  browsers  can  cache
    Revalidate  before  dropping  from  cache
    Don’t  cache  at  all
    Browser  revalidate  before  serving  from  
    cache  
    Proxy  revalidate  before  serving  from  
    cache  

    View Slide

  63. Cache-control in Varnish
    1.Vcl (beresp.ttl)
    2.Cache-control s-maxage
    3.Cache-control max-age
    4.Expires

    View Slide

  64. Let’s talk about Varnish

    View Slide

  65. “Varnish  saves  lives”

    View Slide

  66. Reverse  caching  
    proxy

    View Slide

  67. In  front  of  the  
    webserver

    View Slide

  68. Only get & HEAD
    No cookie & auth headers
    No set-cookie headers
    cache ttl > 0
    No vary “*”
    When will varnish cache?

    View Slide

  69. based on host/ip + urL hash
    By default in Memory
    Optionally on disk
    how will varnish cache?

    View Slide

  70. Request vcl_recv
    In cache?
    vcl_hash Cacheable?
    vcl_hit() vcl_miss()
    vcl_deliver()
    vcl_fetch()
    No
    Yes
    No
    Yes
    Response

    View Slide

  71. #  sub  vcl_recv  {
    #          if  (req.restarts  ==  0)  {
    #              if  (req.http.x-­‐forwarded-­‐for)  {
    #                      set  req.http.X-­‐Forwarded-­‐For  =
    #                              req.http.X-­‐Forwarded-­‐For  +  ",  "  +  client.ip;
    #              }  else  {
    #                      set  req.http.X-­‐Forwarded-­‐For  =  client.ip;
    #              }
    #          }
    #          if  (req.request  !=  "GET"  &&
    #              req.request  !=  "HEAD"  &&
    #              req.request  !=  "PUT"  &&
    #              req.request  !=  "POST"  &&
    #              req.request  !=  "TRACE"  &&
    #              req.request  !=  "OPTIONS"  &&
    #              req.request  !=  "DELETE")  {
    #                  /*  Non-­‐RFC2616  or  CONNECT  which  is  weird.  */
    #                  return  (pipe);
    #          }
    #          if  (req.request  !=  "GET"  &&  req.request  !=  "HEAD")  {
    #                  /*  We  only  deal  with  GET  and  HEAD  by  default  */
    #                  return  (pass);
    #          }
    #          if  (req.http.Authorization  ||  req.http.Cookie)  {
    #                  /*  Not  cacheable  by  default  */
    #                  return  (pass);
    #          }
    #          return  (lookup);
    #  }

    View Slide

  72. #  sub  vcl_hash  {
    #          hash_data(req.url);
    #          if  (req.http.host)  {
    #                  hash_data(req.http.host);
    #          }  else  {
    #                  hash_data(server.ip);
    #          }
    #          return  (hash);
    #  }
    #  sub  vcl_fetch  {
    #          if  (beresp.ttl  <=  0s  ||
    #                  beresp.http.Set-­‐Cookie  ||
    #                  beresp.http.Vary  ==  "*")  {
    #                              /*
    #                                *  Mark  as  "Hit-­‐For-­‐Pass"  for  the  next  2  minutes
    #                                */
    #                              set  beresp.ttl  =  120  s;
    #                              return  (hit_for_pass);
    #          }
    #          return  (deliver);
    #  }

    View Slide

  73. Purge  cache

    View Slide

  74. curl  -­‐X  PURGE  http://example.com/some-­‐page
    acl  purge  {
    "localhost"
    }
    sub  vcl_hit  {
       if  (req.request  ==  "PURGE")  {
           purge;
           error  200  "Purged.";
       }
    }
    sub  vcl_miss  {
       if  (req.request  ==  "PURGE")  {
           purge;
           error  200  "Purged.";
       }
    }

    View Slide

  75. Edge Side Includes

    View Slide

  76. header.php
    menu.php main.php
    footer.php
    TTL  5s
    No  caching
    TTL  10s
    TTL  2s

    View Slide




















  77. View Slide

  78. Surrogate-­‐Capability:  “key=ESI/1.0”
    Surrogate-­‐Control:  “content=ESI/1.0”
    Proxy
    Webserver

    View Slide

  79. ESI  VCL
    sub  vcl_recv  {
           set  req.http.Surrogate-­‐Capability="key=ESI/1.0";
    }
    sub  vcl_fetch  {
      if(beresp.http.Surrogate-­‐Control~"ESI/1.0")  {
                     unset  beresp.http.Surrogate-­‐Control;
             set  beresp.do_esi=true;
           }
    }  

    View Slide

  80. Cookies

    View Slide

  81. Cookies
    Tracking
    Sessions
    Language

    View Slide

  82. Cookies
    HTTP cookie
    request header
    via browser
    HTTP set-cookie
    response header
    via webserver

    View Slide

  83. Varnish doesn’t like them

    View Slide

  84. Don’t vary on
    cookie

    View Slide

  85. Vary on custom
    header
    X-­‐Lang:  nl
    Vary:  X-­‐Lang
    Cookie:  lang=nl

    View Slide

  86. Webserver    
    performance

    View Slide

  87. Apache  vs  Nginx

    View Slide

  88. Prefork  or  worker  mode
    Async,  event-­‐driven  workers
    Apache
    Nginx

    View Slide

  89. Evented  non-­‐blocking  I/O
    We’ll  call  
    you  back!

    View Slide

  90. Blocking
    var  results  =  db.query("SELECT  *  FROM  tablename");
    //use  results  
    Non-­‐blocking
    db.query("SELECT  *  FROM  tablename"  ,  function  (results)  {  
    //use  results  
    });  
    //do  something  else  

    View Slide

  91. Example
    var  http  =  require('http');
    http.createServer(function  (req,  res)  {
       res.writeHead(200,  {'Content-­‐Type':  'text/plain'});
       res.end('Hello  World\n');
    }).listen(1337,  '127.0.0.1');
    console.log('Server  running  at  http://127.0.0.1:1337/');

    View Slide

  92. We  can  do  it  in  PHP  too

    View Slide

  93. Example
    require 'vendor/autoload.php';
    $app = function ($request, $response) {
    $response->writeHead(200, array('Content-
    Type' => 'text/plain'));
    $response->end("Hello World\n");
    };
    $loop = React\EventLoop\Factory::create();
    $socket = new React\Socket\Server($loop);
    $http = new React\Http\Server($socket, $loop);
    $http->on('request', $app);
    echo "Server running at http://127.0.0.1:1337\n";
    $socket->listen(1337);
    $loop->run();

    View Slide

  94. Real  2me  data
    Ajax
    Websockets

    View Slide

  95. Websockets
    var conn = new
    WebSocket('ws://
    some.host');
    conn.onmessage =
    function(e) {
    console.log(e.data);
    };
    conn.send('Hello World!');
    Client

    View Slide

  96. Websockets
    Server
    Ratchet:  
    websocket  
    framework  
    on  top  of  
    React

    View Slide

  97. namespace MyApp;
    use Ratchet\MessageComponentInterface;
    use Ratchet\ConnectionInterface;
    class Chat implements MessageComponentInterface {
    protected $clients;
    public function __construct() {
    $this->clients = new \SplObjectStorage;
    }
    public function onOpen(ConnectionInterface $conn) {
    $this->clients->attach($conn);
    }
    public function onMessage(ConnectionInterface $from, $msg) {
    foreach ($this->clients as $client) {
    if ($from !== $client) {
    $client->send('Response: '.$msg);
    }
    }
    }
    public function onClose(ConnectionInterface $conn) {
    $this->clients->detach($conn);
    }
    public function onError(ConnectionInterface $conn, \Exception $e) {
    echo "An error has occurred: {$e->getMessage()}\n";
    $conn->close();
    }
    }

    View Slide

  98. require __DIR__.'/vendor/autoload.php';
    require 'chat.php';
    use Ratchet\WebSocket\WsServer;
    use Ratchet\Server\IoServer;
    use MyApp\Chat;
    $ws = new WsServer(new Chat());
    $ws->disableVersion(0);
    $server = IoServer::factory($ws,80,'1.2.3.4');
    $server->run();

    View Slide

  99. View Slide

  100. HTTPS

    View Slide

  101. Browser  
    trust
    Client

    View Slide

  102. CSR
    CRT
    Key
    CA  bundle
    Server

    View Slide

  103. Varnish  
    can’t  deal  
    with  it

    View Slide

  104. Terminate  SSL

    View Slide

  105. Is  
    also  a  
    proxy!

    View Slide

  106. upstream  varnish  {
       server  varnish1.example.com;
       server  varnish2.example.com;
    }
    server  {
       listen  443;
       ssl  on;
       ssl_certificate    /etc/nginx/example.ssl.cert;
       ssl_certificate_key    /etc/nginx/example.ssl.key;
       server_name  www.example.com;
       access_log  /var/log/nginx/access.log;
       error_log  /var/log/nginx/error.log;
       location  /  {
           proxy_pass    http://varnish;  
           proxy_redirect  off;
           proxy_set_header  Host  $host;
           proxy_set_header  X-­‐Scheme  $scheme;
           proxy_set_header  X-­‐Forwarded-­‐For  $proxy_add_x_forwarded_for;
           proxy_set_header  CLIENT_CERT  $ssl_client_raw_cert;
    }
    Make  
    your  app  
    aware

    View Slide

  107. Other  
    cool  stuff  
    we  can  
    do!

    View Slide

  108. upstream fpm {
    server php1.server.com:9000;
    server php2.server.com:9000;
    }
    upstream memcached {
    server memcached1.server.com:11211;
    server memcached2.server.com:11211;
    }
    server {
    root /var/www;
    index index.php index.html index.htm;
    server_name nginx.server.com;
    location / {
    try_files $uri $uri/ /index.php;
    }
    location @php {
    fastcgi_pass fpm;
    fastcgi_index index.php;
    include fastcgi_params;
    }

    View Slide

  109. location ~ \.php$ {
    set $memcached_key $request_uri;
    memcached_pass memcached;
    memcached_next_upstream not_found;
    default_type text/html;
    error_page 404 405 502 = @php;
    }
    location ~* \.(?:ico|css|js|gif|jpe?g|png)$ {
    expires max;
    add_header Pragma public;
    add_header Cache-Control "public, must-
    revalidate, proxy-revalidate";
    }
    }

    View Slide

  110. REpresentational  
    State  
    Transfer

    View Slide

  111. Developed  in  
    parallel  with  
    HTTP/1.1

    View Slide

  112. The  way  the  
    web
     *should*  
    work

    View Slide

  113. Resources
    Stateless
    Cacheable
    Layered
    Uniform  interface
    Vocabulary  re-­‐use
    HATEOAS
    Keywords

    View Slide

  114. View Slide

  115. David  
    Zülke
     @dzuelke

    View Slide

  116. ✓A  URL  idenKfies  a  resource.  
    ✓Resources  have  a  hierarchy,  so  you  know  that  
    something  with  addiKonal  slashes  is  a  subordinate  
    resource
    ✓Verbs  are  used  to  perform  operaKons  on  
    resources
    ✓The  operaKon  is  implicit  and  not  part  of  the  URL
    ✓A  hypermedia  format  is  used  to  represent  the  
    data
    ✓Link  relaKons  are  used  to  navigate  a  service
    Quodng  
    David  Zülke
     @dzuelke

    View Slide

  117. GET
    POST
    PUT
    DELETE
    Retrieve
    Insert
    Update
    Remove

    View Slide

  118. HATEOAS
    WTF?

    View Slide

  119. Hypertext
    As
    The
    Engine
    Of
    Application
    State

    View Slide

  120. GET  /products/1234  HTTP/1.1
    Host:  example.com
    Accept:  application/vnd.com.example+xml  
    HTTP/1.1  200  OK
    Content-­‐Type:  application/vnd.com.example+xml;  charset=utf-­‐8
    Allow:  GET,  PUT,  DELETE
     xml  version="1.0"  encoding="utf-­‐8"  ?>
    w3.org/2005/Atom">
       1234
       My  product
       Mijn  product
    5
    6.5
    +xml"  href="http://example.com/products/1234/payment"

     
    Thx  
    David!

    View Slide

  121. And  
    on  that  
    bombshell  ...

    View Slide

  122. View Slide

  123. h"p://joind.in/7581
    Please  
    give  me  
    feedback

    View Slide

  124. Thanks

    View Slide

  125. View Slide