Pro Yearly is on sale from $80 to $50! »

There's this thing called HTTP

Ca901ddcea38854b9783781c91fc87c9?s=47 Thijs Feryn
October 25, 2012

There's this thing called HTTP

First time I've done this one. Presented at a Fronteersbe local user group meeting at Combell.

HTTP is out there and you should know how to use it!

Ca901ddcea38854b9783781c91fc87c9?s=128

Thijs Feryn

October 25, 2012
Tweet

Transcript

  1. There’s   this  thing   called   HTTP By  Thijs

     Feryn
  2. Hi   my  name   is  Thijs

  3. I’m   an  evangelist  at

  4. I’m   a  board  member   at

  5. None
  6. h"p://joind.in/7581 Please   give  me   feedback

  7. None
  8. Name  5   things  you   like  about   HTTP

  9. Name  5   things  you   hate  about   HTTP

  10. What  the   heck  is   HTTP   actually?

  11. It  might  get  a  bit  boring  ...  just  a  bit

  12. Application   level  protocol   in  the  TCP/IP   network

     stack
  13. None
  14. Request & Response

  15. Methods & Status   codes

  16. Headers & Body

  17. GET  /  HTTP/1.1 Host:  fronteers.nl User-­‐Agent:  Mozilla/5.0  (Macintosh;  Intel  

    Mac  OS  X  10.8;  rv:16.0)  Gecko/20100101   Firefox/16.0 Accept:  text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8 Accept-­‐Language:  en,nl;q=0.7,fr-­‐be;q=0.3 Accept-­‐Encoding:  gzip,  deflate Connection:  keep-­‐alive Cookie:  first_pv_60255=1;  _eventqueue=%7B %22heatmap%22%3A%5B%5D%2C%22events%22%3A%5B %5D%7D;  _jsuid=608348910;  unpoco_60255=1
  18. HTTP/1.1  200  OK Date:  Wed,  17  Oct  2012  13:19:58  GMT

    Server:  Apache Content-­‐Encoding:  gzip Vary:  Accept-­‐Encoding X-­‐UA-­‐Compatible:  IE=Edge,chrome=1 Content-­‐Length:  5566 Keep-­‐Alive:  timeout=11,  max=97 Connection:  Keep-­‐Alive Content-­‐Type:  text/html;  charset=UTF-­‐8
  19. Status   codes h"p://www.w3.org/Protocols/rfc2616/rfc2616-­‐sec10.html h"p://en.wikipedia.org/wiki/List_of_HTTP_status_codes

  20. 100  range Magic,   fuggedaboudid!

  21. 200  range Check!

  22. 200 201 202 203 OK Created Accepted Non-­‐Authorita7ve   Informa7on

  23. 204 205 206 No  Content Reset  Content Par7al  Content

  24. 300  range Talk  to  that  guy  over   there

  25. 301 302 303 304 Moved  permanently Found See  other Not

     modified
  26. 305 306 307 Use  Proxy Unused Temporary  Redirect

  27. 400  range Dude,  you  f’d  up!

  28. 400 403 404 405 408 Bad  request Forbidden Not  found

    Method  not  allowed Request  7meout
  29. 400 401 402 403 404 Bad  Request Unauthorized Payment  Required

    Forbidden Not  Found
  30. 405 406 407 408 409 Method  Not  Allowed Not  Acceptable

    Proxy  Authen7ca7on   Required Request  Timeout Conflict
  31. 410 411 412 413 414 Gone Length  Required Precondi7on  Failed

    Request  En7ty  Too  Large Request-­‐URI  Too  Long
  32. 415 416 417 Unsupported  Media  Type Requested  Range  Not  

    Sa7sfiable Expecta7on  Failed
  33. 500  range Sorry  man,  we  f’d   up!

  34. 500 501 502 503 504 505 Internal  server  error Not

     implemented Bad  gateway Service  unavailable Gateway  7meout HTTP  Version  Not   Supported
  35. Request Headers

  36. Accept Accept-­‐Charset Accept-­‐Encoding Accept-­‐language Authorization Cache-­‐control Connection Cookie Content-­‐length Date

    text/plain u^8 gzip,  deflate en,nl;q=0.7,fr-­‐be;q=0.3 Basic  QWxhZGRpbjpvcGVuIHNlc2FtZQ== no-­‐cache close bla=ja;  foo=bar 1024 Tue,  15  Nov  1994  08:12:31  GMT
  37. Host If-­‐None-­‐Match Pragma Proxy-­‐Authorization Referer Upgrade User-­‐Agent Via www.fronteers.nl "737060cd8c284d8af7ad3082f209582d"

    no-­‐cache Basic   QWxhZGRpbjpvcGVuIHNl c2FtZQ== hQp://www.combell.com Mozilla/5.0   websocket 1.0  fred,  1.1  example.com  (Apache/1.1)
  38. Response Headers

  39. Age Allow Cache-­‐control Connection Content-­‐Encoding Content-­‐Language Content-­‐Length Content-­‐Type Date 12

    GET,HEAD max-­‐age=3600 close gzip nl 1024 text/html;  charset=u^-­‐8 Tue,  15  Nov  1994  08:12:31  GMT
  40. ETag Expires Last-­‐Modified Location Pragma Server Set-­‐Cookie Transfer-­‐Encoding Vary 737060cd8c284d8af7ad3082f209582d

    Thu,  01  Dec  1994  16:00:00  GMT Thu,  01  Dec  1994  16:00:00  GMT hkp://www.combell.com no-­‐cache My  magic  server bla=abc;  session=12345 chuncked Accept-­‐encoding
  41. GET  /  HTTP/1.1 Host:  fronteers.nl User-­‐Agent:  Mozilla/5.0  (Macintosh;  Intel  

    Mac  OS  X  10.8;  rv:16.0)  Gecko/20100101   Firefox/16.0 Accept:  text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8 Accept-­‐Language:  en,nl;q=0.7,fr-­‐be;q=0.3 Accept-­‐Encoding:  gzip,  deflate Connection:  keep-­‐alive Cookie:  first_pv_60255=1;  _eventqueue=%7B %22heatmap%22%3A%5B%5D%2C%22events%22%3A%5B %5D%7D;  _jsuid=608348910;  unpoco_60255=1
  42. HTTP/1.1  200  OK Date:  Wed,  17  Oct  2012  13:19:58  GMT

    Server:  Apache Content-­‐Encoding:  gzip Vary:  Accept-­‐Encoding X-­‐UA-­‐Compatible:  IE=Edge,chrome=1 Content-­‐Length:  5566 Keep-­‐Alive:  timeout=11,  max=97 Connection:  Keep-­‐Alive Content-­‐Type:  text/html;  charset=UTF-­‐8
  43. Implementations

  44. Clients +   cURL  &   Lynx

  45. Servers The   classics

  46. The   popular   new  kid Hype   factor  1

  47. NoSQL Hype   factor  2

  48. Varnish Hype   factor   2.5

  49. Hype   factor  3 A  hipster’s  wet  dream

  50. PHP’s  reac2on  to  node.js Pixels   on   purpose

  51. What do we do with HTTP ?

  52. Serve   files

  53. Display  pages

  54. Interact   with  services

  55. Search  &  query

  56. Invalidate  cache

  57. Obstacles

  58. Caching

  59. ETag If-­‐Non-­‐Match:  "3e86-­‐410-­‐3596?bc" ETag:  "3e86-­‐410-­‐3596?bc"

  60. Expires Expires  "Wed,  1  Jan  2014  20:00:00  GMT"

  61. Cache-­‐control Cache-­‐Control  “max-­‐age=3600,  s-­‐ maxage=1000,  public,  must-­‐revalidate”

  62. Max-­‐Age S-­‐maxage Public Private No-­‐cache No-­‐store Must-­‐revalidate Proxy-­‐revalidate TTL  for

     browsers  in  seconds TTL  for  proxies  in  seconds Proxies  &  browsers  can  cache Only  browsers  can  cache Revalidate  before  dropping  from  cache Don’t  cache  at  all Browser  revalidate  before  serving  from   cache   Proxy  revalidate  before  serving  from   cache  
  63. Cache-control in Varnish 1.Vcl (beresp.ttl) 2.Cache-control s-maxage 3.Cache-control max-age 4.Expires

  64. Let’s talk about Varnish

  65. “Varnish  saves  lives”

  66. Reverse  caching   proxy

  67. In  front  of  the   webserver

  68. Only get & HEAD No cookie & auth headers No

    set-cookie headers cache ttl > 0 No vary “*” When will varnish cache?
  69. based on host/ip + urL hash By default in Memory

    Optionally on disk how will varnish cache?
  70. Request vcl_recv In cache? vcl_hash Cacheable? vcl_hit() vcl_miss() vcl_deliver() vcl_fetch()

    No Yes No Yes Response
  71. #  sub  vcl_recv  { #          if

     (req.restarts  ==  0)  { #              if  (req.http.x-­‐forwarded-­‐for)  { #                      set  req.http.X-­‐Forwarded-­‐For  = #                              req.http.X-­‐Forwarded-­‐For  +  ",  "  +  client.ip; #              }  else  { #                      set  req.http.X-­‐Forwarded-­‐For  =  client.ip; #              } #          } #          if  (req.request  !=  "GET"  && #              req.request  !=  "HEAD"  && #              req.request  !=  "PUT"  && #              req.request  !=  "POST"  && #              req.request  !=  "TRACE"  && #              req.request  !=  "OPTIONS"  && #              req.request  !=  "DELETE")  { #                  /*  Non-­‐RFC2616  or  CONNECT  which  is  weird.  */ #                  return  (pipe); #          } #          if  (req.request  !=  "GET"  &&  req.request  !=  "HEAD")  { #                  /*  We  only  deal  with  GET  and  HEAD  by  default  */ #                  return  (pass); #          } #          if  (req.http.Authorization  ||  req.http.Cookie)  { #                  /*  Not  cacheable  by  default  */ #                  return  (pass); #          } #          return  (lookup); #  }
  72. #  sub  vcl_hash  { #          hash_data(req.url);

    #          if  (req.http.host)  { #                  hash_data(req.http.host); #          }  else  { #                  hash_data(server.ip); #          } #          return  (hash); #  } #  sub  vcl_fetch  { #          if  (beresp.ttl  <=  0s  || #                  beresp.http.Set-­‐Cookie  || #                  beresp.http.Vary  ==  "*")  { #                              /* #                                *  Mark  as  "Hit-­‐For-­‐Pass"  for  the  next  2  minutes #                                */ #                              set  beresp.ttl  =  120  s; #                              return  (hit_for_pass); #          } #          return  (deliver); #  }
  73. Purge  cache

  74. curl  -­‐X  PURGE  http://example.com/some-­‐page acl  purge  { "localhost" } sub

     vcl_hit  {    if  (req.request  ==  "PURGE")  {        purge;        error  200  "Purged.";    } } sub  vcl_miss  {    if  (req.request  ==  "PURGE")  {        purge;        error  200  "Purged.";    } }
  75. Edge Side Includes

  76. header.php menu.php main.php footer.php TTL  5s No  caching TTL  10s

    TTL  2s
  77. <html> <body> <table> <tr> <td colspan="2" > <esi:include src="/header.php" />

    </td> </tr> <tr> <td><esi:include src="/menu.php" /></td> <td><esi:include src="/main.php" /></td> </tr> <tr> <td colspan="2" > <esi:include src="/footer.php" /> </td> </tr> </table> </body>
  78. Surrogate-­‐Capability:  “key=ESI/1.0” Surrogate-­‐Control:  “content=ESI/1.0” Proxy Webserver

  79. ESI  VCL sub  vcl_recv  {        set  req.http.Surrogate-­‐Capability="key=ESI/1.0";

    } sub  vcl_fetch  {   if(beresp.http.Surrogate-­‐Control~"ESI/1.0")  {                  unset  beresp.http.Surrogate-­‐Control;          set  beresp.do_esi=true;        } }  
  80. Cookies

  81. Cookies Tracking Sessions Language

  82. Cookies HTTP cookie request header via browser HTTP set-cookie response

    header via webserver
  83. Varnish doesn’t like them

  84. Don’t vary on cookie

  85. Vary on custom header X-­‐Lang:  nl Vary:  X-­‐Lang Cookie:  lang=nl

  86. Webserver     performance

  87. Apache  vs  Nginx

  88. Prefork  or  worker  mode Async,  event-­‐driven  workers Apache Nginx

  89. Evented  non-­‐blocking  I/O We’ll  call   you  back!

  90. Blocking var  results  =  db.query("SELECT  *  FROM  tablename"); //use  results

      Non-­‐blocking db.query("SELECT  *  FROM  tablename"  ,  function  (results)  {   //use  results   });   //do  something  else  
  91. Example var  http  =  require('http'); http.createServer(function  (req,  res)  {  

     res.writeHead(200,  {'Content-­‐Type':  'text/plain'});    res.end('Hello  World\n'); }).listen(1337,  '127.0.0.1'); console.log('Server  running  at  http://127.0.0.1:1337/');
  92. We  can  do  it  in  PHP  too

  93. Example <?php require 'vendor/autoload.php'; $app = function ($request, $response) {

    $response->writeHead(200, array('Content- Type' => 'text/plain')); $response->end("Hello World\n"); }; $loop = React\EventLoop\Factory::create(); $socket = new React\Socket\Server($loop); $http = new React\Http\Server($socket, $loop); $http->on('request', $app); echo "Server running at http://127.0.0.1:1337\n"; $socket->listen(1337); $loop->run();
  94. Real  2me  data Ajax Websockets

  95. Websockets var conn = new WebSocket('ws:// some.host'); conn.onmessage = function(e)

    { console.log(e.data); }; conn.send('Hello World!'); Client
  96. Websockets Server Ratchet:   websocket   framework   on  top

     of   React
  97. <?php namespace MyApp; use Ratchet\MessageComponentInterface; use Ratchet\ConnectionInterface; class Chat implements

    MessageComponentInterface { protected $clients; public function __construct() { $this->clients = new \SplObjectStorage; } public function onOpen(ConnectionInterface $conn) { $this->clients->attach($conn); } public function onMessage(ConnectionInterface $from, $msg) { foreach ($this->clients as $client) { if ($from !== $client) { $client->send('Response: '.$msg); } } } public function onClose(ConnectionInterface $conn) { $this->clients->detach($conn); } public function onError(ConnectionInterface $conn, \Exception $e) { echo "An error has occurred: {$e->getMessage()}\n"; $conn->close(); } }
  98. <?php require __DIR__.'/vendor/autoload.php'; require 'chat.php'; use Ratchet\WebSocket\WsServer; use Ratchet\Server\IoServer; use

    MyApp\Chat; $ws = new WsServer(new Chat()); $ws->disableVersion(0); $server = IoServer::factory($ws,80,'1.2.3.4'); $server->run();
  99. None
  100. HTTPS

  101. Browser   trust Client

  102. CSR CRT Key CA  bundle Server

  103. Varnish   can’t  deal   with  it

  104. Terminate  SSL

  105. Is   also  a   proxy!

  106. upstream  varnish  {    server  varnish1.example.com;    server  varnish2.example.com; }

    server  {    listen  443;    ssl  on;    ssl_certificate    /etc/nginx/example.ssl.cert;    ssl_certificate_key    /etc/nginx/example.ssl.key;    server_name  www.example.com;    access_log  /var/log/nginx/access.log;    error_log  /var/log/nginx/error.log;    location  /  {        proxy_pass    http://varnish;          proxy_redirect  off;        proxy_set_header  Host  $host;        proxy_set_header  X-­‐Scheme  $scheme;        proxy_set_header  X-­‐Forwarded-­‐For  $proxy_add_x_forwarded_for;        proxy_set_header  CLIENT_CERT  $ssl_client_raw_cert; } Make   your  app   aware
  107. Other   cool  stuff   we  can   do!

  108. upstream fpm { server php1.server.com:9000; server php2.server.com:9000; } upstream memcached

    { server memcached1.server.com:11211; server memcached2.server.com:11211; } server { root /var/www; index index.php index.html index.htm; server_name nginx.server.com; location / { try_files $uri $uri/ /index.php; } location @php { fastcgi_pass fpm; fastcgi_index index.php; include fastcgi_params; }
  109. location ~ \.php$ { set $memcached_key $request_uri; memcached_pass memcached; memcached_next_upstream

    not_found; default_type text/html; error_page 404 405 502 = @php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { expires max; add_header Pragma public; add_header Cache-Control "public, must- revalidate, proxy-revalidate"; } }
  110. REpresentational   State   Transfer

  111. Developed  in   parallel  with   HTTP/1.1

  112. The  way  the   web  *should*   work

  113. Resources Stateless Cacheable Layered Uniform  interface Vocabulary  re-­‐use HATEOAS Keywords

  114. None
  115. David   Zülke  @dzuelke

  116. ✓A  URL  idenKfies  a  resource.   ✓Resources  have  a  hierarchy,

     so  you  know  that   something  with  addiKonal  slashes  is  a  subordinate   resource ✓Verbs  are  used  to  perform  operaKons  on   resources ✓The  operaKon  is  implicit  and  not  part  of  the  URL ✓A  hypermedia  format  is  used  to  represent  the   data ✓Link  relaKons  are  used  to  navigate  a  service Quodng   David  Zülke  @dzuelke
  117. GET POST PUT DELETE Retrieve Insert Update Remove

  118. HATEOAS WTF?

  119. Hypertext As The Engine Of Application State

  120. GET  /products/1234  HTTP/1.1 Host:  example.com Accept:  application/vnd.com.example+xml   HTTP/1.1  200

     OK Content-­‐Type:  application/vnd.com.example+xml;  charset=utf-­‐8 Allow:  GET,  PUT,  DELETE <?  xml  version="1.0"  encoding="utf-­‐8"  ?> <product  xmlns="urn:com.example.products"  xmlns:atom="http:// w3.org/2005/Atom">    <id>1234</id>    <name  xml:lang="en">My  product</name>    <name  xml:lang="nl">Mijn  product</name> <price  currency="EUR">5</price> <price  currency="USD">6.5</price> <atom:link  rel="payment"  type="application/vnd.com.example +xml"  href="http://example.com/products/1234/payment" </product>   Thx   David!
  121. And   on  that   bombshell  ...

  122. None
  123. h"p://joind.in/7581 Please   give  me   feedback

  124. Thanks

  125. None