There's this thing called HTTP

First time I've done this one. Presented at a Fronteersbe local user group meeting at Combell.

HTTP is out there and you should know how to use it!

Transcript

1. There’s   this  thing   called   HTTP By  Thijs  Feryn
2. Hi   my  name   is  Thijs
3. I’m   an  evangelist  at
4. I’m   a  board  member   at
6. h"p://joind.in/7581 Please   give  me   feedback
8. Name  5   things  you   like  about   HTTP
9. Name  5   things  you   hate  about   HTTP
10. What  the   heck  is   HTTP   actually?
11. It  might  get  a  bit  boring  ...  just  a  bit
12. Application   level  protocol   in  the  TCP/IP   network  stack
14. Request & Response
15. Methods & Status   codes
16. Headers & Body
17. GET  /  HTTP/1.1 Host:  fronteers.nl User-­‐Agent:  Mozilla/5.0  (Macintosh;  Intel   Mac  OS  X  10.8;  rv:16.0)  Gecko/20100101   Firefox/16.0 Accept:  text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8 Accept-­‐Language:  en,nl;q=0.7,fr-­‐be;q=0.3 Accept-­‐Encoding:  gzip,  deflate Connection:  keep-­‐alive Cookie:  first_pv_60255=1;  _eventqueue=%7B %22heatmap%22%3A%5B%5D%2C%22events%22%3A%5B %5D%7D;  _jsuid=608348910;  unpoco_60255=1
18. HTTP/1.1  200  OK Date:  Wed,  17  Oct  2012  13:19:58  GMT Server:  Apache Content-­‐Encoding:  gzip Vary:  Accept-­‐Encoding X-­‐UA-­‐Compatible:  IE=Edge,chrome=1 Content-­‐Length:  5566 Keep-­‐Alive:  timeout=11,  max=97 Connection:  Keep-­‐Alive Content-­‐Type:  text/html;  charset=UTF-­‐8
19. Status   codes h"p://www.w3.org/Protocols/rfc2616/rfc2616-­‐sec10.html h"p://en.wikipedia.org/wiki/List_of_HTTP_status_codes
20. 100  range Magic,   fuggedaboudid!
21. 200  range Check!
22. 200 201 202 203 OK Created Accepted Non-­‐Authorita7ve   Informa7on
23. 204 205 206 No  Content Reset  Content Par7al  Content
24. 300  range Talk  to  that  guy  over   there
25. 301 302 303 304 Moved  permanently Found See  other Not  modified
26. 305 306 307 Use  Proxy Unused Temporary  Redirect
27. 400  range Dude,  you  f’d  up!
28. 400 403 404 405 408 Bad  request Forbidden Not  found Method  not  allowed Request  7meout
29. 400 401 402 403 404 Bad  Request Unauthorized Payment  Required Forbidden Not  Found
30. 405 406 407 408 409 Method  Not  Allowed Not  Acceptable Proxy  Authen7ca7on   Required Request  Timeout Conflict
31. 410 411 412 413 414 Gone Length  Required Precondi7on  Failed Request  En7ty  Too  Large Request-­‐URI  Too  Long
32. 415 416 417 Unsupported  Media  Type Requested  Range  Not   Sa7sfiable Expecta7on  Failed
33. 500  range Sorry  man,  we  f’d   up!
34. 500 501 502 503 504 505 Internal  server  error Not  implemented Bad  gateway Service  unavailable Gateway  7meout HTTP  Version  Not   Supported
35. Request Headers
36. Accept Accept-­‐Charset Accept-­‐Encoding Accept-­‐language Authorization Cache-­‐control Connection Cookie Content-­‐length Date text/plain u^8 gzip,  deflate en,nl;q=0.7,fr-­‐be;q=0.3 Basic  QWxhZGRpbjpvcGVuIHNlc2FtZQ== no-­‐cache close bla=ja;  foo=bar 1024 Tue,  15  Nov  1994  08:12:31  GMT
37. Host If-­‐None-­‐Match Pragma Proxy-­‐Authorization Referer Upgrade User-­‐Agent Via www.fronteers.nl "737060cd8c284d8af7ad3082f209582d" no-­‐cache Basic   QWxhZGRpbjpvcGVuIHNl c2FtZQ== hQp://www.combell.com Mozilla/5.0   websocket 1.0  fred,  1.1  example.com  (Apache/1.1)
38. Response Headers
39. Age Allow Cache-­‐control Connection Content-­‐Encoding Content-­‐Language Content-­‐Length Content-­‐Type Date 12 GET,HEAD max-­‐age=3600 close gzip nl 1024 text/html;  charset=u^-­‐8 Tue,  15  Nov  1994  08:12:31  GMT
40. ETag Expires Last-­‐Modified Location Pragma Server Set-­‐Cookie Transfer-­‐Encoding Vary 737060cd8c284d8af7ad3082f209582d Thu,  01  Dec  1994  16:00:00  GMT Thu,  01  Dec  1994  16:00:00  GMT hkp://www.combell.com no-­‐cache My  magic  server bla=abc;  session=12345 chuncked Accept-­‐encoding
41. GET  /  HTTP/1.1 Host:  fronteers.nl User-­‐Agent:  Mozilla/5.0  (Macintosh;  Intel   Mac  OS  X  10.8;  rv:16.0)  Gecko/20100101   Firefox/16.0 Accept:  text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8 Accept-­‐Language:  en,nl;q=0.7,fr-­‐be;q=0.3 Accept-­‐Encoding:  gzip,  deflate Connection:  keep-­‐alive Cookie:  first_pv_60255=1;  _eventqueue=%7B %22heatmap%22%3A%5B%5D%2C%22events%22%3A%5B %5D%7D;  _jsuid=608348910;  unpoco_60255=1
42. HTTP/1.1  200  OK Date:  Wed,  17  Oct  2012  13:19:58  GMT Server:  Apache Content-­‐Encoding:  gzip Vary:  Accept-­‐Encoding X-­‐UA-­‐Compatible:  IE=Edge,chrome=1 Content-­‐Length:  5566 Keep-­‐Alive:  timeout=11,  max=97 Connection:  Keep-­‐Alive Content-­‐Type:  text/html;  charset=UTF-­‐8
43. Implementations
44. Clients +   cURL  &   Lynx
45. Servers The   classics
46. The   popular   new  kid Hype   factor  1
47. NoSQL Hype   factor  2
48. Varnish Hype   factor   2.5
49. Hype   factor  3 A  hipster’s  wet  dream
50. PHP’s  reac2on  to  node.js Pixels   on   purpose
51. What do we do with HTTP ?
52. Serve   files
53. Display  pages
54. Interact   with  services
55. Search  &  query
56. Invalidate  cache
57. Obstacles
58. Caching
59. ETag If-­‐Non-­‐Match:  "3e86-­‐410-­‐3596?bc" ETag:  "3e86-­‐410-­‐3596?bc"
60. Expires Expires  "Wed,  1  Jan  2014  20:00:00  GMT"
61. Cache-­‐control Cache-­‐Control  “max-­‐age=3600,  s-­‐ maxage=1000,  public,  must-­‐revalidate”
62. Max-­‐Age S-­‐maxage Public Private No-­‐cache No-­‐store Must-­‐revalidate Proxy-­‐revalidate TTL  for  browsers  in  seconds TTL  for  proxies  in  seconds Proxies  &  browsers  can  cache Only  browsers  can  cache Revalidate  before  dropping  from  cache Don’t  cache  at  all Browser  revalidate  before  serving  from   cache   Proxy  revalidate  before  serving  from   cache  
63. Cache-control in Varnish 1.Vcl (beresp.ttl) 2.Cache-control s-maxage 3.Cache-control max-age 4.Expires
64. Let’s talk about Varnish
65. “Varnish  saves  lives”
66. Reverse  caching   proxy
67. In  front  of  the   webserver
68. Only get & HEAD No cookie & auth headers No set-cookie headers cache ttl > 0 No vary “*” When will varnish cache?
69. based on host/ip + urL hash By default in Memory Optionally on disk how will varnish cache?
70. Request vcl_recv In cache? vcl_hash Cacheable? vcl_hit() vcl_miss() vcl_deliver() vcl_fetch() No Yes No Yes Response
71. #  sub  vcl_recv  { #          if  (req.restarts  ==  0)  { #              if  (req.http.x-­‐forwarded-­‐for)  { #                      set  req.http.X-­‐Forwarded-­‐For  = #                              req.http.X-­‐Forwarded-­‐For  +  ",  "  +  client.ip; #              }  else  { #                      set  req.http.X-­‐Forwarded-­‐For  =  client.ip; #              } #          } #          if  (req.request  !=  "GET"  && #              req.request  !=  "HEAD"  && #              req.request  !=  "PUT"  && #              req.request  !=  "POST"  && #              req.request  !=  "TRACE"  && #              req.request  !=  "OPTIONS"  && #              req.request  !=  "DELETE")  { #                  /*  Non-­‐RFC2616  or  CONNECT  which  is  weird.  */ #                  return  (pipe); #          } #          if  (req.request  !=  "GET"  &&  req.request  !=  "HEAD")  { #                  /*  We  only  deal  with  GET  and  HEAD  by  default  */ #                  return  (pass); #          } #          if  (req.http.Authorization  ||  req.http.Cookie)  { #                  /*  Not  cacheable  by  default  */ #                  return  (pass); #          } #          return  (lookup); #  }
72. #  sub  vcl_hash  { #          hash_data(req.url); #          if  (req.http.host)  { #                  hash_data(req.http.host); #          }  else  { #                  hash_data(server.ip); #          } #          return  (hash); #  } #  sub  vcl_fetch  { #          if  (beresp.ttl  <=  0s  || #                  beresp.http.Set-­‐Cookie  || #                  beresp.http.Vary  ==  "*")  { #                              /* #                                *  Mark  as  "Hit-­‐For-­‐Pass"  for  the  next  2  minutes #                                */ #                              set  beresp.ttl  =  120  s; #                              return  (hit_for_pass); #          } #          return  (deliver); #  }
73. Purge  cache
74. curl  -­‐X  PURGE  http://example.com/some-­‐page acl  purge  { "localhost" } sub  vcl_hit  {    if  (req.request  ==  "PURGE")  {        purge;        error  200  "Purged.";    } } sub  vcl_miss  {    if  (req.request  ==  "PURGE")  {        purge;        error  200  "Purged.";    } }
75. Edge Side Includes
76. header.php menu.php main.php footer.php TTL  5s No  caching TTL  10s TTL  2s
77. <html> <body> <table> <tr> <td colspan="2" > <esi:include src="/header.php" /> </td> </tr> <tr> <td><esi:include src="/menu.php" /></td> <td><esi:include src="/main.php" /></td> </tr> <tr> <td colspan="2" > <esi:include src="/footer.php" /> </td> </tr> </table> </body>
78. Surrogate-­‐Capability:  “key=ESI/1.0” Surrogate-­‐Control:  “content=ESI/1.0” Proxy Webserver
79. ESI  VCL sub  vcl_recv  {        set  req.http.Surrogate-­‐Capability="key=ESI/1.0"; } sub  vcl_fetch  {   if(beresp.http.Surrogate-­‐Control~"ESI/1.0")  {                  unset  beresp.http.Surrogate-­‐Control;          set  beresp.do_esi=true;        } }  
80. Cookies
81. Cookies Tracking Sessions Language
82. Cookies HTTP cookie request header via browser HTTP set-cookie response header via webserver
83. Varnish doesn’t like them
84. Don’t vary on cookie
85. Vary on custom header X-­‐Lang:  nl Vary:  X-­‐Lang Cookie:  lang=nl
86. Webserver     performance
87. Apache  vs  Nginx
88. Prefork  or  worker  mode Async,  event-­‐driven  workers Apache Nginx
89. Evented  non-­‐blocking  I/O We’ll  call   you  back!
90. Blocking var  results  =  db.query("SELECT  *  FROM  tablename"); //use  results   Non-­‐blocking db.query("SELECT  *  FROM  tablename"  ,  function  (results)  {   //use  results   });   //do  something  else  
91. Example var  http  =  require('http'); http.createServer(function  (req,  res)  {    res.writeHead(200,  {'Content-­‐Type':  'text/plain'});    res.end('Hello  World\n'); }).listen(1337,  '127.0.0.1'); console.log('Server  running  at  http://127.0.0.1:1337/');
92. We  can  do  it  in  PHP  too
93. Example <?php require 'vendor/autoload.php'; $app = function ($request, $response) { $response->writeHead(200, array('Content- Type' => 'text/plain')); $response->end("Hello World\n"); }; $loop = React\EventLoop\Factory::create(); $socket = new React\Socket\Server($loop); $http = new React\Http\Server($socket, $loop); $http->on('request', $app); echo "Server running at http://127.0.0.1:1337\n"; $socket->listen(1337); $loop->run();
94. Real  2me  data Ajax Websockets
95. Websockets var conn = new WebSocket('ws:// some.host'); conn.onmessage = function(e) { console.log(e.data); }; conn.send('Hello World!'); Client
96. Websockets Server Ratchet:   websocket   framework   on  top  of   React
97. <?php namespace MyApp; use Ratchet\MessageComponentInterface; use Ratchet\ConnectionInterface; class Chat implements MessageComponentInterface { protected $clients; public function __construct() { $this->clients = new \SplObjectStorage; } public function onOpen(ConnectionInterface $conn) { $this->clients->attach($conn); } public function onMessage(ConnectionInterface $from, $msg) { foreach ($this->clients as $client) { if ($from !== $client) { $client->send('Response: '.$msg); } } } public function onClose(ConnectionInterface $conn) { $this->clients->detach($conn); } public function onError(ConnectionInterface $conn, \Exception $e) { echo "An error has occurred: {$e->getMessage()}\n"; $conn->close(); } }
98. <?php require __DIR__.'/vendor/autoload.php'; require 'chat.php'; use Ratchet\WebSocket\WsServer; use Ratchet\Server\IoServer; use MyApp\Chat; $ws = new WsServer(new Chat()); $ws->disableVersion(0); $server = IoServer::factory($ws,80,'1.2.3.4'); $server->run();
100. HTTPS
101. Browser   trust Client
102. CSR CRT Key CA  bundle Server
103. Varnish   can’t  deal   with  it
104. Terminate  SSL
105. Is   also  a   proxy!
106. upstream  varnish  {    server  varnish1.example.com;    server  varnish2.example.com; } server  {    listen  443;    ssl  on;    ssl_certificate    /etc/nginx/example.ssl.cert;    ssl_certificate_key    /etc/nginx/example.ssl.key;    server_name  www.example.com;    access_log  /var/log/nginx/access.log;    error_log  /var/log/nginx/error.log;    location  /  {        proxy_pass    http://varnish;          proxy_redirect  off;        proxy_set_header  Host  $host;        proxy_set_header  X-­‐Scheme  $scheme;        proxy_set_header  X-­‐Forwarded-­‐For  $proxy_add_x_forwarded_for;        proxy_set_header  CLIENT_CERT  $ssl_client_raw_cert; } Make   your  app   aware
107. Other   cool  stuff   we  can   do!
108. upstream fpm { server php1.server.com:9000; server php2.server.com:9000; } upstream memcached { server memcached1.server.com:11211; server memcached2.server.com:11211; } server { root /var/www; index index.php index.html index.htm; server_name nginx.server.com; location / { try_files $uri $uri/ /index.php; } location @php { fastcgi_pass fpm; fastcgi_index index.php; include fastcgi_params; }
109. location ~ \.php$ { set $memcached_key $request_uri; memcached_pass memcached; memcached_next_upstream not_found; default_type text/html; error_page 404 405 502 = @php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { expires max; add_header Pragma public; add_header Cache-Control "public, must- revalidate, proxy-revalidate"; } }
110. REpresentational   State   Transfer
111. Developed  in   parallel  with   HTTP/1.1
112. The  way  the   web  *should*   work
113. Resources Stateless Cacheable Layered Uniform  interface Vocabulary  re-­‐use HATEOAS Keywords
115. David   Zülke  @dzuelke
116. ✓A  URL  idenKfies  a  resource.   ✓Resources  have  a  hierarchy,  so  you  know  that   something  with  addiKonal  slashes  is  a  subordinate   resource ✓Verbs  are  used  to  perform  operaKons  on   resources ✓The  operaKon  is  implicit  and  not  part  of  the  URL ✓A  hypermedia  format  is  used  to  represent  the   data ✓Link  relaKons  are  used  to  navigate  a  service Quodng   David  Zülke  @dzuelke
117. GET POST PUT DELETE Retrieve Insert Update Remove
118. HATEOAS WTF?
119. Hypertext As The Engine Of Application State
120. GET  /products/1234  HTTP/1.1 Host:  example.com Accept:  application/vnd.com.example+xml   HTTP/1.1  200  OK Content-­‐Type:  application/vnd.com.example+xml;  charset=utf-­‐8 Allow:  GET,  PUT,  DELETE <?  xml  version="1.0"  encoding="utf-­‐8"  ?> <product  xmlns="urn:com.example.products"  xmlns:atom="http:// w3.org/2005/Atom">    <id>1234</id>    <name  xml:lang="en">My  product</name>    <name  xml:lang="nl">Mijn  product</name> <price  currency="EUR">5</price> <price  currency="USD">6.5</price> <atom:link  rel="payment"  type="application/vnd.com.example +xml"  href="http://example.com/products/1234/payment" </product>   Thx   David!
121. And   on  that   bombshell  ...
123. h"p://joind.in/7581 Please   give  me   feedback
124. Thanks