There's this thing called HTTP

by Thijs Feryn

Slide 1

Slide 1 text

There’s this thing called HTTP By Thijs Feryn

Slide 2

Slide 2 text

Hi my name is Thijs

Slide 3

Slide 3 text

I’m an evangelist at

Slide 4

Slide 4 text

I’m a board member at

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

h"p://joind.in/7581 Please give me feedback

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Name 5 things you like about HTTP

Slide 9

Slide 9 text

Name 5 things you hate about HTTP

Slide 10

Slide 10 text

What the heck is HTTP actually?

Slide 11

Slide 11 text

It might get a bit boring ... just a bit

Slide 12

Slide 12 text

Application level protocol in the TCP/IP network stack

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Request & Response

Slide 15

Slide 15 text

Methods & Status codes

Slide 16

Slide 16 text

Headers & Body

Slide 17

Slide 17 text

GET / HTTP/1.1 Host: fronteers.nl User-‐Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0 Accept: text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8 Accept-‐Language: en,nl;q=0.7,fr-‐be;q=0.3 Accept-‐Encoding: gzip, deflate Connection: keep-‐alive Cookie: first_pv_60255=1; _eventqueue=%7B %22heatmap%22%3A%5B%5D%2C%22events%22%3A%5B %5D%7D; _jsuid=608348910; unpoco_60255=1

Slide 18

Slide 18 text

HTTP/1.1 200 OK Date: Wed, 17 Oct 2012 13:19:58 GMT Server: Apache Content-‐Encoding: gzip Vary: Accept-‐Encoding X-‐UA-‐Compatible: IE=Edge,chrome=1 Content-‐Length: 5566 Keep-‐Alive: timeout=11, max=97 Connection: Keep-‐Alive Content-‐Type: text/html; charset=UTF-‐8

Slide 19

Slide 19 text

Status codes h"p://www.w3.org/Protocols/rfc2616/rfc2616-‐sec10.html h"p://en.wikipedia.org/wiki/List_of_HTTP_status_codes

Slide 20

Slide 20 text

100 range Magic, fuggedaboudid!

Slide 21

Slide 21 text

200 range Check!

Slide 22

Slide 22 text

200 201 202 203 OK Created Accepted Non-‐Authorita7ve Informa7on

Slide 23

Slide 23 text

204 205 206 No Content Reset Content Par7al Content

Slide 24

Slide 24 text

300 range Talk to that guy over there

Slide 25

Slide 25 text

301 302 303 304 Moved permanently Found See other Not modified

Slide 26

Slide 26 text

305 306 307 Use Proxy Unused Temporary Redirect

Slide 27

Slide 27 text

400 range Dude, you f’d up!

Slide 28

Slide 28 text

400 403 404 405 408 Bad request Forbidden Not found Method not allowed Request 7meout

Slide 29

Slide 29 text

400 401 402 403 404 Bad Request Unauthorized Payment Required Forbidden Not Found

Slide 30

Slide 30 text

405 406 407 408 409 Method Not Allowed Not Acceptable Proxy Authen7ca7on Required Request Timeout Conﬂict

Slide 31

Slide 31 text

410 411 412 413 414 Gone Length Required Precondi7on Failed Request En7ty Too Large Request-‐URI Too Long

Slide 32

Slide 32 text

415 416 417 Unsupported Media Type Requested Range Not Sa7sﬁable Expecta7on Failed

Slide 33

Slide 33 text

500 range Sorry man, we f’d up!

Slide 34

Slide 34 text

500 501 502 503 504 505 Internal server error Not implemented Bad gateway Service unavailable Gateway 7meout HTTP Version Not Supported

Slide 35

Slide 35 text

Request Headers

Slide 36

Slide 36 text

Accept Accept-‐Charset Accept-‐Encoding Accept-‐language Authorization Cache-‐control Connection Cookie Content-‐length Date text/plain u^8 gzip, deﬂate en,nl;q=0.7,fr-‐be;q=0.3 Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ== no-‐cache close bla=ja; foo=bar 1024 Tue, 15 Nov 1994 08:12:31 GMT

Slide 37

Slide 37 text

Host If-‐None-‐Match Pragma Proxy-‐Authorization Referer Upgrade User-‐Agent Via www.fronteers.nl "737060cd8c284d8af7ad3082f209582d" no-‐cache Basic QWxhZGRpbjpvcGVuIHNl c2FtZQ== hQp://www.combell.com Mozilla/5.0 websocket 1.0 fred, 1.1 example.com (Apache/1.1)

Slide 38

Slide 38 text

Response Headers

Slide 39

Slide 39 text

Age Allow Cache-‐control Connection Content-‐Encoding Content-‐Language Content-‐Length Content-‐Type Date 12 GET,HEAD max-‐age=3600 close gzip nl 1024 text/html; charset=u^-‐8 Tue, 15 Nov 1994 08:12:31 GMT

Slide 40

Slide 40 text

ETag Expires Last-‐Modified Location Pragma Server Set-‐Cookie Transfer-‐Encoding Vary 737060cd8c284d8af7ad3082f209582d Thu, 01 Dec 1994 16:00:00 GMT Thu, 01 Dec 1994 16:00:00 GMT hkp://www.combell.com no-‐cache My magic server bla=abc; session=12345 chuncked Accept-‐encoding

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Implementations

Slide 44

Slide 44 text

Clients + cURL & Lynx

Slide 45

Slide 45 text

Servers The classics

Slide 46

Slide 46 text

The popular new kid Hype factor 1

Slide 47

Slide 47 text

NoSQL Hype factor 2

Slide 48

Slide 48 text

Varnish Hype factor 2.5

Slide 49

Slide 49 text

Hype factor 3 A hipster’s wet dream

Slide 50

Slide 50 text

PHP’s reac2on to node.js Pixels on purpose

Slide 51

Slide 51 text

What do we do with HTTP ?

Slide 52

Slide 52 text

Serve ﬁles

Slide 53

Slide 53 text

Display pages

Slide 54

Slide 54 text

Interact with services

Slide 55

Slide 55 text

Search & query

Slide 56

Slide 56 text

Invalidate cache

Slide 57

Slide 57 text

Obstacles

Slide 58

Slide 58 text

Caching

Slide 59

Slide 59 text

ETag If-‐Non-‐Match: "3e86-‐410-‐3596?bc" ETag: "3e86-‐410-‐3596?bc"

Slide 60

Slide 60 text

Expires Expires "Wed, 1 Jan 2014 20:00:00 GMT"

Slide 61

Slide 61 text

Cache-‐control Cache-‐Control “max-‐age=3600, s-‐ maxage=1000, public, must-‐revalidate”

Slide 62

Slide 62 text

Max-‐Age S-‐maxage Public Private No-‐cache No-‐store Must-‐revalidate Proxy-‐revalidate TTL for browsers in seconds TTL for proxies in seconds Proxies & browsers can cache Only browsers can cache Revalidate before dropping from cache Don’t cache at all Browser revalidate before serving from cache Proxy revalidate before serving from cache

Slide 63

Slide 63 text

Cache-control in Varnish 1.Vcl (beresp.ttl) 2.Cache-control s-maxage 3.Cache-control max-age 4.Expires

Slide 64

Slide 64 text

Let’s talk about Varnish

Slide 65

Slide 65 text

“Varnish saves lives”

Slide 66

Slide 66 text

Reverse caching proxy

Slide 67

Slide 67 text

In front of the webserver

Slide 68

Slide 68 text

Only get & HEAD No cookie & auth headers No set-cookie headers cache ttl > 0 No vary “*” When will varnish cache?

Slide 69

Slide 69 text

based on host/ip + urL hash By default in Memory Optionally on disk how will varnish cache?

Slide 70

Slide 70 text

Request vcl_recv In cache? vcl_hash Cacheable? vcl_hit() vcl_miss() vcl_deliver() vcl_fetch() No Yes No Yes Response

Slide 71

Slide 71 text

# sub vcl_recv { # if (req.restarts == 0) { # if (req.http.x-‐forwarded-‐for) { # set req.http.X-‐Forwarded-‐For = # req.http.X-‐Forwarded-‐For + ", " + client.ip; # } else { # set req.http.X-‐Forwarded-‐For = client.ip; # } # } # if (req.request != "GET" && # req.request != "HEAD" && # req.request != "PUT" && # req.request != "POST" && # req.request != "TRACE" && # req.request != "OPTIONS" && # req.request != "DELETE") { # /* Non-‐RFC2616 or CONNECT which is weird. */ # return (pipe); # } # if (req.request != "GET" && req.request != "HEAD") { # /* We only deal with GET and HEAD by default */ # return (pass); # } # if (req.http.Authorization || req.http.Cookie) { # /* Not cacheable by default */ # return (pass); # } # return (lookup); # }

Slide 72

Slide 72 text

# sub vcl_hash { # hash_data(req.url); # if (req.http.host) { # hash_data(req.http.host); # } else { # hash_data(server.ip); # } # return (hash); # } # sub vcl_fetch { # if (beresp.ttl <= 0s || # beresp.http.Set-‐Cookie || # beresp.http.Vary == "*") { # /* # * Mark as "Hit-‐For-‐Pass" for the next 2 minutes # */ # set beresp.ttl = 120 s; # return (hit_for_pass); # } # return (deliver); # }

Slide 73

Slide 73 text

Purge cache

Slide 74

Slide 74 text

curl -‐X PURGE http://example.com/some-‐page acl purge { "localhost" } sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged."; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged."; } }

Slide 75

Slide 75 text

Edge Side Includes

Slide 76

Slide 76 text

header.php menu.php main.php footer.php TTL 5s No caching TTL 10s TTL 2s

Slide 77

Slide 77 text

Slide 78

Slide 78 text

Surrogate-‐Capability: “key=ESI/1.0” Surrogate-‐Control: “content=ESI/1.0” Proxy Webserver

Slide 79

Slide 79 text

ESI VCL sub vcl_recv { set req.http.Surrogate-‐Capability="key=ESI/1.0"; } sub vcl_fetch { if(beresp.http.Surrogate-‐Control~"ESI/1.0") { unset beresp.http.Surrogate-‐Control; set beresp.do_esi=true; } }

Slide 80

Slide 80 text

Slide 81

Slide 81 text

Cookies Tracking Sessions Language

Slide 82

Slide 82 text

Cookies HTTP cookie request header via browser HTTP set-cookie response header via webserver

Slide 83

Slide 83 text

Varnish doesn’t like them

Slide 84

Slide 84 text

Don’t vary on cookie

Slide 85

Slide 85 text

Vary on custom header X-‐Lang: nl Vary: X-‐Lang Cookie: lang=nl

Slide 86

Slide 86 text

Webserver performance

Slide 87

Slide 87 text

Apache vs Nginx

Slide 88

Slide 88 text

Prefork or worker mode Async, event-‐driven workers Apache Nginx

Slide 89

Slide 89 text

Evented non-‐blocking I/O We’ll call you back!

Slide 90

Slide 90 text

Blocking var results = db.query("SELECT * FROM tablename"); //use results Non-‐blocking db.query("SELECT * FROM tablename" , function (results) { //use results }); //do something else

Slide 91

Slide 91 text

Example var http = require('http'); http.createServer(function (req, res) { res.writeHead(200, {'Content-‐Type': 'text/plain'}); res.end('Hello World\n'); }).listen(1337, '127.0.0.1'); console.log('Server running at http://127.0.0.1:1337/');

Slide 92

Slide 92 text

We can do it in PHP too

Slide 93

Slide 93 text

Example writeHead(200, array('Content- Type' => 'text/plain')); $response->end("Hello World\n"); }; $loop = React\EventLoop\Factory::create(); $socket = new React\Socket\Server($loop); $http = new React\Http\Server($socket, $loop); $http->on('request', $app); echo "Server running at http://127.0.0.1:1337\n"; $socket->listen(1337); $loop->run();

Slide 94

Slide 94 text

Real 2me data Ajax Websockets

Slide 95

Slide 95 text

Websockets var conn = new WebSocket('ws:// some.host'); conn.onmessage = function(e) { console.log(e.data); }; conn.send('Hello World!'); Client

Slide 96

Slide 96 text

Websockets Server Ratchet: websocket framework on top of React

Slide 97

Slide 97 text

clients = new \SplObjectStorage; } public function onOpen(ConnectionInterface $conn) { $this->clients->attach($conn); } public function onMessage(ConnectionInterface $from, $msg) { foreach ($this->clients as $client) { if ($from !== $client) { $client->send('Response: '.$msg); } } } public function onClose(ConnectionInterface $conn) { $this->clients->detach($conn); } public function onError(ConnectionInterface $conn, \Exception $e) { echo "An error has occurred: {$e->getMessage()}\n"; $conn->close(); } }

Slide 98

Slide 98 text

disableVersion(0); $server = IoServer::factory($ws,80,'1.2.3.4'); $server->run();

Slide 99

Slide 99 text

No content

Slide 100

Slide 100 text

HTTPS

Slide 101

Slide 101 text

Browser trust Client

Slide 102

Slide 102 text

CSR CRT Key CA bundle Server

Slide 103

Slide 103 text

Varnish can’t deal with it

Slide 104

Slide 104 text

Terminate SSL

Slide 105

Slide 105 text

Is also a proxy!

Slide 106

Slide 106 text

upstream varnish { server varnish1.example.com; server varnish2.example.com; } server { listen 443; ssl on; ssl_certificate /etc/nginx/example.ssl.cert; ssl_certificate_key /etc/nginx/example.ssl.key; server_name www.example.com; access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; location / { proxy_pass http://varnish; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-‐Scheme $scheme; proxy_set_header X-‐Forwarded-‐For $proxy_add_x_forwarded_for; proxy_set_header CLIENT_CERT $ssl_client_raw_cert; } Make your app aware

Slide 107

Slide 107 text

Other cool stuff we can do!

Slide 108

Slide 108 text

upstream fpm { server php1.server.com:9000; server php2.server.com:9000; } upstream memcached { server memcached1.server.com:11211; server memcached2.server.com:11211; } server { root /var/www; index index.php index.html index.htm; server_name nginx.server.com; location / { try_files $uri $uri/ /index.php; } location @php { fastcgi_pass fpm; fastcgi_index index.php; include fastcgi_params; }

Slide 109

Slide 109 text

location ~ \.php$ { set $memcached_key $request_uri; memcached_pass memcached; memcached_next_upstream not_found; default_type text/html; error_page 404 405 502 = @php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { expires max; add_header Pragma public; add_header Cache-Control "public, must- revalidate, proxy-revalidate"; } }

Slide 110

Slide 110 text

REpresentational State Transfer

Slide 111

Slide 111 text

Developed in parallel with HTTP/1.1

Slide 112

Slide 112 text

The way the web *should* work

Slide 113

Slide 113 text

Resources Stateless Cacheable Layered Uniform interface Vocabulary re-‐use HATEOAS Keywords

Slide 114

Slide 114 text

No content

Slide 115

Slide 115 text

David Zülke @dzuelke

Slide 116

Slide 116 text

✓A URL idenKﬁes a resource. ✓Resources have a hierarchy, so you know that something with addiKonal slashes is a subordinate resource ✓Verbs are used to perform operaKons on resources ✓The operaKon is implicit and not part of the URL ✓A hypermedia format is used to represent the data ✓Link relaKons are used to navigate a service Quodng David Zülke @dzuelke

Slide 117

Slide 117 text

GET POST PUT DELETE Retrieve Insert Update Remove

Slide 118

Slide 118 text

HATEOAS WTF?

Slide 119

Slide 119 text

Hypertext As The Engine Of Application State

Slide 120

Slide 120 text

GET /products/1234 HTTP/1.1 Host: example.com Accept: application/vnd.com.example+xml HTTP/1.1 200 OK Content-‐Type: application/vnd.com.example+xml; charset=utf-‐8 Allow: GET, PUT, DELETE xml version="1.0" encoding="utf-‐8" ?> 1234 My product Mijn product 5 6.5 Thx David!