Slide 1
Slide 1 text
Hoe gebruikt u Varnish Cache
om uw website te versnellen?
Door Thijs Feryn
Slide 2
Slide 2 text
Trage websites sucken
Slide 3
Slide 3 text
Web performance is een
essentieel ondereel van
de user experience
Slide 4
Slide 4 text
Traag ~ down
Slide 5
Slide 5 text
Concurrentie
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
Zware last
Slide 11
Slide 11 text
Mo money
Mo servers
Slide 12
Slide 12 text
Cache
Slide 13
Slide 13 text
Herbereken
niet
als de data
niet
verandert
Slide 14
Slide 14 text
3 x 2 = ?
Slide 15
Slide 15 text
No content
Slide 16
Slide 16 text
Normaal
User Server
Slide 17
Slide 17 text
Met Varnish
User Varnish Server
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
Hallo, ik ben Thijs
Slide 20
Slide 20 text
Ik ben
@ThijsFeryn
op Twitter
Slide 21
Slide 21 text
Ik ben
Evangelist
bij
Slide 22
Slide 22 text
Sinds 2004
Slide 23
Slide 23 text
Support
2004-2007
Slide 24
Slide 24 text
Ex-support lead
2007-2011
Slide 25
Slide 25 text
Evangelist
sinds 2011
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
Een beetje geschiedenis
Slide 28
Slide 28 text
Waarom is het zo goed?
Slide 29
Slide 29 text
Slimme
kernel
Trucs
Slide 30
Slide 30 text
Slaat
HTTP output op
in memory*
Slide 31
Slide 31 text
Respecteert
HTTP
best
practices
Slide 32
Slide 32 text
Varnish
Configuration
Language
Slide 33
Slide 33 text
Websites
met pieken
Slide 34
Slide 34 text
Websites die van
nature traag zijn
Slide 35
Slide 35 text
Wat met TLS/SSL?
Slide 36
Slide 36 text
TLS termineren
User Varnish Server
TLS
HTTPS HTTP
Slide 37
Slide 37 text
Cloud servers
Slide 38
Slide 38 text
Shared hosting
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
No content
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
Varnish spreekt HTTP
Slide 48
Slide 48 text
✓Idempotence
✓State
✓Expiration
✓Conditional requests
✓Cache variaties
Varnish spreekt HTTP
Slide 49
Slide 49 text
✓GET
✓HEAD
-POST
-PUT
-PATCH
-DELETE
-TRACE
-OPTIONS
Idempotence
Cache enkel
GET & HEAD
Slide 50
Slide 50 text
✓Cachet niet wanneer er cookies
meegegeven worden door de browser
✓Cachet niet wanneer er cookies
aangemaakt worden door de backend
✓Cachet niet wanneer er Authorization
headers meegeven worden
State User-specific
content
Beïnvloedt
hit rate
Slide 51
Slide 51 text
Expiration
Expires: Sat, 09 Sep 2017 14:30:00 GMT
Cache-control: public, max-age=3600,
s-maxage=86400
Cache-control: private, no-cache, no-store
Slide 52
Slide 52 text
Expiration voorrang
1. beresp.ttl (VCL)
2. s-maxage
3. max-age
4. expires
5. 120 seconden standaard
Slide 53
Slide 53 text
Conditional requests
HTTP/1.1 200 OK
Host: localhost
Etag: 7c9d70604c6061da9bb9377d3f00eb27
Content-type: text/html; charset=UTF-8
Hello world output
GET /if_none_match.php HTTP/1.1
Host: localhost
User-Agent: curl/7.48.0
Slide 54
Slide 54 text
Conditional requests
HTTP/1.0 304 Not Modified
Host: localhost
Etag: 7c9d70604c6061da9bb9377d3f00eb27
GET /if_none_match.php HTTP/1.1
Host: localhost
User-Agent: curl/7.48.0
If-None-Match:
7c9d70604c6061da9bb9377d3f00eb27
Slide 55
Slide 55 text
Conditional requests
HTTP/1.1 200 OK
Host: localhost
Last-Modified: Fri, 22 Jul 2016 10:11:16 GMT
Content-type: text/html; charset=UTF-8
Hello world output
GET /if_none_match.php HTTP/1.1
Host: localhost
User-Agent: curl/7.48.0
Slide 56
Slide 56 text
Conditional requests
HTTP/1.0 304 Not Modified
Host: localhost
Last-Modified: Fri, 22 Jul 2016 10:11:16 GMT
GET /if_none_match.php HTTP/1.1
Host: localhost
User-Agent: curl/7.48.0
If-Last-Modified: Fri, 22 Jul 2016 10:11:16
GMT
Slide 57
Slide 57 text
✓Hostname (of IP)
✓URL
Cache variaties
Identificeert
objecten in de
cache
Slide 58
Slide 58 text
Cache variations
GET / HTTP/1.1
Host: localhost
Accept-Language: nl
HTTP/1.1 200 OK
Host: localhost
Vary: Accept-Language
Hallo, deze pagina is in
het Nederlands geschreven.
Slide 59
Slide 59 text
Flow
Slide 60
Slide 60 text
No content
Slide 61
Slide 61 text
No content
Slide 62
Slide 62 text
Built-in VCL
cheat sheet
Slide 63
Slide 63 text
sub vcl_recv {
if (req.method == "PRI") {
/* We do not support SPDY or HTTP/2.0 */
return (synth(405));
}
if (req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.method != "GET" && req.method != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
return (hash);
}
Slide 64
Slide 64 text
sub vcl_pipe {
# By default Connection: close is set on all piped requests, to stop
# connection reuse from sending future requests directly to the
# (potentially) wrong backend. If you do want this to happen, you can undo
# it here.
# unset bereq.http.connection;
return (pipe);
}
sub vcl_pass {
return (fetch);
}
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
return (lookup);
}
Slide 65
Slide 65 text
sub vcl_purge {
return (synth(200, "Purged"));
}
sub vcl_hit {
if (obj.ttl >= 0s) {
// A pure unadultered hit, deliver it
return (deliver);
}
if (obj.ttl + obj.grace > 0s) {
// Object is in grace, deliver it
// Automatically triggers a background fetch
return (deliver);
}
// fetch & deliver once we get the result
return (miss);
}
sub vcl_miss {
return (fetch);
}
sub vcl_deliver {
return (deliver);
}
Slide 66
Slide 66 text
sub vcl_backend_fetch {
return (fetch);
}
sub vcl_backend_response {
if (beresp.ttl <= 0s ||
beresp.http.Set-Cookie ||
beresp.http.Surrogate-control ~ "no-store" ||
(!beresp.http.Surrogate-Control &&
beresp.http.Cache-Control ~ "no-cache|no-store|
private") ||
beresp.http.Vary == "*") {
/*
* Mark as "Hit-For-Pass" for the next 2 minutes
*/
set beresp.ttl = 120s;
set beresp.uncacheable = true;
}
return (deliver);
}
Slide 67
Slide 67 text
In een ideale
wereld
Slide 68
Slide 68 text
De realiteit
Slide 69
Slide 69 text
No content
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
Cache-control ?
Slide 72
Slide 72 text
Legacy
Slide 73
Slide 73 text
VCL schrijven
Slide 74
Slide 74 text
Normalizeren
Slide 75
Slide 75 text
vcl 4.0;
import std;
sub vcl_recv {
set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
set req.url = std.querysort(req.url);
if (req.url ~ "\#") {
set req.url = regsub(req.url, "\#.*$", "");
}
if (req.url ~ "\?$") {
set req.url = regsub(req.url, "\?$", "");
}
if (req.restarts == 0) {
if (req.http.Accept-Encoding) {
if (req.http.User-Agent ~ "MSIE 6") {
unset req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
unset req.http.Accept-Encoding;
}
}
}
}
Slide 76
Slide 76 text
sub vcl_recv {
if (req.url ~ "^/status\.php$" ||
req.url ~ "^/update\.php$" ||
req.url ~ "^/admin$" ||
req.url ~ "^/admin/.*$" ||
req.url ~ "^/user$" ||
req.url ~ "^/user/.*$" ||
req.url ~ "^/flag/.*$" ||
req.url ~ "^.*/ajax/.*$" ||
req.url ~ "^.*/ahah/.*$") {
return (pass);
}
}
URL blacklist
Slide 77
Slide 77 text
sub vcl_recv {
if (req.url ~ "^/products/?"
return (hash);
}
}
URL whitelist
Slide 78
Slide 78 text
Cookies
Slide 79
Slide 79 text
vcl 4.0;
sub vcl_recv {
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
}
Tracking cookies verwijderen
Slide 80
Slide 80 text
vcl 4.0;
sub vcl_recv {
if (req.http.Cookie) {
set req.http.Cookie = ";" + req.http.Cookie;
set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
set req.http.Cookie = regsuball(req.http.Cookie, ";(PHPSESSID)=", "; \1=");
set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
}
}
Enkel sessie cookies bewaren
Slide 81
Slide 81 text
sub vcl_recv {
if (req.http.Cookie) {
set req.http.Cookie = ";" + req.http.Cookie;
set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
set req.http.Cookie = regsuball(req.http.Cookie, ";(language)=", "; \1=");
set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
return(pass);
}
return(hash);
}
}
sub vcl_hash {
hash_data(regsub( req.http.Cookie, "^.*language=([^;]*);*.*$", "\1" ));
}
Language cookie cache variatie
Slide 82
Slide 82 text
Alternatieve
language cache
variatie
Slide 83
Slide 83 text
sub vcl_hash {
hash_data(req.http.Accept-Language);
}
HTTP/1.1 200 OK
Host: localhost
Vary: Accept-Language
Hash Accept-Language
Of doe het gewoon zo:
Slide 84
Slide 84 text
Cookie
richtlijnen
Slide 85
Slide 85 text
✓Gooi niet alles in je sessievariabelen
✓Gebruik specifieke cookies
✓Verwijder tracking cookies in uw VCL
✓Make beslissingen op basis van cookies
✓Gebruik cookies voor cache variaties
✓Gebruik enkel de sessie cookie wanneer het echt
nodig is
Cookie richtlijnen
Slide 86
Slide 86 text
Block caching
Slide 87
Slide 87 text
No content
Slide 88
Slide 88 text
Code
rendert
één HTTP
response
Slide 89
Slide 89 text
Grootste
gemene
deler:
Niet cachen
Slide 90
Slide 90 text
Edge Side Includes
✓Placeholder
✓Wordt verwerkt door Varnish
✓Output is een compositie van blocks
✓State per block
✓TTL per block
Slide 91
Slide 91 text
sub vcl_recv {
set req.http.Surrogate-Capability = "key=ESI/1.0";
}
sub vcl_backend_response {
if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
unset beresp.http.Surrogate-Control;
set beresp.do_esi = true;
}
}
Edge Side Includes
Slide 92
Slide 92 text
ESI
vs
AJAX
Slide 93
Slide 93 text
Breaking news is niet aan het “breaken”
Slide 94
Slide 94 text
Purging
Slide 95
Slide 95 text
acl purge {
"localhost";
"127.0.0.1";
"::1";
}
sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, “Not allowed.”));
}
return (purge);
}
}
Purging
Slide 96
Slide 96 text
acl purge {
"localhost";
"127.0.0.1";
"::1";
}
sub vcl_backend_response {
set beresp.http.x-url = bereq.url;
set beresp.http.x-host = bereq.http.host;
}
sub vcl_deliver {
unset resp.http.x-url;
unset resp.http.x-host;
}
sub vcl_recv {
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "Not allowed"));
}
if(req.http.x-purge-regex) {
ban("obj.http.x-host == " + req.http.host + " && obj.http.x-url ~ " + req.http.x-purge-regex);
} else {
ban("obj.http.x-host == " + req.http.host + " && obj.http.x-url == " + req.url);
}
return (synth(200, "Purged"));
}
}
Banning
Slide 97
Slide 97 text
Banning
curl -XPURGE "http://example.com/products"
curl -XPURGE -H "x-purge-regex:/products" \ "http://example.com"
varnishadm> ban obj.http.x-host == example.com && obj.http.x-url ~ ^/
product/[0-9]+/details
Slide 98
Slide 98 text
Snelheid
Slide 99
Slide 99 text
Stabiliteit
Slide 100
Slide 100 text
Developer
empowerment
Slide 101
Slide 101 text
Cachen
Cache-control: public, max-age=3600, s-maxage=86400
Cache-control: private, no-cache, no-store
Niet cachen
Slide 102
Slide 102 text
No content
Slide 103
Slide 103 text
No content
Slide 104
Slide 104 text
https://feryn.eu
https://twitter.com/ThijsFeryn
https://instagram.com/ThijsFeryn
Slide 105
Slide 105 text
No content