Slide 1
Slide 1 text
Varnish
Workshop
By Thijs Feryn
Slide 2
Slide 2 text
Hi#
my#name#
is#Thijs
Slide 3
Slide 3 text
I’m#
an#evangelist#at
Slide 4
Slide 4 text
I’m#
a#board#member#
at
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
h"ps://joind.in/8432
Please#
give#me#
feedback
Slide 8
Slide 8 text
Varnish ?
Slide 9
Slide 9 text
Cache?
Slide 10
Slide 10 text
Reverse proxy?
Slide 11
Slide 11 text
Loadbalancer?
Slide 12
Slide 12 text
Loadbalancer?
HTTP accelerator
Slide 13
Slide 13 text
In#front#of#the#
webserver
Slide 14
Slide 14 text
Primary'purpose
Caching
Slide 15
Slide 15 text
Reverse'caching'
proxy
Slide 16
Slide 16 text
Install &
configure
Slide 17
Slide 17 text
curl%http://repo.varnish4cache.org/debian/GPG4
key.txt%|%apt4key%add%4
apt4get%update
echo%"deb%http://repo.varnish4cache.org/
debian/%squeeze%varnish43.0"%>>%/etc/apt/
sources.list
apt4get%install%varnish
Slide 18
Slide 18 text
Daemon'op)ons
h,ps://www.varnish4cache.org/docs/trunk/reference/varnishd.html
Slide 19
Slide 19 text
Daemon'op)ons
DAEMON_OPTS="4a'127.0.0.1:8080'\
'''''''''''''4T'localhost:6082'\
'''''''''''''4f'/etc/varnish/default.vcl'\
'''''''''''''4s'malloc,3G'\
'''''''''''''4p'thread_pools=8
'''''''''''''4p'thread_pool_min=160'\
'''''''''''''4p'thread_pool_max=3000'\
'''''''''''''4p'thread_pool_add_delay=2'\
'''''''''''''4p'connect_)meout=200s'\
'''''''''''''4p'first_byte_)meout=200s'\
'''''''''''''4p'between_bytes_)meout=50s'\
'''''''''''''4p'session_linger=100'\
'''''''''''''4p'lru_interval=20'\
'''''''''''''4p'listen_depth=8192'\
'''''''''''''4p'sess_workspace=262144'\
'''''''''''''4l'300M"
In#“/etc/default/varnish”
Slide 20
Slide 20 text
Daemon'op)ons
Slide 21
Slide 21 text
varnishadm>%param.show%4l
Shows#all#params
Varnish
parameters
Slide 22
Slide 22 text
Backend
Slide 23
Slide 23 text
Listen%8080
In#“/etc/apache2/ports.conf”
Backend
Slide 24
Slide 24 text
In#vhost#config#when#Varnish#is#
on#same#node#as#webserver
Backend
Slide 25
Slide 25 text
backend%default%{
%%%%%%.host%=%"127.0.0.1";
%%%%%%.port%=%"8080";
}
In#“/etc/varnish/default.vcl”
Backend
Slide 26
Slide 26 text
Threading model
Slide 27
Slide 27 text
Threading params
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
GET / HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X
10.5; fr; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16
Accept: text/html,application/xhtml+xml,application/
xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Cache-Control: max-age=0
Request
Slide 30
Slide 30 text
HTTP/1.1 200 OK
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By: PHP/5.3.2-1ubuntu4.7
Cache-Control: public, max-age=86400
Last-Modified: Mon, 04 Apr 2011 04:13:41 +0000
Expires: Sun, 11 Mar 1984 12:00:00 GMT
Vary: Cookie,Accept-Encoding
ETag: "1301890421"
Content-Type: text/html; charset=utf-8
Content-Length: 23562
Date: Mon, 04 Apr 2011 09:02:26 GMT
X-Varnish: 1886109724 1886107902
Age: 17324
Via: 1.1 varnish
Connection: keep-alive
Response
Slide 31
Slide 31 text
Cache@control
Cache7Control'“max7age=3600,'s7
maxage=1000,'public,'must7revalidate”
Slide 32
Slide 32 text
Max@Age
S@maxage
Public
Private
No@cache
No@store
Must@revalidate
Proxy@revalidate
TTL'for'browsers'in'seconds
TTL'for'proxies'in'seconds
Proxies'&'browsers'can'cache
Only'browsers'can'cache
Revalidate'before'dropping'from'cache
Don’t'cache'at'all
Browser'revalidate'before'serving'from
cache'
Proxy'revalidate'before'serving'from'
cache'
Slide 33
Slide 33 text
Age
Age'“56”
How#old#is#
the#cached#
object?
Slide 34
Slide 34 text
Real#cache#duration
Real'cache'duraNon'for'clients'='max7age'7'age
Real'cache'duraNon'for'proxies'='s7maxage'7'age
Slide 35
Slide 35 text
Expires
Expires'"Wed,'1'Jan'2014'20:00:00'GMT"
Slide 36
Slide 36 text
Expiration
1.Vcl&(beresp.-l)
2.Cache3control&s3maxage
3.Cache3control&max3age
4.Expires
Slide 37
Slide 37 text
The'flow
Slide 38
Slide 38 text
1
Slide 39
Slide 39 text
2
Slide 40
Slide 40 text
3
Slide 41
Slide 41 text
Advanced'
flow
Slide 42
Slide 42 text
1
Slide 43
Slide 43 text
2
Slide 44
Slide 44 text
3
Slide 45
Slide 45 text
Exercise 1
Testing HTTP cache headers
Slide 46
Slide 46 text
Exercise 2
Overriding Time To Live in Varnish
Slide 47
Slide 47 text
Out of the box
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
Varnish obeys caching headers
Slide 50
Slide 50 text
Only get & HEAD
No cookie & auth headers
No set-cookie headers
cache ttl > 0
No vary “*”
When will varnish cache?
Slide 51
Slide 51 text
About cookies
Slide 52
Slide 52 text
About cookies
HTTP cookie
request header
via browser
HTTP set-cookie
response header
via webserver
Slide 53
Slide 53 text
based on host/ip + urL hash
By default in Memory
Optionally on disk
how will varnish cache?
Slide 54
Slide 54 text
200: OK
203: Non-Authoritative Information
300: Multiple Choices
301: Moved Permanently
302: Moved Temporarily
307: Temporary Redirect
410: Gone
404: Not Found
Cacheable status codes
Slide 55
Slide 55 text
Exercise 3
Testing default behaviour
Slide 56
Slide 56 text
Monitoring
&
Logging
Slide 57
Slide 57 text
Varnishstat
Real#time#stats#for#
Varnish#instance
Slide 58
Slide 58 text
4+05:26:25
Hitrate.ratio:.......10......100......254
Hitrate.avg:.....0.8486...0.7619...0.7285
.....1760818.........6.99.........4.82.client_conn.;.Client.connections.accepted
....11088687........25.96........30.36.client_req.;.Client.requests.received
.....8042715........11.98........22.02.cache_hit.;.Cache.hits
.....2609561........11.98.........7.15.cache_miss.;.Cache.misses
.......47104.........1.00.........0.13.backend_conn.;.Backend.conn..success
.........610.........0.00.........0.00.backend_fail.;.Backend.conn..failures
.....2998265........12.98.........8.21.backend_reuse.;.Backend.conn..reuses
.......12081.........0.00.........0.03.backend_toolate.;.Backend.conn..was.closed
.....3010356........13.98.........8.24.backend_recycle.;.Backend.conn..recycles
..........13.........0.00.........0.00.backend_retry.;.Backend.conn..retry
.........520.........0.00.........0.00.fetch_head.;.Fetch.head
.....2857965........11.98.........7.83.fetch_length.;.Fetch.with.Length
......151309.........2.00.........0.41.fetch_chunked.;.Fetch.chunked
........4404.........0.00.........0.01.fetch_close.;.Fetch.wanted.close
.........676.........0.00.........0.00.fetch_failed.;.Fetch.failed
.......31164.........0.00.........0.09.fetch_304.;.Fetch.no.body.(304)
.........220...........................n_sess_mem.;.N.struct.sess_mem
..........53...........................n_sess.;.N.struct.sess
.......29540...........................n_object.;.N.struct.object
.......29561...........................n_objectcore.;.N.struct.objectcore
........5058...........................n_objecthead.;.N.struct.objecthead
........2613...........................n_waitinglist.;.N.struct.waitinglist
...........3...........................n_vbc.;.N.struct.vbc
..........22...........................n_wrk.;.N.worker.threads
........1789.........0.00.........0.00.n_wrk_create.;.N.worker.threads.created
Slide 59
Slide 59 text
.......31164.........0.00.........0.09.fetch_304.;.Fetch.no.body.(304)
.........220...........................n_sess_mem.;.N.struct.sess_mem
..........53...........................n_sess.;.N.struct.sess
.......29540...........................n_object.;.N.struct.object
.......29561...........................n_objectcore.;.N.struct.objectcore
........5058...........................n_objecthead.;.N.struct.objecthead
........2613...........................n_waitinglist.;.N.struct.waitinglist
...........3...........................n_vbc.;.N.struct.vbc
..........22...........................n_wrk.;.N.worker.threads
........1789.........0.00.........0.00.n_wrk_create.;.N.worker.threads.created
.......17204.........0.00.........0.05.n_wrk_queued.;.N.queued.work.requests
...........1...........................n_backend.;.N.backends
.....2579403...........................n_expired.;.N.expired.objects
.....6920374...........................n_lru_moved.;.N.LRU.moved.objects
.....8959782........26.96........24.53.n_objwrite.;.Objects.sent.with.write
.....1760818.........5.99.........4.82.s_sess.;.Total.Sessions
....11088687........25.96........30.36.s_req.;.Total.Requests
......436411.........2.00.........1.20.s_pass.;.Total.pass
.....3044686........13.98.........8.34.s_fetch.;.Total.fetch
..3817818737......8996.58.....10454.48.s_hdrbytes.;.Total.header.bytes
.84764679157....218712.68....232114.35.s_bodybytes.;.Total.body.bytes
......196150.........0.00.........0.54.sess_closed.;.Session.Closed
.......73512.........0.00.........0.20.sess_pipeline.;.Session.Pipeline
.......50339.........0.00.........0.14.sess_readahead.;.Session.Read.Ahead
....10875541........25.96........29.78.sess_linger.;.Session.Linger
.....7865547........25.96........21.54.sess_herd.;.Session.herd
...591440571......1649.54......1619.56.shm_records.;.SHM.records
....30844446.......121.82........84.46.shm_writes.;.SHM.writes
...........6.........0.00.........0.00.shm_flushes.;.SHM.flushes.due.to.overflow
Slide 60
Slide 60 text
$ varnishstat -1 -f n_lru_nuked
n_lru_nuked 0 . N LRU nuked objects
Varnishstat
Removed#from#
cache#due#to#lack#of#
memory
Slide 61
Slide 61 text
Varnishlog
In@memory#logs#of#
Varnish#activity
Slide 62
Slide 62 text
...11.SessionOpen..c.12.12.12.1.53727.:80
...11.ReqStart.....c.12.12.12.1.53727.1401010767
...11.RxRequest....c.GET
...11.RxURL........c./
...11.RxProtocol...c.HTTP/1.1
...11.RxHeader.....c.Host:.12.12.12.6
...11.RxHeader.....c.User;Agent:.Mozilla/5.0.(Macintosh;.Intel.Mac.OS.X.10.8;.rv:17.0).
Gecko/20100101.Firefox/17.0
...11.RxHeader.....c.Accept:.text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8
...11.RxHeader.....c.Accept;Language:.nl,en;q=0.7,fr;be;q=0.3
...11.RxHeader.....c.Accept;Encoding:.gzip,.deflate
...11.RxHeader.....c.Connection:.keep;alive
...11.VCL_call.....c.recv.lookup
...11.VCL_call.....c.hash
...11.Hash.........c./
...11.Hash.........c.12.12.12.6
...11.VCL_return...c.hash
...11.VCL_call.....c.miss.fetch
...11.Backend......c.13.default.default
...11.TTL..........c.1401010767.RFC.0.;1.;1.1357920021.0.1357920020.0.0
...11.VCL_call.....c.fetch
...11.TTL..........c.1401010767.VCL.120.;1.;1.1357920021.;0
...11.VCL_return...c.hit_for_pass
...11.ObjProtocol..c.HTTP/1.1
...11.ObjResponse..c.OK
...11.ObjHeader....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT
...11.ObjHeader....c.Server:.Apache
...11.ObjHeader....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18
...11.ObjHeader....c.Cache;Control:.no;cache,.no;store,.max;age=0
Client#
Slide 63
Slide 63 text
...11.VCL_return...c.hit_for_pass
...11.ObjProtocol..c.HTTP/1.1
...11.ObjResponse..c.OK
...11.ObjHeader....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT
...11.ObjHeader....c.Server:.Apache
...11.ObjHeader....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18
...11.ObjHeader....c.Cache;Control:.no;cache,.no;store,.max;age=0
...11.ObjHeader....c.Vary:.Accept;Encoding
...11.ObjHeader....c.Content;Encoding:.gzip
...11.ObjHeader....c.Content;Length:.119
...11.ObjHeader....c.Content;Type:.text/html
...11.Gzip.........c.u.F.;.119.336.80.80.887
...11.VCL_call.....c.deliver.deliver
...11.TxProtocol...c.HTTP/1.1
...11.TxStatus.....c.200
...11.TxResponse...c.OK
...11.TxHeader.....c.Server:.Apache
...11.TxHeader.....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18
...11.TxHeader.....c.Cache;Control:.no;cache,.no;store,.max;age=0
...11.TxHeader.....c.Vary:.Accept;Encoding
...11.TxHeader.....c.Content;Encoding:.gzip
...11.TxHeader.....c.Content;Type:.text/html
...11.TxHeader.....c.Content;Length:.119
...11.TxHeader.....c.Accept;Ranges:.bytes
...11.TxHeader.....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT
...11.TxHeader.....c.X;Varnish:.1401010767
...11.TxHeader.....c.Age:.0
...11.TxHeader.....c.Via:.1.1.varnish
...11.TxHeader.....c.Connection:.keep;alive
...11.Length.......c.119
...11.ReqEnd.......c.1401010767.1357920020.712090731.1357920020.727306366.0.000087738.
Client#
Slide 64
Slide 64 text
...13.BackendClose.;.default
...13.BackendOpen..b.default.127.0.0.1.51597.127.0.0.1.8080
...13.TxRequest....b.GET
...13.TxURL........b./
...13.TxProtocol...b.HTTP/1.1
...13.TxHeader.....b.Host:.12.12.12.6
...13.TxHeader.....b.User;Agent:.Mozilla/5.0.(Macintosh;.Intel.Mac.OS.X.10.8;.rv:17.0).
Gecko/20100101.Firefox/17.0
...13.TxHeader.....b.Accept:.text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8
...13.TxHeader.....b.Accept;Language:.nl,en;q=0.7,fr;be;q=0.3
...13.TxHeader.....b.X;Forwarded;For:.12.12.12.1
...13.TxHeader.....b.X;Varnish:.1401010767
...13.TxHeader.....b.Accept;Encoding:.gzip
...13.RxProtocol...b.HTTP/1.1
...13.RxStatus.....b.200
...13.RxResponse...b.OK
...13.RxHeader.....b.Date:.Fri,.11.Jan.2013.16:00:20.GMT
...13.RxHeader.....b.Server:.Apache
...13.RxHeader.....b.X;Powered;By:.PHP/5.3.2;1ubuntu4.18
...13.RxHeader.....b.Cache;Control:.no;cache,.no;store,.max;age=0
...13.RxHeader.....b.Vary:.Accept;Encoding
...13.RxHeader.....b.Content;Encoding:.gzip
...13.RxHeader.....b.Content;Length:.119
...13.RxHeader.....b.Content;Type:.text/html
...13.Fetch_Body...b.4(length).cls.0.mklen.1
...13.Length.......b.119
...13.BackendReuse.b.default
Backend#
Slide 65
Slide 65 text
4a!When!writing!to!a!file,!append!to!it!rather!than!overwrite!it.
4b!Include!backend!communication!logs
4c!Include!client!communication!logs
4d!Process!old!logs!on!startup
4I'Perform!regex!filter!on!output
4i'Include!specific!tags!into!output
4m'Perform!regex!filter!on!tags
4r'Read!a!shared!memory!log!dump!from!file
4w!Dump!shared!memory!logs!to!file
4x!Exclude!specific!tags!from!output
4X!Perform!exclusion!regex!filter!on!output
4O!Don’t!group!log!entries.!Don’t!use!with!Bm
Varnishlog
Slide 66
Slide 66 text
Backend
BackendClose
BackendOpen
BackendReuse
BackendXID
CLI
ClientAddr
Debug
Error
ExpBan
ExpKill
ExpPick
Hit
HitPass
HttpError
HttpGarbage
Length
ObjHeader
ObjLostHeader
ObjProtocol
ObjRequest
ObjResponse
ObjStatus
ObjURL
ReqEnd
ReqStart
RxHeader
RxLostHeader
RxProtocol
RxRequest
RxResponse
RxStatus
RxURL
SessionClose
SessionOpen
StatAddr
StatSess
TTL
TxHeader
TxLostHeader
TxProtocol
TxRequest
TxResponse
TxStatus
TxURL
VCL_acl
VCL_call
VCL_return
VCL_trace
WorkThread
Tags
Slide 67
Slide 67 text
varnishlog -c -m VCL_call:hit | grep -i 'RxURL'
Varnishlog
Get#the#URL#for#all#hits
Slide 68
Slide 68 text
varnishlog -b -i TxURL
Varnishlog
Track#all#“missed”#
URL’s
varnishlog -O -b -i TxURL
Ungrouped
Slide 69
Slide 69 text
varnishlog -c -i RxHeader -I User-Agent
Varnishlog
Get#all#user#agents
varnishlog -O -c -i RxHeader -I User-Agent
Ungrouped
Slide 70
Slide 70 text
varnishlog -i RxHeader
Varnishlog
varnishlog -c -i RxHeader
varnishlog -b -i RxHeader
Get#all#request#
headers
Slide 71
Slide 71 text
varnishlog -c -i RxURL -m VCL_call:hit
Varnishlog
Doesn’t#work
@i#is#done#before#@m
Slide 72
Slide 72 text
Why#did#it#miss?
Slide 73
Slide 73 text
11 VCL_return c hit_for_pass
Why'did'it'miss?
TTL#<=#0
13 RxHeader - Cache-Control: no-cache, no-
store, max-age=0
Slide 74
Slide 74 text
11 RxRequest c POST
11 RxURL c /exercises/3/post.php
11 RxProtocol c HTTP/1.1
11 RxHeader c Host: varnish.dev
11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS
X 10.8; rv:18.0) Gecko/20100101 Firefox/18.0
11 RxHeader c Accept: text/html,application/xhtml
+xml,application/xml;q=0.9,*/*;q=0.8
11 RxHeader c Accept-Language: nl,en;q=0.7,fr-be;q=0.3
11 RxHeader c Accept-Encoding: gzip, deflate
11 RxHeader c Referer: http://varnish.dev/exercises/3/post.php
11 RxHeader c Connection: keep-alive
11 RxHeader c Content-Type: application/x-www-form-urlencoded
11 RxHeader c Content-Length: 11
11 VCL_call c recv pass
Why'did'it'miss?
HTTP#POST
Slide 75
Slide 75 text
Was#it#stored#to#
cache?
Slide 76
Slide 76 text
11 TTL c 324425260 RFC 5 -1 -1
1358344107 0 1358344107 0 5
11 VCL_call c fetch deliver
Was'it'stored'to'cache?
Yes
Slide 77
Slide 77 text
11 TTL c 324425261 VCL 120 -1 -1
1358344313 -0
11 VCL_return c hit_for_pass
Was'it'stored'to'cache?
No,#for#
120#seconds#in#
blacklist
Slide 78
Slide 78 text
Exercise 4
Using varnishlog
Slide 79
Slide 79 text
What’s#the#cache#key?
Slide 80
Slide 80 text
11 Hash c /exercises/1/public.php
11 Hash c varnish.dev
What’s'the'cache'key?
Host#+#URL
Slide 81
Slide 81 text
Varnishtop
Incremental#in@
memory#Varnish#logs
Slide 82
Slide 82 text
list.length.109
.....8.12.VCL_return.....deliver
.....4.19.RxRequest......GET
.....4.19.RxProtocol.....HTTP/1.1
.....4.19.RxHeader.......Host:.varnish.dev
.....4.19.RxHeader.......Connection:.keep;alive
.....4.19.VCL_call.......recv
.....4.19.VCL_return.....lookup
.....4.19.VCL_call.......hash
.....4.19.Hash...........varnish.dev
.....4.19.VCL_return.....hash
.....4.19.VCL_call.......hit
.....4.19.VCL_call.......deliver
.....4.19.TxProtocol.....HTTP/1.1
.....4.19.TxHeader.......Server:.Apache
.....4.19.TxHeader.......Content;Encoding:.gzip
.....4.19.TxHeader.......Accept;Ranges:.bytes
.....4.19.TxHeader.......Via:.1.1.varnish
.....4.19.TxHeader.......Connection:.keep;alive
.....3.76.TxHeader.......Age:.26
.....3.12.RxURL........../exercises/5/vary.php
.....3.12.RxHeader.......Accept:.text/html,application/xhtml+xml,application/
xml;q=0.9,*/*;q=0.8
.....3.12.Hash.........../exercises/5/vary.php
.....3.12.TxStatus.......200
.....3.12.TxResponse.....OK
.....3.12.TxHeader.......X;Powered;By:.PHP/5.3.2;1ubuntu4.18
.....3.12.TxHeader.......Vary:.User;Agent,Accept;Encoding
.....3.12.TxHeader.......Cache;Control:.public,must;revalidate,s;maxage=.100
Unfiltered#list
Slide 83
Slide 83 text
4b!Include!backend!communication!logs
4c!Include!client!communication!logs
4d!Process!old!logs!on!startup
4I'Perform!regex!filter!on!output
4i'Include!specific!tags!into!output
4m'Perform!regex!filter!on!tags
4r'Read!a!shared!memory!log!dump!from!file
4p'Specifies!!the!!number!!of!!seconds!!to!!measure!!over
4x!Exclude!specific!tags!from!output
4X!Perform!exclusion!regex!filter!on!output
4O!Don’t!group!log!entries.!Don’t!use!with!Bm
4f!Sort!and!group!only!on!the!first!field!of!each!log!entry
Varnishtop
Slide 84
Slide 84 text
varnishtop -c -i RxHeader -I ^User-Agent
Varnishtop
Get#top#user#agents
Slide 85
Slide 85 text
varnishtop -b -i TxURL
Varnishtop
Get#top#missed#URL’s
Slide 86
Slide 86 text
varnishtop -c -i VCL_call
Varnishtop
Get#top#VCL#calls
Slide 87
Slide 87 text
varnishtop -c -i RxURL
Varnishtop
Most#popular#URL’s
Slide 88
Slide 88 text
Exercise 5
Using varnishtop
Slide 89
Slide 89 text
Cache variations
Slide 90
Slide 90 text
Vary: Accept-Language
Cache'variaNons
Keep#
seperate#cache#objects#
for#each#varation#of#the#
header
Vary: X-myHeader
Vary: Accept-Encoding
Slide 91
Slide 91 text
•Don’t5vary5‘*’
•Don’t5vary5‘cookie’5or5IE5will5freak5out
•Only5vary5request5headers
•Custom5headers5only5work5if5browser5
sends5them
•VCL5magic5(coming!up!later)
Cache'variaNons
Slide 92
Slide 92 text
Exercise 6
Cache variations
Slide 93
Slide 93 text
Varnish#
Configuration#
Language
Slide 94
Slide 94 text
Varnish#
Configuration#
Language
The'Varnish'ConfiguraNon'
Language'allows'you'to'define'
your'caching'policy.'You'write'
VCL'code'which'Varnish'will'
parse,'translate'to'C'code,'
compile'and'link'to.
Slide 95
Slide 95 text
Out#of#the#box#behaviour
Remember#
the#flow!
Slide 96
Slide 96 text
sub vcl_recv {
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + “, ” + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
if (req.request != “GET” &&
req.request != “HEAD” &&
req.request != “PUT” &&
req.request != “POST” &&
req.request != “TRACE” &&
req.request != “OPTIONS” &&
req.request != “DELETE”) {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.request != “GET” && req.request != “HEAD”) {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
return (lookup);
}
Slide 97
Slide 97 text
sub vcl_pipe {
# Note that only the first request to the backend will have
# X-Forwarded-For set. If you use X-Forwarded-For and want to
# have it set for all requests, make sure to have:
# set bereq.http.connection = "close";
# here. It is not set by default as it might break some broken web
# applications, like IIS with NTLM authentication.
return (pipe);
}
sub vcl_pass {
return (pass);
}
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
return (hash);
}
Slide 98
Slide 98 text
sub vcl_hit {
return (deliver);
}
sub vcl_miss {
return (fetch);
}
sub vcl_fetch {
if (beresp.ttl <= 0s ||
beresp.http.Set-Cookie ||
beresp.http.Vary == "*") {
! ! /*
! ! * Mark as "Hit-For-Pass" for the next 2 minutes
! ! */
! ! set beresp.ttl = 120 s;
! ! return (hit_for_pass);
}
return (deliver);
}
sub vcl_deliver {
return (deliver);
}
Slide 99
Slide 99 text
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
set obj.http.Retry-After = "5";
synthetic {"
"} + obj.status + " " + obj.response + {"
Error "} + obj.status + " " + obj.response + {"
"} + obj.response + {"
Guru Meditation:
XID: "} + req.xid + {"
Varnish cache server
"}; return (deliver); } sub vcl_init { ! return (ok); } sub vcl_fini { ! return (ok); }Slide 100
Slide 100 text
VCL'
syntax
Slide 101
Slide 101 text
//This is comment
#This is comment too
/* More comment
A lot more comment*/
Comment
Slide 102
Slide 102 text
if(req.url == "/") {
! error 200 "OK dude";
} elseif(req.url == "/notfound") {
! error 404 "Not found man";
} else {
! error 200 "Else";
}
CondiNonal
Slide 103
Slide 103 text
if(req.url == "/") {
! set req.backend = otherBackend;
}
Set'variables
Slide 104
Slide 104 text
Common%objects
✓Req:5request5object
✓Bereq:5backend5request5object
✓Beresp:5backend5response5object
✓Obj:5cache5object
✓Resp:5output5object
✓Client:5client5information
✓Server:5server5information
Slide 105
Slide 105 text
Variable%availability
Slide 106
Slide 106 text
.host
Host'name'or'IP'address'of'a'backend.
.port
Service'name'or'port'number'of'a'backend.
.connect_timeout
Backend'marked'“unhealthy”'when'a'connection'fails'after'x'time.
.first_byte_timeout
Backend'marked'“unhealthy”'when'the'first'byte'isn’t'received'after'x'time.
.between_byte_timeout
Backend'marked'“unhealthy”'when'the'byte'sequence'isn’t'coming'in'fast'
enough.
Backend'variables
Slide 107
Slide 107 text
.url
Specify'a'URL'to'request'from'the'backend.'Defaults'to'"/".
.request
Specify'a'full'HTTP'request'using'multiple'strings.'.request'will'have'\r\n'
automatically'inserted'after'every'string.'If'specified,'.request'will'take'
precedence'over'.url.
.window
How'many'of'the'latest'polls'we'examine'to'determine'backend'health.'
Defaults'to'8.
.threshold
How'many'of'the'polls'in'.window'must'have'succeeded'for'us'to'consider'the
backend'healthy.'Defaults'to'3.
Probe'variables'(1)
Slide 108
Slide 108 text
.initial
How'many'of'the'probes'are'considered'good'when'Varnish'starts.'Defaults'to
the'same'amount'as'the'threshold.
.expected_response
The'expected'backend'HTTP'response'code.'Defaults'to'200.
.interval
Defines'how'often'the'probe'should'check'the'backend.'Default'is'every'5'
seconds.
.timeout
How'fast'each'probe'times'out.'Default'is'2'seconds.
Probe'variables'(2)
Slide 109
Slide 109 text
client.ip
The'client's'IP'address.
client.identity
Identification'of'the'client,'used'to'load'balance'in'the'client'director.
server.hostname
The'host'name'of'the'server.
server.identity
The'identity'of'the'server,'as'set'by'the'4i'parameter.'If'the'4i'parameter'is'not'
passed'to'varnishd,'server.identity'will'be'set'to'the'name'of'the'instance,'as'
specified'by'the'4n'parameter.
server.ip
The'IP'address'of'the'socket'on'which'the'client'connection'was'received.
server.port
The'port'number'of'the'socket'on'which'the'client'connection'was'received.
Client'&'server'variables
Slide 110
Slide 110 text
req.request
The'request'type'(e.g.'"GET",'"HEAD").
req.url
The'requested'URL.
req.proto
The'HTTP'protocol'version'used'by'the'client.
req.backend
The'backend'to'use'to'service'the'request.
req.backend.healthy
Whether'the'backend'is'healthy'or'not.'Requires'an'active'probe'to'be'set'
on'the'backend.
req.http.header
The'corresponding'HTTP'header.
Request'variables'(1)
Slide 111
Slide 111 text
req.hash_always_miss
Force'a'cache'miss'for'this'request.'If'set'to'true'Varnish'will'disregard'any'
existing'objects'and'always'(re)fetch'from'the'backend.
req.hash_ignore_busy
Ignore'any'busy'object'during'cache'lookup.'You'would'want'to'do'this'if'
you'have'two'server'looking'up'content'from'each'other'to'avoid'potential'
deadlocks.
req.can_gzip
Does'the'client'accept'the'gzip'transfer'encoding.
req.restarts
A'count'of'how'many'times'this'request'has'been'restarted.
Unique'ID'of'this'request.
Request'variables'(2)
Slide 112
Slide 112 text
req.esi
Boolean.'Set'to'false'to'disable'ESI'processing'regardless'of'any'value'in'
beresp.do_esi.'Defaults'to'true.'This'variable'is'subject'to'change'in'future'
versions,'you'should'avoid'using'it.
req.esi_level
A'count'of'how'many'levels'of'ESI'requests'we're'currently'at.
req.grace
Set'to'a'period'to'enable'grace.
req.xid
Unique'ID'of'this'request.
req.hash
The'hash'key'used'to'refer'to'an'object'in'the'cache.'Used'when'both'
reading'from'and'writing'to'the'cache.
Request'variables'(3)
Slide 113
Slide 113 text
bereq.request
The'request'type'(e.g.'"GET",'"HEAD").
bereq.url
The'requested'URL.
bereq.proto
The'HTTP'protocol'version'used'to'talk'to'the'server.
bereq.http.header
The'corresponding'HTTP'header.
bereq.connect_timeout
The'time'in'seconds'to'wait'for'a'backend'connection.
bereq.first_byte_timeout
The'time'in'seconds'to'wait'for'the'first'byte'from'the'backend.'Not'
available'in'pipe'mode.
bereq.between_bytes_timeout
The'time'in'seconds'to'wait'between'each'received'byte'from'the'backend.'
Not'available'in'pipe'mode.
Backend'request'variables
Slide 114
Slide 114 text
beresp.do_stream
Deliver'the'object'to'the'client'directly'without'fetching'the'whole'object'
into'varnish.'If'this'request'is'pass'ed'it'will'not'be'stored'in'memory.'As'of'
Varnish'Cache'3.0'the'object'will'marked'as'busy'as'it'is'delivered'so'only'
client'can'access'the'object.
beresp.do_esi
Boolean.'ESI4process'the'object'after'fetching'it.'Defaults'to'false.'Set'it'to'
true'to'parse'the'object'for'ESI'directives.'Will'only'be'honored'if'req.esi'is'
true.
beresp.do_gzip
Boolean.'Gzip'the'object'before'storing'it.'Defaults'to'false.
beresp.do_gunzip
Boolean.'Unzip'the'object'before'storing'it'in'the'cache.'Defaults'to'false.
beresp.proto
The'HTTP'protocol'version'used'the'backend'replied'with.
beresp.storage
Set'to'force'Varnish'to'save'this'object'to'a'particular'storage'backend.
Backend#response#variables#(1)
Slide 115
Slide 115 text
beresp.status
The'HTTP'status'code'returned'by'the'server.
beresp.response
The'HTTP'status'message'returned'by'the'server.
beresp.ttl
The'object's'remaining'time'to'live,'in'seconds.'beresp.ttl'is'writable.
beresp.grace
Set'to'a'period'to'enable'grace.
beresp.saintmode
Set'to'a'period'to'enable'saint'mode.
beresp.backend.name
Name'of'the'backend'this'response'was'fetched'from.
beresp.backend.ip
IP'of'the'backend'this'response'was'fetched'from.
beresp.backend.port
Port'of'the'backend'this'response'was'fetched'from.
Backend#response#variables#(2)
Slide 116
Slide 116 text
obj.proto
The'HTTP'protocol'version'used'when'the'object'was'retrieved.
obj.status
The'HTTP'status'code'returned'by'the'server.
obj.response
The'HTTP'status'message'returned'by'the'server.
obj.ttl
The'object's'remaining'time'to'live,'in'seconds.'obj.ttl'is'writable.
obj.lastuse
The'approximate'time'elapsed'since'the'object'was'last'requests,'in'
seconds.'This'variable'is'also'available'in'vcl_deliver.
obj.hits
The'approximate'number'of'times'the'object'has'been'delivered.'A'value'of'
0'indicates'a'cache'miss.'This'variable'is'also'available'in'vcl_deliver.
obj.grace
The'object's'grace'period'in'seconds.'obj.grace'is'writable.
obj.http.header
The'corresponding'HTTP'header.
Cache#object#variables
Slide 117
Slide 117 text
resp.proto
The'HTTP'protocol'version'to'use'for'the'response.
resp.status
The'HTTP'status'code'that'will'be'returned.
resp.response
The'HTTP'status'message'that'will'be'returned.
resp.http.header
The'corresponding'HTTP'header.
Client#response#variables
Slide 118
Slide 118 text
Reminder!
Slide 119
Slide 119 text
Working#
with#
backends
Slide 120
Slide 120 text
backend default {
.host = "1.2.3.4";
.port = "http";
}
Backend'config
backend default {
.host = "localhost";
.port = "8080";
}
Simple
Slide 121
Slide 121 text
backend default {
.host = "1.2.3.4";
.port = "http";
.connect_timeout = 1s;
.first_byte_timeout = 5s;
.between_bytes_timeout = 2s;
}
Backend'config
More
Slide 122
Slide 122 text
backend default {
.host = "1.2.3.4";
.port = "http";
}
backend other {
.host = "1.2.3.5";
.port = "http";
}
sub vcl_recv {
if (req.http.host ~ "^(www.)?example.com$") {
set req.backend = other;
}
}
Select'backend
Slide 123
Slide 123 text
Probes
Slide 124
Slide 124 text
backend default {
.host = "1.2.3.4";
.port = "http";
.probe = {
.url = "/test.jpg";
.timeout = 0.3 s;
.window = 8;
.threshold = 3;
.initial = 3;
}
}
Probes
Test#if#
healthy
Slide 125
Slide 125 text
probe healthcheck {
.url = "/status.cgi";
.interval = 60s;
.timeout = 0.3 s;
.window = 8;
.threshold = 3;
.initial = 3;
.expected_response = 200;
}
backend www {
.host = "www.example.com";
.port = "http";
.probe = healthcheck;
}
Probes
Slide 126
Slide 126 text
probe rawprobe {
# NB: \r\n automatically inserted after each
string!
.request =
"GET / HTTP/1.1"
"Host: www.foo.bar"
"Connection: close";
}
Probes
Slide 127
Slide 127 text
Directors
Slide 128
Slide 128 text
A!director!is!a!logical!group!of!
backend!servers!clustered!
together!for!redundancy.!The!
basic!role!of!the!director!is!to!
let!Varnish!choose!a!backend!
server!amongst!several!so!if!one!
is!down!another!can!be!used.
Slide 129
Slide 129 text
Directors
✓Round4robin
✓Random
✓Client
✓Hash
✓Fallback
✓DNS
Slide 130
Slide 130 text
backend.one.{
....host.=.“localhost”;
....port.=.“80”;
}
backend.two.{
....host.=.“127.0.0.1”;
....port.=.“81”;
}
director.localhosts.round;robin.{
....{..backend.=.one;.}
....{..backend.=.two;.}
....{..backend.=.{.
.............host.=.“localhost”;.
.............port.=.“82”;.
........}.
....}
}
sub.vcl_recv.{
....set.req.backend.=.localhosts;
}
Round8robin%director
Slide 131
Slide 131 text
backend.one.{
....host.=.“localhost”;
....port.=.“80”;
}
backend.two.{
....host.=.“127.0.0.1”;
....port.=.“81”;
}
director.localhosts.random.{
....{..backend.=.one;..weight=4;}
....{..backend.=.two;..weight=6;}
....{..backend.=.{.
.............host.=.“localhost”;.
.............port.=.“82”;.
........}.
....}
}
sub.vcl_recv.{
....set.req.backend.=.localhosts;
}
Random%director
Slide 132
Slide 132 text
director.localhosts.client.{
....{..backend.=.one;..weight=1;.}
....{..backend.=.two;..weight=1;.}
}
sub.vcl_recv.{
....set.req.backend.=.localhosts;
....//Load.balance.by.URL
....set.client.identity.=.req.url;
....//Load.balance.by.client.IP,.this.is.the.default.
....set.client.identity.=.client.ip;
....//Load.balance.by.user.agent
....set.client.identity.=.req.http.user;agent;
}
Client%director
Uses#
client.identity
Slide 133
Slide 133 text
director.localhosts.hash.{
....{..backend.=.one;..weight=1;.}
....{..backend.=.two;..weight=1;.}
}
sub.vcl_recv.{
....set.req.backend.=.localhosts;
}
Hash%director
It'will'use'the'value'of'req.hash,'just'as'the'normal'
cache7lookup'methods
This'is'useful'is'you'are'using'Varnish'to'load'
balance'in'front'of'other'Varnish'caches'or'other'
web'accelerators'as'objects'won't'be'duplicated'
across'caches.
Slide 134
Slide 134 text
director.localhosts.fallback.{
....{..backend.=.one;}
....{..backend.=.two;}
}
sub.vcl_recv.{
....set.req.backend.=.localhosts;
}
Fallback%director
Picks#first#
healthy#
backend
Order#
matters
Slide 135
Slide 135 text
director.directorname.dns.{
.........list.=.{
.................host_header.=."www.example.com";
.................port.=."80";
.................connect_timeout.=.0.4s;
................"192.168.15.0"/24;
................"192.168.16.128"/25;
........}
.........ttl.=.5m;
.........suffix.=."internal.example.net";
}
DNS%director
Autogenerates'backends'in'IP'ranges
Slide 136
Slide 136 text
director.directorname.dns.{
.........list.=.{
.................host_header.=."www.example.com";
.................port.=."80";
.................connect_timeout.=.0.4s;
................"192.168.15.0"/24;
................"192.168.16.128"/25;
........}
.........ttl.=.5m;
.........suffix.=."internal.example.net";
}
DNS%director
Hostname'='host_header'+'suffix
www.example.com.internal.example.net
Slide 137
Slide 137 text
director.directorname.dns.{
.........list.=.{
.................host_header.=."www.example.com";
.................port.=."80";
.................connect_timeout.=.0.4s;
................"192.168.15.0"/24;
................"192.168.16.128"/25;
........}
.........ttl.=.5m;
.........suffix.=."internal.example.net";
}
DNS%director
Resolves'hostname'&'caches'result
Backend'selecNon'based'on'result'of'DNS'resolving
Slide 138
Slide 138 text
Exercise 7
Playing with backends, probes &
directors
Slide 139
Slide 139 text
Cache
invalidation
Slide 140
Slide 140 text
Don’t wait for
cache objects
to expire
Slide 141
Slide 141 text
There!are!only!two!hard!
things!in!Computer!Science:!
cache!invalidaKon,!naming!
things,!and!offBbyBone!errors.
Slide 142
Slide 142 text
✓Purge
✓Ban
✓Always'miss
Cache%invalida?on
Slide 143
Slide 143 text
✓Purge'VCL'method
✓Removes'specific'cache'item
✓Frees'up'memory
✓Evicts'variants
✓Next'client'triggers'the'refresh
✓No'backup'if'backend'is'down
Purge
Slide 144
Slide 144 text
sub$vcl_recv${
$$$$$$$$if$(req.request$==$"PURGE")${
$$$$$$$$$$$$$$$$return$(lookup);
$$$$$$$$}
}
sub$vcl_hit${
$$$$$$$$if$(req.request$==$"PURGE")${
$$$$$$$$$$$$$$$$purge;
$$$$$$$$$$$$$$$$error$200$"Purged";
$$$$$$$$}
}
sub$vcl_miss${
$$$$$$$$if$(req.request$==$"PURGE")${
$$$$$$$$$$$$$$$$error$404$"Not$in$cache";
$$$$$$$$}
}
Purge
HTTP#call#
to#purge
Slide 145
Slide 145 text
✓Implement'in'your'CMS
✓HTTP'200':'removed'from'cache
✓HTTP'404:'not'in'cache
✓Some'CMS'plugins'require'200'for'miss()
Purge
curl$IX$PURGE$http://varnish.dev/yourIpage
Slide 146
Slide 146 text
✓Ban'VCL'method'
✓Ban'command'line'action
✓Ban.url'command'line'action
✓Pattern'(regex)'based
✓Adds'ban'to'ban4list
✓No'memory'freed'up'immediately
✓Evicts'variants
✓Extremely'fast'and'powerful
Ban
Slide 147
Slide 147 text
sub$vcl_recv${
$$$$$$$$if$(req.request$==$"PURGE")${
ban("req.http.host$==$"$+$req.http.host$+$"$
&&$req.url$==$"$+$req.url);
error$200$"Banned";
$$$$$$$$}
}
Ban
sub$vcl_recv${
$$$$$$$$if$(req.request$==$"PURGE")${
$$$$$$$$$$$$ban("req.http.host$==$"$+$req.http.host$+$
"$&&$req.url$~$"$+$req.url);
error$200$"Banned";
$$$$$$$$}
}
Ban#URL#
pattern
Ban#URL
Name#
doesn’t#
matter
Slide 148
Slide 148 text
✓Removes'old'bans'from'the'ban4list
✓Separate'thread
✓Do'ban.list'on'CLI'to'see'the'ban4list
✓ban_lurker_sleep'parameter
✓Has'no'access'to'“req”'object
✓Smart'bans'are'ban'lurker'friendly
Ban%lurker
Slide 149
Slide 149 text
✓Ban'lurker'friendly
✓Use'beresp.x4url'&'beresp.x4host'instead'of'
req.url'&'req.http.host
Smart%bans
Slide 150
Slide 150 text
sub$vcl_recv${
$$$$$$$$if$(req.request$==$"PURGE")${
$$$$$$$$$$$$ban("obj.http.xIhost$==$"$+$req.http.host$+$
"$&&$obj.http.xIurl$==$"$+$req.url);
$$$$$$$$$$$$error$200$"Banned";
$$$$$$$$}
}
sub$vcl_fetch${
$$$$$$$$set$beresp.http.xIurl$=$req.url;
$$$$$$$$set$beresp.http.xIhost$=$req.http.host;
}
sub$vcl_deliver${
$$$$$$$$unset$resp.http.xIurl;
$$$$$$$$unset$resp.http.xIhost;
}
Smart%bans
Ban#lurker#
friendly
Slide 151
Slide 151 text
✓Used'for'reloading'content,'not'evicting
✓Uses'set'req.hash_always_miss'='true;
✓Looks'up'object,'but'ignores'it
✓Doesn’t'work'well'with'variations
✓Because'return(pass);'doesn’t'store'fetch'
result'in'cache
✓Because'purge();'will'fail'when'your'
backend'is'down
Always%miss%
Slide 152
Slide 152 text
sub$vcl_recv${
$$$$$$$$if$(req.request$==$"REFRESH")${
$$$$$$$$$$$$$$$$set$req.request$=$"GET";
$$$$$$$$$$$$$$$$set$req.hash_always_miss$=$true;
$$$$$$$$}
}
}
Refresh/always%miss
Slide 153
Slide 153 text
sub$vcl_recv${
$$$$if$(req.request$==$"PURGE")${
$$$$$$return$(lookup);
$$$$}
$$$$if$(req.request$==$"BAN")${
$$$$$$ban("obj.http.xIurl$~$"$+$req.http.xIurl$+$
$$$$$$"$&&$obj.http.xIhost$~$"$+$req.http.xIhost);
$$$$$$error$200$"Banned";
$$$$}
$$$$if$(req.request$==$"REFRESH")${
$$$$$$set$req.request$=$"GET";
$$$$$$set$req.hash_always_miss$=$true;
$$$$}
}
All8in8one%(1)
Slide 154
Slide 154 text
sub$vcl_hit${
$$$$$$$$if$(req.request$==$"PURGE")${
$$$$$$$$$$$$$$$$purge;
$$$$$$$$$$$$$$$$error$200$"Purged";
$$$$$$$$}
}
sub$vcl_miss${
$$$$$$$$if$(req.request$==$"PURGE")${$
$$$$$$$$$$$$$$$$purge;
$$$$$$$$$$$$$$$$error$404$"Not$in$cache";
$$$$$$$$}
}
sub$vcl_pass${
$$$$$$$$if$(req.request$==$"PURGE")${
$$$$$$$$$$$$$$$$error$502$"PURGE$on$a$passed$object";
$$$$$$$$}
}
All8in8one%(2)
Slide 155
Slide 155 text
sub$vcl_fetch${
$$$$$$$$set$beresp.http.xIurl$=$req.url;
$$$$$$$$set$beresp.http.xIhost$=$req.http.host;
}
sub$vcl_deliver${
$$$$$$$$unset$resp.http.xIurl;
$$$$$$$$unset$resp.http.xIhost;
}
All8in8one%(3)
Slide 156
Slide 156 text
varnishstat$I1$If$n_object
Measure%with%varnishstat
Number#
of#objects#in#
cache
Slide 157
Slide 157 text
Protect'your'purges
Slide 158
Slide 158 text
acl$purge${
$$"localhost";
$$"217.21.177.0"/24;
$$"10.10.10.1";
$$"some.host.com";
}
sub$vcl_recv${$$$$
$$if$(req.request$==$"PURGE")${
$$$$if$(!client.ip$~$purge)${
$$$$$$error$405$"Not$allowed.";
$$$$}$$$$
$$$$ban("obj.http.xIurl$==$"$+$req.url$+"$&&
$$$obj.http.xIhost$==$"$+$req.http.host);
$$$$error$200$"Purged";
$$}
}
Protect#your#purges
Use#an#ACL
Slide 159
Slide 159 text
Exercise 8
Invalidation
Slide 160
Slide 160 text
RELOAD !
Slide 161
Slide 161 text
✓/etc/init.d/varnish'reload
✓/etc/init.d/varnish'restart
✓varnishadm'(vcl.load'&'vcl.use)
✓telnet'(vcl.load'&'vcl.use)
Reloading%Varnish%VCL
Slide 162
Slide 162 text
✓/etc/init.d/varnish'reload
✓New'VCL'is'loaded
✓Keeps'objects'in'memory
Service%reload
Slide 163
Slide 163 text
✓/etc/init.d/varnish'restart
✓New'VCL'is'loaded
✓All'objects'are'removed'from'memory
Service%restart
Slide 164
Slide 164 text
✓Call'varnishadm'on'CLI
✓Call'vcl.list'for'an'overview
✓call'vcl.load'bla'/path/vcl/default.vcl
✓On'compilation'failure,'errors'are'shown
✓Call'vcl.use'bla'to'confirm'new'VCL
Varnishadm
Slide 165
Slide 165 text
✓Open'telnet'connection
✓Call'vcl.list'for'an'overview
✓call'vcl.load'bla'/path/vcl/default.vcl
✓On'compilation'failure,'errors'are'shown
✓Call'vcl.use'bla'to'confirm'new'VCL
Telnet
See#
next#slide
Slide 166
Slide 166 text
/usr/sbin/varnishd$IP$/var/run/
varnishd.pid$Ia$:80$If$/home/data/
default.vcl$IT$127.0.0.1:6082$Iu$
varnish$Ig$varnish$It$120$Ip$vcl_dir$/
home/data$Iw$5,500,300$Is$malloc,256m$I
S$/etc/varnish/secret$In$lucid32
Telnet Running#
process
Slide 167
Slide 167 text
$.telnet.127.0.0.1.6082
Telnet
Slide 168
Slide 168 text
$$telnet$127.0.0.1$6082
Trying$::1...
Trying$127.0.0.1...
Connected$to$localhost.
Escape$character$is$'^]'.
107$59$$$$$$
ayfmdmirgrluoqtukkueawfbrpmmvtkw
Authentication$required.
Telnet%authen?ca?on
Auth#
challenge
Slide 169
Slide 169 text
Telnet%authen?ca?on
$.cat./etc/varnish/secret.
ab9cae99;7905;42e8;9f28;
b405e684ce93
Secret#
key
Slide 170
Slide 170 text
Telnet%authen?ca?on
✓Challenge
✓Newline'character'(\x0A)
✓Secret'key
✓Newline'character'(\x0A)
✓Challenge
✓Newline'character'(\x0A)
✓SHA256'string
Compose#
auth#string
Slide 171
Slide 171 text
Telnet%authen?ca?on
Slide 172
Slide 172 text
Telnet%authen?ca?on
$$cat$/etc/varnish/secret$|$php$
secret.php$
ayfmdmirgrluoqtukkueawfbrpmmvtkw$
4fac6b3e568d8b307300d6fa7a1b92ce4e764ce
46cabed4783ffb0e937d86ff3
Slide 173
Slide 173 text
$$telnet$127.0.0.1$6082
Trying$::1...
Trying$127.0.0.1...
Connected$to$localhost.
Escape$character$is$'^]'.
107$59$$$$$$
ayfmdmirgrluoqtukkueawfbrpmmvtkw
Authentication$required.
auth$
4fac6b3e568d8b307300d6fa7a1b92ce4e764ce
46cabed4783ffb0e937d86ff3
200$204
Telnet%authen?ca?on
Slide 174
Slide 174 text
Exercise 9
Load VCL via Varnishadm & telnet
Slide 175
Slide 175 text
Custom#
VCL
Slide 176
Slide 176 text
sub$vcl_recv${
unset$req.http.cookie;
}
sub$vcl_fetch${
unset$beresp.http.setIcookie;
}
Remove%all%cookies
Slide 177
Slide 177 text
sub$vcl_recv${
set$req.http.Cookie$=$regsuball(req.http.Cookie,$
"__utm.=[^;]+(;$)?",$"");
set$req.http.Cookie$=$regsuball(req.http.Cookie,$
"utmctr=[^;]+(;$)?",$"");
set$req.http.Cookie$=$regsuball(req.http.Cookie,$
"utmcmd.=[^;]+(;$)?",$"");
set$req.http.Cookie$=$regsuball(req.http.Cookie,$
"utmccn.=[^;]+(;$)?",$"");
if$(req.http.cookie$~$"^$*$")${
unset$req.http.cookie;
}
}
Remove%GA%cookies
Slide 178
Slide 178 text
sub$vcl_recv${
$$$return(lookup);
}
sub$vcl_hash${
$$$hash_data(req.http.cookie);
}
Cache%cookies
Slide 179
Slide 179 text
sub$vcl_hash${
$$$$if(req.http.Cookie$~$"lang"){
$$$$$$$$hash_data(regsuball(req.http.Cookie,$"^.
+;?$?lang=([aIzAIZ0I9]+)($|;|$;).*$","\1"));
$$$$}
}
Cache%some%cookies
Slide 180
Slide 180 text
sub$vcl_recv${
$$$return(lookup);
}
sub$vcl_fetch${
$$$return(deliver);
}
Ignore%cookies
Slide 181
Slide 181 text
sub$vcl_recv${
$ if$(req.url$~$"^[^?]*\.(bmp|bz2|css|doc|
eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]|
pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff|
xml|zip)(\?.*)?$")${
$$$$$$$$unset$req.http.Cookie;
$$$$$$$$return$(lookup);
$$$$}
}
Cache%sta?c%files
Slide 182
Slide 182 text
sub$vcl_fetch${
$ if$(req.url$~$"^[^?]*\.(bmp|bz2|css|doc|
eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]|
pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff|
xml|zip)(\?.*)?$")${
$$$$$$$$unset$beresp.http.setIcookie;
$$$$}
}
Cache%sta?c%files
Slide 183
Slide 183 text
sub$vcl_deliver${
$$$$set$resp.http.xIcacheIhits$=$obj.hits;
$$$$if$(obj.hits$>$0)${
$$$$$$$$set$resp.http.xIcache$=$"hit";
$$$$}$else${
$$$$$$$$set$resp.http.xIcache$=$"miss";
$$$$}
}
Debug
Slide 184
Slide 184 text
sub$vcl_recv${
set$req.http.Host$=$regsub(req.http.Host,$":[0I9]+",$"");
if$(req.url$~$"\#")${
set$req.url$=$regsub(req.url,$"\#.*$",$"");
}
if$(req.url$~$"\?$")${
set$req.url$=$regsub(req.url,$"\?$",$"");
}
}
Sani?ze%URL
Slide 185
Slide 185 text
sub$vcl_recv${
if$(req.url$~$"(\?|&)(utm_source|utm_medium|utm_campaign|
gclid|cx|ie|cof|siteurl)=")${
set$req.url$=$regsuball(req.url,$"&(utm_source|
utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([AI
z0I9_\I\.%25]+)",$"");
$$$set$req.url$=$regsuball(req.url,$"\?(utm_source|
utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([AIz0I9_
\I\.%25]+)",$"?");
}
}
Sani?ze%URL%(2)
Slide 186
Slide 186 text
Exercise 10
Custom VCL
Slide 187
Slide 187 text
Grace mode
Slide 188
Slide 188 text
✓A'graced'object'is'an'object'that'has'expired,'but'is'
still'kept'in'cache
✓Grace'mode'is'when'Varnish'uses'a'graced'object
✓Used'when'backend'is'slow'or'down
✓req.grace'defines'how'long'an'overdue'an'object'can'
be'for'Varnish'to'still'consider'it'for'grace'mode.
✓beresp.grace'defines'how'long'past'the'beresp.ttl7
time'Varnish'will'keep'an'object
Slide 189
Slide 189 text
sub$vcl_recv{
$$$$if$(req.backend.healthy)${
$$$$$$$$set$req.grace$=$0s;
$$$$}$else${
$$$$$$$$set$req.grace$=$20s;
$$$$}
}
sub$vcl_fetch{
$$$$set$beresp.grace$=$20s;
}
Slide 190
Slide 190 text
probe.healthcheck.{
....url.=."/some/url";
....interval.=.1s;
....timeout.=.1s;
....window.=.1;
....threshold.=.1;
....initial.=.1;
....expected_response.=.200;
}
backend.default.{
...host.=."localhost";
...port.=."8080";
...probe.=.healthcheck;
}
sub.vcl_recv{
....if.(req.backend.healthy).{
........set.req.grace.=.0s;
....}.else.{
........set.req.grace.=.20s;
....}
}
sub.vcl_fetch{
....set.beresp.grace.=.20s;
}
With#
backend#
probe
Slide 191
Slide 191 text
Exercise 11
Grace mode
Slide 192
Slide 192 text
Edge#Side#Includes
Slide 193
Slide 193 text
Slide 194
Slide 194 text
header.php
menu.php main.php
footer.php
TTL#5s
No#caching
TTL#10s
TTL#2s
Slide 195
Slide 195 text
".PHP_EOL;
} else {
include($file);
}
}
ESI#PHP#helper#funcaon#(1)
Slide 196
Slide 196 text
".PHP_EOL;
} else {
include($file);
}
}
ESI#PHP#helper#funcaon#(2)
Slide 197
Slide 197 text
Slide 198
Slide 198 text
ESI#VCL
sub.vcl_recv.{
....set.req.http.Surrogate;Capability="key=ESI/1.0";
}
sub.vcl_fetch.{
. if(beresp.http.Surrogate;Control~"ESI/1.0").{
.........unset.beresp.http.Surrogate;Control;
. ....set.beresp.do_esi=true;
....}
}.
Slide 199
Slide 199 text
Slide 200
Slide 200 text
$content = '
'; if($esi->hasSurrogateEsiCapability($request)){ $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=2', null,false).'
'; $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=0', null,false).'
'; } $content .= '
Back to index'; Symfony#components#(2)
Exercise 12
'; $content .= 'Edge Side Includes
'; $content .= date("Y-m-d H:i:s").''; if($esi->hasSurrogateEsiCapability($request)){ $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=2', null,false).'
'; $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=0', null,false).'
'; } $content .= '
Back to index'; Symfony#components#(2)
Slide 201
Slide 201 text
$response->setContent($content);
$esi->addSurrogateControl($response);
$response->setSharedMaxAge(10);
$response->send();
Symfony#components#(3)
Slide 202
Slide 202 text
And#
on#that#
bombshell#...
Slide 203
Slide 203 text
No content
Slide 204
Slide 204 text
h"ps://joind.in/8432
Please#
give#me#
feedback
Slide 205
Slide 205 text
Thanks
Slide 206
Slide 206 text
No content