Varnish workshop - DPC13

Slides for my Varnish workshop at the Dutch PHP Conference 2013

Transcript

1. Varnish Workshop By Thijs Feryn
2. Hi# my#name# is#Thijs
3. I’m# an#evangelist#at
4. I’m# a#board#member# at
7. h"ps://joind.in/8432 Please# give#me# feedback
8. Varnish ?
9. Cache?
10. Reverse proxy?
11. Loadbalancer?
12. Loadbalancer? HTTP accelerator
13. In#front#of#the# webserver
14. Primary'purpose Caching
15. Reverse'caching' proxy
16. Install & configure
17. curl%http://repo.varnish4cache.org/debian/GPG4 key.txt%|%apt4key%add%4 apt4get%update echo%"deb%http://repo.varnish4cache.org/ debian/%squeeze%varnish43.0"%>>%/etc/apt/ sources.list apt4get%install%varnish
18. Daemon'op)ons h,ps://www.varnish4cache.org/docs/trunk/reference/varnishd.html
19. Daemon'op)ons DAEMON_OPTS="4a'127.0.0.1:8080'\ '''''''''''''4T'localhost:6082'\ '''''''''''''4f'/etc/varnish/default.vcl'\ '''''''''''''4s'malloc,3G'\ '''''''''''''4p'thread_pools=8 '''''''''''''4p'thread_pool_min=160'\ '''''''''''''4p'thread_pool_max=3000'\ '''''''''''''4p'thread_pool_add_delay=2'\ '''''''''''''4p'connect_)meout=200s'\ '''''''''''''4p'first_byte_)meout=200s'\ '''''''''''''4p'between_bytes_)meout=50s'\ '''''''''''''4p'session_linger=100'\ '''''''''''''4p'lru_interval=20'\ '''''''''''''4p'listen_depth=8192'\ '''''''''''''4p'sess_workspace=262144'\ '''''''''''''4l'300M" In#“/etc/default/varnish”
20. Daemon'op)ons
21. varnishadm>%param.show%4l Shows#all#params Varnish parameters
22. Backend
23. Listen%8080 In#“/etc/apache2/ports.conf” Backend
24. <VirtualHost%*:8080> In#vhost#config#when#Varnish#is# on#same#node#as#webserver Backend
25. backend%default%{ %%%%%%.host%=%"127.0.0.1"; %%%%%%.port%=%"8080"; } In#“/etc/varnish/default.vcl” Backend
26. Threading model
27. Threading params
29. GET / HTTP/1.1 Host: localhost User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; fr; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16 Accept: text/html,application/xhtml+xml,application/ xml;q=0.9,*/*;q=0.8 Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Cache-Control: max-age=0 Request
30. HTTP/1.1 200 OK Server: Apache/2.2.14 (Ubuntu) X-Powered-By: PHP/5.3.2-1ubuntu4.7 Cache-Control: public, max-age=86400 Last-Modified: Mon, 04 Apr 2011 04:13:41 +0000 Expires: Sun, 11 Mar 1984 12:00:00 GMT Vary: Cookie,Accept-Encoding ETag: "1301890421" Content-Type: text/html; charset=utf-8 Content-Length: 23562 Date: Mon, 04 Apr 2011 09:02:26 GMT X-Varnish: 1886109724 1886107902 Age: 17324 Via: 1.1 varnish Connection: keep-alive Response
31. Cache@control Cache7Control'“max7age=3600,'s7 maxage=1000,'public,'must7revalidate”
32. Max@Age S@maxage Public Private No@cache No@store Must@revalidate Proxy@revalidate TTL'for'browsers'in'seconds TTL'for'proxies'in'seconds Proxies'&'browsers'can'cache Only'browsers'can'cache Revalidate'before'dropping'from'cache Don’t'cache'at'all Browser'revalidate'before'serving'from cache' Proxy'revalidate'before'serving'from' cache'
33. Age Age'“56” How#old#is# the#cached# object?
34. Real#cache#duration Real'cache'duraNon'for'clients'='max7age'7'age Real'cache'duraNon'for'proxies'='s7maxage'7'age
35. Expires Expires'"Wed,'1'Jan'2014'20:00:00'GMT"
36. Expiration 1.Vcl&(beresp.-l) 2.Cache3control&s3maxage 3.Cache3control&max3age 4.Expires
37. The'flow
41. Advanced' flow
45. Exercise 1 Testing HTTP cache headers
46. Exercise 2 Overriding Time To Live in Varnish
47. Out of the box
49. Varnish obeys caching headers
50. Only get & HEAD No cookie & auth headers No set-cookie headers cache ttl > 0 No vary “*” When will varnish cache?
51. About cookies
52. About cookies HTTP cookie request header via browser HTTP set-cookie response header via webserver
53. based on host/ip + urL hash By default in Memory Optionally on disk how will varnish cache?
54. 200: OK 203: Non-Authoritative Information 300: Multiple Choices 301: Moved Permanently 302: Moved Temporarily 307: Temporary Redirect 410: Gone 404: Not Found Cacheable status codes
55. Exercise 3 Testing default behaviour
56. Monitoring & Logging
57. Varnishstat Real#time#stats#for# Varnish#instance
58. 4+05:26:25 Hitrate.ratio:.......10......100......254 Hitrate.avg:.....0.8486...0.7619...0.7285 .....1760818.........6.99.........4.82.client_conn.;.Client.connections.accepted ....11088687........25.96........30.36.client_req.;.Client.requests.received .....8042715........11.98........22.02.cache_hit.;.Cache.hits .....2609561........11.98.........7.15.cache_miss.;.Cache.misses .......47104.........1.00.........0.13.backend_conn.;.Backend.conn..success .........610.........0.00.........0.00.backend_fail.;.Backend.conn..failures .....2998265........12.98.........8.21.backend_reuse.;.Backend.conn..reuses .......12081.........0.00.........0.03.backend_toolate.;.Backend.conn..was.closed .....3010356........13.98.........8.24.backend_recycle.;.Backend.conn..recycles ..........13.........0.00.........0.00.backend_retry.;.Backend.conn..retry .........520.........0.00.........0.00.fetch_head.;.Fetch.head .....2857965........11.98.........7.83.fetch_length.;.Fetch.with.Length ......151309.........2.00.........0.41.fetch_chunked.;.Fetch.chunked ........4404.........0.00.........0.01.fetch_close.;.Fetch.wanted.close .........676.........0.00.........0.00.fetch_failed.;.Fetch.failed .......31164.........0.00.........0.09.fetch_304.;.Fetch.no.body.(304) .........220...........................n_sess_mem.;.N.struct.sess_mem ..........53...........................n_sess.;.N.struct.sess .......29540...........................n_object.;.N.struct.object .......29561...........................n_objectcore.;.N.struct.objectcore ........5058...........................n_objecthead.;.N.struct.objecthead ........2613...........................n_waitinglist.;.N.struct.waitinglist ...........3...........................n_vbc.;.N.struct.vbc ..........22...........................n_wrk.;.N.worker.threads ........1789.........0.00.........0.00.n_wrk_create.;.N.worker.threads.created
59. .......31164.........0.00.........0.09.fetch_304.;.Fetch.no.body.(304) .........220...........................n_sess_mem.;.N.struct.sess_mem ..........53...........................n_sess.;.N.struct.sess .......29540...........................n_object.;.N.struct.object .......29561...........................n_objectcore.;.N.struct.objectcore ........5058...........................n_objecthead.;.N.struct.objecthead ........2613...........................n_waitinglist.;.N.struct.waitinglist ...........3...........................n_vbc.;.N.struct.vbc ..........22...........................n_wrk.;.N.worker.threads ........1789.........0.00.........0.00.n_wrk_create.;.N.worker.threads.created .......17204.........0.00.........0.05.n_wrk_queued.;.N.queued.work.requests ...........1...........................n_backend.;.N.backends .....2579403...........................n_expired.;.N.expired.objects .....6920374...........................n_lru_moved.;.N.LRU.moved.objects .....8959782........26.96........24.53.n_objwrite.;.Objects.sent.with.write .....1760818.........5.99.........4.82.s_sess.;.Total.Sessions ....11088687........25.96........30.36.s_req.;.Total.Requests ......436411.........2.00.........1.20.s_pass.;.Total.pass .....3044686........13.98.........8.34.s_fetch.;.Total.fetch ..3817818737......8996.58.....10454.48.s_hdrbytes.;.Total.header.bytes .84764679157....218712.68....232114.35.s_bodybytes.;.Total.body.bytes ......196150.........0.00.........0.54.sess_closed.;.Session.Closed .......73512.........0.00.........0.20.sess_pipeline.;.Session.Pipeline .......50339.........0.00.........0.14.sess_readahead.;.Session.Read.Ahead ....10875541........25.96........29.78.sess_linger.;.Session.Linger .....7865547........25.96........21.54.sess_herd.;.Session.herd ...591440571......1649.54......1619.56.shm_records.;.SHM.records ....30844446.......121.82........84.46.shm_writes.;.SHM.writes ...........6.........0.00.........0.00.shm_flushes.;.SHM.flushes.due.to.overflow
60. $ varnishstat -1 -f n_lru_nuked n_lru_nuked 0 . N LRU nuked objects Varnishstat Removed#from# cache#due#to#lack#of# memory
61. Varnishlog In@memory#logs#of# Varnish#activity
62. ...11.SessionOpen..c.12.12.12.1.53727.:80 ...11.ReqStart.....c.12.12.12.1.53727.1401010767 ...11.RxRequest....c.GET ...11.RxURL........c./ ...11.RxProtocol...c.HTTP/1.1 ...11.RxHeader.....c.Host:.12.12.12.6 ...11.RxHeader.....c.User;Agent:.Mozilla/5.0.(Macintosh;.Intel.Mac.OS.X.10.8;.rv:17.0). Gecko/20100101.Firefox/17.0 ...11.RxHeader.....c.Accept:.text/html,application/xhtml+xml,application/xml;q=0.9,*/ *;q=0.8 ...11.RxHeader.....c.Accept;Language:.nl,en;q=0.7,fr;be;q=0.3 ...11.RxHeader.....c.Accept;Encoding:.gzip,.deflate ...11.RxHeader.....c.Connection:.keep;alive ...11.VCL_call.....c.recv.lookup ...11.VCL_call.....c.hash ...11.Hash.........c./ ...11.Hash.........c.12.12.12.6 ...11.VCL_return...c.hash ...11.VCL_call.....c.miss.fetch ...11.Backend......c.13.default.default ...11.TTL..........c.1401010767.RFC.0.;1.;1.1357920021.0.1357920020.0.0 ...11.VCL_call.....c.fetch ...11.TTL..........c.1401010767.VCL.120.;1.;1.1357920021.;0 ...11.VCL_return...c.hit_for_pass ...11.ObjProtocol..c.HTTP/1.1 ...11.ObjResponse..c.OK ...11.ObjHeader....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...11.ObjHeader....c.Server:.Apache ...11.ObjHeader....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...11.ObjHeader....c.Cache;Control:.no;cache,.no;store,.max;age=0 Client#
63. ...11.VCL_return...c.hit_for_pass ...11.ObjProtocol..c.HTTP/1.1 ...11.ObjResponse..c.OK ...11.ObjHeader....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...11.ObjHeader....c.Server:.Apache ...11.ObjHeader....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...11.ObjHeader....c.Cache;Control:.no;cache,.no;store,.max;age=0 ...11.ObjHeader....c.Vary:.Accept;Encoding ...11.ObjHeader....c.Content;Encoding:.gzip ...11.ObjHeader....c.Content;Length:.119 ...11.ObjHeader....c.Content;Type:.text/html ...11.Gzip.........c.u.F.;.119.336.80.80.887 ...11.VCL_call.....c.deliver.deliver ...11.TxProtocol...c.HTTP/1.1 ...11.TxStatus.....c.200 ...11.TxResponse...c.OK ...11.TxHeader.....c.Server:.Apache ...11.TxHeader.....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...11.TxHeader.....c.Cache;Control:.no;cache,.no;store,.max;age=0 ...11.TxHeader.....c.Vary:.Accept;Encoding ...11.TxHeader.....c.Content;Encoding:.gzip ...11.TxHeader.....c.Content;Type:.text/html ...11.TxHeader.....c.Content;Length:.119 ...11.TxHeader.....c.Accept;Ranges:.bytes ...11.TxHeader.....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...11.TxHeader.....c.X;Varnish:.1401010767 ...11.TxHeader.....c.Age:.0 ...11.TxHeader.....c.Via:.1.1.varnish ...11.TxHeader.....c.Connection:.keep;alive ...11.Length.......c.119 ...11.ReqEnd.......c.1401010767.1357920020.712090731.1357920020.727306366.0.000087738. Client#
64. ...13.BackendClose.;.default ...13.BackendOpen..b.default.127.0.0.1.51597.127.0.0.1.8080 ...13.TxRequest....b.GET ...13.TxURL........b./ ...13.TxProtocol...b.HTTP/1.1 ...13.TxHeader.....b.Host:.12.12.12.6 ...13.TxHeader.....b.User;Agent:.Mozilla/5.0.(Macintosh;.Intel.Mac.OS.X.10.8;.rv:17.0). Gecko/20100101.Firefox/17.0 ...13.TxHeader.....b.Accept:.text/html,application/xhtml+xml,application/xml;q=0.9,*/ *;q=0.8 ...13.TxHeader.....b.Accept;Language:.nl,en;q=0.7,fr;be;q=0.3 ...13.TxHeader.....b.X;Forwarded;For:.12.12.12.1 ...13.TxHeader.....b.X;Varnish:.1401010767 ...13.TxHeader.....b.Accept;Encoding:.gzip ...13.RxProtocol...b.HTTP/1.1 ...13.RxStatus.....b.200 ...13.RxResponse...b.OK ...13.RxHeader.....b.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...13.RxHeader.....b.Server:.Apache ...13.RxHeader.....b.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...13.RxHeader.....b.Cache;Control:.no;cache,.no;store,.max;age=0 ...13.RxHeader.....b.Vary:.Accept;Encoding ...13.RxHeader.....b.Content;Encoding:.gzip ...13.RxHeader.....b.Content;Length:.119 ...13.RxHeader.....b.Content;Type:.text/html ...13.Fetch_Body...b.4(length).cls.0.mklen.1 ...13.Length.......b.119 ...13.BackendReuse.b.default Backend#
65. 4a!When!writing!to!a!file,!append!to!it!rather!than!overwrite!it. 4b!Include!backend!communication!logs 4c!Include!client!communication!logs 4d!Process!old!logs!on!startup 4I'Perform!regex!filter!on!output 4i'Include!specific!tags!into!output 4m'Perform!regex!filter!on!tags 4r'Read!a!shared!memory!log!dump!from!file 4w!Dump!shared!memory!logs!to!file 4x!Exclude!specific!tags!from!output 4X!Perform!exclusion!regex!filter!on!output 4O!Don’t!group!log!entries.!Don’t!use!with!Bm Varnishlog
66. Backend BackendClose BackendOpen BackendReuse BackendXID CLI ClientAddr Debug Error ExpBan ExpKill ExpPick Hit HitPass HttpError HttpGarbage Length ObjHeader ObjLostHeader ObjProtocol ObjRequest ObjResponse ObjStatus ObjURL ReqEnd ReqStart RxHeader RxLostHeader RxProtocol RxRequest RxResponse RxStatus RxURL SessionClose SessionOpen StatAddr StatSess TTL TxHeader TxLostHeader TxProtocol TxRequest TxResponse TxStatus TxURL VCL_acl VCL_call VCL_return VCL_trace WorkThread Tags
67. varnishlog -c -m VCL_call:hit | grep -i 'RxURL' Varnishlog Get#the#URL#for#all#hits
68. varnishlog -b -i TxURL Varnishlog Track#all#“missed”# URL’s varnishlog -O -b -i TxURL Ungrouped
69. varnishlog -c -i RxHeader -I User-Agent Varnishlog Get#all#user#agents varnishlog -O -c -i RxHeader -I User-Agent Ungrouped
70. varnishlog -i RxHeader Varnishlog varnishlog -c -i RxHeader varnishlog -b -i RxHeader Get#all#request# headers
71. varnishlog -c -i RxURL -m VCL_call:hit Varnishlog Doesn’t#work @i#is#done#before#@m
72. Why#did#it#miss?
73. 11 VCL_return c hit_for_pass Why'did'it'miss? TTL#<=#0 13 RxHeader - Cache-Control: no-cache, no- store, max-age=0
74. 11 RxRequest c POST 11 RxURL c /exercises/3/post.php 11 RxProtocol c HTTP/1.1 11 RxHeader c Host: varnish.dev 11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:18.0) Gecko/20100101 Firefox/18.0 11 RxHeader c Accept: text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8 11 RxHeader c Accept-Language: nl,en;q=0.7,fr-be;q=0.3 11 RxHeader c Accept-Encoding: gzip, deflate 11 RxHeader c Referer: http://varnish.dev/exercises/3/post.php 11 RxHeader c Connection: keep-alive 11 RxHeader c Content-Type: application/x-www-form-urlencoded 11 RxHeader c Content-Length: 11 11 VCL_call c recv pass Why'did'it'miss? HTTP#POST
75. Was#it#stored#to# cache?
76. 11 TTL c 324425260 RFC 5 -1 -1 1358344107 0 1358344107 0 5 11 VCL_call c fetch deliver Was'it'stored'to'cache? Yes
77. 11 TTL c 324425261 VCL 120 -1 -1 1358344313 -0 11 VCL_return c hit_for_pass Was'it'stored'to'cache? No,#for# 120#seconds#in# blacklist
78. Exercise 4 Using varnishlog
79. What’s#the#cache#key?
80. 11 Hash c /exercises/1/public.php 11 Hash c varnish.dev What’s'the'cache'key? Host#+#URL
81. Varnishtop Incremental#in@ memory#Varnish#logs
82. list.length.109 .....8.12.VCL_return.....deliver .....4.19.RxRequest......GET .....4.19.RxProtocol.....HTTP/1.1 .....4.19.RxHeader.......Host:.varnish.dev .....4.19.RxHeader.......Connection:.keep;alive .....4.19.VCL_call.......recv .....4.19.VCL_return.....lookup .....4.19.VCL_call.......hash .....4.19.Hash...........varnish.dev .....4.19.VCL_return.....hash .....4.19.VCL_call.......hit .....4.19.VCL_call.......deliver .....4.19.TxProtocol.....HTTP/1.1 .....4.19.TxHeader.......Server:.Apache .....4.19.TxHeader.......Content;Encoding:.gzip .....4.19.TxHeader.......Accept;Ranges:.bytes .....4.19.TxHeader.......Via:.1.1.varnish .....4.19.TxHeader.......Connection:.keep;alive .....3.76.TxHeader.......Age:.26 .....3.12.RxURL........../exercises/5/vary.php .....3.12.RxHeader.......Accept:.text/html,application/xhtml+xml,application/ xml;q=0.9,*/*;q=0.8 .....3.12.Hash.........../exercises/5/vary.php .....3.12.TxStatus.......200 .....3.12.TxResponse.....OK .....3.12.TxHeader.......X;Powered;By:.PHP/5.3.2;1ubuntu4.18 .....3.12.TxHeader.......Vary:.User;Agent,Accept;Encoding .....3.12.TxHeader.......Cache;Control:.public,must;revalidate,s;maxage=.100 Unfiltered#list
83. 4b!Include!backend!communication!logs 4c!Include!client!communication!logs 4d!Process!old!logs!on!startup 4I'Perform!regex!filter!on!output 4i'Include!specific!tags!into!output 4m'Perform!regex!filter!on!tags 4r'Read!a!shared!memory!log!dump!from!file 4p'Specifies!!the!!number!!of!!seconds!!to!!measure!!over 4x!Exclude!specific!tags!from!output 4X!Perform!exclusion!regex!filter!on!output 4O!Don’t!group!log!entries.!Don’t!use!with!Bm 4f!Sort!and!group!only!on!the!first!field!of!each!log!entry Varnishtop
84. varnishtop -c -i RxHeader -I ^User-Agent Varnishtop Get#top#user#agents
85. varnishtop -b -i TxURL Varnishtop Get#top#missed#URL’s
86. varnishtop -c -i VCL_call Varnishtop Get#top#VCL#calls
87. varnishtop -c -i RxURL Varnishtop Most#popular#URL’s
88. Exercise 5 Using varnishtop
89. Cache variations
90. Vary: Accept-Language Cache'variaNons Keep# seperate#cache#objects# for#each#varation#of#the# header Vary: X-myHeader Vary: Accept-Encoding
91. •Don’t5vary5‘*’ •Don’t5vary5‘cookie’5or5IE5will5freak5out •Only5vary5request5headers •Custom5headers5only5work5if5browser5 sends5them •VCL5magic5(coming!up!later) Cache'variaNons
92. Exercise 6 Cache variations
93. Varnish# Configuration# Language
94. Varnish# Configuration# Language The'Varnish'ConfiguraNon' Language'allows'you'to'define' your'caching'policy.'You'write' VCL'code'which'Varnish'will' parse,'translate'to'C'code,' compile'and'link'to.
95. Out#of#the#box#behaviour Remember# the#flow!
96. sub vcl_recv { if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + “, ” + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } if (req.request != “GET” && req.request != “HEAD” && req.request != “PUT” && req.request != “POST” && req.request != “TRACE” && req.request != “OPTIONS” && req.request != “DELETE”) { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.request != “GET” && req.request != “HEAD”) { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (lookup); }
97. sub vcl_pipe { # Note that only the first request to the backend will have # X-Forwarded-For set. If you use X-Forwarded-For and want to # have it set for all requests, make sure to have: # set bereq.http.connection = "close"; # here. It is not set by default as it might break some broken web # applications, like IIS with NTLM authentication. return (pipe); } sub vcl_pass { return (pass); } sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } return (hash); }
98. sub vcl_hit { return (deliver); } sub vcl_miss { return (fetch); } sub vcl_fetch { if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { ! ! /* ! ! * Mark as "Hit-For-Pass" for the next 2 minutes ! ! */ ! ! set beresp.ttl = 120 s; ! ! return (hit_for_pass); } return (deliver); } sub vcl_deliver { return (deliver); }
99. sub vcl_error { set obj.http.Content-Type = "text/html; charset=utf-8"; set obj.http.Retry-After = "5"; synthetic {" <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html> <head> <title>"} + obj.status + " " + obj.response + {"</title> </head> <body> <h1>Error "} + obj.status + " " + obj.response + {"</h1> <p>"} + obj.response + {"</p> <h3>Guru Meditation:</h3> <p>XID: "} + req.xid + {"</p> <hr> <p>Varnish cache server</p> </body> </html> "}; return (deliver); } sub vcl_init { ! return (ok); } sub vcl_fini { ! return (ok); }
100. VCL' syntax
101. //This is comment #This is comment too /* More comment A lot more comment*/ Comment
102. if(req.url == "/") { ! error 200 "OK dude"; } elseif(req.url == "/notfound") { ! error 404 "Not found man"; } else { ! error 200 "Else"; } CondiNonal
103. if(req.url == "/") { ! set req.backend = otherBackend; } Set'variables
104. Common%objects ✓Req:5request5object ✓Bereq:5backend5request5object ✓Beresp:5backend5response5object ✓Obj:5cache5object ✓Resp:5output5object ✓Client:5client5information ✓Server:5server5information
105. Variable%availability
106. .host Host'name'or'IP'address'of'a'backend. .port Service'name'or'port'number'of'a'backend. .connect_timeout Backend'marked'“unhealthy”'when'a'connection'fails'after'x'time. .first_byte_timeout Backend'marked'“unhealthy”'when'the'first'byte'isn’t'received'after'x'time. .between_byte_timeout Backend'marked'“unhealthy”'when'the'byte'sequence'isn’t'coming'in'fast' enough. Backend'variables
107. .url Specify'a'URL'to'request'from'the'backend.'Defaults'to'"/". .request Specify'a'full'HTTP'request'using'multiple'strings.'.request'will'have'\r\n' automatically'inserted'after'every'string.'If'specified,'.request'will'take' precedence'over'.url. .window How'many'of'the'latest'polls'we'examine'to'determine'backend'health.' Defaults'to'8. .threshold How'many'of'the'polls'in'.window'must'have'succeeded'for'us'to'consider'the backend'healthy.'Defaults'to'3. Probe'variables'(1)
108. .initial How'many'of'the'probes'are'considered'good'when'Varnish'starts.'Defaults'to the'same'amount'as'the'threshold. .expected_response The'expected'backend'HTTP'response'code.'Defaults'to'200. .interval Defines'how'often'the'probe'should'check'the'backend.'Default'is'every'5' seconds. .timeout How'fast'each'probe'times'out.'Default'is'2'seconds. Probe'variables'(2)
109. client.ip The'client's'IP'address. client.identity Identification'of'the'client,'used'to'load'balance'in'the'client'director. server.hostname The'host'name'of'the'server. server.identity The'identity'of'the'server,'as'set'by'the'4i'parameter.'If'the'4i'parameter'is'not' passed'to'varnishd,'server.identity'will'be'set'to'the'name'of'the'instance,'as' specified'by'the'4n'parameter. server.ip The'IP'address'of'the'socket'on'which'the'client'connection'was'received. server.port The'port'number'of'the'socket'on'which'the'client'connection'was'received. Client'&'server'variables
110. req.request The'request'type'(e.g.'"GET",'"HEAD"). req.url The'requested'URL. req.proto The'HTTP'protocol'version'used'by'the'client. req.backend The'backend'to'use'to'service'the'request. req.backend.healthy Whether'the'backend'is'healthy'or'not.'Requires'an'active'probe'to'be'set' on'the'backend. req.http.header The'corresponding'HTTP'header. Request'variables'(1)
111. req.hash_always_miss Force'a'cache'miss'for'this'request.'If'set'to'true'Varnish'will'disregard'any' existing'objects'and'always'(re)fetch'from'the'backend. req.hash_ignore_busy Ignore'any'busy'object'during'cache'lookup.'You'would'want'to'do'this'if' you'have'two'server'looking'up'content'from'each'other'to'avoid'potential' deadlocks. req.can_gzip Does'the'client'accept'the'gzip'transfer'encoding. req.restarts A'count'of'how'many'times'this'request'has'been'restarted. Unique'ID'of'this'request. Request'variables'(2)
112. req.esi Boolean.'Set'to'false'to'disable'ESI'processing'regardless'of'any'value'in' beresp.do_esi.'Defaults'to'true.'This'variable'is'subject'to'change'in'future' versions,'you'should'avoid'using'it. req.esi_level A'count'of'how'many'levels'of'ESI'requests'we're'currently'at. req.grace Set'to'a'period'to'enable'grace. req.xid Unique'ID'of'this'request. req.hash The'hash'key'used'to'refer'to'an'object'in'the'cache.'Used'when'both' reading'from'and'writing'to'the'cache. Request'variables'(3)
113. bereq.request The'request'type'(e.g.'"GET",'"HEAD"). bereq.url The'requested'URL. bereq.proto The'HTTP'protocol'version'used'to'talk'to'the'server. bereq.http.header The'corresponding'HTTP'header. bereq.connect_timeout The'time'in'seconds'to'wait'for'a'backend'connection. bereq.first_byte_timeout The'time'in'seconds'to'wait'for'the'first'byte'from'the'backend.'Not' available'in'pipe'mode. bereq.between_bytes_timeout The'time'in'seconds'to'wait'between'each'received'byte'from'the'backend.' Not'available'in'pipe'mode. Backend'request'variables
114. beresp.do_stream Deliver'the'object'to'the'client'directly'without'fetching'the'whole'object' into'varnish.'If'this'request'is'pass'ed'it'will'not'be'stored'in'memory.'As'of' Varnish'Cache'3.0'the'object'will'marked'as'busy'as'it'is'delivered'so'only' client'can'access'the'object. beresp.do_esi Boolean.'ESI4process'the'object'after'fetching'it.'Defaults'to'false.'Set'it'to' true'to'parse'the'object'for'ESI'directives.'Will'only'be'honored'if'req.esi'is' true. beresp.do_gzip Boolean.'Gzip'the'object'before'storing'it.'Defaults'to'false. beresp.do_gunzip Boolean.'Unzip'the'object'before'storing'it'in'the'cache.'Defaults'to'false. beresp.proto The'HTTP'protocol'version'used'the'backend'replied'with. beresp.storage Set'to'force'Varnish'to'save'this'object'to'a'particular'storage'backend. Backend#response#variables#(1)
115. beresp.status The'HTTP'status'code'returned'by'the'server. beresp.response The'HTTP'status'message'returned'by'the'server. beresp.ttl The'object's'remaining'time'to'live,'in'seconds.'beresp.ttl'is'writable. beresp.grace Set'to'a'period'to'enable'grace. beresp.saintmode Set'to'a'period'to'enable'saint'mode. beresp.backend.name Name'of'the'backend'this'response'was'fetched'from. beresp.backend.ip IP'of'the'backend'this'response'was'fetched'from. beresp.backend.port Port'of'the'backend'this'response'was'fetched'from. Backend#response#variables#(2)
116. obj.proto The'HTTP'protocol'version'used'when'the'object'was'retrieved. obj.status The'HTTP'status'code'returned'by'the'server. obj.response The'HTTP'status'message'returned'by'the'server. obj.ttl The'object's'remaining'time'to'live,'in'seconds.'obj.ttl'is'writable. obj.lastuse The'approximate'time'elapsed'since'the'object'was'last'requests,'in' seconds.'This'variable'is'also'available'in'vcl_deliver. obj.hits The'approximate'number'of'times'the'object'has'been'delivered.'A'value'of' 0'indicates'a'cache'miss.'This'variable'is'also'available'in'vcl_deliver. obj.grace The'object's'grace'period'in'seconds.'obj.grace'is'writable. obj.http.header The'corresponding'HTTP'header. Cache#object#variables
117. resp.proto The'HTTP'protocol'version'to'use'for'the'response. resp.status The'HTTP'status'code'that'will'be'returned. resp.response The'HTTP'status'message'that'will'be'returned. resp.http.header The'corresponding'HTTP'header. Client#response#variables
118. Reminder!
119. Working# with# backends
120. backend default { .host = "1.2.3.4"; .port = "http"; } Backend'config backend default { .host = "localhost"; .port = "8080"; } Simple
121. backend default { .host = "1.2.3.4"; .port = "http"; .connect_timeout = 1s; .first_byte_timeout = 5s; .between_bytes_timeout = 2s; } Backend'config More
122. backend default { .host = "1.2.3.4"; .port = "http"; } backend other { .host = "1.2.3.5"; .port = "http"; } sub vcl_recv { if (req.http.host ~ "^(www.)?example.com$") { set req.backend = other; } } Select'backend
123. Probes
124. backend default { .host = "1.2.3.4"; .port = "http"; .probe = { .url = "/test.jpg"; .timeout = 0.3 s; .window = 8; .threshold = 3; .initial = 3; } } Probes Test#if# healthy
125. probe healthcheck { .url = "/status.cgi"; .interval = 60s; .timeout = 0.3 s; .window = 8; .threshold = 3; .initial = 3; .expected_response = 200; } backend www { .host = "www.example.com"; .port = "http"; .probe = healthcheck; } Probes
126. probe rawprobe { # NB: \r\n automatically inserted after each string! .request = "GET / HTTP/1.1" "Host: www.foo.bar" "Connection: close"; } Probes
127. Directors
128. A!director!is!a!logical!group!of! backend!servers!clustered! together!for!redundancy.!The! basic!role!of!the!director!is!to! let!Varnish!choose!a!backend! server!amongst!several!so!if!one! is!down!another!can!be!used.
129. Directors ✓Round4robin ✓Random ✓Client ✓Hash ✓Fallback ✓DNS
130. backend.one.{ ....host.=.“localhost”; ....port.=.“80”; } backend.two.{ ....host.=.“127.0.0.1”; ....port.=.“81”; } director.localhosts.round;robin.{ ....{..backend.=.one;.} ....{..backend.=.two;.} ....{..backend.=.{. .............host.=.“localhost”;. .............port.=.“82”;. ........}. ....} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Round8robin%director
131. backend.one.{ ....host.=.“localhost”; ....port.=.“80”; } backend.two.{ ....host.=.“127.0.0.1”; ....port.=.“81”; } director.localhosts.random.{ ....{..backend.=.one;..weight=4;} ....{..backend.=.two;..weight=6;} ....{..backend.=.{. .............host.=.“localhost”;. .............port.=.“82”;. ........}. ....} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Random%director
132. director.localhosts.client.{ ....{..backend.=.one;..weight=1;.} ....{..backend.=.two;..weight=1;.} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; ....//Load.balance.by.URL ....set.client.identity.=.req.url; ....//Load.balance.by.client.IP,.this.is.the.default. ....set.client.identity.=.client.ip; ....//Load.balance.by.user.agent ....set.client.identity.=.req.http.user;agent; } Client%director Uses# client.identity
133. director.localhosts.hash.{ ....{..backend.=.one;..weight=1;.} ....{..backend.=.two;..weight=1;.} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Hash%director It'will'use'the'value'of'req.hash,'just'as'the'normal' cache7lookup'methods This'is'useful'is'you'are'using'Varnish'to'load' balance'in'front'of'other'Varnish'caches'or'other' web'accelerators'as'objects'won't'be'duplicated' across'caches.
134. director.localhosts.fallback.{ ....{..backend.=.one;} ....{..backend.=.two;} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Fallback%director Picks#first# healthy# backend Order# matters
135. director.directorname.dns.{ .........list.=.{ .................host_header.=."www.example.com"; .................port.=."80"; .................connect_timeout.=.0.4s; ................"192.168.15.0"/24; ................"192.168.16.128"/25; ........} .........ttl.=.5m; .........suffix.=."internal.example.net"; } DNS%director Autogenerates'backends'in'IP'ranges
136. director.directorname.dns.{ .........list.=.{ .................host_header.=."www.example.com"; .................port.=."80"; .................connect_timeout.=.0.4s; ................"192.168.15.0"/24; ................"192.168.16.128"/25; ........} .........ttl.=.5m; .........suffix.=."internal.example.net"; } DNS%director Hostname'='host_header'+'suffix www.example.com.internal.example.net
137. director.directorname.dns.{ .........list.=.{ .................host_header.=."www.example.com"; .................port.=."80"; .................connect_timeout.=.0.4s; ................"192.168.15.0"/24; ................"192.168.16.128"/25; ........} .........ttl.=.5m; .........suffix.=."internal.example.net"; } DNS%director Resolves'hostname'&'caches'result Backend'selecNon'based'on'result'of'DNS'resolving
138. Exercise 7 Playing with backends, probes & directors
139. Cache invalidation
140. Don’t wait for cache objects to expire
141. There!are!only!two!hard! things!in!Computer!Science:! cache!invalidaKon,!naming! things,!and!offBbyBone!errors.
142. ✓Purge ✓Ban ✓Always'miss Cache%invalida?on
143. ✓Purge'VCL'method ✓Removes'specific'cache'item ✓Frees'up'memory ✓Evicts'variants ✓Next'client'triggers'the'refresh ✓No'backup'if'backend'is'down Purge
144. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$return$(lookup); $$$$$$$$} } sub$vcl_hit${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$purge; $$$$$$$$$$$$$$$$error$200$"Purged"; $$$$$$$$} } sub$vcl_miss${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$error$404$"Not$in$cache"; $$$$$$$$} } Purge HTTP#call# to#purge
145. ✓Implement'in'your'CMS ✓HTTP'200':'removed'from'cache ✓HTTP'404:'not'in'cache ✓Some'CMS'plugins'require'200'for'miss() Purge curl$IX$PURGE$http://varnish.dev/yourIpage
146. ✓Ban'VCL'method' ✓Ban'command'line'action ✓Ban.url'command'line'action ✓Pattern'(regex)'based ✓Adds'ban'to'ban4list ✓No'memory'freed'up'immediately ✓Evicts'variants ✓Extremely'fast'and'powerful Ban
147. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${ ban("req.http.host$==$"$+$req.http.host$+$"$ &&$req.url$==$"$+$req.url); error$200$"Banned"; $$$$$$$$} } Ban sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$ban("req.http.host$==$"$+$req.http.host$+$ "$&&$req.url$~$"$+$req.url); error$200$"Banned"; $$$$$$$$} } Ban#URL# pattern Ban#URL Name# doesn’t# matter
148. ✓Removes'old'bans'from'the'ban4list ✓Separate'thread ✓Do'ban.list'on'CLI'to'see'the'ban4list ✓ban_lurker_sleep'parameter ✓Has'no'access'to'“req”'object ✓Smart'bans'are'ban'lurker'friendly Ban%lurker
149. ✓Ban'lurker'friendly ✓Use'beresp.x4url'&'beresp.x4host'instead'of' req.url'&'req.http.host Smart%bans
150. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$ban("obj.http.xIhost$==$"$+$req.http.host$+$ "$&&$obj.http.xIurl$==$"$+$req.url); $$$$$$$$$$$$error$200$"Banned"; $$$$$$$$} } sub$vcl_fetch${ $$$$$$$$set$beresp.http.xIurl$=$req.url; $$$$$$$$set$beresp.http.xIhost$=$req.http.host; } sub$vcl_deliver${ $$$$$$$$unset$resp.http.xIurl; $$$$$$$$unset$resp.http.xIhost; } Smart%bans Ban#lurker# friendly
151. ✓Used'for'reloading'content,'not'evicting ✓Uses'set'req.hash_always_miss'='true; ✓Looks'up'object,'but'ignores'it ✓Doesn’t'work'well'with'variations ✓Because'return(pass);'doesn’t'store'fetch' result'in'cache ✓Because'purge();'will'fail'when'your' backend'is'down Always%miss%
152. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"REFRESH")${ $$$$$$$$$$$$$$$$set$req.request$=$"GET"; $$$$$$$$$$$$$$$$set$req.hash_always_miss$=$true; $$$$$$$$} } } Refresh/always%miss
153. sub$vcl_recv${ $$$$if$(req.request$==$"PURGE")${ $$$$$$return$(lookup); $$$$} $$$$if$(req.request$==$"BAN")${ $$$$$$ban("obj.http.xIurl$~$"$+$req.http.xIurl$+$ $$$$$$"$&&$obj.http.xIhost$~$"$+$req.http.xIhost); $$$$$$error$200$"Banned"; $$$$} $$$$if$(req.request$==$"REFRESH")${ $$$$$$set$req.request$=$"GET"; $$$$$$set$req.hash_always_miss$=$true; $$$$} } All8in8one%(1)
154. sub$vcl_hit${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$purge; $$$$$$$$$$$$$$$$error$200$"Purged"; $$$$$$$$} } sub$vcl_miss${ $$$$$$$$if$(req.request$==$"PURGE")${$ $$$$$$$$$$$$$$$$purge; $$$$$$$$$$$$$$$$error$404$"Not$in$cache"; $$$$$$$$} } sub$vcl_pass${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$error$502$"PURGE$on$a$passed$object"; $$$$$$$$} } All8in8one%(2)
155. sub$vcl_fetch${ $$$$$$$$set$beresp.http.xIurl$=$req.url; $$$$$$$$set$beresp.http.xIhost$=$req.http.host; } sub$vcl_deliver${ $$$$$$$$unset$resp.http.xIurl; $$$$$$$$unset$resp.http.xIhost; } All8in8one%(3)
156. varnishstat$I1$If$n_object Measure%with%varnishstat Number# of#objects#in# cache
157. Protect'your'purges
158. acl$purge${ $$"localhost"; $$"217.21.177.0"/24; $$"10.10.10.1"; $$"some.host.com"; } sub$vcl_recv${$$$$ $$if$(req.request$==$"PURGE")${ $$$$if$(!client.ip$~$purge)${ $$$$$$error$405$"Not$allowed."; $$$$}$$$$ $$$$ban("obj.http.xIurl$==$"$+$req.url$+"$&& $$$obj.http.xIhost$==$"$+$req.http.host); $$$$error$200$"Purged"; $$} } Protect#your#purges Use#an#ACL
159. Exercise 8 Invalidation
160. RELOAD !
161. ✓/etc/init.d/varnish'reload ✓/etc/init.d/varnish'restart ✓varnishadm'(vcl.load'&'vcl.use) ✓telnet'(vcl.load'&'vcl.use) Reloading%Varnish%VCL
162. ✓/etc/init.d/varnish'reload ✓New'VCL'is'loaded ✓Keeps'objects'in'memory Service%reload
163. ✓/etc/init.d/varnish'restart ✓New'VCL'is'loaded ✓All'objects'are'removed'from'memory Service%restart
164. ✓Call'varnishadm'on'CLI ✓Call'vcl.list'for'an'overview ✓call'vcl.load'bla'/path/vcl/default.vcl ✓On'compilation'failure,'errors'are'shown ✓Call'vcl.use'bla'to'confirm'new'VCL Varnishadm
165. ✓Open'telnet'connection ✓Call'vcl.list'for'an'overview ✓call'vcl.load'bla'/path/vcl/default.vcl ✓On'compilation'failure,'errors'are'shown ✓Call'vcl.use'bla'to'confirm'new'VCL Telnet See# next#slide
166. /usr/sbin/varnishd$IP$/var/run/ varnishd.pid$Ia$:80$If$/home/data/ default.vcl$IT$127.0.0.1:6082$Iu$ varnish$Ig$varnish$It$120$Ip$vcl_dir$/ home/data$Iw$5,500,300$Is$malloc,256m$I S$/etc/varnish/secret$In$lucid32 Telnet Running# process
167. $.telnet.127.0.0.1.6082 Telnet
168. $$telnet$127.0.0.1$6082 Trying$::1... Trying$127.0.0.1... Connected$to$localhost. Escape$character$is$'^]'. 107$59$$$$$$ ayfmdmirgrluoqtukkueawfbrpmmvtkw Authentication$required. Telnet%authen?ca?on Auth# challenge
169. Telnet%authen?ca?on $.cat./etc/varnish/secret. ab9cae99;7905;42e8;9f28; b405e684ce93 Secret# key
170. Telnet%authen?ca?on ✓Challenge ✓Newline'character'(\x0A) ✓Secret'key ✓Newline'character'(\x0A) ✓Challenge ✓Newline'character'(\x0A) ✓SHA256'string Compose# auth#string
171. Telnet%authen?ca?on <?php $challenge = $argv[1]; $secret = trim(fgets(STDIN)); $pack = $challenge . "\x0A" . $secret . "\x0A" . $challenge . "\x0A"; $key = hash('sha256', $pack); echo $key.PHP_EOL; Challenge Secret# key Result
172. Telnet%authen?ca?on $$cat$/etc/varnish/secret$|$php$ secret.php$ ayfmdmirgrluoqtukkueawfbrpmmvtkw$ 4fac6b3e568d8b307300d6fa7a1b92ce4e764ce 46cabed4783ffb0e937d86ff3
173. $$telnet$127.0.0.1$6082 Trying$::1... Trying$127.0.0.1... Connected$to$localhost. Escape$character$is$'^]'. 107$59$$$$$$ ayfmdmirgrluoqtukkueawfbrpmmvtkw Authentication$required. auth$ 4fac6b3e568d8b307300d6fa7a1b92ce4e764ce 46cabed4783ffb0e937d86ff3 200$204 Telnet%authen?ca?on
174. Exercise 9 Load VCL via Varnishadm & telnet
175. Custom# VCL
176. sub$vcl_recv${ unset$req.http.cookie; } sub$vcl_fetch${ unset$beresp.http.setIcookie; } Remove%all%cookies
177. sub$vcl_recv${ set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "__utm.=[^;]+(;$)?",$""); set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "utmctr=[^;]+(;$)?",$""); set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "utmcmd.=[^;]+(;$)?",$""); set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "utmccn.=[^;]+(;$)?",$""); if$(req.http.cookie$~$"^$*$")${ unset$req.http.cookie; } } Remove%GA%cookies
178. sub$vcl_recv${ $$$return(lookup); } sub$vcl_hash${ $$$hash_data(req.http.cookie); } Cache%cookies
179. sub$vcl_hash${ $$$$if(req.http.Cookie$~$"lang"){ $$$$$$$$hash_data(regsuball(req.http.Cookie,$"^. +;?$?lang=([aIzAIZ0I9]+)($|;|$;).*$","\1")); $$$$} } Cache%some%cookies
180. sub$vcl_recv${ $$$return(lookup); } sub$vcl_fetch${ $$$return(deliver); } Ignore%cookies
181. sub$vcl_recv${ $ if$(req.url$~$"^[^?]*\.(bmp|bz2|css|doc| eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]| pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff| xml|zip)(\?.*)?$")${ $$$$$$$$unset$req.http.Cookie; $$$$$$$$return$(lookup); $$$$} } Cache%sta?c%files
182. sub$vcl_fetch${ $ if$(req.url$~$"^[^?]*\.(bmp|bz2|css|doc| eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]| pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff| xml|zip)(\?.*)?$")${ $$$$$$$$unset$beresp.http.setIcookie; $$$$} } Cache%sta?c%files
183. sub$vcl_deliver${ $$$$set$resp.http.xIcacheIhits$=$obj.hits; $$$$if$(obj.hits$>$0)${ $$$$$$$$set$resp.http.xIcache$=$"hit"; $$$$}$else${ $$$$$$$$set$resp.http.xIcache$=$"miss"; $$$$} } Debug
184. sub$vcl_recv${ set$req.http.Host$=$regsub(req.http.Host,$":[0I9]+",$""); if$(req.url$~$"\#")${ set$req.url$=$regsub(req.url,$"\#.*$",$""); } if$(req.url$~$"\?$")${ set$req.url$=$regsub(req.url,$"\?$",$""); } } Sani?ze%URL
185. sub$vcl_recv${ if$(req.url$~$"(\?|&)(utm_source|utm_medium|utm_campaign| gclid|cx|ie|cof|siteurl)=")${ set$req.url$=$regsuball(req.url,$"&(utm_source| utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([AI z0I9_\I\.%25]+)",$""); $$$set$req.url$=$regsuball(req.url,$"\?(utm_source| utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([AIz0I9_ \I\.%25]+)",$"?"); } } Sani?ze%URL%(2)
186. Exercise 10 Custom VCL
187. Grace mode
188. ✓A'graced'object'is'an'object'that'has'expired,'but'is' still'kept'in'cache ✓Grace'mode'is'when'Varnish'uses'a'graced'object ✓Used'when'backend'is'slow'or'down ✓req.grace'defines'how'long'an'overdue'an'object'can' be'for'Varnish'to'still'consider'it'for'grace'mode. ✓beresp.grace'defines'how'long'past'the'beresp.ttl7 time'Varnish'will'keep'an'object
189. sub$vcl_recv{ $$$$if$(req.backend.healthy)${ $$$$$$$$set$req.grace$=$0s; $$$$}$else${ $$$$$$$$set$req.grace$=$20s; $$$$} } sub$vcl_fetch{ $$$$set$beresp.grace$=$20s; }
190. probe.healthcheck.{ ....url.=."/some/url"; ....interval.=.1s; ....timeout.=.1s; ....window.=.1; ....threshold.=.1; ....initial.=.1; ....expected_response.=.200; } backend.default.{ ...host.=."localhost"; ...port.=."8080"; ...probe.=.healthcheck; } sub.vcl_recv{ ....if.(req.backend.healthy).{ ........set.req.grace.=.0s; ....}.else.{ ........set.req.grace.=.20s; ....} } sub.vcl_fetch{ ....set.beresp.grace.=.20s; } With# backend# probe
191. Exercise 11 Grace mode
192. Edge#Side#Includes
193. <html> <body> <table> <tr> <td colspan="2" > <?php include('header.php') ?> </td> </tr> <tr> <td><?php include('menu.php') ?></td> <td><?php include('main.php') ?></td> </tr> <tr> <td colspan="2" > <?php include('footer.php') ?> </td> </tr> </table> </body> </html>
194. header.php menu.php main.php footer.php TTL#5s No#caching TTL#10s TTL#2s
195. <?php function esi($file){ if(isset($_SERVER['Surrogate-Capability']) && preg_match('/^(.+)="ESI/1.0$/',$_SERVER['Surrogate-Capability'])){ header("Surrogate-Control: content='ESI/1.0'"); $url = 'http://'. $_SERVER['HTTP_HOST'].substr($_SERVER['SCRIPT_NAME'], 0,1+strrpos($_SERVER['SCRIPT_NAME'],'/')).$file; echo "<esi:include src=\"$url\" />".PHP_EOL; } else { include($file); } } ESI#PHP#helper#funcaon#(1)
196. <?php function esi($file){ if(isset($_SERVER['Surrogate-Capability']) && preg_match('/^(.+)="ESI/1.0$/',$_SERVER['Surrogate-Capability'])){ header("Surrogate-Control: content='ESI/1.0'"); $url = 'http://'. $_SERVER['HTTP_HOST'].substr($_SERVER['SCRIPT_NAME'], 0,1+strrpos($_SERVER['SCRIPT_NAME'],'/')).$file; echo "<esi:include src=\"$url\" />".PHP_EOL; } else { include($file); } } ESI#PHP#helper#funcaon#(2)
197. <html> <body> <table> <tr> <td colspan="2" > <esi:include src="/header.php" /> </td> </tr> <tr> <td><esi:include src="/menu.php" /></td> <td><esi:include src="/main.php" /></td> </tr> <tr> <td colspan="2" > <esi:include src="/footer.php" /> </td> </tr> </table> </body>
198. ESI#VCL sub.vcl_recv.{ ....set.req.http.Surrogate;Capability="key=ESI/1.0"; } sub.vcl_fetch.{ . if(beresp.http.Surrogate;Control~"ESI/1.0").{ .........unset.beresp.http.Surrogate;Control; . ....set.beresp.do_esi=true; ....} }.
199. <?php require_once __DIR__.'/vendor/autoload.php'; use Symfony\Component\HttpKernel\HttpCache\Esi; use Symfony\Component\HttpFoundation\Request; use Symfony\Component\HttpFoundation\Response; $esi = new Esi; $request = Request::createFromGlobals(); $response = new Response(); Symfony#components#(1)
200. $content = '<h1>Exercise 12</h1>'; $content .= '<h2>Edge Side Includes</h2>'; $content .= date("Y-m-d H:i:s").'<br/>'; if($esi->hasSurrogateEsiCapability($request)){ $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=2', null,false).'<br />'; $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=0', null,false).'<br />'; } $content .= '<br /><a href="/">Back to index</a>'; Symfony#components#(2)
201. $response->setContent($content); $esi->addSurrogateControl($response); $response->setSharedMaxAge(10); $response->send(); Symfony#components#(3)
202. And# on#that# bombshell#...
204. h"ps://joind.in/8432 Please# give#me# feedback
205. Thanks