Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Varnish workshop - DPC13

Varnish workshop - DPC13

Slides for my Varnish workshop at the Dutch PHP Conference 2013

Ca901ddcea38854b9783781c91fc87c9?s=128

Thijs Feryn

June 06, 2013
Tweet

Transcript

  1. Varnish Workshop By Thijs Feryn

  2. Hi# my#name# is#Thijs

  3. I’m# an#evangelist#at

  4. I’m# a#board#member# at

  5. None
  6. None
  7. h"ps://joind.in/8432 Please# give#me# feedback

  8. Varnish ?

  9. Cache?

  10. Reverse proxy?

  11. Loadbalancer?

  12. Loadbalancer? HTTP accelerator

  13. In#front#of#the# webserver

  14. Primary'purpose Caching

  15. Reverse'caching' proxy

  16. Install & configure

  17. curl%http://repo.varnish4cache.org/debian/GPG4 key.txt%|%apt4key%add%4 apt4get%update echo%"deb%http://repo.varnish4cache.org/ debian/%squeeze%varnish43.0"%>>%/etc/apt/ sources.list apt4get%install%varnish

  18. Daemon'op)ons h,ps://www.varnish4cache.org/docs/trunk/reference/varnishd.html

  19. Daemon'op)ons DAEMON_OPTS="4a'127.0.0.1:8080'\ '''''''''''''4T'localhost:6082'\ '''''''''''''4f'/etc/varnish/default.vcl'\ '''''''''''''4s'malloc,3G'\ '''''''''''''4p'thread_pools=8 '''''''''''''4p'thread_pool_min=160'\ '''''''''''''4p'thread_pool_max=3000'\ '''''''''''''4p'thread_pool_add_delay=2'\ '''''''''''''4p'connect_)meout=200s'\

    '''''''''''''4p'first_byte_)meout=200s'\ '''''''''''''4p'between_bytes_)meout=50s'\ '''''''''''''4p'session_linger=100'\ '''''''''''''4p'lru_interval=20'\ '''''''''''''4p'listen_depth=8192'\ '''''''''''''4p'sess_workspace=262144'\ '''''''''''''4l'300M" In#“/etc/default/varnish”
  20. Daemon'op)ons

  21. varnishadm>%param.show%4l Shows#all#params Varnish parameters

  22. Backend

  23. Listen%8080 In#“/etc/apache2/ports.conf” Backend

  24. <VirtualHost%*:8080> In#vhost#config#when#Varnish#is# on#same#node#as#webserver Backend

  25. backend%default%{ %%%%%%.host%=%"127.0.0.1"; %%%%%%.port%=%"8080"; } In#“/etc/varnish/default.vcl” Backend

  26. Threading model

  27. Threading params

  28. None
  29. GET / HTTP/1.1 Host: localhost User-Agent: Mozilla/5.0 (Macintosh; U; Intel

    Mac OS X 10.5; fr; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16 Accept: text/html,application/xhtml+xml,application/ xml;q=0.9,*/*;q=0.8 Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Cache-Control: max-age=0 Request
  30. HTTP/1.1 200 OK Server: Apache/2.2.14 (Ubuntu) X-Powered-By: PHP/5.3.2-1ubuntu4.7 Cache-Control: public,

    max-age=86400 Last-Modified: Mon, 04 Apr 2011 04:13:41 +0000 Expires: Sun, 11 Mar 1984 12:00:00 GMT Vary: Cookie,Accept-Encoding ETag: "1301890421" Content-Type: text/html; charset=utf-8 Content-Length: 23562 Date: Mon, 04 Apr 2011 09:02:26 GMT X-Varnish: 1886109724 1886107902 Age: 17324 Via: 1.1 varnish Connection: keep-alive Response
  31. Cache@control Cache7Control'“max7age=3600,'s7 maxage=1000,'public,'must7revalidate”

  32. Max@Age S@maxage Public Private No@cache No@store Must@revalidate Proxy@revalidate TTL'for'browsers'in'seconds TTL'for'proxies'in'seconds

    Proxies'&'browsers'can'cache Only'browsers'can'cache Revalidate'before'dropping'from'cache Don’t'cache'at'all Browser'revalidate'before'serving'from cache' Proxy'revalidate'before'serving'from' cache'
  33. Age Age'“56” How#old#is# the#cached# object?

  34. Real#cache#duration Real'cache'duraNon'for'clients'='max7age'7'age Real'cache'duraNon'for'proxies'='s7maxage'7'age

  35. Expires Expires'"Wed,'1'Jan'2014'20:00:00'GMT"

  36. Expiration 1.Vcl&(beresp.-l) 2.Cache3control&s3maxage 3.Cache3control&max3age 4.Expires

  37. The'flow

  38. 1

  39. 2

  40. 3

  41. Advanced' flow

  42. 1

  43. 2

  44. 3

  45. Exercise 1 Testing HTTP cache headers

  46. Exercise 2 Overriding Time To Live in Varnish

  47. Out of the box

  48. None
  49. Varnish obeys caching headers

  50. Only get & HEAD No cookie & auth headers No

    set-cookie headers cache ttl > 0 No vary “*” When will varnish cache?
  51. About cookies

  52. About cookies HTTP cookie request header via browser HTTP set-cookie

    response header via webserver
  53. based on host/ip + urL hash By default in Memory

    Optionally on disk how will varnish cache?
  54. 200: OK 203: Non-Authoritative Information 300: Multiple Choices 301: Moved

    Permanently 302: Moved Temporarily 307: Temporary Redirect 410: Gone 404: Not Found Cacheable status codes
  55. Exercise 3 Testing default behaviour

  56. Monitoring & Logging

  57. Varnishstat Real#time#stats#for# Varnish#instance

  58. 4+05:26:25 Hitrate.ratio:.......10......100......254 Hitrate.avg:.....0.8486...0.7619...0.7285 .....1760818.........6.99.........4.82.client_conn.;.Client.connections.accepted ....11088687........25.96........30.36.client_req.;.Client.requests.received .....8042715........11.98........22.02.cache_hit.;.Cache.hits .....2609561........11.98.........7.15.cache_miss.;.Cache.misses .......47104.........1.00.........0.13.backend_conn.;.Backend.conn..success .........610.........0.00.........0.00.backend_fail.;.Backend.conn..failures .....2998265........12.98.........8.21.backend_reuse.;.Backend.conn..reuses

    .......12081.........0.00.........0.03.backend_toolate.;.Backend.conn..was.closed .....3010356........13.98.........8.24.backend_recycle.;.Backend.conn..recycles ..........13.........0.00.........0.00.backend_retry.;.Backend.conn..retry .........520.........0.00.........0.00.fetch_head.;.Fetch.head .....2857965........11.98.........7.83.fetch_length.;.Fetch.with.Length ......151309.........2.00.........0.41.fetch_chunked.;.Fetch.chunked ........4404.........0.00.........0.01.fetch_close.;.Fetch.wanted.close .........676.........0.00.........0.00.fetch_failed.;.Fetch.failed .......31164.........0.00.........0.09.fetch_304.;.Fetch.no.body.(304) .........220...........................n_sess_mem.;.N.struct.sess_mem ..........53...........................n_sess.;.N.struct.sess .......29540...........................n_object.;.N.struct.object .......29561...........................n_objectcore.;.N.struct.objectcore ........5058...........................n_objecthead.;.N.struct.objecthead ........2613...........................n_waitinglist.;.N.struct.waitinglist ...........3...........................n_vbc.;.N.struct.vbc ..........22...........................n_wrk.;.N.worker.threads ........1789.........0.00.........0.00.n_wrk_create.;.N.worker.threads.created
  59. .......31164.........0.00.........0.09.fetch_304.;.Fetch.no.body.(304) .........220...........................n_sess_mem.;.N.struct.sess_mem ..........53...........................n_sess.;.N.struct.sess .......29540...........................n_object.;.N.struct.object .......29561...........................n_objectcore.;.N.struct.objectcore ........5058...........................n_objecthead.;.N.struct.objecthead ........2613...........................n_waitinglist.;.N.struct.waitinglist ...........3...........................n_vbc.;.N.struct.vbc ..........22...........................n_wrk.;.N.worker.threads ........1789.........0.00.........0.00.n_wrk_create.;.N.worker.threads.created

    .......17204.........0.00.........0.05.n_wrk_queued.;.N.queued.work.requests ...........1...........................n_backend.;.N.backends .....2579403...........................n_expired.;.N.expired.objects .....6920374...........................n_lru_moved.;.N.LRU.moved.objects .....8959782........26.96........24.53.n_objwrite.;.Objects.sent.with.write .....1760818.........5.99.........4.82.s_sess.;.Total.Sessions ....11088687........25.96........30.36.s_req.;.Total.Requests ......436411.........2.00.........1.20.s_pass.;.Total.pass .....3044686........13.98.........8.34.s_fetch.;.Total.fetch ..3817818737......8996.58.....10454.48.s_hdrbytes.;.Total.header.bytes .84764679157....218712.68....232114.35.s_bodybytes.;.Total.body.bytes ......196150.........0.00.........0.54.sess_closed.;.Session.Closed .......73512.........0.00.........0.20.sess_pipeline.;.Session.Pipeline .......50339.........0.00.........0.14.sess_readahead.;.Session.Read.Ahead ....10875541........25.96........29.78.sess_linger.;.Session.Linger .....7865547........25.96........21.54.sess_herd.;.Session.herd ...591440571......1649.54......1619.56.shm_records.;.SHM.records ....30844446.......121.82........84.46.shm_writes.;.SHM.writes ...........6.........0.00.........0.00.shm_flushes.;.SHM.flushes.due.to.overflow
  60. $ varnishstat -1 -f n_lru_nuked n_lru_nuked 0 . N LRU

    nuked objects Varnishstat Removed#from# cache#due#to#lack#of# memory
  61. Varnishlog In@memory#logs#of# Varnish#activity

  62. ...11.SessionOpen..c.12.12.12.1.53727.:80 ...11.ReqStart.....c.12.12.12.1.53727.1401010767 ...11.RxRequest....c.GET ...11.RxURL........c./ ...11.RxProtocol...c.HTTP/1.1 ...11.RxHeader.....c.Host:.12.12.12.6 ...11.RxHeader.....c.User;Agent:.Mozilla/5.0.(Macintosh;.Intel.Mac.OS.X.10.8;.rv:17.0). Gecko/20100101.Firefox/17.0 ...11.RxHeader.....c.Accept:.text/html,application/xhtml+xml,application/xml;q=0.9,*/ *;q=0.8

    ...11.RxHeader.....c.Accept;Language:.nl,en;q=0.7,fr;be;q=0.3 ...11.RxHeader.....c.Accept;Encoding:.gzip,.deflate ...11.RxHeader.....c.Connection:.keep;alive ...11.VCL_call.....c.recv.lookup ...11.VCL_call.....c.hash ...11.Hash.........c./ ...11.Hash.........c.12.12.12.6 ...11.VCL_return...c.hash ...11.VCL_call.....c.miss.fetch ...11.Backend......c.13.default.default ...11.TTL..........c.1401010767.RFC.0.;1.;1.1357920021.0.1357920020.0.0 ...11.VCL_call.....c.fetch ...11.TTL..........c.1401010767.VCL.120.;1.;1.1357920021.;0 ...11.VCL_return...c.hit_for_pass ...11.ObjProtocol..c.HTTP/1.1 ...11.ObjResponse..c.OK ...11.ObjHeader....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...11.ObjHeader....c.Server:.Apache ...11.ObjHeader....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...11.ObjHeader....c.Cache;Control:.no;cache,.no;store,.max;age=0 Client#
  63. ...11.VCL_return...c.hit_for_pass ...11.ObjProtocol..c.HTTP/1.1 ...11.ObjResponse..c.OK ...11.ObjHeader....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...11.ObjHeader....c.Server:.Apache ...11.ObjHeader....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...11.ObjHeader....c.Cache;Control:.no;cache,.no;store,.max;age=0 ...11.ObjHeader....c.Vary:.Accept;Encoding ...11.ObjHeader....c.Content;Encoding:.gzip ...11.ObjHeader....c.Content;Length:.119

    ...11.ObjHeader....c.Content;Type:.text/html ...11.Gzip.........c.u.F.;.119.336.80.80.887 ...11.VCL_call.....c.deliver.deliver ...11.TxProtocol...c.HTTP/1.1 ...11.TxStatus.....c.200 ...11.TxResponse...c.OK ...11.TxHeader.....c.Server:.Apache ...11.TxHeader.....c.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...11.TxHeader.....c.Cache;Control:.no;cache,.no;store,.max;age=0 ...11.TxHeader.....c.Vary:.Accept;Encoding ...11.TxHeader.....c.Content;Encoding:.gzip ...11.TxHeader.....c.Content;Type:.text/html ...11.TxHeader.....c.Content;Length:.119 ...11.TxHeader.....c.Accept;Ranges:.bytes ...11.TxHeader.....c.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...11.TxHeader.....c.X;Varnish:.1401010767 ...11.TxHeader.....c.Age:.0 ...11.TxHeader.....c.Via:.1.1.varnish ...11.TxHeader.....c.Connection:.keep;alive ...11.Length.......c.119 ...11.ReqEnd.......c.1401010767.1357920020.712090731.1357920020.727306366.0.000087738. Client#
  64. ...13.BackendClose.;.default ...13.BackendOpen..b.default.127.0.0.1.51597.127.0.0.1.8080 ...13.TxRequest....b.GET ...13.TxURL........b./ ...13.TxProtocol...b.HTTP/1.1 ...13.TxHeader.....b.Host:.12.12.12.6 ...13.TxHeader.....b.User;Agent:.Mozilla/5.0.(Macintosh;.Intel.Mac.OS.X.10.8;.rv:17.0). Gecko/20100101.Firefox/17.0 ...13.TxHeader.....b.Accept:.text/html,application/xhtml+xml,application/xml;q=0.9,*/ *;q=0.8

    ...13.TxHeader.....b.Accept;Language:.nl,en;q=0.7,fr;be;q=0.3 ...13.TxHeader.....b.X;Forwarded;For:.12.12.12.1 ...13.TxHeader.....b.X;Varnish:.1401010767 ...13.TxHeader.....b.Accept;Encoding:.gzip ...13.RxProtocol...b.HTTP/1.1 ...13.RxStatus.....b.200 ...13.RxResponse...b.OK ...13.RxHeader.....b.Date:.Fri,.11.Jan.2013.16:00:20.GMT ...13.RxHeader.....b.Server:.Apache ...13.RxHeader.....b.X;Powered;By:.PHP/5.3.2;1ubuntu4.18 ...13.RxHeader.....b.Cache;Control:.no;cache,.no;store,.max;age=0 ...13.RxHeader.....b.Vary:.Accept;Encoding ...13.RxHeader.....b.Content;Encoding:.gzip ...13.RxHeader.....b.Content;Length:.119 ...13.RxHeader.....b.Content;Type:.text/html ...13.Fetch_Body...b.4(length).cls.0.mklen.1 ...13.Length.......b.119 ...13.BackendReuse.b.default Backend#
  65. 4a!When!writing!to!a!file,!append!to!it!rather!than!overwrite!it. 4b!Include!backend!communication!logs 4c!Include!client!communication!logs 4d!Process!old!logs!on!startup 4I'Perform!regex!filter!on!output 4i'Include!specific!tags!into!output 4m'Perform!regex!filter!on!tags 4r'Read!a!shared!memory!log!dump!from!file 4w!Dump!shared!memory!logs!to!file 4x!Exclude!specific!tags!from!output

    4X!Perform!exclusion!regex!filter!on!output 4O!Don’t!group!log!entries.!Don’t!use!with!Bm Varnishlog
  66. Backend BackendClose BackendOpen BackendReuse BackendXID CLI ClientAddr Debug Error ExpBan

    ExpKill ExpPick Hit HitPass HttpError HttpGarbage Length ObjHeader ObjLostHeader ObjProtocol ObjRequest ObjResponse ObjStatus ObjURL ReqEnd ReqStart RxHeader RxLostHeader RxProtocol RxRequest RxResponse RxStatus RxURL SessionClose SessionOpen StatAddr StatSess TTL TxHeader TxLostHeader TxProtocol TxRequest TxResponse TxStatus TxURL VCL_acl VCL_call VCL_return VCL_trace WorkThread Tags
  67. varnishlog -c -m VCL_call:hit | grep -i 'RxURL' Varnishlog Get#the#URL#for#all#hits

  68. varnishlog -b -i TxURL Varnishlog Track#all#“missed”# URL’s varnishlog -O -b

    -i TxURL Ungrouped
  69. varnishlog -c -i RxHeader -I User-Agent Varnishlog Get#all#user#agents varnishlog -O

    -c -i RxHeader -I User-Agent Ungrouped
  70. varnishlog -i RxHeader Varnishlog varnishlog -c -i RxHeader varnishlog -b

    -i RxHeader Get#all#request# headers
  71. varnishlog -c -i RxURL -m VCL_call:hit Varnishlog Doesn’t#work @i#is#done#before#@m

  72. Why#did#it#miss?

  73. 11 VCL_return c hit_for_pass Why'did'it'miss? TTL#<=#0 13 RxHeader - Cache-Control:

    no-cache, no- store, max-age=0
  74. 11 RxRequest c POST 11 RxURL c /exercises/3/post.php 11 RxProtocol

    c HTTP/1.1 11 RxHeader c Host: varnish.dev 11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:18.0) Gecko/20100101 Firefox/18.0 11 RxHeader c Accept: text/html,application/xhtml +xml,application/xml;q=0.9,*/*;q=0.8 11 RxHeader c Accept-Language: nl,en;q=0.7,fr-be;q=0.3 11 RxHeader c Accept-Encoding: gzip, deflate 11 RxHeader c Referer: http://varnish.dev/exercises/3/post.php 11 RxHeader c Connection: keep-alive 11 RxHeader c Content-Type: application/x-www-form-urlencoded 11 RxHeader c Content-Length: 11 11 VCL_call c recv pass Why'did'it'miss? HTTP#POST
  75. Was#it#stored#to# cache?

  76. 11 TTL c 324425260 RFC 5 -1 -1 1358344107 0

    1358344107 0 5 11 VCL_call c fetch deliver Was'it'stored'to'cache? Yes
  77. 11 TTL c 324425261 VCL 120 -1 -1 1358344313 -0

    11 VCL_return c hit_for_pass Was'it'stored'to'cache? No,#for# 120#seconds#in# blacklist
  78. Exercise 4 Using varnishlog

  79. What’s#the#cache#key?

  80. 11 Hash c /exercises/1/public.php 11 Hash c varnish.dev What’s'the'cache'key? Host#+#URL

  81. Varnishtop Incremental#in@ memory#Varnish#logs

  82. list.length.109 .....8.12.VCL_return.....deliver .....4.19.RxRequest......GET .....4.19.RxProtocol.....HTTP/1.1 .....4.19.RxHeader.......Host:.varnish.dev .....4.19.RxHeader.......Connection:.keep;alive .....4.19.VCL_call.......recv .....4.19.VCL_return.....lookup .....4.19.VCL_call.......hash .....4.19.Hash...........varnish.dev

    .....4.19.VCL_return.....hash .....4.19.VCL_call.......hit .....4.19.VCL_call.......deliver .....4.19.TxProtocol.....HTTP/1.1 .....4.19.TxHeader.......Server:.Apache .....4.19.TxHeader.......Content;Encoding:.gzip .....4.19.TxHeader.......Accept;Ranges:.bytes .....4.19.TxHeader.......Via:.1.1.varnish .....4.19.TxHeader.......Connection:.keep;alive .....3.76.TxHeader.......Age:.26 .....3.12.RxURL........../exercises/5/vary.php .....3.12.RxHeader.......Accept:.text/html,application/xhtml+xml,application/ xml;q=0.9,*/*;q=0.8 .....3.12.Hash.........../exercises/5/vary.php .....3.12.TxStatus.......200 .....3.12.TxResponse.....OK .....3.12.TxHeader.......X;Powered;By:.PHP/5.3.2;1ubuntu4.18 .....3.12.TxHeader.......Vary:.User;Agent,Accept;Encoding .....3.12.TxHeader.......Cache;Control:.public,must;revalidate,s;maxage=.100 Unfiltered#list
  83. 4b!Include!backend!communication!logs 4c!Include!client!communication!logs 4d!Process!old!logs!on!startup 4I'Perform!regex!filter!on!output 4i'Include!specific!tags!into!output 4m'Perform!regex!filter!on!tags 4r'Read!a!shared!memory!log!dump!from!file 4p'Specifies!!the!!number!!of!!seconds!!to!!measure!!over 4x!Exclude!specific!tags!from!output 4X!Perform!exclusion!regex!filter!on!output

    4O!Don’t!group!log!entries.!Don’t!use!with!Bm 4f!Sort!and!group!only!on!the!first!field!of!each!log!entry Varnishtop
  84. varnishtop -c -i RxHeader -I ^User-Agent Varnishtop Get#top#user#agents

  85. varnishtop -b -i TxURL Varnishtop Get#top#missed#URL’s

  86. varnishtop -c -i VCL_call Varnishtop Get#top#VCL#calls

  87. varnishtop -c -i RxURL Varnishtop Most#popular#URL’s

  88. Exercise 5 Using varnishtop

  89. Cache variations

  90. Vary: Accept-Language Cache'variaNons Keep# seperate#cache#objects# for#each#varation#of#the# header Vary: X-myHeader Vary:

    Accept-Encoding
  91. •Don’t5vary5‘*’ •Don’t5vary5‘cookie’5or5IE5will5freak5out •Only5vary5request5headers •Custom5headers5only5work5if5browser5 sends5them •VCL5magic5(coming!up!later) Cache'variaNons

  92. Exercise 6 Cache variations

  93. Varnish# Configuration# Language

  94. Varnish# Configuration# Language The'Varnish'ConfiguraNon' Language'allows'you'to'define' your'caching'policy.'You'write' VCL'code'which'Varnish'will' parse,'translate'to'C'code,' compile'and'link'to.

  95. Out#of#the#box#behaviour Remember# the#flow!

  96. sub vcl_recv { if (req.restarts == 0) { if (req.http.x-forwarded-for)

    { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + “, ” + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } if (req.request != “GET” && req.request != “HEAD” && req.request != “PUT” && req.request != “POST” && req.request != “TRACE” && req.request != “OPTIONS” && req.request != “DELETE”) { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.request != “GET” && req.request != “HEAD”) { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (lookup); }
  97. sub vcl_pipe { # Note that only the first request

    to the backend will have # X-Forwarded-For set. If you use X-Forwarded-For and want to # have it set for all requests, make sure to have: # set bereq.http.connection = "close"; # here. It is not set by default as it might break some broken web # applications, like IIS with NTLM authentication. return (pipe); } sub vcl_pass { return (pass); } sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } return (hash); }
  98. sub vcl_hit { return (deliver); } sub vcl_miss { return

    (fetch); } sub vcl_fetch { if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") { ! ! /* ! ! * Mark as "Hit-For-Pass" for the next 2 minutes ! ! */ ! ! set beresp.ttl = 120 s; ! ! return (hit_for_pass); } return (deliver); } sub vcl_deliver { return (deliver); }
  99. sub vcl_error { set obj.http.Content-Type = "text/html; charset=utf-8"; set obj.http.Retry-After

    = "5"; synthetic {" <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html> <head> <title>"} + obj.status + " " + obj.response + {"</title> </head> <body> <h1>Error "} + obj.status + " " + obj.response + {"</h1> <p>"} + obj.response + {"</p> <h3>Guru Meditation:</h3> <p>XID: "} + req.xid + {"</p> <hr> <p>Varnish cache server</p> </body> </html> "}; return (deliver); } sub vcl_init { ! return (ok); } sub vcl_fini { ! return (ok); }
  100. VCL' syntax

  101. //This is comment #This is comment too /* More comment

    A lot more comment*/ Comment
  102. if(req.url == "/") { ! error 200 "OK dude"; }

    elseif(req.url == "/notfound") { ! error 404 "Not found man"; } else { ! error 200 "Else"; } CondiNonal
  103. if(req.url == "/") { ! set req.backend = otherBackend; }

    Set'variables
  104. Common%objects ✓Req:5request5object ✓Bereq:5backend5request5object ✓Beresp:5backend5response5object ✓Obj:5cache5object ✓Resp:5output5object ✓Client:5client5information ✓Server:5server5information

  105. Variable%availability

  106. .host Host'name'or'IP'address'of'a'backend. .port Service'name'or'port'number'of'a'backend. .connect_timeout Backend'marked'“unhealthy”'when'a'connection'fails'after'x'time. .first_byte_timeout Backend'marked'“unhealthy”'when'the'first'byte'isn’t'received'after'x'time. .between_byte_timeout Backend'marked'“unhealthy”'when'the'byte'sequence'isn’t'coming'in'fast'

    enough. Backend'variables
  107. .url Specify'a'URL'to'request'from'the'backend.'Defaults'to'"/". .request Specify'a'full'HTTP'request'using'multiple'strings.'.request'will'have'\r\n' automatically'inserted'after'every'string.'If'specified,'.request'will'take' precedence'over'.url. .window How'many'of'the'latest'polls'we'examine'to'determine'backend'health.' Defaults'to'8. .threshold

    How'many'of'the'polls'in'.window'must'have'succeeded'for'us'to'consider'the backend'healthy.'Defaults'to'3. Probe'variables'(1)
  108. .initial How'many'of'the'probes'are'considered'good'when'Varnish'starts.'Defaults'to the'same'amount'as'the'threshold. .expected_response The'expected'backend'HTTP'response'code.'Defaults'to'200. .interval Defines'how'often'the'probe'should'check'the'backend.'Default'is'every'5' seconds. .timeout How'fast'each'probe'times'out.'Default'is'2'seconds.

    Probe'variables'(2)
  109. client.ip The'client's'IP'address. client.identity Identification'of'the'client,'used'to'load'balance'in'the'client'director. server.hostname The'host'name'of'the'server. server.identity The'identity'of'the'server,'as'set'by'the'4i'parameter.'If'the'4i'parameter'is'not' passed'to'varnishd,'server.identity'will'be'set'to'the'name'of'the'instance,'as' specified'by'the'4n'parameter.

    server.ip The'IP'address'of'the'socket'on'which'the'client'connection'was'received. server.port The'port'number'of'the'socket'on'which'the'client'connection'was'received. Client'&'server'variables
  110. req.request The'request'type'(e.g.'"GET",'"HEAD"). req.url The'requested'URL. req.proto The'HTTP'protocol'version'used'by'the'client. req.backend The'backend'to'use'to'service'the'request. req.backend.healthy Whether'the'backend'is'healthy'or'not.'Requires'an'active'probe'to'be'set'

    on'the'backend. req.http.header The'corresponding'HTTP'header. Request'variables'(1)
  111. req.hash_always_miss Force'a'cache'miss'for'this'request.'If'set'to'true'Varnish'will'disregard'any' existing'objects'and'always'(re)fetch'from'the'backend. req.hash_ignore_busy Ignore'any'busy'object'during'cache'lookup.'You'would'want'to'do'this'if' you'have'two'server'looking'up'content'from'each'other'to'avoid'potential' deadlocks. req.can_gzip Does'the'client'accept'the'gzip'transfer'encoding. req.restarts

    A'count'of'how'many'times'this'request'has'been'restarted. Unique'ID'of'this'request. Request'variables'(2)
  112. req.esi Boolean.'Set'to'false'to'disable'ESI'processing'regardless'of'any'value'in' beresp.do_esi.'Defaults'to'true.'This'variable'is'subject'to'change'in'future' versions,'you'should'avoid'using'it. req.esi_level A'count'of'how'many'levels'of'ESI'requests'we're'currently'at. req.grace Set'to'a'period'to'enable'grace. req.xid Unique'ID'of'this'request.

    req.hash The'hash'key'used'to'refer'to'an'object'in'the'cache.'Used'when'both' reading'from'and'writing'to'the'cache. Request'variables'(3)
  113. bereq.request The'request'type'(e.g.'"GET",'"HEAD"). bereq.url The'requested'URL. bereq.proto The'HTTP'protocol'version'used'to'talk'to'the'server. bereq.http.header The'corresponding'HTTP'header. bereq.connect_timeout The'time'in'seconds'to'wait'for'a'backend'connection.

    bereq.first_byte_timeout The'time'in'seconds'to'wait'for'the'first'byte'from'the'backend.'Not' available'in'pipe'mode. bereq.between_bytes_timeout The'time'in'seconds'to'wait'between'each'received'byte'from'the'backend.' Not'available'in'pipe'mode. Backend'request'variables
  114. beresp.do_stream Deliver'the'object'to'the'client'directly'without'fetching'the'whole'object' into'varnish.'If'this'request'is'pass'ed'it'will'not'be'stored'in'memory.'As'of' Varnish'Cache'3.0'the'object'will'marked'as'busy'as'it'is'delivered'so'only' client'can'access'the'object. beresp.do_esi Boolean.'ESI4process'the'object'after'fetching'it.'Defaults'to'false.'Set'it'to' true'to'parse'the'object'for'ESI'directives.'Will'only'be'honored'if'req.esi'is' true. beresp.do_gzip

    Boolean.'Gzip'the'object'before'storing'it.'Defaults'to'false. beresp.do_gunzip Boolean.'Unzip'the'object'before'storing'it'in'the'cache.'Defaults'to'false. beresp.proto The'HTTP'protocol'version'used'the'backend'replied'with. beresp.storage Set'to'force'Varnish'to'save'this'object'to'a'particular'storage'backend. Backend#response#variables#(1)
  115. beresp.status The'HTTP'status'code'returned'by'the'server. beresp.response The'HTTP'status'message'returned'by'the'server. beresp.ttl The'object's'remaining'time'to'live,'in'seconds.'beresp.ttl'is'writable. beresp.grace Set'to'a'period'to'enable'grace. beresp.saintmode Set'to'a'period'to'enable'saint'mode.

    beresp.backend.name Name'of'the'backend'this'response'was'fetched'from. beresp.backend.ip IP'of'the'backend'this'response'was'fetched'from. beresp.backend.port Port'of'the'backend'this'response'was'fetched'from. Backend#response#variables#(2)
  116. obj.proto The'HTTP'protocol'version'used'when'the'object'was'retrieved. obj.status The'HTTP'status'code'returned'by'the'server. obj.response The'HTTP'status'message'returned'by'the'server. obj.ttl The'object's'remaining'time'to'live,'in'seconds.'obj.ttl'is'writable. obj.lastuse The'approximate'time'elapsed'since'the'object'was'last'requests,'in'

    seconds.'This'variable'is'also'available'in'vcl_deliver. obj.hits The'approximate'number'of'times'the'object'has'been'delivered.'A'value'of' 0'indicates'a'cache'miss.'This'variable'is'also'available'in'vcl_deliver. obj.grace The'object's'grace'period'in'seconds.'obj.grace'is'writable. obj.http.header The'corresponding'HTTP'header. Cache#object#variables
  117. resp.proto The'HTTP'protocol'version'to'use'for'the'response. resp.status The'HTTP'status'code'that'will'be'returned. resp.response The'HTTP'status'message'that'will'be'returned. resp.http.header The'corresponding'HTTP'header. Client#response#variables

  118. Reminder!

  119. Working# with# backends

  120. backend default { .host = "1.2.3.4"; .port = "http"; }

    Backend'config backend default { .host = "localhost"; .port = "8080"; } Simple
  121. backend default { .host = "1.2.3.4"; .port = "http"; .connect_timeout

    = 1s; .first_byte_timeout = 5s; .between_bytes_timeout = 2s; } Backend'config More
  122. backend default { .host = "1.2.3.4"; .port = "http"; }

    backend other { .host = "1.2.3.5"; .port = "http"; } sub vcl_recv { if (req.http.host ~ "^(www.)?example.com$") { set req.backend = other; } } Select'backend
  123. Probes

  124. backend default { .host = "1.2.3.4"; .port = "http"; .probe

    = { .url = "/test.jpg"; .timeout = 0.3 s; .window = 8; .threshold = 3; .initial = 3; } } Probes Test#if# healthy
  125. probe healthcheck { .url = "/status.cgi"; .interval = 60s; .timeout

    = 0.3 s; .window = 8; .threshold = 3; .initial = 3; .expected_response = 200; } backend www { .host = "www.example.com"; .port = "http"; .probe = healthcheck; } Probes
  126. probe rawprobe { # NB: \r\n automatically inserted after each

    string! .request = "GET / HTTP/1.1" "Host: www.foo.bar" "Connection: close"; } Probes
  127. Directors

  128. A!director!is!a!logical!group!of! backend!servers!clustered! together!for!redundancy.!The! basic!role!of!the!director!is!to! let!Varnish!choose!a!backend! server!amongst!several!so!if!one! is!down!another!can!be!used.

  129. Directors ✓Round4robin ✓Random ✓Client ✓Hash ✓Fallback ✓DNS

  130. backend.one.{ ....host.=.“localhost”; ....port.=.“80”; } backend.two.{ ....host.=.“127.0.0.1”; ....port.=.“81”; } director.localhosts.round;robin.{ ....{..backend.=.one;.}

    ....{..backend.=.two;.} ....{..backend.=.{. .............host.=.“localhost”;. .............port.=.“82”;. ........}. ....} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Round8robin%director
  131. backend.one.{ ....host.=.“localhost”; ....port.=.“80”; } backend.two.{ ....host.=.“127.0.0.1”; ....port.=.“81”; } director.localhosts.random.{ ....{..backend.=.one;..weight=4;}

    ....{..backend.=.two;..weight=6;} ....{..backend.=.{. .............host.=.“localhost”;. .............port.=.“82”;. ........}. ....} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Random%director
  132. director.localhosts.client.{ ....{..backend.=.one;..weight=1;.} ....{..backend.=.two;..weight=1;.} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; ....//Load.balance.by.URL ....set.client.identity.=.req.url; ....//Load.balance.by.client.IP,.this.is.the.default. ....set.client.identity.=.client.ip;

    ....//Load.balance.by.user.agent ....set.client.identity.=.req.http.user;agent; } Client%director Uses# client.identity
  133. director.localhosts.hash.{ ....{..backend.=.one;..weight=1;.} ....{..backend.=.two;..weight=1;.} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Hash%director It'will'use'the'value'of'req.hash,'just'as'the'normal' cache7lookup'methods

    This'is'useful'is'you'are'using'Varnish'to'load' balance'in'front'of'other'Varnish'caches'or'other' web'accelerators'as'objects'won't'be'duplicated' across'caches.
  134. director.localhosts.fallback.{ ....{..backend.=.one;} ....{..backend.=.two;} } sub.vcl_recv.{ ....set.req.backend.=.localhosts; } Fallback%director Picks#first# healthy#

    backend Order# matters
  135. director.directorname.dns.{ .........list.=.{ .................host_header.=."www.example.com"; .................port.=."80"; .................connect_timeout.=.0.4s; ................"192.168.15.0"/24; ................"192.168.16.128"/25; ........} .........ttl.=.5m; .........suffix.=."internal.example.net";

    } DNS%director Autogenerates'backends'in'IP'ranges
  136. director.directorname.dns.{ .........list.=.{ .................host_header.=."www.example.com"; .................port.=."80"; .................connect_timeout.=.0.4s; ................"192.168.15.0"/24; ................"192.168.16.128"/25; ........} .........ttl.=.5m; .........suffix.=."internal.example.net";

    } DNS%director Hostname'='host_header'+'suffix www.example.com.internal.example.net
  137. director.directorname.dns.{ .........list.=.{ .................host_header.=."www.example.com"; .................port.=."80"; .................connect_timeout.=.0.4s; ................"192.168.15.0"/24; ................"192.168.16.128"/25; ........} .........ttl.=.5m; .........suffix.=."internal.example.net";

    } DNS%director Resolves'hostname'&'caches'result Backend'selecNon'based'on'result'of'DNS'resolving
  138. Exercise 7 Playing with backends, probes & directors

  139. Cache invalidation

  140. Don’t wait for cache objects to expire

  141. There!are!only!two!hard! things!in!Computer!Science:! cache!invalidaKon,!naming! things,!and!offBbyBone!errors.

  142. ✓Purge ✓Ban ✓Always'miss Cache%invalida?on

  143. ✓Purge'VCL'method ✓Removes'specific'cache'item ✓Frees'up'memory ✓Evicts'variants ✓Next'client'triggers'the'refresh ✓No'backup'if'backend'is'down Purge

  144. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$return$(lookup); $$$$$$$$} } sub$vcl_hit${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$purge; $$$$$$$$$$$$$$$$error$200$"Purged"; $$$$$$$$}

    } sub$vcl_miss${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$error$404$"Not$in$cache"; $$$$$$$$} } Purge HTTP#call# to#purge
  145. ✓Implement'in'your'CMS ✓HTTP'200':'removed'from'cache ✓HTTP'404:'not'in'cache ✓Some'CMS'plugins'require'200'for'miss() Purge curl$IX$PURGE$http://varnish.dev/yourIpage

  146. ✓Ban'VCL'method' ✓Ban'command'line'action ✓Ban.url'command'line'action ✓Pattern'(regex)'based ✓Adds'ban'to'ban4list ✓No'memory'freed'up'immediately ✓Evicts'variants ✓Extremely'fast'and'powerful Ban

  147. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${ ban("req.http.host$==$"$+$req.http.host$+$"$ &&$req.url$==$"$+$req.url); error$200$"Banned"; $$$$$$$$} } Ban sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${

    $$$$$$$$$$$$ban("req.http.host$==$"$+$req.http.host$+$ "$&&$req.url$~$"$+$req.url); error$200$"Banned"; $$$$$$$$} } Ban#URL# pattern Ban#URL Name# doesn’t# matter
  148. ✓Removes'old'bans'from'the'ban4list ✓Separate'thread ✓Do'ban.list'on'CLI'to'see'the'ban4list ✓ban_lurker_sleep'parameter ✓Has'no'access'to'“req”'object ✓Smart'bans'are'ban'lurker'friendly Ban%lurker

  149. ✓Ban'lurker'friendly ✓Use'beresp.x4url'&'beresp.x4host'instead'of' req.url'&'req.http.host Smart%bans

  150. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$ban("obj.http.xIhost$==$"$+$req.http.host$+$ "$&&$obj.http.xIurl$==$"$+$req.url); $$$$$$$$$$$$error$200$"Banned"; $$$$$$$$} } sub$vcl_fetch${ $$$$$$$$set$beresp.http.xIurl$=$req.url; $$$$$$$$set$beresp.http.xIhost$=$req.http.host;

    } sub$vcl_deliver${ $$$$$$$$unset$resp.http.xIurl; $$$$$$$$unset$resp.http.xIhost; } Smart%bans Ban#lurker# friendly
  151. ✓Used'for'reloading'content,'not'evicting ✓Uses'set'req.hash_always_miss'='true; ✓Looks'up'object,'but'ignores'it ✓Doesn’t'work'well'with'variations ✓Because'return(pass);'doesn’t'store'fetch' result'in'cache ✓Because'purge();'will'fail'when'your' backend'is'down Always%miss%

  152. sub$vcl_recv${ $$$$$$$$if$(req.request$==$"REFRESH")${ $$$$$$$$$$$$$$$$set$req.request$=$"GET"; $$$$$$$$$$$$$$$$set$req.hash_always_miss$=$true; $$$$$$$$} } } Refresh/always%miss

  153. sub$vcl_recv${ $$$$if$(req.request$==$"PURGE")${ $$$$$$return$(lookup); $$$$} $$$$if$(req.request$==$"BAN")${ $$$$$$ban("obj.http.xIurl$~$"$+$req.http.xIurl$+$ $$$$$$"$&&$obj.http.xIhost$~$"$+$req.http.xIhost); $$$$$$error$200$"Banned"; $$$$} $$$$if$(req.request$==$"REFRESH")${

    $$$$$$set$req.request$=$"GET"; $$$$$$set$req.hash_always_miss$=$true; $$$$} } All8in8one%(1)
  154. sub$vcl_hit${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$purge; $$$$$$$$$$$$$$$$error$200$"Purged"; $$$$$$$$} } sub$vcl_miss${ $$$$$$$$if$(req.request$==$"PURGE")${$ $$$$$$$$$$$$$$$$purge; $$$$$$$$$$$$$$$$error$404$"Not$in$cache";

    $$$$$$$$} } sub$vcl_pass${ $$$$$$$$if$(req.request$==$"PURGE")${ $$$$$$$$$$$$$$$$error$502$"PURGE$on$a$passed$object"; $$$$$$$$} } All8in8one%(2)
  155. sub$vcl_fetch${ $$$$$$$$set$beresp.http.xIurl$=$req.url; $$$$$$$$set$beresp.http.xIhost$=$req.http.host; } sub$vcl_deliver${ $$$$$$$$unset$resp.http.xIurl; $$$$$$$$unset$resp.http.xIhost; } All8in8one%(3)

  156. varnishstat$I1$If$n_object Measure%with%varnishstat Number# of#objects#in# cache

  157. Protect'your'purges

  158. acl$purge${ $$"localhost"; $$"217.21.177.0"/24; $$"10.10.10.1"; $$"some.host.com"; } sub$vcl_recv${$$$$ $$if$(req.request$==$"PURGE")${ $$$$if$(!client.ip$~$purge)${ $$$$$$error$405$"Not$allowed.";

    $$$$}$$$$ $$$$ban("obj.http.xIurl$==$"$+$req.url$+"$&& $$$obj.http.xIhost$==$"$+$req.http.host); $$$$error$200$"Purged"; $$} } Protect#your#purges Use#an#ACL
  159. Exercise 8 Invalidation

  160. RELOAD !

  161. ✓/etc/init.d/varnish'reload ✓/etc/init.d/varnish'restart ✓varnishadm'(vcl.load'&'vcl.use) ✓telnet'(vcl.load'&'vcl.use) Reloading%Varnish%VCL

  162. ✓/etc/init.d/varnish'reload ✓New'VCL'is'loaded ✓Keeps'objects'in'memory Service%reload

  163. ✓/etc/init.d/varnish'restart ✓New'VCL'is'loaded ✓All'objects'are'removed'from'memory Service%restart

  164. ✓Call'varnishadm'on'CLI ✓Call'vcl.list'for'an'overview ✓call'vcl.load'bla'/path/vcl/default.vcl ✓On'compilation'failure,'errors'are'shown ✓Call'vcl.use'bla'to'confirm'new'VCL Varnishadm

  165. ✓Open'telnet'connection ✓Call'vcl.list'for'an'overview ✓call'vcl.load'bla'/path/vcl/default.vcl ✓On'compilation'failure,'errors'are'shown ✓Call'vcl.use'bla'to'confirm'new'VCL Telnet See# next#slide

  166. /usr/sbin/varnishd$IP$/var/run/ varnishd.pid$Ia$:80$If$/home/data/ default.vcl$IT$127.0.0.1:6082$Iu$ varnish$Ig$varnish$It$120$Ip$vcl_dir$/ home/data$Iw$5,500,300$Is$malloc,256m$I S$/etc/varnish/secret$In$lucid32 Telnet Running# process

  167. $.telnet.127.0.0.1.6082 Telnet

  168. $$telnet$127.0.0.1$6082 Trying$::1... Trying$127.0.0.1... Connected$to$localhost. Escape$character$is$'^]'. 107$59$$$$$$ ayfmdmirgrluoqtukkueawfbrpmmvtkw Authentication$required. Telnet%authen?ca?on Auth#

    challenge
  169. Telnet%authen?ca?on $.cat./etc/varnish/secret. ab9cae99;7905;42e8;9f28; b405e684ce93 Secret# key

  170. Telnet%authen?ca?on ✓Challenge ✓Newline'character'(\x0A) ✓Secret'key ✓Newline'character'(\x0A) ✓Challenge ✓Newline'character'(\x0A) ✓SHA256'string Compose# auth#string

  171. Telnet%authen?ca?on <?php $challenge = $argv[1]; $secret = trim(fgets(STDIN)); $pack =

    $challenge . "\x0A" . $secret . "\x0A" . $challenge . "\x0A"; $key = hash('sha256', $pack); echo $key.PHP_EOL; Challenge Secret# key Result
  172. Telnet%authen?ca?on $$cat$/etc/varnish/secret$|$php$ secret.php$ ayfmdmirgrluoqtukkueawfbrpmmvtkw$ 4fac6b3e568d8b307300d6fa7a1b92ce4e764ce 46cabed4783ffb0e937d86ff3

  173. $$telnet$127.0.0.1$6082 Trying$::1... Trying$127.0.0.1... Connected$to$localhost. Escape$character$is$'^]'. 107$59$$$$$$ ayfmdmirgrluoqtukkueawfbrpmmvtkw Authentication$required. auth$ 4fac6b3e568d8b307300d6fa7a1b92ce4e764ce

    46cabed4783ffb0e937d86ff3 200$204 Telnet%authen?ca?on
  174. Exercise 9 Load VCL via Varnishadm & telnet

  175. Custom# VCL

  176. sub$vcl_recv${ unset$req.http.cookie; } sub$vcl_fetch${ unset$beresp.http.setIcookie; } Remove%all%cookies

  177. sub$vcl_recv${ set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "__utm.=[^;]+(;$)?",$""); set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "utmctr=[^;]+(;$)?",$""); set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "utmcmd.=[^;]+(;$)?",$""); set$req.http.Cookie$=$regsuball(req.http.Cookie,$ "utmccn.=[^;]+(;$)?",$""); if$(req.http.cookie$~$"^$*$")${

    unset$req.http.cookie; } } Remove%GA%cookies
  178. sub$vcl_recv${ $$$return(lookup); } sub$vcl_hash${ $$$hash_data(req.http.cookie); } Cache%cookies

  179. sub$vcl_hash${ $$$$if(req.http.Cookie$~$"lang"){ $$$$$$$$hash_data(regsuball(req.http.Cookie,$"^. +;?$?lang=([aIzAIZ0I9]+)($|;|$;).*$","\1")); $$$$} } Cache%some%cookies

  180. sub$vcl_recv${ $$$return(lookup); } sub$vcl_fetch${ $$$return(deliver); } Ignore%cookies

  181. sub$vcl_recv${ $ if$(req.url$~$"^[^?]*\.(bmp|bz2|css|doc| eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]| pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff| xml|zip)(\?.*)?$")${ $$$$$$$$unset$req.http.Cookie; $$$$$$$$return$(lookup); $$$$} }

    Cache%sta?c%files
  182. sub$vcl_fetch${ $ if$(req.url$~$"^[^?]*\.(bmp|bz2|css|doc| eot|flv|gif|gz|ico|jpeg|jpg|js|less|mp[34]| pdf|png|rar|rtf|swf|tar|tgz|txt|wav|woff| xml|zip)(\?.*)?$")${ $$$$$$$$unset$beresp.http.setIcookie; $$$$} } Cache%sta?c%files

  183. sub$vcl_deliver${ $$$$set$resp.http.xIcacheIhits$=$obj.hits; $$$$if$(obj.hits$>$0)${ $$$$$$$$set$resp.http.xIcache$=$"hit"; $$$$}$else${ $$$$$$$$set$resp.http.xIcache$=$"miss"; $$$$} } Debug

  184. sub$vcl_recv${ set$req.http.Host$=$regsub(req.http.Host,$":[0I9]+",$""); if$(req.url$~$"\#")${ set$req.url$=$regsub(req.url,$"\#.*$",$""); } if$(req.url$~$"\?$")${ set$req.url$=$regsub(req.url,$"\?$",$""); } } Sani?ze%URL

  185. sub$vcl_recv${ if$(req.url$~$"(\?|&)(utm_source|utm_medium|utm_campaign| gclid|cx|ie|cof|siteurl)=")${ set$req.url$=$regsuball(req.url,$"&(utm_source| utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([AI z0I9_\I\.%25]+)",$""); $$$set$req.url$=$regsuball(req.url,$"\?(utm_source| utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=([AIz0I9_ \I\.%25]+)",$"?"); }

    } Sani?ze%URL%(2)
  186. Exercise 10 Custom VCL

  187. Grace mode

  188. ✓A'graced'object'is'an'object'that'has'expired,'but'is' still'kept'in'cache ✓Grace'mode'is'when'Varnish'uses'a'graced'object ✓Used'when'backend'is'slow'or'down ✓req.grace'defines'how'long'an'overdue'an'object'can' be'for'Varnish'to'still'consider'it'for'grace'mode. ✓beresp.grace'defines'how'long'past'the'beresp.ttl7 time'Varnish'will'keep'an'object

  189. sub$vcl_recv{ $$$$if$(req.backend.healthy)${ $$$$$$$$set$req.grace$=$0s; $$$$}$else${ $$$$$$$$set$req.grace$=$20s; $$$$} } sub$vcl_fetch{ $$$$set$beresp.grace$=$20s; }

  190. probe.healthcheck.{ ....url.=."/some/url"; ....interval.=.1s; ....timeout.=.1s; ....window.=.1; ....threshold.=.1; ....initial.=.1; ....expected_response.=.200; } backend.default.{

    ...host.=."localhost"; ...port.=."8080"; ...probe.=.healthcheck; } sub.vcl_recv{ ....if.(req.backend.healthy).{ ........set.req.grace.=.0s; ....}.else.{ ........set.req.grace.=.20s; ....} } sub.vcl_fetch{ ....set.beresp.grace.=.20s; } With# backend# probe
  191. Exercise 11 Grace mode

  192. Edge#Side#Includes

  193. <html> <body> <table> <tr> <td colspan="2" > <?php include('header.php') ?>

    </td> </tr> <tr> <td><?php include('menu.php') ?></td> <td><?php include('main.php') ?></td> </tr> <tr> <td colspan="2" > <?php include('footer.php') ?> </td> </tr> </table> </body> </html>
  194. header.php menu.php main.php footer.php TTL#5s No#caching TTL#10s TTL#2s

  195. <?php function esi($file){ if(isset($_SERVER['Surrogate-Capability']) && preg_match('/^(.+)="ESI/1.0$/',$_SERVER['Surrogate-Capability'])){ header("Surrogate-Control: content='ESI/1.0'"); $url =

    'http://'. $_SERVER['HTTP_HOST'].substr($_SERVER['SCRIPT_NAME'], 0,1+strrpos($_SERVER['SCRIPT_NAME'],'/')).$file; echo "<esi:include src=\"$url\" />".PHP_EOL; } else { include($file); } } ESI#PHP#helper#funcaon#(1)
  196. <?php function esi($file){ if(isset($_SERVER['Surrogate-Capability']) && preg_match('/^(.+)="ESI/1.0$/',$_SERVER['Surrogate-Capability'])){ header("Surrogate-Control: content='ESI/1.0'"); $url =

    'http://'. $_SERVER['HTTP_HOST'].substr($_SERVER['SCRIPT_NAME'], 0,1+strrpos($_SERVER['SCRIPT_NAME'],'/')).$file; echo "<esi:include src=\"$url\" />".PHP_EOL; } else { include($file); } } ESI#PHP#helper#funcaon#(2)
  197. <html> <body> <table> <tr> <td colspan="2" > <esi:include src="/header.php" />

    </td> </tr> <tr> <td><esi:include src="/menu.php" /></td> <td><esi:include src="/main.php" /></td> </tr> <tr> <td colspan="2" > <esi:include src="/footer.php" /> </td> </tr> </table> </body>
  198. ESI#VCL sub.vcl_recv.{ ....set.req.http.Surrogate;Capability="key=ESI/1.0"; } sub.vcl_fetch.{ . if(beresp.http.Surrogate;Control~"ESI/1.0").{ .........unset.beresp.http.Surrogate;Control; . ....set.beresp.do_esi=true;

    ....} }.
  199. <?php require_once __DIR__.'/vendor/autoload.php'; use Symfony\Component\HttpKernel\HttpCache\Esi; use Symfony\Component\HttpFoundation\Request; use Symfony\Component\HttpFoundation\Response; $esi

    = new Esi; $request = Request::createFromGlobals(); $response = new Response(); Symfony#components#(1)
  200. $content = '<h1>Exercise 12</h1>'; $content .= '<h2>Edge Side Includes</h2>'; $content

    .= date("Y-m-d H:i:s").'<br/>'; if($esi->hasSurrogateEsiCapability($request)){ $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=2', null,false).'<br />'; $content .= $esi->renderIncludeTag( $request->getBasePath().'/edge.php?ttl=0', null,false).'<br />'; } $content .= '<br /><a href="/">Back to index</a>'; Symfony#components#(2)
  201. $response->setContent($content); $esi->addSurrogateControl($response); $response->setSharedMaxAge(10); $response->send(); Symfony#components#(3)

  202. And# on#that# bombshell#...

  203. None
  204. h"ps://joind.in/8432 Please# give#me# feedback

  205. Thanks

  206. None