Software Engineering Manager at CaringBridge MNPHP Organizer Open Source Contributor (Zend Framework and various others) Where you can find me: • Twitter: mwillbanks G+: Mike Willbanks • IRC (freenode): mwillbanks Blog: http://blog.digitalstruct.com • GitHub: https://github.com/mwillbanks Housekeeping…
“I don’t want to listen to you but only want to play on my laptop.” The quick, easy and uninformed way... You listening over there? • The Awesome VCL’s, Directors and more… now we’re going places. • The Crazy ESI, Purging, VCL C, and VMOD… my head hurts, stop! • Varnish Command Line Apps varnishtop, varnishstat, etc. • Questions Although you can bring them up at anytime! Agenda
using memory and keeping in mind cookies, request headers and more… •It caches pages so that your web server can RELAX! What about my apache, tomcat, uhhh… (mongrel|thin|goliath….) Generally caching by TTL + HTTP Headers (cookies too!) •A load banancer, proxy and more… What? …. Yes, it can do that! #winning What The Hell? Tell me!
message to our mobile users! The system is down, or we want to be able to communicate a message to them about some subject… maybe a campaign. The apps and mobile site rely on an API • Trouble in paradise? Few and far in between. • We want to talk to our users? Of course! Let an API talk to a server… A story on crashing and burning A story on a bad thing making a huge success! A General Use Case
EPEL: yum install varnish • only 6.x otherwise you’ll be out of date! WOOT Compiling #winning #git • git clone git://git.varnish-cache.org/varnish-cache • cd varnish-cache • sh autogen.sh • ./configure • Make && make install Installment
book version of the reference manual https://www.varnish-cache.org/docs/3.0/tutorial/index.html •Knock yourselves out! There is a ton of documentation • Yes, this makes happy developers. Documentation is very accurate, read carefully. Focus heavily on VCL’s, that is generally what you need. I’m attempting to show you some of how this works but you will require the documentation to assist you. Documentation
Hi wordpress and drupal! • https://www.varnish-cache.org/trac/wiki/VarnishAndWordpress • https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal Examples of all sorts of crazy • https://www.varnish-cache.org/trac/wiki/VCLExamples Existing VCL’s – The truly lazy…
& Independently States are Isolated but are Related Return statements exit one state and start another VCL defaults are ALWAYS appended below your own VCL •VCL can be complex, but… Two main subroutines; vcl_recv and vcl_fetch Common actions: pass, hit_for_pass, lookup, pipe, deliver Common variables: req, beresp and obj More subroutines, functions and complexity can arise dependent on condition. VCL – Varnish Configuration Language
VMOD initialization • vcl_recv – Beginning of request, req is in scope • vcl_pipe – Client & backend data passed unaltered • vcl_pass – Request goes to backend and not cached • vcl_hash – call hash_data to add to the hash • vcl_hit – called on request found in the cache • vcl_miss – called on request not found in the cache • vcl_fetch – called on document retrieved from backend • vcl_deliver – called prior to delivery of cached object • vcl_error – called on errors • vcl_fini – all requests have exited VCL, cleanup of VMOD’s VCL - Subroutines
input. Request host and URL is default from the default vcl. • regsub(string, regex, sub) – substitution on first occurance sub can contain numbers 0-n to inject matches from the regex. • regsuball(string, regex, sub) – substitution on all occurances • ban(expression) – Ban all objects in cache that match • ban(regex) – Ban all objects in cache that have a URL match VCL - Functions
servers • Directors are a glorified reverse proxy Allows for certain types of load balancing Allows for talking to a cluster “A director is a logical group of backend servers clustered together for redundancy. The basic role of the director is to let Varnish choose a backend server amongst several so if one is down another can be used.” Directors
•Client Director – picks a backend by client identity •Hash Director – picks a backend by URL hash value •Round-Robin Director – picks a backend in order •DNS Director – picks a backend by means of DNS Random OR Round-Robin •Fallback – picks the first “healthy” backend Directors – The Types
(server side includes) to include fragments (or dynamic content for that matter). •If don’t think you can use varnish because you have say, user information displayed on every page; think again! •Think of it as replacing regions inside of a page as if you were using XHR (AJAX). •Three Statements are Implemented esi:include – Include a page esi:remove – Remove content <!-- esi --> - ESI disabled, execute normally ESI – Edge Side Includes
set beresp.do_esi = true; By default, ESI will still cache, so add an exclusion if you need it • if (req.url == “/show_username.php”) { return (pass); } Varnish refuses to parse content for ESI if it does not look like XML • This is by default; so check varnishstat and varnishlog Using ESI
/> <!-- Don't do this as you'd lose the advantage of varnish --> <!--esi <?php include 'user_header.php'; ?> --> </header> <section id="main"></section <footer></footer> </body> </html> ESI – By Example
too much tonight, ok? • If you do… find a DD or a cab. •The various ways of purging varnishadm – command line utility • It’s the ole finger in the back of the throat Sockets (port 6082) – everyone likes a good socket wrench • Sure, Ipecac is likely overkill. HTTP – now that is the sexiness • See, now we’re not forcing the point! Purging – Beer anyone?
a message queue (or gearman job server) Have a worker that knows about the varnish servers Submit the request to clear the cache in the asynchronously or synchronously depending on your use case. • Have enough workers to make this effective at purging the cache quickly. Sexy Purging
embed C into the VCL for varnish? •Want to do something crazy fast or leverage a C library for pre or post processing? •I know… you’re thinking that’s useless.. On to the example; and a good one from the Varnish WIKI! Embedding C in VCL – you must be crazy
you to extend varnish and create new functions •Now, if you are writing modules for varnish you have a specialty use case! Go read up on it! https://www.varnish-cache.org/docs/trunk/reference/vmod.html VMOD – Varnish Modules / Extensions
sever in the queue or load test an environment? varnishreplay –r log.txt •Replaying logs can allow you to do this. This is great for when you are going to be deploying code to check for performance issues. Cache Warmup
mwillbanks
itsmedreamwalker
sanfrecce_osaka
kangnux
luisasofie_xoxo
takefumiyoshii
irof
izumin5210
kishida
aethiopicuschan
bkuhlmann
sullis
hanyudaeiiti
shpigford
lauravandoore
swwweet
frogandcode
sugarenia
philhawksworth
zakiwarfel
shlominoach
chrisshort
sonatard
trishagee
cassininazir
