Share
Embed
Accelerating Web Applications with Varnish
The Varnish web accelerator has been the center of attention recently as more and more content providers realize its power and flexibility. More than just a simple caching proxy, Varnish lets us crack open the request - response cycle and take control of precisely how each and every request is served. Through practical examples, I'll demonstrate how websites of every size are using this powerful tool to maximize performance, improving the user's experience while eliminating back-end complexity. We'll explore the unique Varnish Control Language and discover how it can make your site delivery lighter, leaner, and lightning fast!
Samantha Quiñones
May 21, 2014
Transcript
78 slides
-
About Me @ieatkillerbees http://tembies.com Decade+ as a SW engineer at
Visa International Lead PHP developer at POLITICO since March, 2013 Software engineer since 1996 working primarily with Python and PHP since 2005 -
In the beginning… The web was born as a platform
for mostly static content Most content, once published, changes rarely Static content continues to dominate the web -
What is static? Even though most content is static, changes
must be available very rapidly Static content is cheap to deliver but expensive to maintain Static content is... static. Stale. Boring. -
What is dynamic? Content is curated for each consumer Dynamic
content is expensive to deliver but cheap to maintain. Dynamic content makes the web engaging. -
“The computer is incredibly fast, accurate, and stupid. Man is
unbelievably slow, inaccurate, and brilliant. The marriage of the two is a force beyond calculation.” –Leo Cherne -
Basic content-delivery via HTTP Files on disk Passive benefit from
OS memory mapping and caching Prone to I/O blocking Nearly impossible to scale out -
The Relational DBMS High degree of data organization and integrity
Common, well-supported interfaces SQL Designed to scale on mainframes and “enterprise” servers (read: more CPUs/engines, more RAM/real storage) -
The Relational DBMS SELECT DISTINCT t1.CU_ship_name1, t1.CU_ship_name2, t1.CU_email FROM customers
AS t1 WHERE t1.CU_solicit=1 AND t1.CU_cdate >= 20100725000000 AND t1.CU_cdate <= 20100801000000 AND EXISTS( SELECT NULL FROM orders AS t2 INNER JOIN item AS t3 ON t2.O_ref = t3.I_oref INNER JOIN product AS t4 ON t3.I_pid = t4.P_id INNER JOIN ( SELECT C_id FROM category WHERE C_store_type = 2 ) AS t5 ON t4.P_cat = t5.C_id WHERE t1.CU_id = t2.O_cid); -
How do we leverage the power of a DBMS to
manage content while also building applications that are scalable, performant, and durable? -
“The advantage of adding cache…is that they have the potential
to… eliminate some interactions, improving efficiency, scalability, and user-perceived performance by reducing the average latency of a series of interactions.” –Roy Fielding -
Caches Amortize the cost of expensive operations across many requests
Vastly increase the performance of an application at the cost of responsiveness to change Caches can scale horizontally in front of a smaller DBMS implementation -
Caches of all Flavors In-memory object caches Content-delivery networks (CDNs)
Static pre-rendering (yeah, this happens) Key-value stores & Document Databases Caching proxies -
Caches of all Flavors Memcached is among the most popular
caching tools In-memory key-value store Operates on the predominant caching pattern -
In-Memory Object Caches $cache = new CacheClient(['127.0.0.1:4242']); ! $widget =
$cache->get('widget-1'); if (!$widget) { $widget = $database->find('widget-1'); $cache->set('widget-1', $widget, CACHE_EXPIRY); } ! $gizmo = new Gizmo(); $id = $database->save($gizmo); $cache->set($id, $gizmo, CACHE_EXPIRY); -
In-Memory Object Caches Tools like memcached are developer-focused The onus
is on developers to make sure that they are caching data efficiently Usually this means that absolutely everything gets cached Or, the caching platform is underutilized to the point of near- irrelevance. -
Content Delivery Networks Akamai, Cloudflare, etc Fantastic for caching truly
static content Extremely robust Largely invisible to developers Not at all flexible -
Static Pre-Rendering Very annoying to maintain When combined with other
methods, it can greatly reduce the “freshness” of content Hides extremely poor performance Extremely robust -
Caching Reverse Proxies Almost like a “local” CDN Maintain a
cache of content from the origin server(s) Invisible to developers Squid, Varnish, and other “web accelerators” fall in this category -
All roads lead… We're trying to make our sites “faster”…
But what does that mean? Most dynamic content is pretty static -
The reality of media publishing Content is overwhelmingly static Yet
extremely time-sensitive Slow delivery of content is a huge revenue risk Load is inconsistent and unpredictable Editorial and engineering requirements are rarely in sync Failure can be devastating -
Case Study: A Norwegian Tabloid Verdens Gang is one of
Norway's most popular newspapers Suffered the same problems of all media platforms Poul-Henning Kamp, a BSD core developer, was the lead developer and application architect for Verdens Gang As a kernel developer, Kamp has a particular set of skills that allowed him to approach this problem from a new angle -
Memory & Storage In the olden days, there was a
line of demarcation between primary and secondary storage In short, primary storage (RAM in modern computers) can be accessed directly by the CPU Secondary storage is accessed via an I/O channel or controller -
Memory and Storage As early as the 1950s, computer scientists
were experimenting with virtual memory. By the 1970s, virtual memory was common in commercial computers Virtual memory is an abstraction that allows secondary storage to extend primary storage The operating system cooperates with specialized hardware to manage the paging of data in and out of virtual memory. -
Virtual Memory is a Cache In essence, virtual memory is
a cache The operating system swaps data between high-speed primary storage and slower secondary storage based on factors like age and access frequency Commonly accessed data is kept warm and ready while rarely- needed data can be quickly retrieved when called for -
Caching-Reverse Proxies Traditional caching reverse proxies allocate memory and fill
it with objects Less-used objects are written to disk. Objects on disk are written to memory when requested Sounds familiar, right? -
Varnish is different The operating system is already paging data
between primary and secondary storage So why reinvent the wheel? Varnish gets out of the operating system's way and lets the kernel do what it's best at! -
Varnish in a nutshell At start-up, Varnish allocates a big,
empty chunk of memory Within that space, Varnish maintains a workspace with pointers to cached objects, headers, etc Varnish prioritizes worker threads by most recently used These factors combine to reduce overall memory ops -
Getting Started Open-source: http://www.varnish-cache.org Commercial: https://www.varnish-software.com/ DEB: apt-get install varnish
RPM: Available from EPEL Open BSD: pkg_add varnish Source: git://git.varnish-cache.org/varnish-cache -
Varnish Config Language Varnish has its own DSL, VCL Modeled
after C (and allows for inline C) VCL allows us to modify requests and responses “in-flight” Configure “backends” and how to interact with them Compiled in to binary objects and loaded in to memory -
Starting varnishd -f : Specify a VCL file to load
-s : Configure the storage (malloc, file, etc) -T : Specify where Varnish should listen for admin connections -a : Specify where Varnish should listen for HTTP requests -
Backends Backends are origin servers (Apache, NGINX, etc) that will
be serving your content Varnish can proxy a single backend, a cluster of them, or multiple clusters of them The configuration of backends and request routing could fill an hour on its own -
Directors Logical clusters of backends that allow for load balancing
and redundancy Monitor the health of individual backends Random directors route requests to origin servers randomly Client directors route based on the identity of the client Hash directors route based on the URL of the request Round-robin directors round robin requests -
DNS Directors director originpool dns { .list = { .host_header
= "origin.foo.com"; .port = "80"; .connect_timeout = 0.5s; "10.42.42.0"/24; } .ttl = 5m; } -
Probes Used by Varnish determines if a backend is healthy
or not Specifies a check interval, timeout, expected response, etc. Can be set as part of a backend definition, or standalone. -
Probes probe healthcheck { .url = "/health.php"; .interval = 60s;
.timeout = 0.3 s; .window = 8; .threshold = 3; .initial = 3; .expected_response = 200; } backend origin { .host = "origin.foo.com"; .port = "http"; .probe = healthcheck; } -
ACLs Used to identify client addresses Can allow bypassing the
proxy for local clients Restrict URLs to certain clients -
ACLs acl admin { "localhost"; "10.42.42.42"; } sub vcl_recv {
if (req.url ~ "^/admin") { if (client.ip ~ admin) { return(pass); } else { error 405 "Not allowed in admin area."; } } } -
Hooks Hooks allow the execution of VCL at a number
of pre-defined points in the request-response cycle. vcl_recv – Called after a request has been received vcl_pipe – Called when entering pipe mode vcl_pass – Called when entering pass mode vcl_hit – Called when an object is found in the cache vcl_miss – Called when an object is not found in the cache -
Hooks vcl_fetch* – Called when a response has been received
from an origin server vcl_deliver – Called before a cached object is returned to a client vcl_error – Called when an error happens -
Hooks in Varnish 4 vcl_fetch has been replaced by: vcl_backend_fetch
- Called before sending the backend request vcl_backend_response - Called after a response has been successfully retrieved from the backend -
Grace Mode Allows varnish to serve an expired object while
a fresh object is being generated by the backend. Doesn't require the user to pay a “first-hit” tax Avoids threads piling up Protects against stampeding popular resources Some users may get expired data even though fresh data is “technically” available. -
Saint Mode Like grace mode, but more awesome. Allows Varnish
to serve expired objects when we don't like what the backends are returning. 200 OK with no response body? Serve from cache 500 errors? Serve from cache Merged in to Grace Mode in Varnish 4 -
Saint Mode Receive request for expired resource Request resource from
origin-1 Receive 503 Request resource from origin-2 Receive 503 Increase TTL for resource by 30 seconds and restart request -
Managing Varnish Varnish has a simple command prompt accessible by
telnetting to the configured port List loaded VCL files, load new VCL files, switch between active VCL files Get the status of backends“ban” URLs (force them to pass requests to a backend) -
Purging Objects Varnish understands a nonstandard HTTP method “PURGE” PURGE
/obsolete/resource Configurable via ACL Frees memory Next client request will refresh the content -
Purging Objects sub vcl_recv { if (req.request == "PURGE") {
if (!client.ip ~ purge) { error 405 "Not allowed."; } return (lookup); } } sub vcl_hit { if (req.request == "PURGE") { purge; error 200 "Purged."; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "Purged."; } } -
Banning Objects Banning applies a filter to cached objects Another
way to invalidate content Applies instantly Does NOT free up memory Large ban lists can degrade performance Understands regular expressions -
Vary Special header used by the origin server to indicate
one resource is a variation of another Variations are common for different encodings (gzip, deflate) Or based on user agent (but it’s important to normalize!) -
Vary if (req.http.Accept-Encoding) { if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") { #
No point in compressing these remove req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate"; } else { # unknown algorithm remove req.http.Accept-Encoding; } } -
Replaying Traffic Takes a Varnish log and replays all of
the traffic Powerful tool to quickly warm up an empty cache when starting new Varnish instances -
Varnish is complex Varnish is an incredibly complex and powerful
tool Varnish is a dynamic caching framework This has been an overview of its features.Install it, read the docs, play, experiment, and explore! -
Varnish killed the memcached star Politico's PHP stack uses a
distributed service-oriented architecture Nearly all operations are abstracted behind RESTful APIs REST services from MongoDB, MySQL, other internal and external APIs We don't use memcached We don't use anything LIKE memcached -
Varnish killed the memcached star Our services network sits behind
a cluster of Varnish instances. All calls, even internally, go through those Varnish clusters. Every service returns appropriate HTTP status codes, including a 502s, 503s, and 504s. Every service returns Cache-Control headers which instruct Varnish to set the TTLs for each resource -
We consume a lot of stuff Stored content from our
internal CMS Static assets in S3 Data feeds from other reporting agencies Every external dependency is abstracted behind an interface that we define and control -
What to do when an external dependency fails? Option 1:
Fail gracefully Option 2: Fail ungracefully Option 3: Don't fail, because you're a BEAST! -
Summary Varnish is powerful. It's simple to get started, and
difficult to master Varnish has the potential to not only accelerate applications, but to simplify infrastructures -
Contact Info & Feedback Twitter: @ieatkillerbees Blog: http://tembies.com Email: [email protected]
Joind.in: https://joind.in/10648