Getting Started with Varnish

Transcript

1. Getting Started with Varnish php[world] 2014 - Tutorials Day Samantha Quiñones
2. @ieatkillerbees http://tembies.com
5. —Roy Fielding “The advantage of adding cache…is that they have the potential to… eliminate some interactions, improving efficiency, scalability, and user-perceived performance by reducing the average latency of a series of interactions.”
6. VM Setup ! (open port 6081)
7. What is Varnish • Web Application Accelerator • Caching Reverse Proxy • Written in C, initially built by Poul Henning Kamp • Open-source at http://www.varnish-cache.org • Supported at http://www.varnish-cache.com
8. Digital Media Publishing • Very low write volume • Cannot tolerate high write latency • Most resources are never updated after creation
9. Caching & Digital Media • Caching is great for digital media if… • Cache-busting is inexpensive • Management of “hot” and “cold” objects is efficient
10. Case Study: Verdens Gang • Verdens Gang is one of Norway's most popular newspapers • Suffered the same problems of all digital media platforms • Poul Henning Kamp, a BSD core developer, was the lead developer and application architect for Verdens Gang • As a kernel developer, Kamp has a particular set of skills that allowed him to approach this problem from a new angle
13. Computer Storage • In the olden days, there was a line of demarcation between primary and secondary storage • In short, primary storage (RAM in modern computers) can be accessed directly by the CPU • Secondary storage is accessed via an I/O channel or controller
14. Virtual Memory Management • As early as the 1950s, computer scientists were experimenting with virtual memory. • By the 1970s, virtual memory was common in commercial computers • Virtual memory is an abstraction that allows secondary storage to extend primary storage • The operating system cooperates with specialized hardware to manage the paging of data in and out of virtual memory.
15. Format Time (s) Equivalent Distance Equivalent Time 1 CPU Cycle 0.3 ns 1 m (1 step) 1 second L1 Cache 0.9 ns 3 m (3 steps) 3 seconds Main Memory 120 ns 360 m (to the highway) 6 minutes SSD 50 µs 170 km (Richmond, VA) 2 days HDD 5 ms 13,000 km (Hong Kong) 5 months
17. Virtual Memory is a Cache • In essence, virtual memory is a cache • The operating system swaps data between high-speed primary storage and slower secondary storage based on factors like age and access frequency • Commonly accessed data is kept “hot” and ready while rarely-needed data can be quickly retrieved when called for
18. Caching Reverse Proxies • CRPs work by retaining a copy of the data they proxy • Copies can be retained in memory or on disk • Copies have an expiration time (TTL) after which they are abandoned
19. Memory-Backed CRP • Traditional caching reverse proxies allocate memory and fill it with objects • Less-used objects are written to disk • Objects on disk are written to memory when requested • Sounds familiar, right?
20. Varnish’s Difference • Varnish allocates a heap of memory up front • Objects stored in that heap are managed by the OS • OS Virtual Memory Managers are very sophisticated • Why reinvent the wheel?
21. How Varnish Works • Varnish creates a “workspace” in its memory space • Workspace contains pointers to cached objects, headers, etc • Varnish prioritizes worker threads by most recently used • These factors combine to reduce overall disk & memory ops
23. varnishd • varnishd has two processes • manager runs as root and starts the child (which does all the work) • manager monitors child and restarts it if it fails • manager interacts with the varnish cli interface (varnishadm) • child runs with more limited permissions and handles traffic
24. varnishadm (varnish CLI) • Allows administrators to interact with a running varnish • Secured by PSK • Designed to be “scriptable”
25. Varnish Config Language • Configuration DSL that is translated to C and compiled • We do not “configure” Varnish so much as write policies for handling types of traffic
26. varnishlog • Provide access to logs stored in memory
27. varnishstats • Provides access to in-memory statistics (cache hit/miss rate, resource usage, etc)
29. Storage Backends • Malloc (memory-based) • File (disk-based)
30. Malloc Storage • Memory is allocated as startup in KB, Mb, Gb, or Tb • Defaults to unlimited • Overflows to swap • Extremely fast performance
31. File Storage • Space allocated in KB, Mb, Gb, Tb, or as a percentage of available space on the device • Defaults to 50% of available space
32. Transient Storage • Special storage space for short-lived objects • Defaults to an unlimited malloc • Threshold TTL is configurable (default: 10s)
33. Sizing • Understand the size of your “hot” dataset • Size of homepage (including images) + size of linked pages/objects • Cost to produce objects
34. First Steps
35. Installing Varnish • https://www.varnish-cache.org/installation/ubuntu
36. Installing Varnish # apt-get install apt-transport-https # curl https://repo.varnish-cache.org/ubuntu/GPG-key.txt | apt-key add - # echo "deb https://repo.varnish-cache.org/ubuntu/ precise varnish-4.0" >> /etc/apt/sources.list.d/varnish-cache.list # apt-get update # apt-get install varnish
37. Important Commands • service varnish restart — Stops and restarts Varnish. Clears all cache • service varnish reload — Reloads the currently active VCL • varnishadm vcl.load <name> <filename> — Loads a VCL • varnishadm vcl.use <name> — Makes VCL named <name> active • varnishadm param.set <param> <value> — Sets parameters
38. Default Config Edit /etc/default/varnish ! DAEMON_OPTS="-a :6081 \ # Listen Address -T localhost:6082 \ # Manage Address -f /etc/varnish/default.vcl \ # Config File -S /etc/varnish/secret \ # PSK -s malloc,256m" # Storage Config
39. Default VCL Edit /etc/varnish/default.vcl ! # Default backend definition. Set this to point to your content server. backend default { .host = “127.0.0.1"; .port = "8080"; }
40. Default VCL Change to… ! backend server1 { .host = "varnish.tembies.com:8180"; }
41. Exercise: Install Varnish
42. Serving Requests
43. Varnish & Cookies • By default, varnish will not cache if the request has a Cookie header or if the response has a Set-Cookie header • NB: It is better to not cache content, or to cache multiple copies, than to deliver content to the wrong person.
44. Dealing with Cookies • If possible, strip any cookies you do not need. If there are none left, cache • Create url schemes based on whether cookies are needed or not • Never cache Set-Cookie
45. Stripping Cookies sub strip_req_cookies { if (req.url !~ "^/admin") { set req.http.X-Orig-Cookie = req.http.Cookie; unset req.http.Cookie; } } ! sub strip_client_cc_headers { if (req.http.cache-control) { unset req.http.cache-control; } } ! sub vcl_recv { call strip_req_cookies; call strip_client_cc_headers; }
46. Exercise: Making Requests
47. Varnish Log & Stats
48. Varnish’s Log • Log is stored in memory and streamed to connected log clients • No I/O overhead for logging • Logs can be viewed/filtered in real-time
49. # varnishlog … - VCL_call HASH - VCL_return lookup - Hit 2147483657 - VCL_call HIT - VCL_return deliver - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK …
50. varnishlog • -b — Only show log lines from traffic going between Varnish and the backend servers. This will be useful when we want to optimize cache hit rates. • -c — Same as '-b' but for client side traffic. • -m tag:<regex> — Only list transactions where the tag matches a regular expression. If it matches you will get the whole transaction.
51. varnishstat • Window in to the health and performance of varnish • Hundreds of counters with current and running-average values
52. # varnishstat
53. Exercise: Inspecting Requests
54. HTTP Refresh
55. Hypertext Transfer Protocol • Current Version 1.1 (RFC 2616, 7230, 7231, 7232, 7233, 7234, & 7235) • Requests consist of a method, headers, and sometimes a body • Methods — GET, POST, HEAD, OPTIONS, PUT, DELETE, TRACE, or CONNECT • Responses consist of a status, headers, and sometimes a body • Many requests can be sent over a single connection
56. Request Format [method] [uri] HTTP/1.1 [<header1>: <value1>] [<headern>: <valuen>] ! [body]
57. GET /cached HTTP/1.1 Host: localhost:6081 Accept: text/html,application/xhtml+xml,application/ xml;q=0.9,image/webp,*/*;q=0.8 Accept-Encoding: gzip, deflate, sdch Accept-Language: en-US,en;q=0.8 Cookie: _ga=GA1.1.344169523.1415468951 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.52 Safari/537.36 X-DevTools-Emulate-Network-Conditions-Client-Id: 49CA66DB-5415-4490-B960-962423964A4F
58. Response Format HTTP/1.1 [status code] [status message] [<header1>: <value1>] [<headern>: <valuen>] ! [body]
59. HTTP/1.1 200 OK Accept-Ranges: bytes Age: 0 Cache-Control: max-age=30, public, s-maxage=30 Connection: keep-alive Content-Encoding: gzip Content-Type: text/html; charset=UTF-8 Date: Mon, 10 Nov 2014 02:24:58 GMT Server: Apache/2.4.10 (Ubuntu) Transfer-Encoding: chunked Vary: Accept-Encoding Via: 1.1 varnish-v4 X-Backend-Port: 6081 X-Powered-By: PHP/5.5.12-2ubuntu4 X-Varnish: 32772
60. Response Codes • 1xx — Informational • 2xx — Success • 3xx — Redirection • 4xx — Client Error • 5xx — Server Error
61. Cache Response Headers • Expires — Date & time after which a resource is considered stale • Last-Modified — Date & time when the resource was updated • Etag — “Entity tag,” a unique value for a resource’s contents. Usually a hash • Age — Number of seconds a resource has been in cache
62. Cache Request Headers • If-Modified-Since — Requests a fresh resource if the resource’s last modified date/time is more recent than the date/time specified • If-None-Match — If the resource’s Etag differs from the Etag specified, server should sent a fresh resource
63. Vary • Response header that indicates the response may differ based on the header specified. (i.e. Vary: User-Agent)
64. Cache-Control • Specified directives that control cache behavior • public — Any cache may cache the content • no-store — No cache should store the content • no-cache — Store the content, but don’t serve it without validating it • max-age — Number of seconds a cache may store content for • s-maxage — Same as max-age, but applies to public caches only • must-revalidate — Stale content cannot be served without validating it
65. Break: Questions?
66. VCL Building Blocks
67. Varnish Config Language • C-Derived Domain-Specific State Engine • Processes requests in isolation • return(action) exits one state and moves to the next • Default VCL is present beneath your code and is appended during compilation
68. VCL Syntax • Comments — //, #, /* */ • Subroutines — sub [name] { … } • Loops — Nope! • Termination — return() • Objects — Struct-like objects that map a name to a group of client addresses acl local { "localhost"; // myself "192.0.2.0"/24; // and everyone on the local network ! "192.0.2.23"; // except for the dialin router }
70. VCL Functions • regsub(<str>, <regex>, <sub>) — Replace the first match of <regex> in <str> with <sub> • regsuball(<str>, <regex>, <sub>) — Replace all matches of <regex> in <str> with <sub> • ban(<regex>) — Invalidate all cached objects that match <regex> • call(<subroutine>) — Call a subroutine • hash_data(<input>) — Adds data to the hash input. By default, Host and URL of the request are used • new() — Creates a new object • rollback() — Restore request headers • synthetic(<string>) — Prepares a synthetic response • return(<action>) — Terminate a subroutine
71. vcl_recv • Called at the start of a request after the request has been parsed. • Access to request object • Normalize input • Make backend routing decisions • Re-write client data • Manage caching policy • Access controls & security
72. vcl_recv - State Transitions • pass (→vcl_pass) — Bypass the cache, send request to the backend and return the response • pipe (→vcl_pipe) — Switch to a proxy-like mode • hash (→vcl_hash) — Attempt a cache lookup, possibly entering new data in the cache • synth — Generate a synthetic error response and abandons the request • purge (→vcl_hash→vcl_purge) — Purge the object and any variants
73. sub vcl_recv { if (req.method != "GET" && req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.method != "GET" && req.method != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (hash); }
74. Request Object • req.backend_hint — Set backend to this if we attempt to fetch • req.hash_always_miss — (bool) Force a cache miss for this request. If set to true Varnish will disregard any existing objects and always (re)fetch from the backend • req.http.[header] — The corresponding HTTP header • req.method — The request type (e.g. "GET", "HEAD") • req.restarts — Count of how many times this request has been restarted • req.url — The requested URL • req.xid — Unique ID of this request
75. Exercise: Vary on OS
76. vcl_backend_response • Called after the response headers have been received from a backend • deliver (→vcl_deliver) — Deliver the response, possibly caching it • abandon — Abandons the request and returns an error • retry — Retries the backend request. When the number of retries exceeds max_retries, Varnish will return an error.
77. sub vcl_backend_response { if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Surrogate-control ~ "no-store" || (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~ "no-cache|no-store|private") || beresp.http.Vary == "*") { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ set beresp.ttl = 120s; set beresp.uncacheable = true; } return (deliver); }
78. Backend Response Object • beresp.backend.ip — IP of the backend this response was fetched from • beresp.backend.name — Name of the backend this response was fetched from • beresp.grace — Set to a period to enable grace • beresp.http.[HEADER] — The corresponding HTTP header • beresp.proto — The HTTP protocol version used the backend replied with • beresp.reason — The HTTP status message returned by the server • beresp.status — The HTTP status code returned by the server • beresp.storage_hint — Hint to Varnish that you want to save this object to a particular storage backend • beresp.ttl — The object's remaining time to live, in seconds. beresp.ttl is writable • beresp.uncacheable — (bool) Marks the response as uncacheable
79. Calculating TTL • The s-maxage variable in the Cache-Control response header • The max-age variable in the Cache-Control response header • The Expires response header • The default_ttl parameter. • Cached Statuses: 200, 203, 300, 301, 302, 307, 404, 410
80. Exercise: Force TTL for PNGs
81. vcl_hit • Called when a cache lookup is successful • deliver(→vcl_deliver) — Deliver the object. Control passes to vcl_deliver • synth(status code, reason) — Return the specified status code to the client and abandon the request. • restart — Restart the transaction
82. sub vcl_hit { if (obj.ttl >= 0s) { // A pure unadultered hit, deliver it return (deliver); } if (obj.ttl + obj.grace > 0s) { // Object is in grace, deliver it // Automatically triggers a background fetch return (deliver); } // fetch & deliver once we get the result return (fetch); }
83. vcl_miss • Called after a cache lookup if the requested document was not found in the cache • synth(status code, reason) — Return the specified status code to the client and abandon the request • pass (→vcl_pass) — Switch to pass mode • fetch (→vcl_backend_fetch) — Retrieve the requested object from the backend • restart — Restart the transaction sub vcl_miss { return (fetch); }
84. vcl_hash • Defines the unique characteristics of a request ! sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } return (lookup); }
85. vcl_pass • Called upon entering pass mode. In this mode, the request is passed on to the backend, and the backend's response is passed on to the client, but is not entered into the cache • synth(status code, reason — Return the specified status code to the client and abandon the request • pass — Proceed with pass mode • restart — Restart the transaction
86. vcl_deliver • Called before a cached object is delivered to the client • deliver — Deliver the object to the client • restart — Restart the transaction
87. vcl_backend_fetch • Called before sending the backend request • fetch — Fetch the object from the backend. • abandon — Abandon the backend request and generates an error.
88. vcl_backend_error • This subroutine is called if we fail the backend fetch • deliver — Deliver the error • retry — Retry the backend transaction
89. Exercise: Add Hit/Miss Header
90. Expiring Cache
91. Cache Invalidation • Purging — Removing an object (and its variants) from the cache • Banning — Filter cached objects, preventing them from being served
92. Purging • Implement a special PURGE HTTP method acl purge { "localhost"; } ! sub vcl_recv { if (req.method == "PURGE") { if (!client.ip ~ purge) { return(synth(405,"Forbidden")); // Bail with an error } return (purge); } }
93. Banning • Bans act as filters on objects which tell Varnish not to return cached objects that meet certain criteria • Bans are checked when a cache hit is made • Bans can be set from CLI or with custom VCL • varnishadm ban req.req.url ~ “\.png$” bans all *.png files • Banned content remains in cache, memory is not freed
94. Exercise: Implement Purging
95. Directors & Backends
96. Backends backend default { .host = "varnish.tembies.com"; .port = "8180"; }
97. Multiple Backends backend default { .host = "varnish.tembies.com"; .port = "8180"; } backend varnishorg { .host = “www.varnish-cache.org“; .port = "80"; }
98. Hinting & Routing backend foo { .host = “foo.com"; .port = "80"; } backend bar { .host = “bar.com”; .port = "80"; } sub vcl_recv { if (req.http.host ~ "foo.com") { set req.backend_hint = foo; } elsif (req.http.host ~ "bar.com") { set req.backend_hint = bar; } }
99. Directors • Logical groupings of backends • Random or round-robin routing of requests • Set periodic health checks and manage health status of backends
100. probe health_check { .url = "/health"; .timeout = 1s; .interval = 5s; } ! backend server1 { .host = "varnish.tembies.com:8180"; .probe = health_check; } backend server2 { .host = "varnish.tembies.com:8181"; .probe = health_check; } backend server3 { .host = "varnish.tembies.com:8182"; .probe = health_check; }
101. import directors; ! sub vcl_init { new vdir = directors.round_robin(); vdir.add_backend(server1); vdir.add_backend(server2); vdir.add_backend(server3); } ! sub vcl_recv { #… call strip_req_headers; set req.backend_hint = vdir.backend(); }
102. Exercise: Create & Monitor Backends
103. Grace Mode
104. Grace Mode • “Backup” TTL set on objects that lets Varnish serve them even when they are stale, under certain circumstances • When “graced” content is served, Varnish automatically attempts to refresh it
106. Exercise: Grace all objects for 2 minutes
107. Rescuing Requests
108. Retry • Returning “retry” from (nearly) anywhere starts the VCL state engine from the top, with any changes to the request saved • Varnish can make intelligent decisions about whether or not to serve questionable content
109. sub vcl_backend_response { if (beresp.status == 200 && beresp.http.content-length == 0) { return(retry) } }
110. Edge Side Includes
111. What is ESI? • Simple markup language that enables content composition • Allows the combination of cached and uncached resources in to a single whole • Varnish implements a small subset, esi:include and esi:remove
112. Exercise: Turn on ESI
113. VMODs & Embedded C
114. VMODs • Extensions that provide additional functionality, loaded in VCL with the “load” command • Varnish comes with the VMOD “std” which provides useful helper functions • https://www.varnish-cache.org/vmods
115. vmod_std • std.querysort(req.url) — Sorts the query string • std.healthy(backend) — Returns TRUE is a backend is healthy • strstr(stringA, stringB) — Returns the substring if the second string is a substring of the first string • man vmod_std
116. Embedded C • C{ #include <dragons.h> }C • C code included in VCL is compiled with the VCL and dynamically linked to Varnish in memory. • Embedded C essentially becomes part of the Varnish process • If your code produces a segfault, Varnish will crash • Holy crap, don’t do this, why are you still reading this?
117. Further Reading • https://www.varnish-cache.org/docs/4.0/index.html • http://www.mobify.com/blog/beginners-guide-to-http-cache-headers/ • https://www.varnish-cache.org/trac/wiki/VCLExamples
118. Samantha Quiñones • samantha@tembies.com or samantha.quinones@teamaol.com • @ieatkillerbees • http://tembies.com • Feedback: https://joind.in/11907
119. Image Credits • Antonis Arestis photo - © 2012 Selene Alexia Christodoulou - CC • Futurama - © 1999 Twentieth-Century Fox • Modern Warehouse - © Axisdaman - CC • Begging Maltese - © Ed Yourdon - CC • Mud Bricks - © Whiteghost.ink - CC • Tetly’s Beer - © Reedy - CC • Film Director & Crew - © AiClassEland - CC • Ballerina - © David R. Tribble - CC • Stampede - © Andy Docker - CC • Beachy Head - © Papa Lima Whiskey - CC • Nuclear Test - © US NNSA