The Power of Cache in a Slow World

Most of us have some form of cache anxiety. We’re good at caching static content like images and scripts, but taking the next step - caching dynamic and user-specific content - is confusing and often requires a leap of faith. In this talk you’ll learn new and old strategies for caching dynamic content, HTTP performance rules to live by, and a better understanding of how to accelerate any application. This is just what the doctor ordered: a prescription for cache money.

Transcript

1. Michael May | Atlanta, GA | 2015-04-23 THE POWER OF CACHE IN A SLOW WORLD
2. @ohaimmay © 2015 All rights Reserved Listen up, ya’ll Cool things on The Internet: Routing, TCP, DNS Hypertext Transfer Protocol: Caching, Cookies Accelerating Apps: Varnish, Cache Strategies
3. @ohaimmay © 2015 All rights Reserved Customer Summit 2015 © 2015 All rights Reserved
4. @ohaimmay © 2015 All rights Reserved global content delivery network real-time stats, logging, & purge built on varnish, infinitely configurable HIRING! (come say hi) about:
5. @ohaimmay © 2015 All rights Reserved each cache node: 18 TB SSD 768 GB RAM 4x 10GbE ports > 50k requests/sec looking for a good time? watch “Scaling Networks through Software” from SREcon ‘15 Infrastructure
6. Background on The Internet
7. @ohaimmay © 2015 All rights Reserved Autonomous Systems (AS) Network(s) controlled by single operator for single entity e.g. Level3, Comcast, Fastly Each AS has an Autonomous System Number (ASN) ASN key component in route selection
8. @ohaimmay © 2015 All rights Reserved Internet Routing: BGP Border Gate Protocol (BGP) is used to choose “the best” path through the internet How it works: 1. Internet routes pulled from neighbors 2. Routes are analyzed to find “shortest AS path” (i.e. traverses the least number of AS) 3. “Shortest AS path” next-hop stored in routing table
9. @ohaimmay © 2015 All rights Reserved Internet Exchange (IX) Physical point where autonomous systems exchange traffic e.g. DE-CIX, AMS-IX, PAIX Peering Internet routes are exchanged between AS using a BGP session Internet Routing Peering Agreements Define amount of traffic that can be sent and how much it costs FYI: NANOG totally controls the internet :-)
10. @ohaimmay © 2015 All rights Reserved Internet Routing mmay:~ $ netstat -nr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 10.199.0.1 UGSc 42 0 en0 10.199/20 link#4 UCS 0 0 en0 10.199.0.1/32 link#4 UCS 1 0 en0 10.199.8.133/32 link#4 UCS 0 0 en0 10.215/24 172.16.102.129 UGSc 1 0 vmnet8 127 127.0.0.1 UCS 0 0 lo0 127.0.0.1 127.0.0.1 UH 2 3174 lo0 169.254 link#4 UCS 0 0 en0 172.16.102/24 link#14 UC 1 0 vmnet8 172.16.102.129 0:c:29:ec:24:c4 UHLWIi 3 239 vmnet8 1154 172.16.114/24 link#13 UC 1 0 vmnet1
11. @ohaimmay © 2015 All rights Reserved #justTCPthings Slow connection initialization 3-way Handshake - SYN, SYN-ACK, ACK TCP Window size Controls amount of data sent to client Slow Start Iteratively increase window size Server won’t overload client
12. @ohaimmay © 2015 All rights Reserved Domain Name System: DNS Human-style name to IP Address TTL - The length of time to cache a record Root name servers controlled by ICANN `dig` is handy to debug DNS issues
13. @ohaimmay © 2015 All rights Reserved mmay:~ $ dig fastly.com ; <<>> DiG 9.8.3-P1 <<>> fastly.com ;; QUESTION SECTION: ;fastly.com. IN A ;; ANSWER SECTION: fastly.com. 29 IN A 23.235.37.129 fastly.com. 29 IN A 23.235.33.129 fastly.com. 29 IN A 104.156.85.129 fastly.com. 29 IN A 104.156.81.129 ;; Query time: 126 msec ;; SERVER: 8.8.8.8#53
14. @ohaimmay © 2015 All rights Reserved mmay:~ $ dig api.fastly.com ; <<>> DiG 9.8.3-P1 <<>> api.fastly.com ;; QUESTION SECTION: ;api.fastly.com. IN A ;; ANSWER SECTION: api.fastly.com. 29 IN CNAME global-ssl.fastly.net. global-ssl.fastly.net. 10 IN CNAME fallback.global.fastly.net. fallback.global.fastly.net. 10 IN A 23.235.46.249 fallback.global.fastly.net. 10 IN A 199.27.76.249 ;; Query time: 77 msec ;; SERVER: 8.8.4.4#53(8.8.4.4)
15. @ohaimmay © 2015 All rights Reserved
16. @ohaimmay © 2015 All rights Reserved Moving data around the world takes time!
17. @ohaimmay © 2015 All rights Reserved Moving Data around the world takes time! Round Trip Time: Atlanta to Syndney (approx. 10,000 miles) Ideal: 80ms (speed of light) Real: ??
18. @ohaimmay © 2015 All rights Reserved ATL to SYD RTT mmay:~ $ ping cache-sydney001.hosts.fastly.net PING cache-sydney001.hosts.fastly.net: 56 data bytes 64 bytes received: icmp_seq=0 ttl=44 time=273.722 ms 64 bytes received: icmp_seq=1 ttl=43 time=347.763 ms 64 bytes received: icmp_seq=2 ttl=44 time=276.129 ms 64 bytes received: icmp_seq=3 ttl=43 time=369.563 ms 64 bytes received: icmp_seq=4 ttl=44 time=269.116 ms --- cache-sydney001.hosts.fastly.net ping statistics — 6 packets transmitted, 6 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 269.116/293.231/503.991/77.754ms
19. @ohaimmay © 2015 All rights Reserved From an airplane mmay:~ $ ping api.fastly.com PING fallback.global-ssl.fastly.net : 56 data bytes 64 bytes from host: icmp_seq=0 ttl=53 time=816.572 ms 64 bytes from host: icmp_seq=1 ttl=53 time=700.514 ms 64 bytes from host: icmp_seq=2 ttl=53 time=862.110 ms 64 bytes from host: icmp_seq=3 ttl=53 time=680.817 ms --- fallback.global-ssl.fastly.net ping statistics --- 5 packets transmitted, 4 packets received, 20.0% packet loss round-trip min/avg/max/stddev = 680.817/765.003/862.110/76.379 ms
20. @ohaimmay © 2015 All rights Reserved From an airplane mmay:~ $ traceroute api.fastly.com 1 cru.ltv (172.27.1.1) 0.888 ms 0.753 ms 0.709 ms 2 * * * 3 10.90.255.53 (10.90.255.53) 644.224 ms 649.387 ms 620.989 ms 4 10.90.1.32 (10.90.1.32) 777.188 ms 640.297 ms * 5 10.90.1.2 (10.90.1.2) 707.036 ms 899.143 ms 669.061 ms 6 10.90.10.2 (10.90.10.2) 640.856 ms 633.943 ms 743.556 ms 7 10.90.252.42 (10.90.252.42) 825.843 ms 716.490 ms 716.694 ms 8 10.90.252.41 (10.90.252.41) 819.711 ms 636.821 ms 796.497 ms 9 10.90.255.210 (10.90.255.210) 625.594 ms 668.321 ms 638.774 ms 10 10.90.255.221 (10.90.255.221) 629.221 ms 653.113 ms 967.769 ms 11 10.71.255.17 (10.71.255.17) 817.891 ms 844.275 ms 703.073 ms 12 edge.denver1.level3.net 729.202 ms 813.494 ms 825.576 ms
21. @ohaimmay © 2015 All rights Reserved ATL to SYD RTT (with 1337 h4x0rz) mmay:~ $ ping fast.mmay.net PING fallback.global-ssl.fastly.net (23.235.39.249): 56 data bytes 64 bytes from 23.235.39.249: icmp_seq=0 ttl=57 time=22.411 ms 64 bytes from 23.235.39.249: icmp_seq=2 ttl=57 time=30.053 ms 64 bytes from 23.235.39.249: icmp_seq=3 ttl=57 time=23.592 ms 64 bytes from 23.235.39.249: icmp_seq=4 ttl=57 time=23.180 ms 64 bytes from 23.235.39.249: icmp_seq=5 ttl=57 time=24.993 ms --- fallback.global-ssl.fastly.net ping statistics --- 6 packets transmitted, 6 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 22.411/27.042/38.025/5.510 ms
22. @ohaimmay © 2015 All rights Reserved Did we just beat the speed of light?
23. @ohaimmay © 2015 All rights Reserved Did we just beat the speed of light? OMG Tachyons are real!!!!!!
24. @ohaimmay © 2015 All rights Reserved NO
25. @ohaimmay © 2015 All rights Reserved NO We cheated
26. @ohaimmay © 2015 All rights Reserved CACHING
27. HTTP Hyper-wat?
28. @ohaimmay © 2015 All rights Reserved HTTP Headers: Cache-Control Cache-Control: public, max-age=3600, s-maxage=86400 s-maxage - surrogate caches only! (e.g. CDN) Grace Mode: stale-while-revalidate - serve stale while background revalidation stale-if-error - serve stale if origin 5xx’s
29. @ohaimmay © 2015 All rights Reserved HTTP Headers: Vary Vary: Accept-Encoding
30. @ohaimmay © 2015 All rights Reserved HTTP Headers: Vary Vary: Accept-Encoding This checks out
31. @ohaimmay © 2015 All rights Reserved HTTP Headers: Vary Vary: User-Agent
32. @ohaimmay © 2015 All rights Reserved HTTP Headers: Vary Vary: User-Agent NO, PLS!
33. @ohaimmay © 2015 All rights Reserved HTTP Cookies Set-Cookie - sent by server Cookie - sent by client If cookies exists in response, it will not be cached
34. HTTP Acceleration with Varnish
35. @ohaimmay © 2015 All rights Reserved Varnish - HTTP Reverse Proxy Cache Varnish ProTip: CNAME TLD to proxy cache/load balancer
36. @ohaimmay © 2015 All rights Reserved Varnish - HTTP Reverse Proxy Cache mmay:~ $ curl -svo /dev/null https://app.fastly.com/ * Connected to app.fastly.com (23.235.39.249) port 443 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.37.1 > Host: app.fastly.com > Accept: */* > < HTTP/1.1 200 OK < Status: 200 OK < Connection: keep-alive < Content-Type: text/html;charset=utf-8 < Via: 1.1 varnish < Cache-Control: max-age=0;s-maxage=0;private < X-Served-By: cache-atl6230-ATL < X-Cache: MISS < X-Cache-Hits: 0 < Vary: Accept-Encoding [data not shown] * Connection #0 to host app.fastly.com left intact
37. @ohaimmay © 2015 All rights Reserved Varnish - HTTP Reverse Proxy Cache mmay:~ $ curl -svo /dev/null https://app.fastly.com/ * Connected to app.fastly.com (23.235.39.249) port 443 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.37.1 > Host: app.fastly.com > Accept: */* > < HTTP/1.1 200 OK < Status: 200 OK < Connection: keep-alive < Content-Type: text/html;charset=utf-8 < Via: 1.1 varnish < Cache-Control: max-age=0;s-maxage=0;private < X-Served-By: cache-atl6230-ATL < X-Cache: MISS < X-Cache-Hits: 0 < Vary: Accept-Encoding [data not shown] * Connection #0 to host app.fastly.com left intact Proxied through varnish
38. @ohaimmay © 2015 All rights Reserved Varnish - HTTP Reverse Proxy Cache mmay:~ $ curl -svo /dev/null https://app.fastly.com/ * Connected to app.fastly.com (23.235.39.249) port 443 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.37.1 > Host: app.fastly.com > Accept: */* > < HTTP/1.1 200 OK < Status: 200 OK < Connection: keep-alive < Content-Type: text/html;charset=utf-8 < Via: 1.1 varnish < Cache-Control: max-age=0;s-maxage=0;private < X-Served-By: cache-atl6230-ATL < X-Cache: MISS < X-Cache-Hits: 0 < Vary: Accept-Encoding [data not shown] * Connection #0 to host app.fastly.com left intact Special cache Headers added by the proxy
39. @ohaimmay © 2015 All rights Reserved Varnish - HTTP Reverse Proxy Cache mmay:~ $ curl -svo /dev/null https://app.fastly.com/ * Connected to app.fastly.com (23.235.39.249) port 443 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.37.1 > Host: app.fastly.com > Accept: */* > < HTTP/1.1 200 OK < Status: 200 OK < Connection: keep-alive < Content-Type: text/html;charset=utf-8 < Via: 1.1 varnish < Cache-Control: max-age=0;s-maxage=0;private < X-Served-By: cache-atl6230-ATL < X-Cache: MISS < X-Cache-Hits: 0 < Vary: Accept-Encoding [data not shown] * Connection #0 to host app.fastly.com left intact Connection: Keep-Alive (DSA)
40. @ohaimmay © 2015 All rights Reserved Varnish Config Language (VCL) DSL for defining interactions with HTTP traffic Translated to C, executed on the cache Negligible performance impact! Lots of built-in libs for geoIP, time, etc.
41. @ohaimmay © 2015 All rights Reserved Varnish State Machine hello world
42. @ohaimmay © 2015 All rights Reserved Varnish Built-in Subroutines hello world vcl_recv - first routine called vcl_deliver - last routine called vcl_fetch - called when cache miss vcl_hit - called when cache hit vcl_error - called when fetch errors, can be used to do other cool stuff
43. @ohaimmay © 2015 All rights Reserved Varnish VCL Example hello world # block requests to Admin pages without Authorization sub vcl_recv { if (req.url ~ "^/admin" && !req.http.Authorization) { error 401 "Unauthorized"; } }
44. Synthetic Responses
45. @ohaimmay © 2015 All rights Reserved Cache a “Like” button
46. @ohaimmay © 2015 All rights Reserved Cache a “Like” button
47. @ohaimmay © 2015 All rights Reserved Traditional Reverse Proxy Varnish
48. @ohaimmay © 2015 All rights Reserved Reverse Proxy with Synthetic Response Varnish
49. @ohaimmay © 2015 All rights Reserved Synthetic Response VCL sub vcl_recv { if (req.url ~ "^/like" && req.request == "POST") { error 666 "OK"; } } sub vcl_error { if (obj.status == 666) { set obj.status = 200; set obj.http.Content-Type = "text/plain"; synthetic {"OK"}; } return (deliver); }
50. Dynamic Content Cache Strategies
51. @ohaimmay © 2015 All rights Reserved “Cacheable” Dynamic Content Changes unpredictably Static for some amount of time Change requires immediate update e.g. News articles, user comments, Inventory levels, wiki pages
52. @ohaimmay © 2015 All rights Reserved Dynamic Content Caching Strategies Short TTLs Edge Side Includes (ESI) AJAX API caching with event-driven purge
53. @ohaimmay © 2015 All rights Reserved Short TTLs Set Cache-Control max-age to something small Cache-Control: public, max-age=1 Pretty easy to implement and reason about Decent solution if you can’t invalidate content
54. @ohaimmay © 2015 All rights Reserved Edge Side Includes (ESI) Assemble dynamic content from independently generated and cached fragments Allows for “recursive” includes Think Rails ‘Russian Doll Caching’, but assembled at the proxy caching layer
55. @ohaimmay © 2015 All rights Reserved ESI: HTML <html> <navbar style=‘dat-nav-tho’> <a href=‘/stuff’>Cool Stuff</a> <a href=‘/lolz>Cats or something</a> <esi:include src=‘/profile’ /> </navbar> </html> — Request made by cache — $ curl -H ‘Cookie: my-cookie’ http://mmay.net/profile <h2>Michael May</h2> <img href=‘/profile/mmay.jpg’/>
56. @ohaimmay © 2015 All rights Reserved ESI: JSON { thing: { user: { <esi:include src=‘/profile.json’ /> } } } — Request made by cache — $ curl -H “Cookie: mmmtasty” https://mmay.net/profile.json { name: ‘mmay’, avatar: ‘static.mmay.net/cat.gif' }
57. @ohaimmay © 2015 All rights Reserved AJAX Move dynamic content to APIs Fetch data asynchronously client-side
58. @ohaimmay © 2015 All rights Reserved API Caching with event-driven purge Set Cache-Control with a reasonably long TTL Set Surrogate-Key(s) with unique cache key(s) Invalidate cached responses by Surrogate-Key purge
59. @ohaimmay © 2015 All rights Reserved API Caching with event-driven purge class LolController def index @lols = Lol.last(100) set_surrogate_key_header @products.table_key respond_with @products end end def update @lol = Lol.find(params[:id]) if @lol.update(params) @product.purge render @product end end github.com/fastly/fastly-rails
60. @ohaimmay © 2015 All rights Reserved Oh snap, what about CSRF? <%= csrf_meta_tag %> Can’t cache when CSRF token rendered in page server-side Alternatives: ESI the Cookie into the page Add CSRF token into cookie AJAX async CSRF tokens
61. @ohaimmay © 2015 All rights Reserved Caching when cookies present 1. Remove cookie and save in temp variable 2. Do normal cache lookup 3. Set cookie from temp var right before delivery hello world hello world
62. @ohaimmay © 2015 All rights Reserved Caching when cookies present (VCL) # Remove cookie, save temporarily sub vcl_recv { if (req.http.Cookie ~ "mycookie=") { set req.http.Tmp-Set-Cookie = req.http.Cookie; unset req.http.Cookie; } else { set req.hash_always_miss = true; } } # Add back saved cookie sub vcl_deliver { if (req.http.Tmp-Set-Cookie) { set resp.http.Set-Cookie = req.http.Tmp-Set-Cookie; } }
63. @ohaimmay © 2015 All rights Reserved Caching when cookies present (VCL) # If cache miss # Remove and save cookie from backend response sub vcl_fetch { if (beresp.http.Set-Cookie) { set req.http.Tmp-Set-Cookie = beresp.http.Set-Cookie; unset beresp.http.Set-Cookie; } }
64. @ohaimmay © 2015 All rights Reserved
65. @ohaimmay © 2015 All rights Reserved We made it!
66. @ohaimmay © 2015 All rights Reserved Take Aways Tools like curl, dig, traceroute are handy for debugging Tune Cache-Control headers (s-maxage, stale-if-error) Try replacing middleware with VCL for faster response time Choose a caching strategy that makes you comfortable Strip cookies to cache otherwise public content
67. Q & A @ohaimmay michael@fastly.com for questions, comments, jobs special thanks to: