Edge Caching Dynamic Apps

slides from GORUCO 2014

Transcript

1. Edge Caching Dynamic Apps GORUCO | NYC | June 21, 2014 Michael May | @ehm_may
2. The World’s Smartest CDN
3. GoRuCo | 6/21/2014 | @ehm_may Overview • Caching • Content Delivery Networks (CDNs) • Modern Innovations in CDNs (dynamic edge caching) • Dynamic edge caching with Ruby/Rails
4. GoRuCo | 6/21/2014 | @ehm_may Caching Foundations • I have data stored somewhere (HDD, DB, Cloud) • I access that data often • Move data closer to where I am • Move data to faster storage ! • Side effect: Reduce load on original storage location
5. GoRuCo | 6/21/2014 | @ehm_may Terminology
6. GoRuCo | 6/21/2014 | @ehm_may Terminology CACHE HIT! requested data is contained in the cache
7. GoRuCo | 6/21/2014 | @ehm_may Terminology CACHE MISS! requested data is NOT in the cache !
8. GoRuCo | 6/21/2014 | @ehm_may Terminology HIT RATIO! percentage of accesses that result in cache hits
9. GoRuCo | 6/21/2014 | @ehm_may Terminology PURGE! data is removed from the cache !
10. GoRuCo | 6/21/2014 | @ehm_may Terminology Origin! HTTP application server (e.g. Rails server) !
11. GoRuCo | 6/21/2014 | @ehm_may Terminology Edge! HTTP caching server !
12. GoRuCo | 6/21/2014 | @ehm_may Caches operate with a limited amount of storage
13. GoRuCo | 6/21/2014 | @ehm_may Need a strategy for what to keep in cache
14. GoRuCo | 6/21/2014 | @ehm_may Least Recently Used is one type of strategy
15. GoRuCo | 6/21/2014 | @ehm_may LRU Interface • set(object) - O(1) - stores object, calls prune( ) (if cache is full)
16. GoRuCo | 6/21/2014 | @ehm_may LRU Interface • set(object) - O(1) - stores object, calls prune( ) (if cache is full) • get(key) - O(1) - returns object, moves it to the “head” of the cache
17. GoRuCo | 6/21/2014 | @ehm_may LRU Interface • set(object) - O(1) - stores object, calls prune( ) (if cache is full) • get(key) - O(1) - returns object, moves it to the “head” of the cache • prune( ) - removes object from “tail” of the cache
18. GoRuCo | 6/21/2014 | @ehm_may LRU Implementation • HashTable to store data - get( ) and set( ) functions • Doubly Linked List for tracking “time since last use”
19. GoRuCo | 6/21/2014 | @ehm_may Cache Hierarchy CPU/HARDWARE CACHE (L1-L4) MAIN MEMORY MEMCACHED / VARNISH CACHE CONTENT DELIVERY NETWORK SOFTWARE/APPLICATION CACHE (e.g. Rails Cache)
20. GoRuCo | 6/21/2014 | @ehm_may Cache Hierarchy CPU/HARDWARE CACHE (L1-L4) MAIN MEMORY MEMCACHED / VARNISH CACHE CONTENT DELIVERY NETWORK SOFTWARE/APPLICATION CACHE (e.g. Rails Cache)
21. GoRuCo | 6/21/2014 | @ehm_may Content Delivery Network is a globally distributed network of cache servers
22. GoRuCo | 6/21/2014 | @ehm_may
23. GoRuCo | 6/21/2014 | @ehm_may
24. GoRuCo | 6/21/2014 | @ehm_may
25. GoRuCo | 6/21/2014 | @ehm_may
26. GoRuCo | 6/21/2014 | @ehm_may
27. GoRuCo | 6/21/2014 | @ehm_may
28. GoRuCo | 6/21/2014 | @ehm_may
29. GoRuCo | 6/21/2014 | @ehm_may Content Delivery Networks
30. GoRuCo | 6/21/2014 | @ehm_may Traditionally CDNs only cache static content
31. GoRuCo | 6/21/2014 | @ehm_may Static Content •Images •Javascripts •Stylesheets
32. GoRuCo | 6/21/2014 | @ehm_may Edge caching static content is easy! The Asset Pipeline builds links with the asset_host ! Rails: # config/environments/production.rb config.action_controller.asset_host = “mycdn.coolcdn.com” ! Sinatra: (with something like sinatra_asset_pipeline) # in app.rb set :assets_host, ‘mycdn.coolcdn.com’
33. GoRuCo | 6/21/2014 | @ehm_may Cache invalidation is hard. Especially when your cache is globally distributed
34. GoRuCo | 6/21/2014 | @ehm_may Content Delivery Networks • How data/requests get to the edge • Controlling content with HTTP Headers • Lesser known CDN features
35. GoRuCo | 6/21/2014 | @ehm_may How data gets to “The Edge”
36. GoRuCo | 6/21/2014 | @ehm_may Push | Pull
37. GoRuCo | 6/21/2014 | @ehm_may PUSH CDN • You “push” content to the CDN • Usually happens when an asset is updated • Harder to keep assets in sync • Can strain origin server when pushing
38. GoRuCo | 6/21/2014 | @ehm_may PULL CDN • CDN “pulls” content from your origin server • Seamless updates when assets change • Small latency cost on first request • Acts as a reverse proxy
39. GoRuCo | 6/21/2014 | @ehm_may Reverse Proxy
40. GoRuCo | 6/21/2014 | @ehm_may 1st request 2nd, …, n requests
41. GoRuCo | 6/21/2014 | @ehm_may How requests get to “The Edge” * The edge closest to you…usually
42. GoRuCo | 6/21/2014 | @ehm_may Request Routing • Request made by client
43. GoRuCo | 6/21/2014 | @ehm_may Request Routing • Request made by client • Request triggers DNS lookup
44. GoRuCo | 6/21/2014 | @ehm_may Request Routing • Request made by client • Request triggers DNS lookup • DNS resolves request to a geographical region
45. GoRuCo | 6/21/2014 | @ehm_may Request Routing • Request made by client • Request triggers DNS lookup • DNS resolves request to a geographical region • Request forwarded to CDN Edge nearest to that region
46. GoRuCo | 6/21/2014 | @ehm_may Controlling content on your Content Delivery Network Using HTTP Headers
47. GoRuCo | 6/21/2014 | @ehm_may Cache-Control HTTP Headers
48. GoRuCo | 6/21/2014 | @ehm_may Cache-Control • Specify behavior intended to prevent caches from interfering with the request or response • Override the default caching algorithm • Specify what can be cached • Specify how long to cache
49. GoRuCo | 6/21/2014 | @ehm_may Cache-Control: max-age=3600
50. GoRuCo | 6/21/2014 | @ehm_may Cache-Control: max-age! ! Indicates how long to keep response fresh. Respected by all shared caches (CDN) and private caches (browser).
51. GoRuCo | 6/21/2014 | @ehm_may Use Cache-Control: max-age as an umbrella strategy to cache responses
52. GoRuCo | 6/21/2014 | @ehm_may Cache-Control: private, max-age=3600
53. GoRuCo | 6/21/2014 | @ehm_may Cache-Control: private! ! Indicates response MUST NOT be cached by shared caches. However, private caches may cache.
54. GoRuCo | 6/21/2014 | @ehm_may Use Cache-Control: private when responses contain sensitive user- specific data
55. GoRuCo | 6/21/2014 | @ehm_may Cache-Control: s-maxage=3600
56. GoRuCo | 6/21/2014 | @ehm_may Cache-Control: s-maxage! ! Indicates how long to keep response fresh. Respected ONLY by shared caches.
57. GoRuCo | 6/21/2014 | @ehm_may Use Cache-Control: s-maxage to specify different times to cache between shared and private caches
58. GoRuCo | 6/21/2014 | @ehm_may Cache-Control:no-cache Cache-Control:max-age=0 DON’T CACHE ME!
59. GoRuCo | 6/21/2014 | @ehm_may Surrogate-Control
60. GoRuCo | 6/21/2014 | @ehm_may “Surrogate” refers to reverse proxies
61. GoRuCo | 6/21/2014 | @ehm_may Surrogate-Control: max-age=3600
62. GoRuCo | 6/21/2014 | @ehm_may Surrogate-Control: max-age! ! Indicates how long to keep response fresh. Respected ONLY by reverse proxies.
63. GoRuCo | 6/21/2014 | @ehm_may Sneaky Example Cache-Control: max-age=3600 Surrogate-Control: max-age=2592000
64. GoRuCo | 6/21/2014 | @ehm_may Sneaky Example • Surrogate-Control takes priority over Cache-Control • Surrogate will cache for 1 year • Browser will cache for 30 minutes • Sometimes Surrogate-Control is stripped by the Surrogate
65. GoRuCo | 6/21/2014 | @ehm_may Interesting features in modern CDNs
66. GoRuCo | 6/21/2014 | @ehm_may Transmission Control Protocol
67. GoRuCo | 6/21/2014 | @ehm_may Transmission Control Protocol What web browsers use to connect to servers
68. GoRuCo | 6/21/2014 | @ehm_may How TCP establishes connections
69. GoRuCo | 6/21/2014 | @ehm_may TCP Handshake aka “3-Way” Handshake
70. GoRuCo | 6/21/2014 | @ehm_may HTTP Keep-Alive
71. GoRuCo | 6/21/2014 | @ehm_may HTTP Keep-Alive Header • Connection: Keep-Alive • TCP Session is kept open for an extended period • Eliminate costly TCP Handshake • In CDN Land edge nodes may keep connections alive with origin => faster cache misses (Dynamic Site Acceleration)
72. GoRuCo | 6/21/2014 | @ehm_may Instant Purging
73. GoRuCo | 6/21/2014 | @ehm_may
74. GoRuCo | 6/21/2014 | @ehm_may Instant Purging • PURGE request comes in to edge node • Edge node purges and passes to on to nearest neighbors • Recurse until everyone has purged brucespang.com/bimodal
75. GoRuCo | 6/21/2014 | @ehm_may Instant Purging • Purges happen globally in < 300ms on 95th percentile • Enables dynamic content caching at the edge brucespang.com/bimodal www.fastly.com/blog/building-fast-and-reliable-purging-system
76. GoRuCo | 6/21/2014 | @ehm_may Dynamic Caching • Explain process • Example using Rails Fragment caching • Map fragment caching to edge caching
77. GoRuCo | 6/21/2014 | @ehm_may Dynamic Content
78. GoRuCo | 6/21/2014 | @ehm_may Dynamic Content • Data that changes frequently
79. GoRuCo | 6/21/2014 | @ehm_may Dynamic Content • Data that changes frequently • Frequently does not mean continuously
80. GoRuCo | 6/21/2014 | @ehm_may Dynamic Content • Data that changes frequently • Frequently does not mean continuously • Periods of time where data is not being changed
81. GoRuCo | 6/21/2014 | @ehm_may Dynamic Caching is when you cache data in-between regeneration
82. GoRuCo | 6/21/2014 | @ehm_may Beneficial when dynamic data is requested multiple times
83. GoRuCo | 6/21/2014 | @ehm_may Dynamic Caching • API requests • Comments • News article headlines • E-commerce inventory • Search results
84. GoRuCo | 6/21/2014 | @ehm_may Dynamic Caching - How To 1. Create unique cache keys
85. GoRuCo | 6/21/2014 | @ehm_may Dynamic Caching - How To 1. Create unique cache keys 2. Bind data to cache keys
86. GoRuCo | 6/21/2014 | @ehm_may Dynamic Caching - How To 1. Create unique cache keys 2. Bind data to cache keys 3. Purge with cache key on data updates
87. GoRuCo | 6/21/2014 | @ehm_may Rails Fragment Caching
88. GoRuCo | 6/21/2014 | @ehm_may # products/index.html.erb <% Product.all.each do |p| %> <% cache(p.cache_key) do %> <%= link_to p.name, product_url(p) %> <% end %> <% end %> ! # in products_controller.rb expire_fragment(cache_key)
89. GoRuCo | 6/21/2014 | @ehm_may If you can Rails Fragment Cache, you can Dynamic Edge Cache
90. GoRuCo | 6/21/2014 | @ehm_may
91. GoRuCo | 6/21/2014 | @ehm_may
92. GoRuCo | 6/21/2014 | @ehm_may
93. GoRuCo | 6/21/2014 | @ehm_may
94. GoRuCo | 6/21/2014 | @ehm_may
95. GoRuCo | 6/21/2014 | @ehm_may How dynamic caching is done on the edge
96. GoRuCo | 6/21/2014 | @ehm_may 1. Create unique cache keys In CDN land, Surrogate-Keys are your cache keys
97. GoRuCo | 6/21/2014 | @ehm_may 1. Create unique cache keys Surrogate-Key is a HTTP Header that tells the cache to associate key(s) with response data
98. GoRuCo | 6/21/2014 | @ehm_may 2. Bind data to cache keys Use Surrogate-Key on HTTP GETs GET /product/123 ! HTTP/1.1 200 OK Content-Type: text/json Surrogate-Control: max-age=86400 Surrogate-Key: products product/123 {id: 123, name: “Kewl Keyboard”, price: “$124.99”}
99. GoRuCo | 6/21/2014 | @ehm_may 3. Purge with cache key on data updates PURGE Surrogate-Key on ! HTTP POST, PUT, DELETE PUT /product/123 {price: “$1.00”} ! # In PUT request handler PURGE /product/123
100. GoRuCo | 6/21/2014 | @ehm_may Implementing edge caching in Ruby on Rails
101. GoRuCo | 6/21/2014 | @ehm_may fastly-rails! github.com/fastly/fastly-rails ! Contains helpers for integrating with dynamic edge caching
102. GoRuCo | 6/21/2014 | @ehm_may Dynamic Edge Caching on Rails 1. Extend models with surrogate key instance methods
103. GoRuCo | 6/21/2014 | @ehm_may class Product < ActiveRecord::Base ! def resource_key “#{table_key}/#{id}" # e.g. “products/1234” end ! def table_key self.class.table_name # e.g. “products” end end
104. GoRuCo | 6/21/2014 | @ehm_may Dynamic Edge Caching on Rails 1. Extend models with surrogate key instance methods 2. Add ActionController helpers to set up headers
105. GoRuCo | 6/21/2014 | @ehm_may class ProductsController < ApplicationController before_filter :set_cache_control_headers, only: [:index, :show] ! def index @products = Product.all set_surrogate_key_header @product.table_key respond_with @products end ! def show @product = Products.find(params[:id]) set_surrogate_key_header @product.record_key respond_with @product end end
106. GoRuCo | 6/21/2014 | @ehm_may class ProductsController < ApplicationController before_filter :set_cache_control_headers, only: [:index, :show]! ! def index @products = Product.all set_surrogate_key_header @product.table_key respond_with @products end ! def show @product = Products.find(params[:id]) set_surrogate_key_header @product.record_key respond_with @product end end
107. GoRuCo | 6/21/2014 | @ehm_may def set_cache_control_headers(max_age = 3600) response.headers['Cache-Control'] = "public, no-cache" response.headers['Surrogate-Control'] = "max-age=#{max_age}" request.session_options[:skip] = true # don’t cache cookies end
108. GoRuCo | 6/21/2014 | @ehm_may class ProductsController < ApplicationController before_filter :set_cache_control_headers, only: [:index, :show] ! def index @products = Product.all set_surrogate_key_header @product.table_key respond_with @products end ! def show @product = Products.find(params[:id]) set_surrogate_key_header @product.record_key respond_with @product end end
109. GoRuCo | 6/21/2014 | @ehm_may class ProductsController < ApplicationController before_filter :set_cache_control_headers, only: [:index, :show] ! def index @products = Product.all set_surrogate_key_header @product.table_key! respond_with @products end ! def show @product = Products.find(params[:id]) set_surrogate_key_header @product.record_key! respond_with @product end end
110. GoRuCo | 6/21/2014 | @ehm_may def set_surrogate_key_header(key) response.headers['Surrogate-Key'] = key end
111. GoRuCo | 6/21/2014 | @ehm_may Dynamic Edge Caching on Rails 1. Extend models with surrogate key instance methods 2. Add ActionController helpers to set up headers 3. Add purge/purge_all instance methods to models
112. GoRuCo | 6/21/2014 | @ehm_may class ProductsController < ApplicationController ! def create @product = Product.new(params) if @product.save @product.purge_all! render @product end end
113. GoRuCo | 6/21/2014 | @ehm_may def update @product = Product.find(params[:id]) if @product.update(params) @product.purge! render @product end end
114. GoRuCo | 6/21/2014 | @ehm_may def delete @product = Product.find(params[:id]) if @product.destroy @product.purge_all! end end end
115. GoRuCo | 6/21/2014 | @ehm_may Dynamic Edge Caching on Rails 1. Extend models with surrogate key instance methods 2. Add ActionController helpers to set up headers 3. Add purge/purge_all instance methods to models
116. GoRuCo | 6/21/2014 | @ehm_may fastly-rails! github.com/fastly/fastly-rails
117. GoRuCo | 6/21/2014 | @ehm_may Use fastly-rails to iteratively set up dynamic edge caching
118. GoRuCo | 6/21/2014 | @ehm_may In Summary • Offload static assets onto a content delivery network
119. GoRuCo | 6/21/2014 | @ehm_may In Summary • Offload static assets onto a content delivery network • CDN Instant purging enables dynamic edge caching
120. GoRuCo | 6/21/2014 | @ehm_may In Summary • Offload static assets onto a content delivery network • CDN Instant purging enables dynamic edge caching • Use fastly-rails to integrate dynamic edge caching into Rails
121. Thank You github.com/fastly/fastly-rails michael@fastly.com | @ehm_may