Caching the Uncacheable [Long Version]

(Surge 2014) This is a longer version of our Velocity 2014 slides around caching dynamic content. Topic: In the past, CDNs have been used to cache and distribute static objects. But issues around invalidation, staleness, and lack of visibility have prevented us from using CDNs to fully leverage the benefits of caching when it comes to dynamic content. Today, using a real-time, modern CDN that provides instant cache invalidation and real-time analytics allows for instantaneous control over dynamic content caching.

Transcript

1. Caching  The  Uncacheable     Leveraging  Your  CDN  To  Cache  Dynamic  Content   Hooman  Behesh+,  VP  Technology  
2. Dynamic  Content  Is  Really   Interes<ng!  
3. What  Is  Dynamic  Content?   •  Stuff  that’s  not  sta+c!   •  With  web  traffic,  generally  the  base  HTML   –  Big  deal  because  it’s  blocking   –  And  some+mes  a  large  object  à  longer  download  
5. Blocking  
8. What  Is  Dynamic  Content?   •  Stuff  that’s  not  sta+c!   •  With  web  traffic,  generally  the  base  HTML   –  Big  deal  because  it’s  blocking   –  And  some+mes  a  large  object  à  longer  download   •  Could  be  other  things  too   –  AJAX  calls   –  API  calls   •  More…  
9. Classically,  with  dynamic  content…   Caching  
10. Dynamic  Content  Caching  Problems   •  Serving  stale  pages   – Lack  of  good  invalida+on  framework    
11. Caching     vs.     Invalida<on  
15. We  tried…  
16. Dynamic  Content  Caching  Problems   •  Serving  stale  pages   – Lack  of  good  invalida+on  framework    
17. Dynamic  Content  Caching  Problems   •  Serving  stale  pages   – Lack  of  good  invalida+on  framework   •  Real-­‐+me  visibility   – Real-­‐+me  analy+cs/stats   – Real-­‐+me  logging/tracking    
18. So…   Caching  
19. CDNs  and  Dynamic  Content   •  Generally,  handling  dynamic  content  has  been   a  maRer  of  transport   – Op+mize  from-­‐origin  delivery   – “DSA”  (Dynamic  Site  Accelera+on)   – Middle  mile  op+miza+ons   – TCP  tweaks  
20. Dynamic  Content,  Tradi<onally   CDN  Node   Client   Origin   Some  TCP  Tweaks  
21. Dynamic  Content,  Tradi<onally   CDN  Node   CDN  Node   Client   Origin   Lots  of  TCP  Tweaks  
22. Dynamic  Content,  Tradi<onally   •  We  some+mes  do  micro  caching  of  HTML   –  Short  TTL  for  HTML  content  via  Cache-­‐Control   –  Not  full  proof   •  Ex:  news  stories  faux-­‐pas!   •  ESI  (Edge  Side  Includes)   –  Par+al  caching   –  Hard  and  onerous    
23. Actually…   •  Dynamic  content  is  more  cacheable  than  we   think   •  Sta+c  for  shorter  periods  of  +me   •  Unpredictable  invalida+on   –  Standard  HTTP  caching  rules  aren’t  good  enough  
24. A  Lot  BeLer!   CDN  Node   CDN  Node   Client   Origin  
25. No  CDN  
26. CDN   (sta<c)  
27. Blocking   CDN   (sta<c)  
28. CDN   (sta<c)  
29. CDN   (sta<c   +   dynamic)  
30. So  Many  Benefits!   •  Performance   –  Faster  +me  to  first  byte   –  Faster  start  render   –  Happy  users!   •  Offload   –  Less  work  for  our  servers   –  Less  bandwidth  at  origin  
31. What  would  make  it  beLer?  
32. Programma<c  Invalida<on   •  Invalida+on  API   •  Granular   –  Specific  URL   –  URL  groups   –  All   –  Etc…  
33. Purge   •  As  a  page  gets  published/updated,  a  purge  command   also  gets  published   •  Must  be  Instant!   •  Big  problem  with  classic  CDNs     –  mul+-­‐minute  purges!   –  Ex:  News  story,  part  2  
34. Instant?   •  IS  NOT:   –  12  minutes!   –  quick  acknowledgment!   •  IS:   –  <1sec   –  predictable  and  determinis+c  behavior   –  (has  to  also  include  propaga+on)  
35. Power  of  the  Purge!   •  Purge  dependencies   – Surrogate  Keys   – Using  tags/labels  to  purge  en+re  chunks  of   content  at  once    
36. Not  totally  obsolete!  
37. Example:   CMS  +  Purge  
38. WordPress:  Before   CDN  Node  
39. WordPress:  Before   CDN  Node  
40. WordPress:  Before   CDN  Node  
41. WordPress:  Before   CDN  Node  
42. WordPress:  Before   CDN  Node   Cache  
44. WordPress:  AVer   CDN  Node  
45. WordPress:  AVer   CDN  Node   HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!
46. WordPress:  AVer   CDN  Node   HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!
47. WordPress:  AVer   CDN  Node  
48. WordPress:  AVer   CDN  Node  
49. WordPress:  AVer   CDN  Node   PURGE  
50. WordPress:  AVer   CDN  Node   PURGE  
51. WordPress:  AVer   CDN  Node   (Has  to  be  instantaneous!)   PURGE  
52. WordPress:  AVer   CDN  Node   HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!
53. WordPress:  AVer   CDN  Node  
54. Example:   customer1.js  
55. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Referer: www.customer1.com)
56. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js
57. Before   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
58. Before   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
59. Before   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
60. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ager  1  min)  
61. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js VALIDATION If-Modified-Since: Wed, 24 Sep 2014 19:51:30 GMT (Ager  1  min)  
62. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js 304 Not Modified (Ager  1  min)  
63. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js 304 Not Modified (Ager  1  min)  
64. AVer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js
65. AVer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js
66. AVer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
67. AVer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
68. AVer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
69. AVer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ager  1  min)  
70. AVer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ager  1  min)  
71. This  happens  many  many  <mes!   (many  many  happy  visitors!)  
72. AVer   CDN  Node   Origin   Customer1   changes  config   (Ager  1  min)  
73. AVer   CDN  Node   Origin   PURGE  customer1.js   Customer1   changes  config   (Ager  1  min)  
74. AVer   CDN  Node   Origin   PURGE  customer1.js   Customer1   changes  config   (Ager  1  min)  
75. AVer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ager  1  min)  
76. AVer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 (Ager  1  min)  
77. AVer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js (Ager  1  min)  
78. More  than  just  Invalida<on…  
79. The  Influence  of  Clouds   •  The  CDN  is  an  extension  of  the  app   •  No  longer  a  black  box   •  Real-­‐+me  integra+on  with  the  app   •  Infrastructure  as  code   – Your  content  =>  You  need  control  
80. Control   •  Programmability   – Configura+on  API   – Invalida+on  API   – Instantaneous  and  real  +me  
81. Caching  Control   •  Granular  caching   – Dynamic  caching   – flexible  cache  keys   – etc   •  Ex:  Geo-­‐based  caching  
82. Control  at  the  Edge   •  Moving  app  logic  to  the  edge   •  VCL   – Varnish  Configura+on  Language   – Script-­‐like  configura+on  for  func+onality  at  the   edge  
83. Visibility   •  Real  +me  analy+cs   –  Network  stats   –  HTTP  stats  (status  codes  ,  etc)   –  Caching  stats  (hits,  misses,  etc)   –  Stats  API   •  Logging   –  Real  +me  logs   –  Streaming  to  various  log  des+na+ons  
84. Example:   Beacon  Termina<on  at  the  Edge  
85. Before   CDN  Node   Origin   Log  Analysis   http://collector.site.com/beacon.img?a=1&b=2&c=3
86. Before   CDN  Node   Origin   Log  Analysis   http://collector.site.com/beacon.img?a=1&b=2&c=3
87. Before   CDN  Node   Origin   Log  Analysis   HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3
88. Before   CDN  Node   Origin   Log  Analysis   HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3
89. AVer   CDN  Node   Origin   http://collector.site.com/beacon.img?a=1&b=2&c=3
90. AVer   CDN  Node   Origin   HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3
91. AVer   CDN  Node   Origin   HTTP/1.1 204 No Content Date: Sat, 21 Jun 2014 23:21:12 GMT Server: Awesome Server Content-Length: 0 http://collector.site.com/beacon.img?a=1&b=2&c=3
92. AVer   CDN  Node   Origin   Syslog  /  S3  /  FTP/etc   http://collector.site.com/beacon.img?a=1&b=2&c=3
93. Example:   Edge-­‐generated  Content  
94. JSON  Data  Center  ID   CDN  Node   Origin   http://www.site.com/which_datacenter.js
95. JSON  Data  Center  ID   CDN  Node   Origin   { ‘datacenter’ : ‘SJC’ } http://www.site.com/which_datacenter.js
96. VCL  Snippet  
97. JSON  Geo  IP   CDN  Node   Origin   { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” } http://www.site.com/geo_ip.js
98. More  Examples   •  Caching  with  tracking  cookies:   –  hRp://www.fastly.com/blog/how-­‐to-­‐cache-­‐with-­‐ tracking-­‐cookies   •  API  Caching:   –  hRp://www.fastly.com/blog/api-­‐caching-­‐part-­‐iii  (part   3,  with  links  to  previous  two  parts)   •  Log  Streaming:   –  hRp://www.fastly.com/blog/+ps-­‐for-­‐streaming-­‐logs    
99. Let’s  Sum  Up!  
100. Summary   •  Dynamic  content  can  be  cached   –  We  need  instant  purging   –  We  need  real-­‐+me  logs  and  stats   •  Real-­‐+me  integra+on  of  our  CDN  with  our  app  is  cool!   –  Extensive/granular  API  to  control  the  CDN   –  Control  and  visibility  at  the  edge  lets  us  be  really  crea+ve   •  Never  use  “Totally Long Time!” in  a  Cache-Control header!    
101. Thank  you!   hooman@fastly.com