Caching the Uncacheable [Long Version]

Slide 1

Slide 1 text

Caching The Uncacheable Leveraging Your CDN To Cache Dynamic Content Hooman Behesh+, VP Technology

Slide 2

Slide 2 text

Dynamic Content Is Really Interes

Slide 3

Slide 3 text

What Is Dynamic Content? •  Stuﬀ that’s not sta+c! •  With web traﬃc, generally the base HTML –  Big deal because it’s blocking –  And some+mes a large object à longer download

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Blocking

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

What Is Dynamic Content? •  Stuﬀ that’s not sta+c! •  With web traﬃc, generally the base HTML –  Big deal because it’s blocking –  And some+mes a large object à longer download •  Could be other things too –  AJAX calls –  API calls •  More…

Slide 9

Slide 9 text

Classically, with dynamic content… Caching

Slide 10

Slide 10 text

Dynamic Content Caching Problems •  Serving stale pages – Lack of good invalida+on framework

Slide 11

Slide 11 text

Caching vs. Invalida

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

We tried…

Slide 16

Slide 16 text

Dynamic Content Caching Problems •  Serving stale pages – Lack of good invalida+on framework

Slide 17

Slide 17 text

Dynamic Content Caching Problems •  Serving stale pages – Lack of good invalida+on framework •  Real-‐+me visibility – Real-‐+me analy+cs/stats – Real-‐+me logging/tracking

Slide 18

Slide 18 text

So… Caching

Slide 19

Slide 19 text

CDNs and Dynamic Content •  Generally, handling dynamic content has been a maRer of transport – Op+mize from-‐origin delivery – “DSA” (Dynamic Site Accelera+on) – Middle mile op+miza+ons – TCP tweaks

Slide 20

Slide 20 text

Dynamic Content, Tradi

Slide 21

Slide 21 text

Dynamic Content, Tradi

Slide 22

Slide 22 text

Dynamic Content, Tradi

Slide 23

Slide 23 text

Actually… •  Dynamic content is more cacheable than we think •  Sta+c for shorter periods of +me •  Unpredictable invalida+on –  Standard HTTP caching rules aren’t good enough

Slide 24

Slide 24 text

A Lot BeLer! CDN Node CDN Node Client Origin

Slide 25

Slide 25 text

No CDN

Slide 26

Slide 26 text

CDN (sta

Slide 27

Slide 27 text

Blocking CDN (sta

Slide 28

Slide 28 text

CDN (sta

Slide 29

Slide 29 text

CDN (sta

Slide 30

Slide 30 text

So Many Benefits! •  Performance –  Faster +me to first byte –  Faster start render –  Happy users! •  Offload –  Less work for our servers –  Less bandwidth at origin

Slide 31

Slide 31 text

What would make it beLer?

Slide 32

Slide 32 text

Programma

Slide 33

Slide 33 text

Purge •  As a page gets published/updated, a purge command also gets published •  Must be Instant! •  Big problem with classic CDNs –  mul+-‐minute purges! –  Ex: News story, part 2

Slide 34

Slide 34 text

Instant? •  IS NOT: –  12 minutes! –  quick acknowledgment! •  IS: –  <1sec –  predictable and determinis+c behavior –  (has to also include propaga+on)

Slide 35

Slide 35 text

Power of the Purge! •  Purge dependencies – Surrogate Keys – Using tags/labels to purge en+re chunks of content at once

Slide 36

Slide 36 text

Not totally obsolete!

Slide 37

Slide 37 text

Example: CMS + Purge

Slide 38

Slide 38 text

WordPress: Before CDN Node

Slide 39

Slide 39 text

WordPress: Before CDN Node

Slide 40

Slide 40 text

WordPress: Before CDN Node

Slide 41

Slide 41 text

WordPress: Before CDN Node

Slide 42

Slide 42 text

WordPress: Before CDN Node Cache

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

WordPress: AVer CDN Node

Slide 45

Slide 45 text

WordPress: AVer CDN Node HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!

Slide 46

Slide 46 text

WordPress: AVer CDN Node HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!

Slide 47

Slide 47 text

WordPress: AVer CDN Node

Slide 48

Slide 48 text

WordPress: AVer CDN Node

Slide 49

Slide 49 text

WordPress: AVer CDN Node PURGE

Slide 50

Slide 50 text

WordPress: AVer CDN Node PURGE

Slide 51

Slide 51 text

WordPress: AVer CDN Node (Has to be instantaneous!) PURGE

Slide 52

Slide 52 text

WordPress: AVer CDN Node HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!

Slide 53

Slide 53 text

WordPress: AVer CDN Node

Slide 54

Slide 54 text

Example: customer1.js

Slide 55

Slide 55 text

Before CDN Node Origin http://www.3rdparty.com/customer1.js (Referer: www.customer1.com)

Slide 56

Slide 56 text

Before CDN Node Origin http://www.3rdparty.com/customer1.js

Slide 57

Slide 57 text

Before CDN Node Origin HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Before CDN Node Origin http://www.3rdparty.com/customer1.js (Ager 1 min)

Slide 61

Slide 61 text

Before CDN Node Origin http://www.3rdparty.com/customer1.js VALIDATION If-Modified-Since: Wed, 24 Sep 2014 19:51:30 GMT (Ager 1 min)

Slide 62

Slide 62 text

Before CDN Node Origin http://www.3rdparty.com/customer1.js 304 Not Modified (Ager 1 min)

Slide 63

Slide 63 text

Before CDN Node Origin http://www.3rdparty.com/customer1.js 304 Not Modified (Ager 1 min)

Slide 64

Slide 64 text

AVer CDN Node Origin http://www.3rdparty.com/customer1.js

Slide 65

Slide 65 text

AVer CDN Node Origin http://www.3rdparty.com/customer1.js

Slide 66

Slide 66 text

AVer CDN Node Origin HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js

Slide 67

Slide 67 text

Slide 68

Slide 68 text

Slide 69

Slide 69 text

AVer CDN Node Origin http://www.3rdparty.com/customer1.js (Ager 1 min)

Slide 70

Slide 70 text

AVer CDN Node Origin http://www.3rdparty.com/customer1.js (Ager 1 min)

Slide 71

Slide 71 text

This happens many many

Slide 72

Slide 72 text

AVer CDN Node Origin Customer1 changes conﬁg (Ager 1 min)

Slide 73

Slide 73 text

AVer CDN Node Origin PURGE customer1.js Customer1 changes conﬁg (Ager 1 min)

Slide 74

Slide 74 text

AVer CDN Node Origin PURGE customer1.js Customer1 changes conﬁg (Ager 1 min)

Slide 75

Slide 75 text

AVer CDN Node Origin http://www.3rdparty.com/customer1.js (Ager 1 min)

Slide 76

Slide 76 text

AVer CDN Node Origin http://www.3rdparty.com/customer1.js HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 (Ager 1 min)

Slide 77

Slide 77 text

Slide 78

Slide 78 text

More than just Invalida

Slide 79

Slide 79 text

The Inﬂuence of Clouds •  The CDN is an extension of the app •  No longer a black box •  Real-‐+me integra+on with the app •  Infrastructure as code – Your content => You need control

Slide 80

Slide 80 text

Control •  Programmability – Conﬁgura+on API – Invalida+on API – Instantaneous and real +me

Slide 81

Slide 81 text

Caching Control •  Granular caching – Dynamic caching – ﬂexible cache keys – etc •  Ex: Geo-‐based caching

Slide 82

Slide 82 text

Control at the Edge •  Moving app logic to the edge •  VCL – Varnish Conﬁgura+on Language – Script-‐like conﬁgura+on for func+onality at the edge

Slide 83

Slide 83 text

Visibility •  Real +me analy+cs –  Network stats –  HTTP stats (status codes , etc) –  Caching stats (hits, misses, etc) –  Stats API •  Logging –  Real +me logs –  Streaming to various log des+na+ons

Slide 84

Slide 84 text

Example: Beacon Termina

Slide 85

Slide 85 text

Before CDN Node Origin Log Analysis http://collector.site.com/beacon.img?a=1&b=2&c=3

Slide 86

Slide 86 text

Before CDN Node Origin Log Analysis http://collector.site.com/beacon.img?a=1&b=2&c=3

Slide 87

Slide 87 text

Before CDN Node Origin Log Analysis HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3

Slide 88

Slide 88 text

Slide 89

Slide 89 text

AVer CDN Node Origin http://collector.site.com/beacon.img?a=1&b=2&c=3

Slide 90

Slide 90 text

AVer CDN Node Origin HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3

Slide 91

Slide 91 text

AVer CDN Node Origin HTTP/1.1 204 No Content Date: Sat, 21 Jun 2014 23:21:12 GMT Server: Awesome Server Content-Length: 0 http://collector.site.com/beacon.img?a=1&b=2&c=3

Slide 92

Slide 92 text

AVer CDN Node Origin Syslog / S3 / FTP/etc http://collector.site.com/beacon.img?a=1&b=2&c=3

Slide 93

Slide 93 text

Example: Edge-‐generated Content

Slide 94

Slide 94 text

JSON Data Center ID CDN Node Origin http://www.site.com/which_datacenter.js

Slide 95

Slide 95 text

JSON Data Center ID CDN Node Origin { ‘datacenter’ : ‘SJC’ } http://www.site.com/which_datacenter.js

Slide 96

Slide 96 text

VCL Snippet

Slide 97

Slide 97 text

JSON Geo IP CDN Node Origin { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” } http://www.site.com/geo_ip.js

Slide 98

Slide 98 text

More Examples •  Caching with tracking cookies: –  hRp://www.fastly.com/blog/how-‐to-‐cache-‐with-‐ tracking-‐cookies •  API Caching: –  hRp://www.fastly.com/blog/api-‐caching-‐part-‐iii (part 3, with links to previous two parts) •  Log Streaming: –  hRp://www.fastly.com/blog/+ps-‐for-‐streaming-‐logs

Slide 99

Slide 99 text

Let’s Sum Up!

Slide 100

Slide 100 text

Summary •  Dynamic content can be cached –  We need instant purging –  We need real-‐+me logs and stats •  Real-‐+me integra+on of our CDN with our app is cool! –  Extensive/granular API to control the CDN –  Control and visibility at the edge lets us be really crea+ve •  Never use “Totally Long Time!” in a Cache-Control header!

Slide 101

Slide 101 text

Thank you! [email protected]