Fortifying Craft for High Traffic

by Prateek Rungta

Slide 1

Slide 1 text

Fortifying Craft for High Traﬀic

Slide 2

Slide 2 text

Fortifying Craft for ⚡

Slide 3

Slide 3 text

Fortifying Craft for

Slide 4

Slide 4 text

Credits: Bo-Yi Wu (Flickr)

Slide 5

Slide 5 text

1 VPS

Slide 6

Slide 6 text

#perf Combine CSS & JS Minification Unused CSS CDN GZIP HTTP/2 Automation Bundling Caching VPS Image Optimisation DNS prefetch Responsive Images Lazy loading Image Sprites Async Service Workers Server Push Code Splitting No .htaccess

Slide 7

Slide 7 text

server #perf CDN GZIP HTTP/2 Caching VPS No .htaccess

Slide 8

Slide 8 text

Caching

Slide 9

Slide 9 text

Cash-ing / Kaysh-ing )

Slide 10

Slide 10 text

⚠ Caching cannot be simply flipped on, like a switch.

Slide 11

Slide 11 text

✋ What is Caching?

Slide 12

Slide 12 text

Do Work Output Input

Slide 13

Slide 13 text

Do Work Output Save in cache Input

Slide 14

Slide 14 text

Do Work Output Save in cache Input Cached ✅

Slide 15

Slide 15 text

Do Work Output Save in cache Input Cached Uncached ✅

Slide 16

Slide 16 text

Do Work Output Save in cache Input Cached Uncached

Slide 17

Slide 17 text

Cache Output Input

Slide 18

Slide 18 text

Cache HTML Request

Slide 19

Slide 19 text

Cache HTML Request

Slide 20

Slide 20 text

Cache Update? What?

Slide 21

Slide 21 text

Store Filter Invalidate Cache Process

Slide 22

Slide 22 text

1. Filter Identify which responses should be cached

Slide 23

Slide 23 text

Types of responses

Slide 24

Slide 24 text

Static responses remain  the same for all requests e.g. a news article, portfolio site

Slide 25

Slide 25 text

Dynamic responses are unique for each request e.g. CSRF token (form page)

Slide 26

Slide 26 text

Contextual responses diﬀer based on the request parameters e.g. Mobile/desktop variants, language variants

Slide 27

Slide 27 text

Private responses should bypass the cache e.g. live preview, control panel, user sessions

Slide 28

Slide 28 text

Dynamic Static Contextual Private e.g. contact form e.g. live preview e.g. news article e.g. mobile/desktop variants

Slide 29

Slide 29 text

2. Store Which part of the stack should we cache?

Slide 30

Slide 30 text

Other Layers e.g. Database queries Network e.g. CDN Web Server e.g. Nginx e.g. Craft – full pages, template partials Application

Slide 31

Slide 31 text

3. Invalidate When & how to delete & update caches?

Slide 32

Slide 32 text

“There are only two hard things in Computer Science: cache invalidation and naming things.” Phil Karlton

Slide 33

Slide 33 text

Targeted All at once Time based Tags

Slide 34

Slide 34 text

Store Filter Invalidate Cache Process

Slide 35

Slide 35 text

Store Filter Invalidate Caching Strategy + +

Slide 36

Slide 36 text

Many possible strategies.

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Two approaches for Craft sites

Slide 39

Slide 39 text

Full page caching in Craft Strategy #1

Slide 40

Slide 40 text

Reduce server response time Goal

Slide 41

Slide 41 text

Strategy #1 Store Filter Invalidate

Slide 42

Slide 42 text

Strategy #1 Store Filter Invalidate

Slide 43

Slide 43 text

Page Template Request HTML home.twig contact.twig news/_entry.twig Craft Lifecycle

Slide 44

Slide 44 text

Page Template Request HTML home.twig Template  Database Template Database Template  Craft Lifecycle

Slide 45

Slide 45 text

{% cache %} Page Template Request HTML home.twig Template  Database Template Database

Slide 46

Slide 46 text

home.twig contact.twig news/ _entry.twig {% cache %} {% cache %} {% cache %} {% endcache %} {% endcache %} {% endcache %}

Slide 47

Slide 47 text

Strategy #1 Store Filter Invalidate

Slide 48

Slide 48 text

Strategy #1 Store Filter Invalidate {% cache %}

Slide 49

Slide 49 text

Strategy #1 Store Filter Invalidate {% cache %}

Slide 50

Slide 50 text

“Your caches will automatically clear when any elements (entries, assets, etc.) within the tags are saved or deleted.” Craft Docs for {% cache %}

Slide 51

Slide 51 text

Strategy #1 Store Filter Invalidate {% cache %}

Slide 52

Slide 52 text

Strategy #1 Store Filter Invalidate {% cache %} Targeted

Slide 53

Slide 53 text

“Possible side eﬀects include stale content, excessively long-running background tasks, stuck tasks, and in rare cases, death.” Craft Docs for {% cache %} ⚠

Slide 54

Slide 54 text

Death is, well, out of scope

Slide 55

Slide 55 text

Let’s examine “excessively long-running background tasks, stuck tasks”

Slide 56

Slide 56 text

#1 Happy Lager Craft Demo Site 20 pages 1 author, editing content monthly

Slide 57

Slide 57 text

Small Website 20 pages 1 author, editing content monthly

Slide 58

Slide 58 text

#2 Guiding Tech Web Publication 9000+ pages 3 million+ monthly visitors 10+ authors editing content daily

Slide 59

Slide 59 text

Large Website 9000+ pages 3 million+ monthly visitors 10+ authors editing content daily

Slide 60

Slide 60 text

15 tracked elements 09 tracked queries 110 tracked elements 080 tracked queries (per page average)

Slide 61

Slide 61 text

360,000+ tracked queries 92 tracked queries If half the site is cached…

Slide 62

Slide 62 text

Deleting stale template caches Deleting stale template caches An Entry is edited… 92 tracked queries 360,000+ tracked queries

Slide 63

Slide 63 text

☑ Deleting stale … An Entry is edited… ⏳ Deleting stale … Deleting stale … Deleting stale … Deleting stale … Deleting stale … Deleting stale … Deleting stale … ☑ ☑ ☑

Slide 64

Slide 64 text

⏳ Deleting stale … Deleting stale … Deleting stale … Deleting stale … “excessively long- running background tasks, stuck tasks”

Slide 65

Slide 65 text

Need better cache invalidation

Slide 66

Slide 66 text

Switch to {% cacheflag %}

Slide 67

Slide 67 text

Similar to {% cache %}, but with tag based invalidation

Slide 68

Slide 68 text

1. Assign tags to Content & Caches 2. When a piece of content is edited, go through its tags 3. Find all caches with any matching tags 4. Delete these caches

Slide 69

Slide 69 text

• Entry • Category • Global • Asset • User, etc. • Sections • Category Groups • Global Sets • Asset Volumes • User Groups, etc. Element Types Groups Tag Options

Slide 70

Slide 70 text

Content tags are auto assigned Element: Entries Section: News News Entry Global Set Work Entry Element: Globals GlobalS: Footer Element: Entrie Section: Work

Slide 71

Slide 71 text

Manually specify template tags

Slide 72

Slide 72 text

news/ _entry.twig home.twig {% cache %} {% cache %} {% cache %} {% endcache %} {% endcache %} {% endcache %} contact.twig

Slide 73

Slide 73 text

{% cacheflag %} {% cacheflag %} {% cacheflag %} {% endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig

Slide 74

Slide 74 text

{% cacheflag %} {% cacheflag %} {% cacheflag %} {% endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: News Element: Globals Element: Entries Element: Globals Section: Contact Element: Globals

Slide 75

Slide 75 text

Slide 76

Slide 76 text

{% cacheflag %} {% cacheflag %} {% cacheflag %} {% endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: Contact Element: Globals If a news entry is edited Element: Entries Element: Globals Section: News Element: Globals

Slide 77

Slide 77 text

Slide 78

Slide 78 text

{% cacheflag %} {% cacheflag %} {% cacheflag %} {% endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: Contact Element: Globals If a work entry is edited Element: Entries Element: Globals Section: News Element: Globals

Slide 79

Slide 79 text

Slide 80

Slide 80 text

{% cacheflag %} {% cacheflag %} {% cacheflag %} {% endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: Contact Element: Globals If the footer globalset is edited Element: Entries Element: Globals Section: News Element: Globals

Slide 81

Slide 81 text

Strategy #1 Store Filter Invalidate {% cache %} Targeted

Slide 82

Slide 82 text

Strategy #1 Store Filter Invalidate {% cacheflag %} Tags

Slide 83

Slide 83 text

Strategy #1 Store Filter Invalidate {% cacheflag %} Tags

Slide 84

Slide 84 text

{% cacheflag %} {% cacheflag %} {% cacheflag %} {% endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig

Slide 85

Slide 85 text

news/ _entry.twig home.twig contact.twig {% extends '_init' %} {% extends '_init' %} {% extends '_init' %} _init.twig {% cacheflag %} {% endcacheflag %}

Slide 86

Slide 86 text

Dynamic Static Contextual Private e.g. contact form e.g. Live Preview e.g. news article e.g. mobile / desktop variants

Slide 87

Slide 87 text

1. Static – same response for every request Simply extend _init.twig e.g. news article

Slide 88

Slide 88 text

2. Dynamic – unique response for each request {# contact.twig #} {% nocache %}  {{ csrfInput() }}  {% endnocache %} Use the No-Cache plugin e.g. contact form

Slide 89

Slide 89 text

3. Contextual – diﬀer based on request params Use a cache key e.g. mobile / desktop version, language versions {# _init.twig #} {% cacheflag … using cacheKey … %}

Slide 90

Slide 90 text

{# _init.twig #}  {% set cacheKey = {   device: craft.app.request.isMobileBrowser  ? 'mobile' : 'nonmobile',   user: currentUser  ? 'user' : 'guest',   }|join(',') %}

Slide 91

Slide 91 text

4. Private – bypass the cache Cache Flag natively bypasses Live Preview and Draft URLs e.g. Live preview, Draft

Slide 92

Slide 92 text

4. Private – bypass the cache Use a boolean flag e.g. Form submissions, Live preview, Draft {# _init.twig #} {% cacheflag … if isCacheable %}

Slide 93

Slide 93 text

{# _init.twig #}    {% set isCacheable =  not craft.app.request.isPost  and isCacheable|default(true)  %} {# _private.twig #}    {% set isCacheable = false %}

Slide 94

Slide 94 text

Complete _init.twig

Slide 95

Slide 95 text

{# Cache Config #}  {% set cacheflags = cacheflags ?? 'entries|assets|globals|categories|users' %}  {% set cacheKeyPrefix = {  device: craft.app.request.isMobileBrowser ? 'mobile' : 'nonmobile',  user: currentUser ? 'user' : 'guest',  }|join(',') %}  {% set cacheableEnv = craft.app.request.isPost  and not (doNotCache ?? false)  and not craft.app.session.hasFlash('error')  and not craft.app.session.hasFlash('notice')  %}    {# Figure out if page should be cached #}  {%- if cacheableEnv %}    {# If a cacheKey is set, use that to globally cache the rendered page #}  {% if cacheKey ?? false %}  {% cacheflag flagged cacheflags globally using key (cacheKeyPrefix ~ ':' ~ cacheKey) for 1 month %}  {%- minify html %}  {{ block('html') }}  {% endminify -%}  {% endcacheflag %}   {% else %}   {# No cacheKey set, cache the rendered page by url (not globally) #}  {% cacheflag flagged cacheflags using key cacheKeyPrefix for 1 month %}  {%- minify html %}  {{ block('html') }}  {% endminify -%}  {% endcacheflag %}  {% endif %}    {% else %}  {% block html %}{% endblock %}  {% endif %}

Slide 96

Slide 96 text

Strategy #1 Store Filter Invalidate {% cacheflag %} Tags

Slide 97

Slide 97 text

Strategy #1 Store Filter Invalidate {% cacheflag %} Tags _init.twig

Slide 98

Slide 98 text

Strategy #1 Store Filter Invalidate {% cacheflag %} Tags _init.twig

Slide 99

Slide 99 text

⚡ What about response times?

Slide 100

Slide 100 text

Regular Cached 60ms 250ms Regular Cached 110ms 1550ms 4x 14x

Slide 101

Slide 101 text

Advantages • Speed (up to 14 times faster TTFB) • Content is always fresh, never stale • Scales up to thousands of entries • Scales up to tens of editors / hour

Slide 102

Slide 102 text

• Cache is busted frequently • Some users will encounter uncached responses • Developers have to specify tags Drawbacks

Slide 103

Slide 103 text

Microcaching in Nginx Strategy #2

Slide 104

Slide 104 text

Fast & stable handling  of high traﬀic Goal

Slide 105

Slide 105 text

Reduce processing for a single request as much as possible CPU Memory Time

Slide 106

Slide 106 text

Ensure as many requests are served by the cache as possible

Slide 107

Slide 107 text

Strategy #2 Store Filter Invalidate

Slide 108

Slide 108 text

Strategy #2 Store Filter Invalidate

Slide 109

Slide 109 text

PHP Craft Template  Database Template Database ... NGINX Request HTML Server Lifecycle

Slide 110

Slide 110 text

FastCGI Cache PHP Craft Template  Database Template NGINX Request HTML

Slide 111

Slide 111 text

Strategy #2 Store Filter Invalidate

Slide 112

Slide 112 text

Strategy #2 Store Filter Invalidate FastCGI Cache

Slide 113

Slide 113 text

Strategy #2 Store Filter Invalidate FastCGI Cache

Slide 114

Slide 114 text

Once a cache is busted, it has to be re-created when requested…

Slide 115

Slide 115 text

… making that response slow.

Slide 116

Slide 116 text

PHP Response Update cache Request Cached Uncached ✅ ⏳⏳⏳

Slide 117

Slide 117 text

Instead, we could serve stale content while the cache regenerates

Slide 118

Slide 118 text

PHP ⏳⏳⏳ Response Update cache Request Cached Stale Use Stale Cache ✅

Slide 119

Slide 119 text

All responses (except the very first) will now be fast…

Slide 120

Slide 120 text

…as long as stale caches are available & not deleted.

Slide 121

Slide 121 text

Time based expiry allows us to keep stale caches around

Slide 122

Slide 122 text

But, no one wants to wait an hour for edits to show up

Slide 123

Slide 123 text

What if we brought the cache duration down to… one second?

Slide 124

Slide 124 text

Slide 125

Slide 125 text

OK — /home (expired) GET /home Visitors Nginx Craft GET /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) ⏳ ⏳ GET /home Every request gets a fast response, no waiting

Slide 126

Slide 126 text

Slide 127

Slide 127 text

This is called Microcaching

Slide 128

Slide 128 text

Strategy #2 Store Filter Invalidate FastCGI Cache

Slide 129

Slide 129 text

Strategy #2 Store Filter Invalidate FastCGI Cache Every Second

Slide 130

Slide 130 text

Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background updates

Slide 131

Slide 131 text

Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background updates

Slide 132

Slide 132 text

nginx.conf http { … server { … location ~ \.php$ { … } } }

Slide 133

Slide 133 text

http { … server { … # Setup FastCGI Cache # , , etc. fastcgi_cache_lock on; fastcgi_cache_use_stale updating; fastcgi_cache_background_update on; location ~ \.php$ { … # Qualifiers } } }

Slide 134

Slide 134 text

Dynamic Static Contextual Private e.g. contact form e.g. Live Preview e.g. news article e.g. mobile / desktop variants

Slide 135

Slide 135 text

1. Static – same response for each request Enable FastCGI Cache in the PHP location block e.g. news article

Slide 136

Slide 136 text

1. Static – same response for each request Enable FastCGI Cache in the PHP location block e.g. news article fastcgi_cache_valid 200 301 404 1s;

Slide 137

Slide 137 text

2. Dynamic – unique response for each request FastCGI automatically ignores responses with cookies e.g. contact form

Slide 138

Slide 138 text

2. Dynamic – unique response for each request FastCGI automatically ignores responses with cookies e.g. contact form fastcgi_ignore_headers Cache-Control Expires Set-Cookie; fastcgi_hide_header Set-Cookie;

Slide 139

Slide 139 text

2. Dynamic – unique response for each request Also, bypass cache by sending a custom header from Craft e.g. page that shows current time, contact form

Slide 140

Slide 140 text

2. Dynamic – unique response for each request Also, bypass cache by sending a custom header from Craft e.g. page that shows current time, contact form {# _init.twig #} {% header "X-Accel-Expires: 0" %}

Slide 141

Slide 141 text

3. Contextual – diﬀer based on request params Add relevant request parameters to the cache key e.g. desktop / mobile version

Slide 142

Slide 142 text

3. Contextual – diﬀer based on request params Add relevant request parameters to the cache key e.g. desktop / mobile version fastcgi_cache_key "$scheme$request_method$host$request_uri";

Slide 143

Slide 143 text

3. Contextual – diﬀer based on request-er Add relevant request parameters to the cache key e.g. desktop / mobile version fastcgi_cache_key "$is_mobile$scheme$request_method$host$request

Slide 144

Slide 144 text

map $http_user_agent $is_mobile {  default 0;  "~*android.+mobile|avant[…]" 1;  "~*^(1207|6310|6590|3gso[…]" 1;  } map $http_accept_language $lang { default en; ~es es; ~fr fr; }

Slide 145

Slide 145 text

4. Private – bypass the cache Don’t cache POST requests a) Form submissions

Slide 146

Slide 146 text

4. Private – bypass the cache Don’t cache POST requests a) Form submissions fastcgi_cache_methods GET HEAD POST;

Slide 147

Slide 147 text

4. Private – bypass the cache Exclude URLs with `token` param a) Form submissions b) Live Preview, Drafts

Slide 148

Slide 148 text

4. Private – bypass the cache Exclude URLs with `token` param a) Form submissions b) Live Preview, Drafts fastcgi_cache_bypass $arg_token; fastcgi_no_cache $arg_token;

Slide 149

Slide 149 text

4. Private – bypass the cache Add a second location block for PHP without FastCGI caching a) Form submissions b) Live Preview, Drafts c) Control Panel

Slide 150

Slide 150 text

location ^~ /admin { try_files $uri $uri/ @phpfpm_nocache; } location ^~ /actions/ { … } location ^~ /index.php/admin { … } location ^~ /index.php/actions { … } location @phpfpm_nocache { # PHP # no FastCGI Cache }

Slide 151

Slide 151 text

4. Private – bypass the cache Bypass if Craft’s session cookie is set a) Form submissions b) Live Preview, Drafts c) Control Panel d) Logged-in users

Slide 152

Slide 152 text

4. Private – bypass the cache Bypass if Craft’s session cookie is set a) Form submissions b) Live Preview, Drafts c) Control Panel d) Logged-in users fastcgi_cache_bypass $cookie_1031b8c4[…]; fastcgi_no_cache $cookie_1031b8c4[…];

Slide 153

Slide 153 text

Complete nginx.conf

Slide 154

Slide 154 text

http {  ...  # Configure FastCGI cache  fastcgi_cache_path /var/run/fastcgicache levels=1:2 keys_zone=fastcgicache:100m inactive=1d;  # Cache config  fastcgi_cache_lock on;  fastcgi_cache_use_stale updating error timeout invalid_header http_500;  fastcgi_cache_background_update on;  fastcgi_cache_methods GET HEAD;  fastcgi_cache_key "$scheme$request_method$host$request_uri";      server {  ...    # Craft-specific location handlers to ensure AdminCP requests route through index.php  # If you change your `cpTrigger`, change it here as well  location ^~ /admin {  try_files $uri $uri/ @phpfpm_nocache;  }  location ^~ /actions/ {  try_files $uri $uri/ @phpfpm_nocache;  }  location ^~ /index.php/admin {  try_files $uri $uri/ @phpfpm_nocache;  }  location ^~ /index.php/actions {  try_files $uri $uri/ @phpfpm_nocache;  }  location ~ \.php$ {  # Enable cache  fastcgi_cache fastcgicache;  fastcgi_ignore_headers Cache-Control Expires;  fastcgi_cache_valid 200 301 302 404 1s;  fastcgi_cache_bypass $arg_token $cookie_1031b8c41dfff97a311a7ac99863bdc5_identity;  fastcgi_no_cache $arg_token $cookie_1031b8c41dfff97a311a7ac99863bdc5_identity;  ...  }    location @phpfpm_nocache {  # No FastCGI Cache  fastcgi_cache_bypass 1;  fastcgi_no_cache 1;    # PHP  include fastcgi_params;  fastcgi_index index.php;  fastcgi_param SCRIPT_NAME /index.php;  fastcgi_param SCRIPT_FILENAME $document_root/index.php;  fastcgi_pass unix:/var/run/$APP_NAME.sock;  fastcgi_intercept_errors off;  fastcgi_connect_timeout 300;  fastcgi_send_timeout 300;  fastcgi_read_timeout 300;  }  }  }

Slide 155

Slide 155 text

Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background updates

Slide 156

Slide 156 text

Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf

Slide 157

Slide 157 text

Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf

Slide 158

Slide 158 text

What about ⚡and

Slide 159

Slide 159 text

We ran some tests $ ab  -c 10 #concurrency  -t 30 #timelimit 

Slide 160

Slide 160 text

Longest Response Time Uncached Microcached 96 milliseconds 526 milliseconds

Slide 161

Slide 161 text

Average Response Time Uncached Microcached 10 milliseconds 247 milliseconds

Slide 162

Slide 162 text

Completed Requests Uncached Microcached 29,389 1,214

Slide 163

Slide 163 text

Average Requests per Second Uncached Microcached 979.63 40.47

Slide 164

Slide 164 text

All on a $40/month  (or cheaper ) VPS

Slide 165

Slide 165 text

• Near instant responses • Exponentially scales up capacity to handle traﬀic & spikes • All* requests get fast responses • Can make a cheap VPS fly *Except the very first Advantages

Slide 166

Slide 166 text

• Content may be stale • Nginx tinkering, not easy to debug Drawbacks

Slide 167

Slide 167 text

Recap

Slide 168

Slide 168 text

#1 Full Page Caching in Craft Store Filter Invalidate {% cacheflag %} Tags _init.twig

Slide 169

Slide 169 text

#2 Microcaching in Nginx Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf

Slide 170

Slide 170 text

• Both strategies are great #perf boosters • Combined, they can work wonders on Craft sites • Drastically reduces infrastructure cost • No negative impact on AX • No user or client complaints about stale content • No JavaScript usage or dependency

Slide 171

Slide 171 text

Thank You @rungta