Fortifying Craft for High Traffic

5d201d801f5a6e5a54c1178a20eb913d?s=47 Prateek Rungta
September 20, 2019

Fortifying Craft for High Traffic

In this talk, we go through a multi-tiered caching strategy using Craft CMS and NGINX that enables a single VPS to consistently deliver sub-200ms response times, even while handling loads of 10 to 100 concurrent requests per second.

Delivered in Montréal at Dot All 2019.

5d201d801f5a6e5a54c1178a20eb913d?s=128

Prateek Rungta

September 20, 2019
Tweet

Transcript

  1. Fortifying Craft for High Traffic

  2. Fortifying Craft for ⚡

  3. Fortifying Craft for

  4. Credits: Bo-Yi Wu (Flickr)

  5. 1 VPS

  6. #perf Combine CSS & JS Minification Unused CSS CDN GZIP

    HTTP/2 Automation Bundling Caching VPS Image Optimisation DNS prefetch Responsive Images Lazy loading Image Sprites Async Service Workers Server Push Code Splitting No .htaccess
  7. server #perf CDN GZIP HTTP/2 Caching VPS No .htaccess

  8. Caching

  9. Cash-ing / Kaysh-ing )

  10. ⚠ Caching cannot be simply flipped on, like a switch.

  11. ✋ What is Caching?

  12. Do Work Output Input

  13. Do Work Output Save in cache Input

  14. Do Work Output Save in cache Input Cached ✅

  15. Do Work Output Save in cache Input Cached Uncached ✅

  16. Do Work Output Save in cache Input Cached Uncached

  17. Cache Output Input

  18. Cache HTML Request

  19. Cache HTML Request

  20. Cache Update? What?

  21. Store Filter Invalidate Cache Process

  22. 1. Filter Identify which responses should be cached

  23. Types of responses

  24. Static responses remain
 the same for all requests e.g. a

    news article, portfolio site
  25. Dynamic responses are unique for each request e.g. CSRF token

    (form page)
  26. Contextual responses differ based on the request parameters e.g. Mobile/desktop

    variants, language variants
  27. Private responses should bypass the cache e.g. live preview, control

    panel, user sessions
  28. Dynamic Static Contextual Private e.g. contact form e.g. live preview

    e.g. news article e.g. mobile/desktop variants
  29. 2. Store Which part of the stack should we cache?

  30. Other Layers e.g. Database queries Network e.g. CDN Web Server

    e.g. Nginx e.g. Craft – full pages, template partials Application
  31. 3. Invalidate When & how to delete & update caches?

  32. “There are only two hard things in Computer Science: cache

    invalidation and naming things.” Phil Karlton
  33. Targeted All at once Time based Tags

  34. Store Filter Invalidate Cache Process

  35. Store Filter Invalidate Caching Strategy + +

  36. Many possible strategies.

  37. None
  38. Two approaches for Craft sites

  39. Full page caching in Craft Strategy #1

  40. Reduce server response time Goal

  41. Strategy #1 Store Filter Invalidate

  42. Strategy #1 Store Filter Invalidate

  43. Page Template Request HTML home.twig contact.twig news/_entry.twig Craft Lifecycle

  44. Page Template Request HTML home.twig Template
 Database Template Database Template


    Craft Lifecycle
  45. {% cache %} Page Template Request HTML home.twig Template
 Database

    Template Database
  46. home.twig contact.twig news/ _entry.twig {% cache %} {% cache %}

    {% cache %} {% endcache %} {% endcache %} {% endcache %}
  47. Strategy #1 Store Filter Invalidate

  48. Strategy #1 Store Filter Invalidate {% cache %}

  49. Strategy #1 Store Filter Invalidate {% cache %}

  50. “Your caches will automatically clear when any elements (entries, assets,

    etc.) within the tags are saved or deleted.” Craft Docs for {% cache %}
  51. Strategy #1 Store Filter Invalidate {% cache %}

  52. Strategy #1 Store Filter Invalidate {% cache %} Targeted

  53. “Possible side effects include stale content, excessively long-running background tasks,

    stuck tasks, and in rare cases, death.” Craft Docs for {% cache %} ⚠
  54. Death is, well, out of scope

  55. Let’s examine “excessively long-running background tasks, stuck tasks”

  56. #1 Happy Lager Craft Demo Site 20 pages 1 author,

    editing content monthly
  57. Small Website 20 pages 1 author, editing content monthly

  58. #2 Guiding Tech Web Publication 9000+ pages 3 million+ monthly

    visitors 10+ authors editing content daily
  59. Large Website 9000+ pages 3 million+ monthly visitors 10+ authors

    editing content daily
  60. 15 tracked elements 09 tracked queries 110 tracked elements 080

    tracked queries (per page average)
  61. 360,000+ tracked queries 92 tracked queries If half the site

    is cached…
  62. Deleting stale template caches Deleting stale template caches An Entry

    is edited… 92 tracked queries 360,000+ tracked queries
  63. ☑ Deleting stale … An Entry is edited… ⏳ Deleting

    stale … Deleting stale … Deleting stale … Deleting stale … Deleting stale … Deleting stale … Deleting stale … ☑ ☑ ☑
  64. ⏳ Deleting stale … Deleting stale … Deleting stale …

    Deleting stale … “excessively long- running background tasks, stuck tasks”
  65. Need better cache invalidation

  66. Switch to {% cacheflag %}

  67. Similar to {% cache %}, but with tag based invalidation

  68. 1. Assign tags to Content & Caches 2. When a

    piece of content is edited, go through its tags 3. Find all caches with any matching tags 4. Delete these caches
  69. • Entry • Category • Global • Asset • User,

    etc. • Sections • Category Groups • Global Sets • Asset Volumes • User Groups, etc. Element Types Groups Tag Options
  70. Content tags are auto assigned Element: Entries Section: News News

    Entry Global Set Work Entry Element: Globals GlobalS: Footer Element: Entrie Section: Work
  71. Manually specify template tags

  72. news/ _entry.twig home.twig {% cache %} {% cache %} {%

    cache %} {% endcache %} {% endcache %} {% endcache %} contact.twig
  73. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig
  74. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: News Element: Globals Element: Entries Element: Globals Section: Contact Element: Globals
  75. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: News Element: Globals Element: Entries Element: Globals Section: Contact Element: Globals If a news entry is edited
  76. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: Contact Element: Globals If a news entry is edited Element: Entries Element: Globals Section: News Element: Globals
  77. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: News Element: Globals Element: Entries Element: Globals Section: Contact Element: Globals If a work entry is edited
  78. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: Contact Element: Globals If a work entry is edited Element: Entries Element: Globals Section: News Element: Globals
  79. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: News Element: Globals Element: Entries Element: Globals Section: Contact Element: Globals If the footer globalset is edited
  80. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig Section: Contact Element: Globals If the footer globalset is edited Element: Entries Element: Globals Section: News Element: Globals
  81. Strategy #1 Store Filter Invalidate {% cache %} Targeted

  82. Strategy #1 Store Filter Invalidate {% cacheflag %} Tags

  83. Strategy #1 Store Filter Invalidate {% cacheflag %} Tags

  84. {% cacheflag %} {% cacheflag %} {% cacheflag %} {%

    endcacheflag %} {% endcacheflag %} {% endcacheflag %} news/ _entry.twig home.twig contact.twig
  85. news/ _entry.twig home.twig contact.twig {% extends '_init' %} {% extends

    '_init' %} {% extends '_init' %} _init.twig {% cacheflag %} {% endcacheflag %}
  86. Dynamic Static Contextual Private e.g. contact form e.g. Live Preview

    e.g. news article e.g. mobile / desktop variants
  87. 1. Static – same response for every request Simply extend

    _init.twig e.g. news article
  88. 2. Dynamic – unique response for each request {# contact.twig

    #} {% nocache %}
 {{ csrfInput() }}
 {% endnocache %} Use the No-Cache plugin e.g. contact form
  89. 3. Contextual – differ based on request params Use a

    cache key e.g. mobile / desktop version, language versions {# _init.twig #} {% cacheflag … using cacheKey … %}
  90. {# _init.twig #}
 {% set cacheKey = { 
 device:

    craft.app.request.isMobileBrowser
 ? 'mobile' : 'nonmobile', 
 user: currentUser
 ? 'user' : 'guest', 
 }|join(',') %}
  91. 4. Private – bypass the cache Cache Flag natively bypasses

    Live Preview and Draft URLs e.g. Live preview, Draft
  92. 4. Private – bypass the cache Use a boolean flag

    e.g. Form submissions, Live preview, Draft {# _init.twig #} {% cacheflag … if isCacheable %}
  93. {# _init.twig #}
 
 {% set isCacheable =
 not craft.app.request.isPost


    and isCacheable|default(true)
 %} {# _private.twig #}
 
 {% set isCacheable = false %}
  94. Complete _init.twig

  95. {# Cache Config #}
 {% set cacheflags = cacheflags ??

    'entries|assets|globals|categories|users' %}
 {% set cacheKeyPrefix = {
 device: craft.app.request.isMobileBrowser ? 'mobile' : 'nonmobile',
 user: currentUser ? 'user' : 'guest',
 }|join(',') %}
 {% set cacheableEnv = craft.app.request.isPost
 and not (doNotCache ?? false)
 and not craft.app.session.hasFlash('error')
 and not craft.app.session.hasFlash('notice')
 %}
 
 {# Figure out if page should be cached #}
 {%- if cacheableEnv %}
 
 {# If a cacheKey is set, use that to globally cache the rendered page #}
 {% if cacheKey ?? false %}
 {% cacheflag flagged cacheflags globally using key (cacheKeyPrefix ~ ':' ~ cacheKey) for 1 month %}
 {%- minify html %}
 {{ block('html') }}
 {% endminify -%}
 {% endcacheflag %} 
 {% else %} 
 {# No cacheKey set, cache the rendered page by url (not globally) #}
 {% cacheflag flagged cacheflags using key cacheKeyPrefix for 1 month %}
 {%- minify html %}
 {{ block('html') }}
 {% endminify -%}
 {% endcacheflag %}
 {% endif %}
 
 {% else %}
 {% block html %}{% endblock %}
 {% endif %}
  96. Strategy #1 Store Filter Invalidate {% cacheflag %} Tags

  97. Strategy #1 Store Filter Invalidate {% cacheflag %} Tags _init.twig

  98. Strategy #1 Store Filter Invalidate {% cacheflag %} Tags _init.twig

  99. ⚡ What about response times?

  100. Regular Cached 60ms 250ms Regular Cached 110ms 1550ms 4x 14x

  101. Advantages • Speed (up to 14 times faster TTFB) •

    Content is always fresh, never stale • Scales up to thousands of entries • Scales up to tens of editors / hour
  102. • Cache is busted frequently • Some users will encounter

    uncached responses • Developers have to specify tags Drawbacks
  103. Microcaching in Nginx Strategy #2

  104. Fast & stable handling
 of high traffic Goal

  105. Reduce processing for a single request as much as possible

    CPU Memory Time
  106. Ensure as many requests are served by the cache as

    possible
  107. Strategy #2 Store Filter Invalidate

  108. Strategy #2 Store Filter Invalidate

  109. PHP Craft Template
 Database Template Database ... NGINX Request HTML

    Server Lifecycle
  110. FastCGI Cache PHP Craft Template
 Database Template NGINX Request HTML

  111. Strategy #2 Store Filter Invalidate

  112. Strategy #2 Store Filter Invalidate FastCGI Cache

  113. Strategy #2 Store Filter Invalidate FastCGI Cache

  114. Once a cache is busted, it has to be re-created

    when requested…
  115. … making that response slow.

  116. PHP Response Update cache Request Cached Uncached ✅ ⏳⏳⏳

  117. Instead, we could serve stale content while the cache regenerates

  118. PHP ⏳⏳⏳ Response Update cache Request Cached Stale Use Stale

    Cache ✅
  119. All responses (except the very first) will now be fast…

  120. …as long as stale caches are available & not deleted.

  121. Time based expiry allows us to keep stale caches around

  122. But, no one wants to wait an hour for edits

    to show up
  123. What if we brought the cache duration down to… one

    second?
  124. OK — /home (expired) GET /home Visitors Nginx Craft GET

    /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) ⏳ ⏳ GET /home
  125. OK — /home (expired) GET /home Visitors Nginx Craft GET

    /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) ⏳ ⏳ GET /home Every request gets a fast response, no waiting
  126. OK — /home (expired) GET /home Visitors Nginx Craft GET

    /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) ⏳ ⏳ GET /home Every request gets a fast response, no waiting Craft handles a fraction of the traffic
  127. This is called Microcaching

  128. Strategy #2 Store Filter Invalidate FastCGI Cache

  129. Strategy #2 Store Filter Invalidate FastCGI Cache Every Second

  130. Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background

    updates
  131. Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background

    updates
  132. nginx.conf http { … server { … location ~ \.php$

    { … } } }
  133. http { … server { … # Setup FastCGI Cache

    # <Path>, <Zone>, etc. fastcgi_cache_lock on; fastcgi_cache_use_stale updating; fastcgi_cache_background_update on; location ~ \.php$ { … # Qualifiers } } }
  134. Dynamic Static Contextual Private e.g. contact form e.g. Live Preview

    e.g. news article e.g. mobile / desktop variants
  135. 1. Static – same response for each request Enable FastCGI

    Cache in the PHP location block e.g. news article
  136. 1. Static – same response for each request Enable FastCGI

    Cache in the PHP location block e.g. news article fastcgi_cache_valid 200 301 404 1s;
  137. 2. Dynamic – unique response for each request FastCGI automatically

    ignores responses with cookies e.g. contact form
  138. 2. Dynamic – unique response for each request FastCGI automatically

    ignores responses with cookies e.g. contact form fastcgi_ignore_headers Cache-Control Expires Set-Cookie; fastcgi_hide_header Set-Cookie;
  139. 2. Dynamic – unique response for each request Also, bypass

    cache by sending a custom header from Craft e.g. page that shows current time, contact form
  140. 2. Dynamic – unique response for each request Also, bypass

    cache by sending a custom header from Craft e.g. page that shows current time, contact form {# _init.twig #} {% header "X-Accel-Expires: 0" %}
  141. 3. Contextual – differ based on request params Add relevant

    request parameters to the cache key e.g. desktop / mobile version
  142. 3. Contextual – differ based on request params Add relevant

    request parameters to the cache key e.g. desktop / mobile version fastcgi_cache_key "$scheme$request_method$host$request_uri";
  143. 3. Contextual – differ based on request-er Add relevant request

    parameters to the cache key e.g. desktop / mobile version fastcgi_cache_key "$is_mobile$scheme$request_method$host$request
  144. map $http_user_agent $is_mobile {
 default 0;
 "~*android.+mobile|avant[…]" 1;
 "~*^(1207|6310|6590|3gso[…]" 1;


    } map $http_accept_language $lang { default en; ~es es; ~fr fr; }
  145. 4. Private – bypass the cache Don’t cache POST requests

    a) Form submissions
  146. 4. Private – bypass the cache Don’t cache POST requests

    a) Form submissions fastcgi_cache_methods GET HEAD POST;
  147. 4. Private – bypass the cache Exclude URLs with `token`

    param a) Form submissions b) Live Preview, Drafts
  148. 4. Private – bypass the cache Exclude URLs with `token`

    param a) Form submissions b) Live Preview, Drafts fastcgi_cache_bypass $arg_token; fastcgi_no_cache $arg_token;
  149. 4. Private – bypass the cache Add a second location

    block for PHP without FastCGI caching a) Form submissions b) Live Preview, Drafts c) Control Panel
  150. location ^~ /admin { try_files $uri $uri/ @phpfpm_nocache; } location

    ^~ /actions/ { … } location ^~ /index.php/admin { … } location ^~ /index.php/actions { … } location @phpfpm_nocache { # PHP # no FastCGI Cache }
  151. 4. Private – bypass the cache Bypass if Craft’s session

    cookie is set a) Form submissions b) Live Preview, Drafts c) Control Panel d) Logged-in users
  152. 4. Private – bypass the cache Bypass if Craft’s session

    cookie is set a) Form submissions b) Live Preview, Drafts c) Control Panel d) Logged-in users fastcgi_cache_bypass $cookie_1031b8c4[…]; fastcgi_no_cache $cookie_1031b8c4[…];
  153. Complete nginx.conf

  154. http {
 ...
 # Configure FastCGI cache
 fastcgi_cache_path /var/run/fastcgicache levels=1:2

    keys_zone=fastcgicache:100m inactive=1d;
 # Cache config
 fastcgi_cache_lock on;
 fastcgi_cache_use_stale updating error timeout invalid_header http_500;
 fastcgi_cache_background_update on;
 fastcgi_cache_methods GET HEAD;
 fastcgi_cache_key "$scheme$request_method$host$request_uri";
 
 
 server {
 ...
 
 # Craft-specific location handlers to ensure AdminCP requests route through index.php
 # If you change your `cpTrigger`, change it here as well
 location ^~ /admin {
 try_files $uri $uri/ @phpfpm_nocache;
 }
 location ^~ /actions/ {
 try_files $uri $uri/ @phpfpm_nocache;
 }
 location ^~ /index.php/admin {
 try_files $uri $uri/ @phpfpm_nocache;
 }
 location ^~ /index.php/actions {
 try_files $uri $uri/ @phpfpm_nocache;
 }
 location ~ \.php$ {
 # Enable cache
 fastcgi_cache fastcgicache;
 fastcgi_ignore_headers Cache-Control Expires;
 fastcgi_cache_valid 200 301 302 404 1s;
 fastcgi_cache_bypass $arg_token $cookie_1031b8c41dfff97a311a7ac99863bdc5_identity;
 fastcgi_no_cache $arg_token $cookie_1031b8c41dfff97a311a7ac99863bdc5_identity;
 ...
 }
 
 location @phpfpm_nocache {
 # No FastCGI Cache
 fastcgi_cache_bypass 1;
 fastcgi_no_cache 1;
 
 # PHP
 include fastcgi_params;
 fastcgi_index index.php;
 fastcgi_param SCRIPT_NAME /index.php;
 fastcgi_param SCRIPT_FILENAME $document_root/index.php;
 fastcgi_pass unix:/var/run/$APP_NAME.sock;
 fastcgi_intercept_errors off;
 fastcgi_connect_timeout 300;
 fastcgi_send_timeout 300;
 fastcgi_read_timeout 300;
 }
 }
 }
  155. Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background

    updates
  156. Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background

    updates Nginx Conf
  157. Strategy #2 Store Filter Invalidate FastCGI Cache Every Second* *Background

    updates Nginx Conf
  158. What about ⚡and

  159. We ran some tests $ ab
 -c 10 #concurrency
 -t

    30 #timelimit
 <url>
  160. Longest Response Time Uncached Microcached 96 milliseconds 526 milliseconds

  161. Average Response Time Uncached Microcached 10 milliseconds 247 milliseconds

  162. Completed Requests Uncached Microcached 29,389 1,214

  163. Average Requests per Second Uncached Microcached 979.63 40.47

  164. All on a $40/month
 (or cheaper ) VPS

  165. • Near instant responses • Exponentially scales up capacity to

    handle traffic & spikes • All* requests get fast responses • Can make a cheap VPS fly *Except the very first Advantages
  166. • Content may be stale • Nginx tinkering, not easy

    to debug Drawbacks
  167. Recap

  168. #1 Full Page Caching in Craft Store Filter Invalidate {%

    cacheflag %} Tags _init.twig
  169. #2 Microcaching in Nginx Store Filter Invalidate FastCGI Cache Every

    Second* *Background updates Nginx Conf
  170. • Both strategies are great #perf boosters • Combined, they

    can work wonders on Craft sites • Drastically reduces infrastructure cost • No negative impact on AX • No user or client complaints about stale content • No JavaScript usage or dependency
  171. Thank You @rungta