Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Micro-caching in Nginx for High Performance

Micro-caching in Nginx for High Performance

A talk on optimising server side performance using a micro-caching strategy implemented in Nginx. Watch on YouTube. The sample Nginx config file from this talk is available on GitHub.

Originally delivered at the 6th edition of the Bangalore Site Speed meetup group in August 2021.

Prateek Rungta

August 30, 2021
Tweet

More Decks by Prateek Rungta

Other Decks in Technology

Transcript

  1. Micro-caching in Nginx

  2. Micro-caching for ⚡

  3. Micro-caching for 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩

    👴👵👴👵👴👵👴
  4. Credits: Bo-Yi Wu (Flickr)

  5. 😛

  6. 1 VPS

  7. #perf Combine CSS & JS Minification Unused CSS CDN GZIP

    HTTP/2 Automation Bundling Caching VPS Image Optimisation DNS prefetch Responsive Images Lazy loading Image Sprites Async Service Workers Server Push Code Splitting No .htaccess
  8. server #perf CDN GZIP HTTP/2 Caching VPS No .htaccess

  9. A traditional CMS-powered server that generates pages

  10. Caching

  11. ✋ What is Caching?

  12. Work Output Input

  13. 🧑🍳 A Recipe Shop

  14. 
 Kitchen 
 
 Recipe 
 
 Input 
 🍅🧀🍖

    🧑🍳🍳 🍕
  15. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 


    🍅🧀🍖 🧑🍳🍳 🍕 🍅🧀🍖 🍕
  16. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 


    🍅🧀🍖 😴 🍕 🍅🧀🍖 🍕 ✅
  17. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 


    🍇💧 🍅🧀🍖 🍕 ❌
  18. 🧑🍳🍳 
 Kitchen 
 
 Recipe 
 Cache 
 Input

    
 🍇💧 🍷 🍅🧀🍖 🍕 🍇💧 🍷
  19. Cache Output Input

  20. 🧑🍳🌐🧑💻 A Recipe Website

  21. Cache HTML Request

  22. ⚠ Caching cannot be simply flipped on, like a switch.

  23. Update? Where? What? Cache Process

  24. Store Filter Invalidate Cache Process

  25. 1. Filter Identify which responses should be cached

  26. Types of responses

  27. Static responses remain 
 the same for all requests e.g.

    a news article, portfolio site
  28. Dynamic responses are unique for each request e.g. CSRF token

    (form page)
  29. Contextual responses di ff er based on the request parameters

    e.g. Mobile/desktop variants, language variants
  30. Private responses should bypass the cache e.g. control panel, user

    sessions
  31. Dynamic Static Contextual Private e.g. contact form e.g. user sessions

    e.g. news article e.g. mobile/desktop variants
  32. 2. Store Which part of the stack should we cache?

  33. Other Layers e.g. Database queries Network e.g. CDN Web Server

    e.g. Nginx e.g. CMS – full pages, template partials Application
  34. 3. Invalidate When & how to delete & update caches?

  35. “There are only two hard things in Computer Science: cache

    invalidation and naming things.” Phil Karlton
  36. Targeted All at once Time based Tags 🔥 ⏳ 🎯

    🏷
  37. Store Filter Invalidate Cache Process

  38. Store Filter Invalidate Caching Strategy + +

  39. Many possible strategies.

  40. Microcaching in Nginx Strategy

  41. Reduce server response time Goal

  42. Handle high tra ff ic Goal

  43. Reduce processing for a single request as much as possible

    CPU Memory Time
  44. Ensure as many requests are served by the cache as

    possible
  45. Strategy Store Filter Invalidate

  46. Strategy Store Filter Invalidate

  47. Language (eg PHP) CMS Template 
 Database Template NGINX Request

    HTML Server Lifecycle
  48. FastCGI Cache Language (eg PHP) CMS Template 
 Database Template

    NGINX Request HTML
  49. Strategy Store Filter Invalidate FastCGI Cache

  50. Strategy Store Filter Invalidate FastCGI Cache

  51. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 


    🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  52. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request

    
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  53. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request

    
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕
  54. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request

    
 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕
  55. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request

    
 🍅🧀🍖 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌
  56. 
 CMS 
 ⏳⏳⏳ 
 HTML 
 FastCGI Cache 


    Request 
 🍅🧀🍖 😑 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  57. If we delete invalid caches, the next requester has to

    wait… ⏳⏳⏳
  58. … making that response slow. 😑

  59. What if we let invalid caches 
 persist instead? 🤔

  60. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request

    
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕
  61. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request

    
 ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  62. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request

    
 🍅🧀🍖 ⚠ ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  63. 
 CMS 
 ⏳ 
 HTML 
 FastCGI Cache 


    Request 
 🍅🧀🍖 🚀 ⚠ ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  64. 
 CMS 
 ⏳⏳ 
 HTML 
 FastCGI Cache 


    Request 
 ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  65. 
 CMS 
 ⏳⏳⏳ 
 HTML 
 FastCGI Cache 


    Request 
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
  66. All responses* will now be fast… *Except the very first

  67. When to invalidate?

  68. Targeted All at once Time based Tags 🔥 ⏳ 🎯

    🏷
  69. Time based expiry allows us to keep stale caches around

  70. But, no one wants to wait an hour for edits

    to show up
  71. What if we brought the cache duration down to… one

    second?
  72. OK — /home (expired) GET /home Visitors Nginx CMS GET

    /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) 🔒⏳ 🔒⏳ GET /home
  73. OK — /home (expired) GET /home Visitors Nginx CMS GET

    /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) 🔒⏳ 🔒⏳ GET /home Every request gets a fast response, no waiting
  74. OK — /home (expired) GET /home Visitors Nginx CMS GET

    /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) 🔒⏳ 🔒⏳ GET /home Every request gets a fast response, no waiting CMS handles a fraction of the tra ff ic
  75. This is called Microcaching 🎉

  76. Strategy Store Filter Invalidate FastCGI Cache

  77. Strategy Store Filter Invalidate FastCGI Cache Every Second

  78. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

  79. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

  80. nginx.conf http { … server { … location ~ \.php$

    { … } } }
  81. http { … server { … # Setup FastCGI Cach

    e # <Path>, <Zone>, etc . fastcgi_cache_lock on; fastcgi_cache_use_stale updating; fastcgi_cache_background_update on; location ~ \.php$ { … # Qualifiers } } }
  82. Dynamic Static Contextual Private e.g. contact form e.g. Control Panel

    e.g. news article e.g. mobile / desktop variants
  83. 1. Static – same response for each request Enable FastCGI

    Cache in the PHP location block e.g. news article
  84. 1. Static – same response for each request Enable FastCGI

    Cache in the PHP location block e.g. news article fastcgi_cache_valid 200 301 404 1s;
  85. 2. Dynamic – unique response for each request FastCGI automatically

    ignores responses with cookies e.g. contact form
  86. 2. Dynamic – unique response for each request FastCGI automatically

    ignores responses with cookies e.g. contact form fastcgi_ignore_headers Cache-Contro l Expires Set-Cookie ; fastcgi_hide_header Set-Cookie;
  87. 2. Dynamic – unique response for each request Also, bypass

    cache by sending a custom header from your CMS e.g. page that shows current time, contact form
  88. 2. Dynamic – unique response for each request Also, bypass

    cache by sending a custom header from your CMS e.g. page that shows current time, contact form # eg. (in Twig ) {% header "X-Accel-Expires: 0" %}
  89. 3. Contextual – di ff er based on request params

    Add relevant request parameters to the cache key e.g. desktop / mobile version
  90. 3. Contextual – di y "$scheme$request_method$host$request_uri";

  91. 3. Contextual – di y "$is_mobile$scheme$request_method$host$request

  92. map $http_user_agent $is_mobile {
 default 0;
 "~*android.+mobile|avant[…]" 1;
 "~*^(1207|6310|6590|3gso[…]" 1;


    } map $http_accept_language $lang { default en ; ~es es ; ~fr fr ; }
  93. 4. Private – bypass the cache Don’t cache POST requests

    a) Form submissions
  94. 4. Private – bypass the cache Don’t cache POST requests

    a) Form submissions fastcgi_cache_methods GET HEAD POST;
  95. 4. Private – bypass the cache Exclude URLs with `dra

    ft ` param a) Form submissions b) Dra ft s
  96. 4. Private – bypass the cache Exclude URLs with `dra

    ; fastcgi_no_cache $arg_draft;
  97. 4. Private – bypass the cache Add a second location

    block for PHP without FastCGI caching a) Form submissions b) Dra ft s c) Control Panel
  98. location ^~ /admin { try_files $uri $uri/ @phpfpm_nocache ; }

    location @phpfpm_nocache { # PH P # no FastCGI Cach e }
  99. 4. Private – bypass the cache Bypass if the session

    cookie is set a) Form submissions b) Dra ft s c) Control Panel d) Logged-in users
  100. 4. Private – bypass the cache Bypass if the session

    cookie is set a) Form submissions ; fastcgi_no_cache $cookie_1031b8c4[…];
  101. Complete nginx.conf

  102. http {
 .. . 
 # Configure FastCGI cache
 fastcgi_cache_path

    /var/run/fastcgicache levels=1:2 keys_zone=fastcgicache:100m inactive=1d ; 
 # Cache config
 fastcgi_cache_lock on;
 fastcgi_cache_use_stale updating error timeout invalid_header http_500;
 fastcgi_cache_background_update on;
 fastcgi_cache_methods GET HEAD;
 fastcgi_cache_key "$scheme$request_method$host$request_uri";
 
 server {
 ...
 
 # Ensure admin / user-specific requests skip the cache
 # using a custom location handler
 location ^~ /admin {
 try_files $uri $uri/ @phpfpm_nocache;
 } # continued... # ...continued location ~ \.php$ {
 # Enable cache
 fastcgi_cache fastcgicache;
 fastcgi_ignore_headers Cache-Control Expires;
 fastcgi_cache_valid 200 301 404 1s;
 fastcgi_cache_bypass $arg_token $cookie_session_cookie;
 fastcgi_no_cache $arg_token $cookie_session_cookie;
 # Regular PHP handling ...
 }
 
 location @phpfpm_nocache {
 # No FastCGI Cache
 fastcgi_cache_bypass 1;
 fastcgi_no_cache 1;
 # Regular PHP handling ... 
 }
 }
 }
  103. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

  104. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

    Nginx Conf
  105. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

    Nginx Conf
  106. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

    Nginx Conf
  107. What about ⚡and 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧 👩👨👩👨👩👨👩👨

    👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧
  108. We ran some tests $ ab 
 -c 10 #concurrency

    
 -t 30 #timelimit 
 guidingtech.com
  109. Longest Response Time Uncached Microcached 96 milliseconds 526 milliseconds

  110. Average Response Time Uncached Microcached 10 milliseconds 247 milliseconds

  111. Completed Requests Uncached Microcached 29,389 1,214

  112. Average Requests per Second Uncached Microcached 979.63 40.47

  113. All on a $40/month 
 (or cheaper ) VPS

  114. • Near instant responses • Exponentially scales up capacity to

    handle tra ff ic & spikes • All* requests get fast responses • Can make a cheap VPS fly *Except the very first Advantages
  115. • Content may be stale • Nginx tinkering, not easy

    to debug Drawbacks
  116. Recap

  117. Microcaching in Nginx Store Filter Invalidate FastCGI Cache Every Second*

    *Background updates Nginx Conf
  118. • Great #perf booster • Drastically reduces infrastructure cost •

    No negative impact on AX • No user or client complaints about stale content • No JavaScript usage or dependency
  119. 🙏 @rungta @miranj