Micro-caching in Nginx for High Performance

A talk on optimising server side performance using a micro-caching strategy implemented in Nginx. Watch on YouTube. The sample Nginx config file from this talk is available on GitHub.

Originally delivered at the 6th edition of the Bangalore Site Speed meetup group in August 2021.

Transcript

1. Micro-caching in Nginx
2. Micro-caching for ⚡
3. Micro-caching for 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴
4. Credits: Bo-Yi Wu (Flickr)
5. 😛
6. 1 VPS
7. #perf Combine CSS & JS Minification Unused CSS CDN GZIP HTTP/2 Automation Bundling Caching VPS Image Optimisation DNS prefetch Responsive Images Lazy loading Image Sprites Async Service Workers Server Push Code Splitting No .htaccess
8. server #perf CDN GZIP HTTP/2 Caching VPS No .htaccess
9. A traditional CMS-powered server that generates pages
10. Caching
11. ✋ What is Caching?
12. Work Output Input
13. 🧑🍳 A Recipe Shop
14. 
 Kitchen 
 
 Recipe 
 
 Input 
 🍅🧀🍖 🧑🍳🍳 🍕
15. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 
 🍅🧀🍖 🧑🍳🍳 🍕 🍅🧀🍖 🍕
16. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 
 🍅🧀🍖 😴 🍕 🍅🧀🍖 🍕 ✅
17. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 
 🍇💧 🍅🧀🍖 🍕 ❌
18. 🧑🍳🍳 
 Kitchen 
 
 Recipe 
 Cache 
 Input 
 🍇💧 🍷 🍅🧀🍖 🍕 🍇💧 🍷
19. Cache Output Input
20. 🧑🍳🌐🧑💻 A Recipe Website
21. Cache HTML Request
22. ⚠ Caching cannot be simply flipped on, like a switch.
23. Update? Where? What? Cache Process
24. Store Filter Invalidate Cache Process
25. 1. Filter Identify which responses should be cached
26. Types of responses
27. Static responses remain 
 the same for all requests e.g. a news article, portfolio site
28. Dynamic responses are unique for each request e.g. CSRF token (form page)
29. Contextual responses di ff er based on the request parameters e.g. Mobile/desktop variants, language variants
30. Private responses should bypass the cache e.g. control panel, user sessions
31. Dynamic Static Contextual Private e.g. contact form e.g. user sessions e.g. news article e.g. mobile/desktop variants
32. 2. Store Which part of the stack should we cache?
33. Other Layers e.g. Database queries Network e.g. CDN Web Server e.g. Nginx e.g. CMS – full pages, template partials Application
34. 3. Invalidate When & how to delete & update caches?
35. “There are only two hard things in Computer Science: cache invalidation and naming things.” Phil Karlton
36. Targeted All at once Time based Tags 🔥 ⏳ 🎯 🏷
37. Store Filter Invalidate Cache Process
38. Store Filter Invalidate Caching Strategy + +
39. Many possible strategies.
40. Microcaching in Nginx Strategy
41. Reduce server response time Goal
42. Handle high tra ff ic Goal
43. Reduce processing for a single request as much as possible CPU Memory Time
44. Ensure as many requests are served by the cache as possible
45. Strategy Store Filter Invalidate
46. Strategy Store Filter Invalidate
47. Language (eg PHP) CMS Template 
 Database Template NGINX Request HTML Server Lifecycle
48. FastCGI Cache Language (eg PHP) CMS Template 
 Database Template NGINX Request HTML
49. Strategy Store Filter Invalidate FastCGI Cache
50. Strategy Store Filter Invalidate FastCGI Cache
51. 
 Kitchen 
 
 Recipe 
 Cache 
 Input 
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
52. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
53. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕
54. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request 
 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕
55. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌
56. 
 CMS 
 ⏳⏳⏳ 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 😑 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
57. If we delete invalid caches, the next requester has to wait… ⏳⏳⏳
58. … making that response slow. 😑
59. What if we let invalid caches 
 persist instead? 🤔
60. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕
61. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request 
 ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
62. 
 CMS 
 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 ⚠ ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
63. 
 CMS 
 ⏳ 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 🚀 ⚠ ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
64. 
 CMS 
 ⏳⏳ 
 HTML 
 FastCGI Cache 
 Request 
 ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
65. 
 CMS 
 ⏳⏳⏳ 
 HTML 
 FastCGI Cache 
 Request 
 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟
66. All responses* will now be fast… *Except the very first
67. When to invalidate?
68. Targeted All at once Time based Tags 🔥 ⏳ 🎯 🏷
69. Time based expiry allows us to keep stale caches around
70. But, no one wants to wait an hour for edits to show up
71. What if we brought the cache duration down to… one second?
72. OK — /home (expired) GET /home Visitors Nginx CMS GET /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) 🔒⏳ 🔒⏳ GET /home
73. OK — /home (expired) GET /home Visitors Nginx CMS GET /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) 🔒⏳ 🔒⏳ GET /home Every request gets a fast response, no waiting
74. OK — /home (expired) GET /home Visitors Nginx CMS GET /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) 🔒⏳ 🔒⏳ GET /home Every request gets a fast response, no waiting CMS handles a fraction of the tra ff ic
75. This is called Microcaching 🎉
76. Strategy Store Filter Invalidate FastCGI Cache
77. Strategy Store Filter Invalidate FastCGI Cache Every Second
78. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates
79. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates
80. nginx.conf http { … server { … location ~ \.php$ { … } } }
81. http { … server { … # Setup FastCGI Cach e # <Path>, <Zone>, etc . fastcgi_cache_lock on; fastcgi_cache_use_stale updating; fastcgi_cache_background_update on; location ~ \.php$ { … # Qualifiers } } }
82. Dynamic Static Contextual Private e.g. contact form e.g. Control Panel e.g. news article e.g. mobile / desktop variants
83. 1. Static – same response for each request Enable FastCGI Cache in the PHP location block e.g. news article
84. 1. Static – same response for each request Enable FastCGI Cache in the PHP location block e.g. news article fastcgi_cache_valid 200 301 404 1s;
85. 2. Dynamic – unique response for each request FastCGI automatically ignores responses with cookies e.g. contact form
86. 2. Dynamic – unique response for each request FastCGI automatically ignores responses with cookies e.g. contact form fastcgi_ignore_headers Cache-Contro l Expires Set-Cookie ; fastcgi_hide_header Set-Cookie;
87. 2. Dynamic – unique response for each request Also, bypass cache by sending a custom header from your CMS e.g. page that shows current time, contact form
88. 2. Dynamic – unique response for each request Also, bypass cache by sending a custom header from your CMS e.g. page that shows current time, contact form # eg. (in Twig ) {% header "X-Accel-Expires: 0" %}
89. 3. Contextual – di ff er based on request params Add relevant request parameters to the cache key e.g. desktop / mobile version
90. 3. Contextual – di y "$scheme$request_method$host$request_uri";
91. 3. Contextual – di y "$is_mobile$scheme$request_method$host$request
92. map $http_user_agent $is_mobile {
 default 0;
 "~*android.+mobile|avant[…]" 1;
 "~*^(1207|6310|6590|3gso[…]" 1;
 } map $http_accept_language $lang { default en ; ~es es ; ~fr fr ; }
93. 4. Private – bypass the cache Don’t cache POST requests a) Form submissions
94. 4. Private – bypass the cache Don’t cache POST requests a) Form submissions fastcgi_cache_methods GET HEAD POST;
95. 4. Private – bypass the cache Exclude URLs with `dra ft ` param a) Form submissions b) Dra ft s
96. 4. Private – bypass the cache Exclude URLs with `dra ; fastcgi_no_cache $arg_draft;
97. 4. Private – bypass the cache Add a second location block for PHP without FastCGI caching a) Form submissions b) Dra ft s c) Control Panel
98. location ^~ /admin { try_files $uri $uri/ @phpfpm_nocache ; } location @phpfpm_nocache { # PH P # no FastCGI Cach e }
99. 4. Private – bypass the cache Bypass if the session cookie is set a) Form submissions b) Dra ft s c) Control Panel d) Logged-in users
100. 4. Private – bypass the cache Bypass if the session cookie is set a) Form submissions ; fastcgi_no_cache $cookie_1031b8c4[…];
101. Complete nginx.conf
102. http {
 .. . 
 # Configure FastCGI cache
 fastcgi_cache_path /var/run/fastcgicache levels=1:2 keys_zone=fastcgicache:100m inactive=1d ; 
 # Cache config
 fastcgi_cache_lock on;
 fastcgi_cache_use_stale updating error timeout invalid_header http_500;
 fastcgi_cache_background_update on;
 fastcgi_cache_methods GET HEAD;
 fastcgi_cache_key "$scheme$request_method$host$request_uri";
 
 server {
 ...
 
 # Ensure admin / user-specific requests skip the cache
 # using a custom location handler
 location ^~ /admin {
 try_files $uri $uri/ @phpfpm_nocache;
 } # continued... # ...continued location ~ \.php$ {
 # Enable cache
 fastcgi_cache fastcgicache;
 fastcgi_ignore_headers Cache-Control Expires;
 fastcgi_cache_valid 200 301 404 1s;
 fastcgi_cache_bypass $arg_token $cookie_session_cookie;
 fastcgi_no_cache $arg_token $cookie_session_cookie;
 # Regular PHP handling ...
 }
 
 location @phpfpm_nocache {
 # No FastCGI Cache
 fastcgi_cache_bypass 1;
 fastcgi_no_cache 1;
 # Regular PHP handling ... 
 }
 }
 }
103. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates
104. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf
105. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf
106. Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf
107. What about ⚡and 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧
108. We ran some tests $ ab 
 -c 10 #concurrency 
 -t 30 #timelimit 
 guidingtech.com
109. Longest Response Time Uncached Microcached 96 milliseconds 526 milliseconds
110. Average Response Time Uncached Microcached 10 milliseconds 247 milliseconds
111. Completed Requests Uncached Microcached 29,389 1,214
112. Average Requests per Second Uncached Microcached 979.63 40.47
113. All on a $40/month 
 (or cheaper ) VPS
114. • Near instant responses • Exponentially scales up capacity to handle tra ff ic & spikes • All* requests get fast responses • Can make a cheap VPS fly *Except the very first Advantages
115. • Content may be stale • Nginx tinkering, not easy to debug Drawbacks
116. Recap
117. Microcaching in Nginx Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf
118. • Great #perf booster • Drastically reduces infrastructure cost • No negative impact on AX • No user or client complaints about stale content • No JavaScript usage or dependency
119. 🙏 @rungta @miranj