Micro-caching in Nginx for High Performance

Slide 1

Slide 1 text

Micro-caching in Nginx

Slide 2

Slide 2 text

Micro-caching for ⚡

Slide 3

Slide 3 text

Micro-caching for 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴 👦👧👦👧👦👧👦 👩👨👩👨👩👨👩 👴👵👴👵👴👵👴

Slide 4

Slide 4 text

Credits: Bo-Yi Wu (Flickr)

Slide 5

Slide 5 text

😛

Slide 6

Slide 6 text

1 VPS

Slide 7

Slide 7 text

#perf Combine CSS & JS Minification Unused CSS CDN GZIP HTTP/2 Automation Bundling Caching VPS Image Optimisation DNS prefetch Responsive Images Lazy loading Image Sprites Async Service Workers Server Push Code Splitting No .htaccess

Slide 8

Slide 8 text

server #perf CDN GZIP HTTP/2 Caching VPS No .htaccess

Slide 9

Slide 9 text

A traditional CMS-powered server that generates pages

Slide 10

Slide 10 text

Caching

Slide 11

Slide 11 text

✋ What is Caching?

Slide 12

Slide 12 text

Work Output Input

Slide 13

Slide 13 text

🧑🍳 A Recipe Shop

Slide 14

Slide 14 text

  Kitchen     Recipe     Input   🍅🧀🍖 🧑🍳🍳 🍕

Slide 15

Slide 15 text

  Kitchen     Recipe   Cache   Input   🍅🧀🍖 🧑🍳🍳 🍕 🍅🧀🍖 🍕

Slide 16

Slide 16 text

  Kitchen     Recipe   Cache   Input   🍅🧀🍖 😴 🍕 🍅🧀🍖 🍕 ✅

Slide 17

Slide 17 text

  Kitchen     Recipe   Cache   Input   🍇💧 🍅🧀🍖 🍕 ❌

Slide 18

Slide 18 text

🧑🍳🍳   Kitchen     Recipe   Cache   Input   🍇💧 🍷 🍅🧀🍖 🍕 🍇💧 🍷

Slide 19

Slide 19 text

Cache Output Input

Slide 20

Slide 20 text

🧑🍳🌐🧑💻 A Recipe Website

Slide 21

Slide 21 text

Cache HTML Request

Slide 22

Slide 22 text

⚠ Caching cannot be simply flipped on, like a switch.

Slide 23

Slide 23 text

Update? Where? What? Cache Process

Slide 24

Slide 24 text

Store Filter Invalidate Cache Process

Slide 25

Slide 25 text

1. Filter Identify which responses should be cached

Slide 26

Slide 26 text

Types of responses

Slide 27

Slide 27 text

Static responses remain   the same for all requests e.g. a news article, portfolio site

Slide 28

Slide 28 text

Dynamic responses are unique for each request e.g. CSRF token (form page)

Slide 29

Slide 29 text

Contextual responses di ff er based on the request parameters e.g. Mobile/desktop variants, language variants

Slide 30

Slide 30 text

Private responses should bypass the cache e.g. control panel, user sessions

Slide 31

Slide 31 text

Dynamic Static Contextual Private e.g. contact form e.g. user sessions e.g. news article e.g. mobile/desktop variants

Slide 32

Slide 32 text

2. Store Which part of the stack should we cache?

Slide 33

Slide 33 text

Other Layers e.g. Database queries Network e.g. CDN Web Server e.g. Nginx e.g. CMS – full pages, template partials Application

Slide 34

Slide 34 text

3. Invalidate When & how to delete & update caches?

Slide 35

Slide 35 text

“There are only two hard things in Computer Science: cache invalidation and naming things.” Phil Karlton

Slide 36

Slide 36 text

Targeted All at once Time based Tags 🔥 ⏳ 🎯 🏷

Slide 37

Slide 37 text

Store Filter Invalidate Cache Process

Slide 38

Slide 38 text

Store Filter Invalidate Caching Strategy + +

Slide 39

Slide 39 text

Many possible strategies.

Slide 40

Slide 40 text

Microcaching in Nginx Strategy

Slide 41

Slide 41 text

Reduce server response time Goal

Slide 42

Slide 42 text

Handle high tra ff ic Goal

Slide 43

Slide 43 text

Reduce processing for a single request as much as possible CPU Memory Time

Slide 44

Slide 44 text

Ensure as many requests are served by the cache as possible

Slide 45

Slide 45 text

Strategy Store Filter Invalidate

Slide 46

Slide 46 text

Strategy Store Filter Invalidate

Slide 47

Slide 47 text

Language (eg PHP) CMS Template   Database Template NGINX Request HTML Server Lifecycle

Slide 48

Slide 48 text

FastCGI Cache Language (eg PHP) CMS Template   Database Template NGINX Request HTML

Slide 49

Slide 49 text

Strategy Store Filter Invalidate FastCGI Cache

Slide 50

Slide 50 text

Strategy Store Filter Invalidate FastCGI Cache

Slide 51

Slide 51 text

  Kitchen     Recipe   Cache   Input   🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 52

Slide 52 text

  CMS     HTML   FastCGI Cache   Request   🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 53

Slide 53 text

  CMS     HTML   FastCGI Cache   Request   🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕

Slide 54

Slide 54 text

  CMS     HTML   FastCGI Cache   Request   🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕

Slide 55

Slide 55 text

  CMS     HTML   FastCGI Cache   Request   🍅🧀🍖 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌

Slide 56

Slide 56 text

  CMS   ⏳⏳⏳   HTML   FastCGI Cache   Request   🍅🧀🍖 😑 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 57

Slide 57 text

If we delete invalid caches, the next requester has to wait… ⏳⏳⏳

Slide 58

Slide 58 text

… making that response slow. 😑

Slide 59

Slide 59 text

What if we let invalid caches   persist instead? 🤔

Slide 60

Slide 60 text

  CMS     HTML   FastCGI Cache   Request   🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟 ❌ Invalidate 🍕

Slide 61

Slide 61 text

  CMS     HTML   FastCGI Cache   Request   ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 62

Slide 62 text

  CMS     HTML   FastCGI Cache   Request   🍅🧀🍖 ⚠ ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 63

Slide 63 text

  CMS   ⏳   HTML   FastCGI Cache   Request   🍅🧀🍖 🚀 ⚠ ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 64

Slide 64 text

  CMS   ⏳⏳   HTML   FastCGI Cache   Request   ⚠ 🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 65

Slide 65 text

  CMS   ⏳⏳⏳   HTML   FastCGI Cache   Request   🍅🧀🍖 🍕 🍇💧 🍷 🍞🥬🥔🧀 🍔🍟

Slide 66

Slide 66 text

All responses* will now be fast… *Except the very first

Slide 67

Slide 67 text

When to invalidate?

Slide 68

Slide 68 text

Targeted All at once Time based Tags 🔥 ⏳ 🎯 🏷

Slide 69

Slide 69 text

Time based expiry allows us to keep stale caches around

Slide 70

Slide 70 text

But, no one wants to wait an hour for edits to show up

Slide 71

Slide 71 text

What if we brought the cache duration down to… one second?

Slide 72

Slide 72 text

OK — /home (expired) GET /home Visitors Nginx CMS GET /home OK — /home GET /home GET /home OK — /home (expired) OK — /home (expired) GET /home OK — /home (expired on arrival) 🔒⏳ 🔒⏳ GET /home

Slide 73

Slide 73 text

Slide 74

Slide 74 text

Slide 75

Slide 75 text

This is called Microcaching 🎉

Slide 76

Slide 76 text

Strategy Store Filter Invalidate FastCGI Cache

Slide 77

Slide 77 text

Strategy Store Filter Invalidate FastCGI Cache Every Second

Slide 78

Slide 78 text

Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

Slide 79

Slide 79 text

Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

Slide 80

Slide 80 text

nginx.conf http { … server { … location ~ \.php$ { … } } }

Slide 81

Slide 81 text

http { … server { … # Setup FastCGI Cach e # , , etc . fastcgi_cache_lock on; fastcgi_cache_use_stale updating; fastcgi_cache_background_update on; location ~ \.php$ { … # Qualifiers } } }

Slide 82

Slide 82 text

Dynamic Static Contextual Private e.g. contact form e.g. Control Panel e.g. news article e.g. mobile / desktop variants

Slide 83

Slide 83 text

1. Static – same response for each request Enable FastCGI Cache in the PHP location block e.g. news article

Slide 84

Slide 84 text

1. Static – same response for each request Enable FastCGI Cache in the PHP location block e.g. news article fastcgi_cache_valid 200 301 404 1s;

Slide 85

Slide 85 text

2. Dynamic – unique response for each request FastCGI automatically ignores responses with cookies e.g. contact form

Slide 86

Slide 86 text

2. Dynamic – unique response for each request FastCGI automatically ignores responses with cookies e.g. contact form fastcgi_ignore_headers Cache-Contro l Expires Set-Cookie ; fastcgi_hide_header Set-Cookie;

Slide 87

Slide 87 text

2. Dynamic – unique response for each request Also, bypass cache by sending a custom header from your CMS e.g. page that shows current time, contact form

Slide 88

Slide 88 text

2. Dynamic – unique response for each request Also, bypass cache by sending a custom header from your CMS e.g. page that shows current time, contact form # eg. (in Twig ) {% header "X-Accel-Expires: 0" %}

Slide 89

Slide 89 text

3. Contextual – di ff er based on request params Add relevant request parameters to the cache key e.g. desktop / mobile version

Slide 90

Slide 90 text

3. Contextual – di y "$scheme$request_method$host$request_uri";

Slide 91

Slide 91 text

3. Contextual – di y "$is_mobile$scheme$request_method$host$request

Slide 92

Slide 92 text

map $http_user_agent $is_mobile {  default 0;  "~*android.+mobile|avant[…]" 1;  "~*^(1207|6310|6590|3gso[…]" 1;  } map $http_accept_language $lang { default en ; ~es es ; ~fr fr ; }

Slide 93

Slide 93 text

4. Private – bypass the cache Don’t cache POST requests a) Form submissions

Slide 94

Slide 94 text

4. Private – bypass the cache Don’t cache POST requests a) Form submissions fastcgi_cache_methods GET HEAD POST;

Slide 95

Slide 95 text

4. Private – bypass the cache Exclude URLs with `dra ft ` param a) Form submissions b) Dra ft s

Slide 96

Slide 96 text

4. Private – bypass the cache Exclude URLs with `dra ; fastcgi_no_cache $arg_draft;

Slide 97

Slide 97 text

4. Private – bypass the cache Add a second location block for PHP without FastCGI caching a) Form submissions b) Dra ft s c) Control Panel

Slide 98

Slide 98 text

location ^~ /admin { try_files $uri $uri/ @phpfpm_nocache ; } location @phpfpm_nocache { # PH P # no FastCGI Cach e }

Slide 99

Slide 99 text

4. Private – bypass the cache Bypass if the session cookie is set a) Form submissions b) Dra ft s c) Control Panel d) Logged-in users

Slide 100

Slide 100 text

4. Private – bypass the cache Bypass if the session cookie is set a) Form submissions ; fastcgi_no_cache $cookie_1031b8c4[…];

Slide 101

Slide 101 text

Complete nginx.conf

Slide 102

Slide 102 text

http {  .. .   # Configure FastCGI cache  fastcgi_cache_path /var/run/fastcgicache levels=1:2 keys_zone=fastcgicache:100m inactive=1d ;   # Cache config  fastcgi_cache_lock on;  fastcgi_cache_use_stale updating error timeout invalid_header http_500;  fastcgi_cache_background_update on;  fastcgi_cache_methods GET HEAD;  fastcgi_cache_key "$scheme$request_method$host$request_uri";    server {  ...    # Ensure admin / user-specific requests skip the cache  # using a custom location handler  location ^~ /admin {  try_files $uri $uri/ @phpfpm_nocache;  } # continued... # ...continued location ~ \.php$ {  # Enable cache  fastcgi_cache fastcgicache;  fastcgi_ignore_headers Cache-Control Expires;  fastcgi_cache_valid 200 301 404 1s;  fastcgi_cache_bypass $arg_token $cookie_session_cookie;  fastcgi_no_cache $arg_token $cookie_session_cookie;  # Regular PHP handling ...  }    location @phpfpm_nocache {  # No FastCGI Cache  fastcgi_cache_bypass 1;  fastcgi_no_cache 1;  # Regular PHP handling ...   }  }  }

Slide 103

Slide 103 text

Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates

Slide 104

Slide 104 text

Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf

Slide 105

Slide 105 text

Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf

Slide 106

Slide 106 text

Strategy Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf

Slide 107

Slide 107 text

What about ⚡and 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧 👩👨👩👨👩👨👩👨 👴👵👴👵👴👵👴👵 👦👧👦👧👦👧👦👧

Slide 108

Slide 108 text

We ran some tests $ ab   -c 10 #concurrency   -t 30 #timelimit   guidingtech.com

Slide 109

Slide 109 text

Longest Response Time Uncached Microcached 96 milliseconds 526 milliseconds

Slide 110

Slide 110 text

Average Response Time Uncached Microcached 10 milliseconds 247 milliseconds

Slide 111

Slide 111 text

Completed Requests Uncached Microcached 29,389 1,214

Slide 112

Slide 112 text

Average Requests per Second Uncached Microcached 979.63 40.47

Slide 113

Slide 113 text

All on a $40/month   (or cheaper ) VPS

Slide 114

Slide 114 text

• Near instant responses • Exponentially scales up capacity to handle tra ff ic & spikes • All* requests get fast responses • Can make a cheap VPS fly *Except the very first Advantages

Slide 115

Slide 115 text

• Content may be stale • Nginx tinkering, not easy to debug Drawbacks

Slide 116

Slide 116 text

Recap

Slide 117

Slide 117 text

Microcaching in Nginx Store Filter Invalidate FastCGI Cache Every Second* *Background updates Nginx Conf

Slide 118

Slide 118 text

• Great #perf booster • Drastically reduces infrastructure cost • No negative impact on AX • No user or client complaints about stale content • No JavaScript usage or dependency

Slide 119

Slide 119 text

🙏 @rungta @miranj