New Options for Caching Dynamic Content with Cloudflare Workers KOITABASHI Yoshitaka 2023/8/24 yoshii0110 1

2023/8/24 yoshii0110 2 KOITABASHI Yoshitaka @yoshii0110 Momento - community advocate KDDI Agile Development Center – software engineer Amazon EC2, AWS Fargate, AWS App Runner 🥷 🏢 💕

Static content vs Dynamic content • static content delivery web page The same content is delivered each time a user views it, without changing the displayed content. Common LP and personal blogs are also treated as static content, ex, HTML files, images, etc…. • Dynamic content delivery web page Deliver content that changes based on user-specific factors such as timing of access to a page, page location, device, etc… ex, Services where a lot of users publish/view articles, such as technical article platforms. 2023/8/24 yoshii0110 3

Modern front-end page rendering • Reactive Web Application • Client Side Rendering (CSR) => Sigle Page Application (SPA) - React, Vue, Angular or something like CSR • Server Side Rendering (SSR) – Next.js, Nuxt.js • Static Site Generator (SSG) Jamstack system, Gatsby, Hugo, Next.js, Nuxt.js, etc. • Incremental Static Regeneration (ISR) – Next.js (Features added in version 9.4) 2023/8/24 yoshii0110 4

SSR (Server Side Rendering) • Returns dynamically generated HTML in response to requests. • JavaScript, virtual DOM etc... are also used on the server side. • The disadvantage is that the server side is heavy. if API communication, etc... is used, response time is long. 2023/8/24 yoshii0110 5

SSG (Static Site Generator) • Pre-generated HTML is returned in response to a request. • SSG generates HTML at build time. • Delivery is very fast, but page content cannot be changed dynamically after deployment. 2023/8/24 yoshii0110 6

ISR (Incremental Static Regeneration) • Returns statically built pages in response to requests. • When the expiration date is exceeded, the static page is regenerated asynchronously by SSR. • While utilizing the cache, static pages can be automatically updated, and if a request is made again after a certain period of time, the content of the next and subsequent pages will be built and updated. 2023/8/24 yoshii0110 7

Caching of static content • For example, a caching strategy to consider when you want to shorten the response time to return static content such as HTML, CSS, and JS images created by SSG is to use a CDN (content delivery network). • CDN is an image that prevents the concentration of access to the origin server and directs them to geographically close edge locations (edge servers). • Edge locations are located around the world and are more distributed than data centers in public cloud regions. 2023/8/24 yoshii0110 8

Standard CDN Usage • Often, when an edge location is accessed by a user, it refers to the origin server and caches the content for the next and subsequent accesses. • On subsequent accesses, the cached content is delivered to the user. 2023/8/24 yoshii0110 9

Modern Web Applications and CDN • In recent years, with the many page rendering mechanisms introduced earlier, the work done by javascript on the browser side has increased, and the size of js files has also grown. • Even if SSG is used, CDN is important when delivering HTML and javascript, which have grown in size, to the user more quickly. • Recently, there are services that can be used without the need to be aware of CDN. • Vercel • AWS Amplify 2023/8/24 yoshii0110 10

Cache dynamic content • For example, there are patterns such as SSR/ISR where the content changes each time it is accessed, or where the content on a web application is changed and generated by some script or trigger. • For dynamic content delivery, in order to reduce response time to the user, it would be nice if scripts could be executed on the CDN, rather than having to access a more distant origin server each time the user accesses the site, wouldn't it? • This speeds up web applications that handle dynamic content because dynamic content is "cached" and thus response time to client requests is reduced. 2023/8/24 yoshii0110 11

Application execution on CDN edge servers • In recent years, services that allow javascript and other applications to run on the edge of a CDN have become popular. • The reason we are talking about this is that the ability to run applications on the edge affects the caching of dynamic content. • It is possible to generate dynamic content based on user-initiated events, web server-initiated events, etc., and serve it to the client, modify static content, and manipulate the cache. • Cloudflare Workers • Vercel Edge Functions • AWS CloudFront Functions • AWS Lambda@Edge 2023/8/24 yoshii0110 12 引用: https://blog.cloudflare.com/eliminating-cold-starts-with-cloudflare-workers/

What we want to achieve this time • Follow @yusukebe's "Dynamic Content Storing" inspired by Stale-While- Revalidate • Implement this using hono, Cloudflare Workers, and momento cache 2023/8/24 yoshii0110 13 https://yusukebe.com/posts/2022/dcs/ https://datatracker.ietf.org/doc/html/rfc5861

Cache Strategy: Stale-While-Revalidate • The web uses HTTP headers to control caching to control browsers and proxies. • Stale-While-Revalidate is a specification for cache control using such headers. • We often use Cache-Control, Expires, Etag, Last-Modified, and so on in cache specification. • When Stale-While-Revalidate is used, the movement is such that the content is displayed from the cache, but the cache is updated asynchronously behind the scenes. 2023/8/24 yoshii0110 14 https://docs.fastly.com/ja/guides/serving-stale-content https://datatracker.ietf.org/doc/html/rfc5861

Stale-While-Revalidate flow 2023/8/24 yoshii0110 15 https://blog.jxck.io/entries/2016-04-16/stale-while-revalidate.html 従来: max-age Stale-While-Revalidate

What is Cloudflare Workers / hono? • Cloudflare Workers • Cloudflare's Serverless Services • Runs code on CDN edge servers, unlike AWS Lambda, Google Cloud Functions, etc. • Provides an independent execution environment for each request using isolate, a V8 engine • hono • A small, simple, and super-fast web framework developed for the edge • repo: https://github.com/honojs/hono • docs: https://hono.dev/ 2023/8/24 yoshii0110 16

What is Momento? • Momento is a serverless service that creates a comfortable world for application engineers and provides a mechanism for faster, more secure, and more stable delivery of the services created by them. • Three functions are currently available. ①: momento Cache ②: momento Topics (Pub/Sub) ③: momento Vector Index (AI / ML) 2023/8/24 yoshii0110 17 https://www.gomomento.com/

Cache strategy using Momento chache 2023/8/24 yoshii0110 18

Momento cache simplicity 2023/8/24 yoshii0110 19

Cache strategy using Momento chache - basic sequence of operation diagram 2023/8/24 yoshii0110 20 1): When a user views a web page, Cloudflare workers retrieve dynamic content from momento cache, not origin. 2): If there is no content cache in the momento cache, fetch to origin.

Cache strategy using Momento chache - Sequence diagram focusing on cache fine flow 2023/8/24 yoshii0110 21 The assumption is that two identical dynamic contents are cached in the momento cache. Cache the same response with keys prefixed "fresh" and "stale" Set the ttl for the "stale" one longer 1): First, return cached value with "fresh" prefix key 2): When the TTL of the "fresh" cache expires, return the "stale" cache and replace the fresh cache with the fetched one.

verification 2023/8/24 yoshii0110 22 Even if the TTL has expired, if the Stale exists, the fetch will be executed and then cached. repo: https://github.com/Yoshiitaka/momento-cloudflare-swr

Check cache hits and misses and other metrics from the console 2023/8/24 yoshii0110 23

Momento and Cloudflare Integration • blog: https://www.gomomento.com/blog/momento-cache-is-now-accessible-at-the- edge • docs: https://docs.momentohq.com/ja/develop/api-reference/http-api • docs: https://docs.momentohq.com/ja/develop/integrations/cloudflare • Sample code: https://github.com/momentohq/client-sdk- javascript/tree/main/examples/cloudflare-workers 2023/8/24 yoshii0110 24

Other options as cache • Cache API • KVストア • Upstash • etc… 2023/8/24 yoshii0110 25