Practical Serverless & Edge Computing

Slide 1

Slide 1 text

practical serverless & edge computing. alexey taktarov · @mlfrg

Slide 2

Slide 2 text

2 alexey taktarov / @mlfrg λ. co-founded resume.io, 8.5M+ resumes online λ. I write code (JS / Ruby) occasionally these days… λ. I love designing backends!

Slide 3

Slide 3 text

3 wouter: a tiny 1.3KB React router github.com/molefrog/wouter react-colorful: a 1.8KB color picker github.com/omgovich/react-colorful an amateur OSS contributor react-colorful — a micro color picker component for React

Slide 4

Slide 4 text

1. the state of serverless in 2020

Slide 5

Slide 5 text

5 fundamental problems in networking increasing throughput horizontal scaling, network configuration

Slide 6

Slide 6 text

5 fundamental problems in networking increasing throughput horizontal scaling, network configuration reducing latency caching, geo-distribution

Slide 7

Slide 7 text

6 serverless* aims to solve both * Functions-as-a-Service → https://martinfowler.com/articles/serverless.html

Slide 8

Slide 8 text

7 λ

Slide 9

Slide 9 text

7 λ function calls (http requests / events)

Slide 10

Slide 10 text

λ http reqs λ. ∞-scalable λ. no ops needed λ. pay for the compute you need (down to 100ms) 8

Slide 11

Slide 11 text

λ http reqs λ. ∞-scalable λ. no ops needed λ. pay for the compute you need (down to 100ms) λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ λ 8

Slide 12

Slide 12 text

9 AWS Lambda

Slide 13

Slide 13 text

9 AWS Lambda

Slide 14

Slide 14 text

9 general-purpose FaaS providers AWS Lambda, Google Cloud, Azure, IBM, Yandex …

Slide 15

Slide 15 text

9 general-purpose FaaS providers AWS Lambda, Google Cloud, Azure, IBM, Yandex … → https://fauna.com/blog/comparison-faas-providers deployment platforms + SmartCDNs Vercel (formerly Zeit / Now), Netlify better DX, less configurable built on internally

Slide 16

Slide 16 text

Slide 17

Slide 17 text

10 Vercel and Cloudflare λ. CDN-based λ. great developer experience* * Serverless Framework is another great tech worth looking at

Slide 18

Slide 18 text

2. general-purpose lambdas with Vercel 11

Slide 19

Slide 19 text

import fetch from 'node-fetch' export default async (req, res) "=> { const name = req.query.name "|| 'wouter' const response = await fetch(githubRepo(name)) const repo = await response.json() res.json({ stars: `${repo.stargazers_count} stars!` }) } api/stars.js 12

Slide 20

Slide 20 text

1. NPM deps are bundled import fetch from 'node-fetch' export default async (req, res) "=> { const name = req.query.name "|| 'wouter' const response = await fetch(githubRepo(name)) const repo = await response.json() res.json({ stars: `${repo.stargazers_count} stars!` }) } 1 api/stars.js 12

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

13 > vercel zero-config deployment with just one command https:"//gh-stars.vercel.app/api/stars

Slide 24

Slide 24 text

14 a case study server-side PDF rendering microservice

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

16 λ. the preview must update in the real-time λ. react-pdf renders preview in a WebWorker λ. IE isn’t supported → need an SSR fallback

Slide 28

Slide 28 text

17 λ : (json) "-> pdf

Slide 29

Slide 29 text

18 challenge #1 how to authorise requests? backend browser lambda → →

Slide 30

Slide 30 text

19 authorisation decoupled JSON Web Tokens signed with asymmetric ciphers (RSA/ECD) → https://vercel.com/docs/solutions/authentication

Slide 31

Slide 31 text

20 eyJhbGciOiJIUzI1NiI sInR5cCI6IkpXVCJ9. eyJ1aWQiOiIxIiwicGx uIjoicHJlbWl1bSIsIm lhdCI6MTUxNjIzOTAyM n0. o1ayTFNg0eVZSQvd4T RUNRgSBa2MK2LCAUtj yIx1ydk ← header → https://jwt.io/

Slide 32

Slide 32 text

21 eyJhbGciOiJIUzI1NiI sInR5cCI6IkpXVCJ9. eyJ1aWQiOiIxIiwicGx uIjoicHJlbWl1bSIsIm lhdCI6MTUxNjIzOTAyM n0. o1ayTFNg0eVZSQvd4T RUNRgSBa2MK2LCAUtj yIx1ydk ← payload → https://jwt.io/ { "uid": "1", "pln": "premium", "iat": 1516239022 }

Slide 33

Slide 33 text

22 eyJhbGciOiJIUzI1NiI sInR5cCI6IkpXVCJ9. eyJ1aWQiOiIxIiwicGx uIjoicHJlbWl1bSIsIm lhdCI6MTUxNjIzOTAyM n0. o1ayTFNg0eVZSQvd4T RUNRgSBa2MK2LCAUtj yIx1ydk ← signature → https://jwt.io/ issue: private key verify: public key

Slide 34

Slide 34 text

23 backend issues JWT signs with the secret key browser passes JWT to the lambda Authorization header lambda user info token authenticity token lifetime (iat)

Slide 35

Slide 35 text

24 challenge #2 how to reuse already allocated resources?

Slide 36

Slide 36 text

25 export default async (req, res) "=> { const db = await connectToDB() const user = await db.getUserById(req.query.id) }

Slide 37

Slide 37 text

25 export default async (req, res) "=> { const db = await connectToDB() const user = await db.getUserById(req.query.id) } ← latency!

Slide 38

Slide 38 text

26 reusing resources between calls λ. one lambda can only process 1 request at a time λ. a lambda could be reused between requests λ λ λ λ λ λ lambda instance #1 lambda instance #2 λ

Slide 39

Slide 39 text

27 let dbPool = null const connectToDB = async () "=> { if (!dbPool) { dbPool = createConnectionPool() } return await dbPool.acquire() } ← could be reused between requests

Slide 40

Slide 40 text

28 challenge #3 DRY-ing up the repetitive code

Slide 41

Slide 41 text

29 const useAuthMiddleware = fn "=> (req, res, ""...args) "=> { if (!jwtValid) { const errorMessage = "Not Authorized!" return res.status(402).json({ errorMessage }) } return fn(req, res, ""...args) } export default useAuthMiddleware(lambda) Remember: lambdas are just plain functions!

Slide 42

Slide 42 text

30 PDF rendering lambda performance λ. responds within 400ms–1.6s (depending on a document size) λ. no downtime since March, 2019 λ. processes ~300GB/m worth of requests λ. conversion rate increased by 0.2%

Slide 43

Slide 43 text

31 a case study instant resume image preview lambda

Slide 44

Slide 44 text

32 ← image preview

Slide 45

Slide 45 text

33 λ : (id) "-> jpeg

Slide 46

Slide 46 text

33 λ : (id) "-> jpeg could take up to 2s to render the image…

Slide 47

Slide 47 text

3. Smart CDNs

Slide 48

Slide 48 text

35 Now Vercel is a CDN that is also capable of executing code. Guillermo Rauch, Vercel

Slide 49

Slide 49 text

36 response.setHeader('Cache-Control', 's-maxage=86400') Vercel will cache the image in 70+ locations around the world for 24 hours → https://vercel.com/docs/edge-network/caching

Slide 50

Slide 50 text

37 → https://vercel.com/docs/edge-network/regions λ SFO1 760ms cache MISS: call lambda

Slide 51

Slide 51 text

37 → https://vercel.com/docs/edge-network/regions λ SFO1 760ms cache MISS: call lambda cache HIT! SFO1 20ms

Slide 52

Slide 52 text

37 → https://vercel.com/docs/edge-network/regions λ SFO1 760ms cache MISS: call lambda ARN1 800ms cache MISS: call lambda cache HIT! SFO1 20ms

Slide 53

Slide 53 text

38 https:"//ssr.resume.tools/to-image/ssid-vGP2z-1.jpeg?version=105324 page number jpeg / png / webp change to invalidate

Slide 54

Slide 54 text

39 another example that uses edge caching in Vercel random avatar generator

Slide 55

Slide 55 text

40 λ. a Vercel lambda builds SVGs from the set of pre-made parts: heads, eyes, hairs λ. the result is cached forever in SmartCDN λ. PRNG is initialized with the name as a seed https:"//headsome.resume.tools/alex.svg

Slide 56

Slide 56 text

Slide 57

Slide 57 text

41 challenge #4 but the first request is still slow!

Slide 58

Slide 58 text

A B B B B 42 → https://vercel.com/docs/edge-network/caching#stale-while-revalidate Cache-Control: ’s-maxage=60, stale-while-revalidate=600' serve cached version, but revalidate in the background A 1 min ← revalidates without blocking the request 10 min A A C D E 0 min

Slide 59

Slide 59 text

λ. request time is critical λ. the content changes infrequently λ. does not have to be always fresh when to use stale-while-revalidate OG image preview rarely changes, but has to be fast!

Slide 60

Slide 60 text

4. Edge Computing w/ Cloudflare Workers

Slide 61

Slide 61 text

origin SFO

Slide 62

Slide 62 text

origin SFO client

Slide 63

Slide 63 text

origin SFO edge FRA client

Slide 64

Slide 64 text

origin SFO edge FRA client Cloudflare has 200+ edge locations around the globe → https://www.cloudflare.com/network/

Slide 65

Slide 65 text

46 running JS on the edge λ. V8 Isolate: cold starts in 5ms λ. can modify origin request / response λ. no filesystem, no Node modules λ. only pure JS packages λ. Web APIs: Crypto, fetch, URL etc. → https://developers.cloudflare.com/workers/

Slide 66

Slide 66 text

47 a case study Injecting «Above-the-Fold» CSS on the fly

Slide 67

Slide 67 text

48 λ. CSS files are render-blocking λ. «Above-the-Fold» CSS could be inlined to speed up the page load λ. higher PageSpeed → better SEO → https://web.dev/defer-non-critical-css/

Slide 68

Slide 68 text

Slide 69

Slide 69 text

50 .hero{background-color: #ADD8E6;color: #444;cursor: pointer;""... "

Slide 70

Slide 70 text

50 .hero{background-color: #ADD8E6;color: #444;cursor: pointer;""... " «Critical» CSS of the above-the-fold content

Slide 71

Slide 71 text

51 challenge #1 modify origin’s HTML? → https://developers.cloudflare.com/workers/runtime-apis/html-rewriter

Slide 72

Slide 72 text

51 challenge #1 modify origin’s HTML? → https://developers.cloudflare.com/workers/runtime-apis/html-rewriter HTMLRewriter class! based on lol-html — a low output latency rewriter written in Rust

Slide 73

Slide 73 text

52 async function handleRequest(req) { const originResponse = await fetch(req) return new HTMLRewriter() .on("div", new MyDivHandler()) .transform(originResponse) } addEventListener("fetch", event "=> { event.respondWith(handleRequest(event.request)) })

Slide 74

Slide 74 text

1. inspired by ServiceWorkers 1 52 async function handleRequest(req) { const originResponse = await fetch(req) return new HTMLRewriter() .on("div", new MyDivHandler()) .transform(originResponse) } addEventListener("fetch", event "=> { event.respondWith(handleRequest(event.request)) })

Slide 75

Slide 75 text

1. inspired by ServiceWorkers 1 2. fetches the response from the origin 52 async function handleRequest(req) { const originResponse = await fetch(req) return new HTMLRewriter() .on("div", new MyDivHandler()) .transform(originResponse) } addEventListener("fetch", event "=> { event.respondWith(handleRequest(event.request)) }) 2

Slide 76

Slide 76 text

Slide 77

Slide 77 text

53 request → https://github.com/addyosmani/critical

Slide 78

Slide 78 text

53 request fetch origin → → https://github.com/addyosmani/critical

Slide 79

Slide 79 text

53 request fetch origin → critical CSS → → https://github.com/addyosmani/critical

Slide 80

Slide 80 text

53 request fetch origin → critical CSS → → rewrite HTML → https://github.com/addyosmani/critical

Slide 81

Slide 81 text

53 request fetch origin → critical CSS → → rewrite HTML response → → https://github.com/addyosmani/critical

Slide 82

Slide 82 text

53 request fetch origin → critical CSS → → rewrite HTML response → → https://github.com/addyosmani/critical headless-chrome? not supported by workers… →

Slide 83

Slide 83 text

53 request fetch origin → critical CSS → → rewrite HTML response → → https://github.com/addyosmani/critical headless-chrome? not supported by workers… → Vercel lambda running headless-chrome! λ

Slide 84

Slide 84 text

called with waitUntil() 54 → https://developers.cloudflare.com/workers/learning/fetch-event-lifecycle origin server Rails app CF worker fetch the origin KV → CSS client Vercel lambda extract CSS w/ Chrome

Slide 85

Slide 85 text

«Critic» performance λ. CPU time ~5.3ms (99 pct) λ. processes 3M+ reqs/m λ. SEO clicks doubled Core Web Vitals report, green shows the number of webpages that are considered “good” → https://search.google.com/ worker

Slide 86

Slide 86 text

5. Conclusion

Slide 87

Slide 87 text

57 Cloudflare Workers λ. SSR, APIs, websites λ. fine-grained app proxies Vercel, Netlify and alternatives λ. server-side rendering λ. APIs λ. dynamic websites with infrequent updates: blogs, SEO pages λ. things that use canvas, headless chrome, native packages etc. λ. low-latency / HA APIs λ. real-time apps w/ WebSockets* * https://blog.cloudflare.com/introducing-workers-durable-objects/

Slide 88

Slide 88 text

58 For the longest time, CDNs have been treating the "origin" as an opaque black box. It's now possible to push content directly to the network and design frameworks that optimize for this capability. Guillermo Rauch, Vercel

Slide 89

Slide 89 text

59 λ. deno.land website built w/ Vercel & CF Workers λ. Using node-canvas with Vercel λ. chrome-aws-lambda package additional resources appendix font Inter · icons Radix Icons Alexey Taktarov · molefrog.com · github.com/molefrog · @mlfrg