Practical Serverless & Edge Computing

Video: https://youtu.be/kKfW9LLL7Rc?t=196

While serverless is way beyond its hype point, it is still an underestimated technology that could open up new approaches in how we build our apps. In this talk I'm going to give an overview of practical applications of serverless JS. We're going to cover serverless frontend microservices, authorization methods, Smart CDNs, caching (including stale-while-revalidate) with Vercel and Cloudflare Workers. I'll illustrate these methods with practical examples from the tools we build internally at resume.io OG image generation, PDF/DOCX rendering, automatic Critical CSS injection and more.

Transcript

1. practical serverless &
   edge computing.
   alexey taktarov · @mlfrg
   

   View Slide

2. 2
   alexey taktarov / @mlfrg
   λ. co-founded resume.io, 8.5M+ resumes online
   λ. I write code (JS / Ruby) occasionally these days…
   λ. I love designing backends!
   

   View Slide

3. 3
   wouter: a tiny 1.3KB React router
   github.com/molefrog/wouter
   react-colorful: a 1.8KB color picker
   github.com/omgovich/react-colorful
   an amateur OSS contributor
   react-colorful — a micro color
   picker component for React
   

   View Slide

4. 1. the state of serverless in 2020
   

   View Slide

5. 5
   fundamental problems in networking
   increasing throughput
   horizontal scaling, network configuration
   

   View Slide

6. 5
   fundamental problems in networking
   increasing throughput
   horizontal scaling, network configuration
   reducing latency
   caching, geo-distribution
   

   View Slide

7. 6
   serverless* aims to solve both
   * Functions-as-a-Service
   → https://martinfowler.com/articles/serverless.html
   

   View Slide

8. 7
   λ
   

   View Slide

9. 7
   λ
   function calls
   (http requests / events)
   

   View Slide

10. λ
    http reqs
    λ. ∞-scalable
    λ. no ops needed
    λ. pay for the compute you need
    (down to 100ms)
    8
    

    View Slide

11. λ
    http reqs
    λ. ∞-scalable
    λ. no ops needed
    λ. pay for the compute you need
    (down to 100ms)
    λ λ
    λ λ λ
    λ
    λ λ λ λ λ λ λ
    λ
    λ
    λ
    λ
    λ
    λ
    λ
    8
    

    View Slide

12. 9
    AWS Lambda
    

    View Slide

13. 9
    AWS Lambda
    

    View Slide

14. 9
    general-purpose FaaS providers
    AWS Lambda, Google Cloud,
    Azure, IBM, Yandex …
    

    View Slide

15. 9
    general-purpose FaaS providers
    AWS Lambda, Google Cloud,
    Azure, IBM, Yandex …
    → https://fauna.com/blog/comparison-faas-providers
    deployment platforms + SmartCDNs
    Vercel (formerly Zeit / Now), Netlify
    better DX, less configurable
    built on internally
    

    View Slide

16. 9
    general-purpose FaaS providers
    AWS Lambda, Google Cloud,
    Azure, IBM, Yandex …
    → https://fauna.com/blog/comparison-faas-providers
    deployment platforms + SmartCDNs
    Vercel (formerly Zeit / Now), Netlify
    better DX, less configurable
    built on internally
    edge computing
    Cloudflare Workers
    AWS Lambda@Edge
    EdgeEngine
    

    View Slide

17. 10
    Vercel and Cloudflare
    λ. CDN-based
    λ. great developer experience*
    * Serverless Framework is another great tech worth looking at
    

    View Slide

18. 2. general-purpose lambdas with Vercel
    11
    

    View Slide

19. import fetch from 'node-fetch'
    
    export default async (req, res) "=> {
    
    const name = req.query.name "|| 'wouter'
    
    const response = await fetch(githubRepo(name))
    
    const repo = await response.json()
    
    res.json({
    
    stars: `${repo.stargazers_count} stars!`
    
    })
    
    }
    api/stars.js
    12
    

    View Slide

20. 1. NPM deps are bundled
    import fetch from 'node-fetch'
    
    export default async (req, res) "=> {
    
    const name = req.query.name "|| 'wouter'
    
    const response = await fetch(githubRepo(name))
    
    const repo = await response.json()
    
    res.json({
    
    stars: `${repo.stargazers_count} stars!`
    
    })
    
    }
    1
    api/stars.js
    12
    

    View Slide

21. 1. NPM deps are bundled
    import fetch from 'node-fetch'
    
    export default async (req, res) "=> {
    
    const name = req.query.name "|| 'wouter'
    
    const response = await fetch(githubRepo(name))
    
    const repo = await response.json()
    
    res.json({
    
    stars: `${repo.stargazers_count} stars!`
    
    })
    
    }
    1
    2. async/await is supported
    2
    api/stars.js
    12
    

    View Slide

22. 1. NPM deps are bundled
    import fetch from 'node-fetch'
    
    export default async (req, res) "=> {
    
    const name = req.query.name "|| 'wouter'
    
    const response = await fetch(githubRepo(name))
    
    const repo = await response.json()
    
    res.json({
    
    stars: `${repo.stargazers_count} stars!`
    
    })
    
    }
    1
    2. async/await is supported
    3. req and res provide
    convenient helper methods
    3
    2
    api/stars.js
    12
    

    View Slide

23. 13
    > vercel
    zero-config deployment with just one command
    https:"//gh-stars.vercel.app/api/stars
    

    View Slide

24. 14
    a case study
    server-side PDF rendering microservice
    

    View Slide

25. View Slide

26. View Slide

27. 16
    λ. the preview must update in the real-time
    λ. react-pdf renders preview in a WebWorker
    λ. IE isn’t supported → need an SSR fallback
    

    View Slide

28. 17
    λ : (json) "-> pdf
    

    View Slide

29. 18
    challenge #1
    how to authorise requests?
    backend browser lambda
    → →
    

    View Slide

30. 19
    authorisation decoupled
    JSON Web Tokens signed with asymmetric ciphers (RSA/ECD)
    → https://vercel.com/docs/solutions/authentication
    

    View Slide

31. 20
    eyJhbGciOiJIUzI1NiI
    sInR5cCI6IkpXVCJ9.
    eyJ1aWQiOiIxIiwicGx
    uIjoicHJlbWl1bSIsIm
    lhdCI6MTUxNjIzOTAyM
    n0.
    o1ayTFNg0eVZSQvd4T
    RUNRgSBa2MK2LCAUtj
    yIx1ydk
    ← header
    → https://jwt.io/
    

    View Slide

32. 21
    eyJhbGciOiJIUzI1NiI
    sInR5cCI6IkpXVCJ9.
    eyJ1aWQiOiIxIiwicGx
    uIjoicHJlbWl1bSIsIm
    lhdCI6MTUxNjIzOTAyM
    n0.
    o1ayTFNg0eVZSQvd4T
    RUNRgSBa2MK2LCAUtj
    yIx1ydk
    ← payload
    → https://jwt.io/
    {
    
    "uid": "1",
    
    "pln": "premium",
    
    "iat": 1516239022
    
    }
    
    

    View Slide

33. 22
    eyJhbGciOiJIUzI1NiI
    sInR5cCI6IkpXVCJ9.
    eyJ1aWQiOiIxIiwicGx
    uIjoicHJlbWl1bSIsIm
    lhdCI6MTUxNjIzOTAyM
    n0.
    o1ayTFNg0eVZSQvd4T
    RUNRgSBa2MK2LCAUtj
    yIx1ydk
    ← signature
    → https://jwt.io/
    issue: private key
    verify: public key
    

    View Slide

34. 23
    backend
    issues JWT
    signs with the secret key
    browser
    passes JWT to the lambda
    Authorization header
    lambda
    user info
    token authenticity
    token lifetime (iat)
    

    View Slide

35. 24
    challenge #2
    how to reuse already allocated resources?
    

    View Slide

36. 25
    export default async (req, res) "=> {
    
    const db = await connectToDB()
    
    const user = await db.getUserById(req.query.id)
    
    }
    
    

    View Slide

37. 25
    export default async (req, res) "=> {
    
    const db = await connectToDB()
    
    const user = await db.getUserById(req.query.id)
    
    }
    
    ← latency!
    

    View Slide

38. 26
    reusing resources between calls
    λ. one lambda can only process 1 request at a time
    λ. a lambda could be reused between requests
    λ λ λ λ λ λ
    lambda instance #1 lambda instance #2
    λ
    

    View Slide

39. 27
    let dbPool = null
    
    const connectToDB = async () "=> {
    
    if (!dbPool) {
    
    dbPool = createConnectionPool()
    
    }
    
    return await dbPool.acquire()
    
    }
    
    ← could be reused between requests
    

    View Slide

40. 28
    challenge #3
    DRY-ing up the repetitive code
    

    View Slide

41. 29
    const useAuthMiddleware = fn "=>
    
    (req, res, ""...args) "=> {
    
    if (!jwtValid) {
    
    const errorMessage = "Not Authorized!"
    
    return res.status(402).json({ errorMessage })
    
    }
    
    return fn(req, res, ""...args)
    
    }
    
    export default useAuthMiddleware(lambda)
    
    Remember: lambdas are just plain functions!
    

    View Slide

42. 30
    PDF rendering lambda performance
    λ. responds within 400ms–1.6s (depending on a document size)
    λ. no downtime since March, 2019
    λ. processes ~300GB/m worth of requests
    λ. conversion rate increased by 0.2%
    

    View Slide

43. 31
    a case study
    instant resume image preview lambda
    

    View Slide

44. 32
    ← image preview
    

    View Slide

45. 33
    λ : (id) "-> jpeg
    

    View Slide

46. 33
    λ : (id) "-> jpeg
    could take up to 2s to render the image…
    

    View Slide

47. 3. Smart CDNs
    

    View Slide

48. 35
    Now Vercel is a CDN that is also capable of executing code.
    Guillermo Rauch, Vercel
    

    View Slide

49. 36
    response.setHeader('Cache-Control', 's-maxage=86400')
    
    Vercel will cache the image in 70+ locations around the world for 24 hours
    → https://vercel.com/docs/edge-network/caching
    

    View Slide

50. 37
    → https://vercel.com/docs/edge-network/regions
    λ
    SFO1 760ms cache MISS: call lambda
    

    View Slide

51. 37
    → https://vercel.com/docs/edge-network/regions
    λ
    SFO1 760ms cache MISS: call lambda
    cache HIT!
    SFO1 20ms
    

    View Slide

52. 37
    → https://vercel.com/docs/edge-network/regions
    λ
    SFO1 760ms cache MISS: call lambda
    ARN1 800ms cache MISS: call lambda
    cache HIT!
    SFO1 20ms
    

    View Slide

53. 38
    https:"//ssr.resume.tools/to-image/ssid-vGP2z-1.jpeg?version=105324
    page number
    jpeg / png / webp
    change to invalidate
    

    View Slide

54. 39
    another example that uses edge caching in Vercel
    random avatar generator
    

    View Slide

55. 40
    λ. a Vercel lambda builds SVGs from the set of
    pre-made parts: heads, eyes, hairs
    λ. the result is cached forever in SmartCDN
    λ. PRNG is initialized with the name as a seed
    https:"//headsome.resume.tools/alex.svg
    

    View Slide

56. 40
    λ. a Vercel lambda builds SVGs from the set of
    pre-made parts: heads, eyes, hairs
    λ. the result is cached forever in SmartCDN
    λ. PRNG is initialized with the name as a seed
    https:"//headsome.resume.tools/alex.svg
    

    View Slide

57. 41
    challenge #4
    but the first request is still slow!
    

    View Slide

58. A B B
    B
    B
    42
    → https://vercel.com/docs/edge-network/caching#stale-while-revalidate
    Cache-Control: ’s-maxage=60, stale-while-revalidate=600'
    
    serve cached version, but revalidate in the background
    A
    1 min
    ← revalidates without blocking the request
    10 min
    A A C D E
    0 min
    

    View Slide

59. λ. request time is critical
    λ. the content changes infrequently
    λ. does not have to be always fresh
    when to use stale-while-revalidate
    OG image preview rarely changes, but has to be fast!
    

    View Slide

60. 4. Edge Computing w/ Cloudflare Workers
    

    View Slide

61. origin
    SFO
    

    View Slide

62. origin
    SFO
    client
    

    View Slide

63. origin
    SFO
    edge
    FRA
    client
    

    View Slide

64. origin
    SFO
    edge
    FRA
    client
    Cloudflare has 200+ edge locations around the globe
    → https://www.cloudflare.com/network/
    

    View Slide

65. 46
    running JS on the edge
    λ. V8 Isolate: cold starts in 5ms
    λ. can modify origin request / response
    λ. no filesystem, no Node modules
    λ. only pure JS packages
    λ. Web APIs: Crypto, fetch, URL etc.
    → https://developers.cloudflare.com/workers/
    

    View Slide

66. 47
    a case study
    Injecting «Above-the-Fold» CSS on the fly
    

    View Slide

67. 48
    λ. CSS files are render-blocking
    λ. «Above-the-Fold» CSS could be
    inlined to speed up the page load
    λ. higher PageSpeed → better SEO
    → https://web.dev/defer-non-critical-css/
    

    View Slide

68. View Slide

69. 50
    <br/><br/>.hero{background-color: #ADD8E6;color: #444;cursor: pointer;""...<br/><br/>"
    
    
    onload="this.onload=null;this.rel='stylesheet'">
    
    

    View Slide

70. 50
    <br/><br/>.hero{background-color: #ADD8E6;color: #444;cursor: pointer;""...<br/><br/>"
    
    
    onload="this.onload=null;this.rel='stylesheet'">
    
    «Critical» CSS of the above-the-fold content
    

    View Slide

71. 51
    challenge #1
    modify origin’s HTML?
    → https://developers.cloudflare.com/workers/runtime-apis/html-rewriter
    

    View Slide

72. 51
    challenge #1
    modify origin’s HTML?
    → https://developers.cloudflare.com/workers/runtime-apis/html-rewriter
    HTMLRewriter class!
    based on lol-html — a low output latency rewriter
    written in Rust
    

    View Slide

73. 52
    async function handleRequest(req) {
    
    const originResponse = await fetch(req)
    
    return new HTMLRewriter()
    
    .on("div", new MyDivHandler())
    
    .transform(originResponse)
    
    }
    
    addEventListener("fetch", event "=> {
    
    event.respondWith(handleRequest(event.request))
    
    })
    
    

    View Slide

74. 1. inspired by ServiceWorkers
    1
    52
    async function handleRequest(req) {
    
    const originResponse = await fetch(req)
    
    return new HTMLRewriter()
    
    .on("div", new MyDivHandler())
    
    .transform(originResponse)
    
    }
    
    addEventListener("fetch", event "=> {
    
    event.respondWith(handleRequest(event.request))
    
    })
    
    

    View Slide

75. 1. inspired by ServiceWorkers
    1
    2. fetches the response
    from the origin
    52
    async function handleRequest(req) {
    
    const originResponse = await fetch(req)
    
    return new HTMLRewriter()
    
    .on("div", new MyDivHandler())
    
    .transform(originResponse)
    
    }
    
    addEventListener("fetch", event "=> {
    
    event.respondWith(handleRequest(event.request))
    
    })
    
    2
    

    View Slide

76. 1. inspired by ServiceWorkers
    1
    2. fetches the response
    from the origin
    52
    async function handleRequest(req) {
    
    const originResponse = await fetch(req)
    
    return new HTMLRewriter()
    
    .on("div", new MyDivHandler())
    
    .transform(originResponse)
    
    }
    
    addEventListener("fetch", event "=> {
    
    event.respondWith(handleRequest(event.request))
    
    })
    
    2
    3
    2. rewriter allows to modify
    individual elements using
    a «visitor» pattern
    

    View Slide

77. 53
    request
    → https://github.com/addyosmani/critical
    

    View Slide

78. 53
    request fetch origin
    →
    → https://github.com/addyosmani/critical
    

    View Slide

79. 53
    request fetch origin
    → critical CSS
    →
    → https://github.com/addyosmani/critical
    

    View Slide

80. 53
    request fetch origin
    → critical CSS
    → → rewrite HTML
    → https://github.com/addyosmani/critical
    

    View Slide

81. 53
    request fetch origin
    → critical CSS
    → → rewrite HTML response
    →
    → https://github.com/addyosmani/critical
    

    View Slide

82. 53
    request fetch origin
    → critical CSS
    → → rewrite HTML response
    →
    → https://github.com/addyosmani/critical
    headless-chrome? not supported by workers…
    →
    

    View Slide

83. 53
    request fetch origin
    → critical CSS
    → → rewrite HTML response
    →
    → https://github.com/addyosmani/critical
    headless-chrome? not supported by workers…
    →
    Vercel lambda running headless-chrome!
    λ
    

    View Slide

84. called with waitUntil()
    54
    → https://developers.cloudflare.com/workers/learning/fetch-event-lifecycle
    origin server
    Rails app
    CF worker
    fetch the origin KV → CSS
    client
    Vercel lambda
    extract CSS w/ Chrome
    

    View Slide

85. «Critic» performance
    λ. CPU time ~5.3ms (99 pct)
    λ. processes 3M+ reqs/m
    λ. SEO clicks doubled
    Core Web Vitals report, green shows the number
    of webpages that are considered “good”
    → https://search.google.com/
    worker
    

    View Slide

86. 5. Conclusion
    

    View Slide

87. 57
    Cloudflare Workers
    λ. SSR, APIs, websites
    λ. fine-grained app proxies
    Vercel, Netlify and alternatives
    λ. server-side rendering
    λ. APIs
    λ. dynamic websites with infrequent
    updates: blogs, SEO pages
    λ. things that use canvas, headless
    chrome, native packages etc.
    λ. low-latency / HA APIs
    λ. real-time apps w/ WebSockets*
    * https://blog.cloudflare.com/introducing-workers-durable-objects/
    

    View Slide

88. 58
    For the longest time, CDNs have been treating the "origin" as an
    opaque black box.
    It's now possible to push content directly to the network and
    design frameworks that optimize for this capability.
    Guillermo Rauch, Vercel
    

    View Slide

89. 59
    λ. deno.land website built w/ Vercel & CF Workers
    λ. Using node-canvas with Vercel
    λ. chrome-aws-lambda package
    additional resources
    appendix
    font Inter · icons Radix Icons
    Alexey Taktarov · molefrog.com · github.com/molefrog · @mlfrg
    

    View Slide