$30 off During Our Annual Pro Sale. View Details »

SymfonyWorld 2021 Summer Edition: Feedback on our use of Varnish

SymfonyWorld 2021 Summer Edition: Feedback on our use of Varnish

On a large French e-commerce site, we use Varnish as HTTP cache.

We use it for the API as well as for the HTML front ends.

We use ESI, with user-context-hashes allowing us to do
authenticated caching. We mix invalidation by tags but also by ttl.

This is a feedback to explain the choices we had to make, how we use it, what it brings us.

We'll also answer a lot of questions that people have about HTTP caching in practice.

Bastien Jaillot

June 17, 2021
Tweet

More Decks by Bastien Jaillot

Other Decks in Programming

Transcript

  1. Feedback on our use of
    Varnish
    SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic
    Architect @JoliCode

    View Slide

  2. Scale
    symfony/demo
    https://github.com/symfony/demo

    View Slide

  3. TL;DL (too long; didn’t listen)
    using Varnish gives us stellar
    performance with simpler code

    View Slide

  4. Monitore All. The. Thing
    - never have to guess: test and monitor
    - graphs
    - logs
    - business metrics
    - load benchmark
    - profiling

    View Slide

  5. 01. Typical Web architecture

    View Slide

  6. View Slide

  7. 01. Typical Web architecture
    - same request : same impact
    on infrastructure
    - to scale, increase everything
    (Web server, php-fpm, db)
    - at the end it will be resources
    like database which will be the
    bottleneck

    View Slide

  8. 01. Typical Web architecture
    Concurrency rq/s 90% ms Load webserver
    1 40 25ms 20% (1)
    10 150 90ms 95% (2)
    100 135 850ms 100% (12)
    300 100 3s 100% (20)
    already
    dead

    View Slide

  9. 02. Add some servers

    View Slide

  10. 02. Add some servers
    # servers Concurrency rq/s 90% ms Load webserver
    1 10 150 90ms 95% (2)
    4 10 372 30ms 50% (1)
    1 100 135 850ms 100% (12)
    4 100 550 440ms 100% (12)
    1 300 100 3s 100% (20)
    4 300 550 910ms 100% (20)
    Way
    better
    still
    dead

    View Slide

  11. 03. With app cache
    - use redis or memcached
    - compute less
    - reduce pressure

    View Slide

  12. View Slide

  13. findLatest seems to be a good
    candidate for caching.

    View Slide

  14. We add cache, but we do not see
    any performance improvement??

    View Slide

  15. Paginator is not serializable with an
    iterator and closures.
    Not a good candidate for caching.
    (no thanks to the silent fail)

    View Slide

  16. View Slide

  17. Next attempt, cache the response.
    Performance is now better.
    But do you see the new issue?
    Same cache whether we have a tag
    or not.

    View Slide

  18. 03. With app cache
    Concurrency rq/s 90% ms Load webserver
    1 130 7ms 15% (0.1)
    10 500 34ms 40% (3)
    100 500 254ms 100% (12)
    300 350 1,5s 100% (25)
    100 (4 servers) 1700 85 ms 31% (2)
    300 (4 servers) 1900 309ms 35% (2)
    still dead,
    but nicer
    nice, but not fast enough

    View Slide

  19. 03. With app cache: cache
    lifetime after
    create/edit/delete?

    View Slide

  20. View Slide

  21. View Slide

  22. https://symfony.com/blog/new-in-symfony-5-2-async-cache-recomputing
    https://github.com/nicolas-grekas/cache-bus/tree/b8de8b7ed70da28f3a403d451f5195da77ec6ff2

    View Slide

  23. 04. HTTP cache with Symfony built-in reverse
    proxy

    View Slide

  24. View Slide

  25. 04. HTTP cache
    with Symfony
    built-in reverse
    proxy

    View Slide

  26. 04. HTTP cache with Symfony built-in reverse
    proxy
    Concurrency rq/s 90% ms Load webserver
    1 300 3ms 15% (0.4)
    10 1350 11ms 40% (2)
    100 1400 80ms 100% (12)
    300 1100 150ms 100% (25)
    dead
    Nice!

    View Slide

  27. Edge Side Include
    - fragment with different cache
    rules
    - shared across pages
    - awesome

    View Slide

  28. View Slide

  29. 05. HTTP cache
    with a dedicated
    reverse proxy:
    Varnish

    View Slide

  30. 05. HTTP cache with a dedicated reverse proxy:
    Varnish
    Concurrency rq/s 90% ms Load webserver Load varnish
    1 2550 1ms 0 (0) 0
    10 6700 3ms 0 (0) 0
    100 6800 30ms 0 (0) 0
    300 6900 64ms 0 (0) 0
    1000 6900 154ms 0 (0) 0,15
    Woooooooooooottt! (not dead / very much alive and bored)

    View Slide

  31. Varnish is VERY hard to
    benchmark, it just doesn’t care

    View Slide

  32. Varnish is very very very fast
    … but it’s as tricky as it’s
    powerful… (have you met VCL?)

    View Slide

  33. 05. HTTP cache with a dedicated reverse proxy:
    Varnish
    - Varnish = memory, volatile
    - Programmable via VCL (not nice)
    - Grace mode 💚
    - ESI = separate request to backend

    View Slide

  34. 06. HTTP cache
    with authenticated
    traffic
    usually, authenticated = not cached
    request
    we can change that

    View Slide

  35. View Slide

  36. User context hash
    User Session id rôle user context hash
    Alice anonymous 1
    Bob authenticated 1
    Bastien a paid subscriber 2
    Xavier b paid subscriber 2
    Nicolas c moderator 3
    Fabien d the boss 3

    View Slide

  37. View Slide

  38. View Slide

  39. … paste some VCL stuff
    https://github.com/FriendsOfSymfony/FOSHttpCache/blob/master/resources/config/v
    arnish/fos_user_context.vcl

    View Slide

  40. View Slide

  41. 07. Varnish: invalidate
    grace (always
    performant)
    accuracy async? support regexp
    TTL x x
    Purge x
    Ban x x x

    View Slide

  42. 08. Complicate everything: api first
    - api first
    - own varnish layer
    - complex cascade of sync and async invalidation

    View Slide

  43. View Slide

  44. View Slide

  45. View Slide

  46. 09. Varnish multi zone
    - no way to communicate with all varnish in sync mode
    - the api stays in one zone
    - let the ttl do its stuff
    - invalidate via an async message that notifies all stacks to ban everything

    View Slide

  47. More caveats while scaling

    View Slide

  48. Consistent order params
    - http://www.symfony.com/?a=1&b=2&c=3
    - http://www.symfony.com/?b=2&a=1&c=3
    - http://www.symfony.com/?c=3&b=2?a=1
    => three different caches to compute
    https://github.com/nytimes/libvmod-queryfilter

    View Slide

  49. Crash redis
    - redis is very robust, but can still crash
    - key always appending, size increase non stop
    - monitor!
    BEFORE
    AFTER

    View Slide

  50. Ban tags with regex
    rq/s * ban/s * size of the cache tags
    header * complexity of the regexp
    not much I can do, all real load, let’s check the
    regexp

    View Slide

  51. View Slide

  52. View Slide

  53. User context hash

    View Slide

  54. View Slide

  55. Cookies sent to API, preventing cache
    - cookies on .symfony.com
    - api.symfony.com called from a backoffice
    - no http cache (no user context hash on our api)
    - strip all cookies

    View Slide

  56. Cheat code: use a commercial
    varnish will prevent a lot of
    these issues

    View Slide

  57. Varnish Cache Plus
    - by the maintainers of Varnish
    - share cache between varnish
    - better cookies management
    - better invalidation across cluster
    - VCL review by experts
    - …
    - and a lot more

    View Slide

  58. Fastly
    >_Fastly's cache servers run an evolution of Varnish which diverged from the
    community project at version 2.1. Varnish Configuration Language (VCL)
    remains the primary way to configure our cache behaviors, and our VCL syntax
    derives from Varnish 2.1.5, but has been significantly upgraded and extended for
    Fastly-specific features.
    > While it's possible that VCL written for the open source Varnish will work on
    Fastly, our VCL variant should be seen as a separate language. See using VCL for
    detailed information about the architecture of VCL configurations.

    View Slide

  59. Fastly
    Powers a LOT of sites
    Awesome
    Varnish could have a cleaner
    503 page

    View Slide

  60. Funny story:
    Symfony 5.2 vs 5.3
    - bug on translations
    - json parsed at each requests
    - slows everything down

    View Slide

  61. Typical Web architecture… on 5.3
    Concurrency rq/s 90% ms Load webserver
    1 40 60 25ms 17ms 20% (1)
    10 150 165 90ms 75ms 95% (2)
    100 135 158 850ms 600ms 100% (6)
    300 100 172 3s 1.5s 100% (12)
    dying

    View Slide

  62. 1/ Perf improvement for free
    by updating dependencies
    2/ Hidden by Http Cache

    View Slide

  63. Include the performance in
    your CI

    View Slide

  64. Takeaways
    … you get use to <70ms response

    View Slide

  65. Performance is addictive
    ● monitore!
    ● Varnish protect you and can hide a lot of issues
    ● once you are in <80ms, you want to stay there

    View Slide

  66. Thanks.
    SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic

    View Slide