SymfonyWorld 2021 Summer Edition: Feedback on our use of Varnish

On a large French e-commerce site, we use Varnish as HTTP cache.

We use it for the API as well as for the HTML front ends.

We use ESI, with user-context-hashes allowing us to do
authenticated caching. We mix invalidation by tags but also by ttl.

This is a feedback to explain the choices we had to make, how we use it, what it brings us.

We'll also answer a lot of questions that people have about HTTP caching in practice.

Transcript

1. Feedback on our use of Varnish SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic Architect @JoliCode
2. Scale symfony/demo https://github.com/symfony/demo
3. TL;DL (too long; didn’t listen) using Varnish gives us stellar performance with simpler code
4. Monitore All. The. Thing - never have to guess: test and monitor - graphs - logs - business metrics - load benchmark - profiling
5. 01. Typical Web architecture
7. 01. Typical Web architecture - same request : same impact on infrastructure - to scale, increase everything (Web server, php-fpm, db) - at the end it will be resources like database which will be the bottleneck
8. 01. Typical Web architecture Concurrency rq/s 90% ms Load webserver 1 40 25ms 20% (1) 10 150 90ms 95% (2) 100 135 850ms 100% (12) 300 100 3s 100% (20) already dead
9. 02. Add some servers
10. 02. Add some servers # servers Concurrency rq/s 90% ms Load webserver 1 10 150 90ms 95% (2) 4 10 372 30ms 50% (1) 1 100 135 850ms 100% (12) 4 100 550 440ms 100% (12) 1 300 100 3s 100% (20) 4 300 550 910ms 100% (20) Way better still dead
11. 03. With app cache - use redis or memcached - compute less - reduce pressure
13. findLatest seems to be a good candidate for caching.
14. We add cache, but we do not see any performance improvement??
15. Paginator is not serializable with an iterator and closures. Not a good candidate for caching. (no thanks to the silent fail)
17. Next attempt, cache the response. Performance is now better. But do you see the new issue? Same cache whether we have a tag or not.
18. 03. With app cache Concurrency rq/s 90% ms Load webserver 1 130 7ms 15% (0.1) 10 500 34ms 40% (3) 100 500 254ms 100% (12) 300 350 1,5s 100% (25) 100 (4 servers) 1700 85 ms 31% (2) 300 (4 servers) 1900 309ms 35% (2) still dead, but nicer nice, but not fast enough
19. 03. With app cache: cache lifetime after create/edit/delete?
22. https://symfony.com/blog/new-in-symfony-5-2-async-cache-recomputing https://github.com/nicolas-grekas/cache-bus/tree/b8de8b7ed70da28f3a403d451f5195da77ec6ff2
23. 04. HTTP cache with Symfony built-in reverse proxy
25. 04. HTTP cache with Symfony built-in reverse proxy
26. 04. HTTP cache with Symfony built-in reverse proxy Concurrency rq/s 90% ms Load webserver 1 300 3ms 15% (0.4) 10 1350 11ms 40% (2) 100 1400 80ms 100% (12) 300 1100 150ms 100% (25) dead Nice!
27. Edge Side Include - fragment with different cache rules - shared across pages - awesome
29. 05. HTTP cache with a dedicated reverse proxy: Varnish
30. 05. HTTP cache with a dedicated reverse proxy: Varnish Concurrency rq/s 90% ms Load webserver Load varnish 1 2550 1ms 0 (0) 0 10 6700 3ms 0 (0) 0 100 6800 30ms 0 (0) 0 300 6900 64ms 0 (0) 0 1000 6900 154ms 0 (0) 0,15 Woooooooooooottt! (not dead / very much alive and bored)
31. Varnish is VERY hard to benchmark, it just doesn’t care
32. Varnish is very very very fast … but it’s as tricky as it’s powerful… (have you met VCL?)
33. 05. HTTP cache with a dedicated reverse proxy: Varnish - Varnish = memory, volatile - Programmable via VCL (not nice) - Grace mode 💚 - ESI = separate request to backend
34. 06. HTTP cache with authenticated traffic usually, authenticated = not cached request we can change that
36. User context hash User Session id rôle user context hash Alice anonymous 1 Bob authenticated 1 Bastien a paid subscriber 2 Xavier b paid subscriber 2 Nicolas c moderator 3 Fabien d the boss 3
39. … paste some VCL stuff https://github.com/FriendsOfSymfony/FOSHttpCache/blob/master/resources/config/v arnish/fos_user_context.vcl
41. 07. Varnish: invalidate grace (always performant) accuracy async? support regexp TTL x x Purge x Ban x x x
42. 08. Complicate everything: api first - api first - own varnish layer - complex cascade of sync and async invalidation
46. 09. Varnish multi zone - no way to communicate with all varnish in sync mode - the api stays in one zone - let the ttl do its stuff - invalidate via an async message that notifies all stacks to ban everything
47. More caveats while scaling
48. Consistent order params - http://www.symfony.com/?a=1&b=2&c=3 - http://www.symfony.com/?b=2&a=1&c=3 - http://www.symfony.com/?c=3&b=2?a=1 => three different caches to compute https://github.com/nytimes/libvmod-queryfilter
49. Crash redis - redis is very robust, but can still crash - key always appending, size increase non stop - monitor! BEFORE AFTER
50. Ban tags with regex rq/s * ban/s * size of the cache tags header * complexity of the regexp not much I can do, all real load, let’s check the regexp
53. User context hash
55. Cookies sent to API, preventing cache - cookies on .symfony.com - api.symfony.com called from a backoffice - no http cache (no user context hash on our api) - strip all cookies
56. Cheat code: use a commercial varnish will prevent a lot of these issues
57. Varnish Cache Plus - by the maintainers of Varnish - share cache between varnish - better cookies management - better invalidation across cluster - VCL review by experts - … - and a lot more
58. Fastly >_Fastly's cache servers run an evolution of Varnish which diverged from the community project at version 2.1. Varnish Configuration Language (VCL) remains the primary way to configure our cache behaviors, and our VCL syntax derives from Varnish 2.1.5, but has been significantly upgraded and extended for Fastly-specific features. > While it's possible that VCL written for the open source Varnish will work on Fastly, our VCL variant should be seen as a separate language. See using VCL for detailed information about the architecture of VCL configurations.
59. Fastly Powers a LOT of sites Awesome Varnish could have a cleaner 503 page
60. Funny story: Symfony 5.2 vs 5.3 - bug on translations - json parsed at each requests - slows everything down
61. Typical Web architecture… on 5.3 Concurrency rq/s 90% ms Load webserver 1 40 60 25ms 17ms 20% (1) 10 150 165 90ms 75ms 95% (2) 100 135 158 850ms 600ms 100% (6) 300 100 172 3s 1.5s 100% (12) dying
62. 1/ Perf improvement for free by updating dependencies 2/ Hidden by Http Cache
63. Include the performance in your CI
64. Takeaways … you get use to <70ms response
65. Performance is addictive • monitore! • Varnish protect you and can hide a lot of issues • once you are in <80ms, you want to stay there
66. Thanks. SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic