SymfonyWorld 2021 Summer Edition: Feedback on our use of Varnish

On a large French e-commerce site, we use Varnish as HTTP cache.

We use it for the API as well as for the HTML front ends.

We use ESI, with user-context-hashes allowing us to do
authenticated caching. We mix invalidation by tags but also by ttl.

This is a feedback to explain the choices we had to make, how we use it, what it brings us.

We'll also answer a lot of questions that people have about HTTP caching in practice.

Transcript

1. Feedback on our use of
   Varnish
   SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic
   Architect @JoliCode
   

   View Slide

2. Scale
   symfony/demo
   https://github.com/symfony/demo
   

   View Slide

3. TL;DL (too long; didn’t listen)
   using Varnish gives us stellar
   performance with simpler code
   

   View Slide

4. Monitore All. The. Thing
   - never have to guess: test and monitor
   - graphs
   - logs
   - business metrics
   - load benchmark
   - profiling
   

   View Slide

5. 01. Typical Web architecture
   

   View Slide

6. View Slide

7. 01. Typical Web architecture
   - same request : same impact
   on infrastructure
   - to scale, increase everything
   (Web server, php-fpm, db)
   - at the end it will be resources
   like database which will be the
   bottleneck
   

   View Slide

8. 01. Typical Web architecture
   Concurrency rq/s 90% ms Load webserver
   1 40 25ms 20% (1)
   10 150 90ms 95% (2)
   100 135 850ms 100% (12)
   300 100 3s 100% (20)
   already
   dead
   

   View Slide

9. 02. Add some servers
   

   View Slide

10. 02. Add some servers
    # servers Concurrency rq/s 90% ms Load webserver
    1 10 150 90ms 95% (2)
    4 10 372 30ms 50% (1)
    1 100 135 850ms 100% (12)
    4 100 550 440ms 100% (12)
    1 300 100 3s 100% (20)
    4 300 550 910ms 100% (20)
    Way
    better
    still
    dead
    

    View Slide

11. 03. With app cache
    - use redis or memcached
    - compute less
    - reduce pressure
    

    View Slide

12. View Slide

13. findLatest seems to be a good
    candidate for caching.
    

    View Slide

14. We add cache, but we do not see
    any performance improvement??
    

    View Slide

15. Paginator is not serializable with an
    iterator and closures.
    Not a good candidate for caching.
    (no thanks to the silent fail)
    

    View Slide

16. View Slide

17. Next attempt, cache the response.
    Performance is now better.
    But do you see the new issue?
    Same cache whether we have a tag
    or not.
    

    View Slide

18. 03. With app cache
    Concurrency rq/s 90% ms Load webserver
    1 130 7ms 15% (0.1)
    10 500 34ms 40% (3)
    100 500 254ms 100% (12)
    300 350 1,5s 100% (25)
    100 (4 servers) 1700 85 ms 31% (2)
    300 (4 servers) 1900 309ms 35% (2)
    still dead,
    but nicer
    nice, but not fast enough
    

    View Slide

19. 03. With app cache: cache
    lifetime after
    create/edit/delete?
    

    View Slide

20. View Slide

21. View Slide

22. https://symfony.com/blog/new-in-symfony-5-2-async-cache-recomputing
    https://github.com/nicolas-grekas/cache-bus/tree/b8de8b7ed70da28f3a403d451f5195da77ec6ff2
    

    View Slide

23. 04. HTTP cache with Symfony built-in reverse
    proxy
    

    View Slide

24. View Slide

25. 04. HTTP cache
    with Symfony
    built-in reverse
    proxy
    

    View Slide

26. 04. HTTP cache with Symfony built-in reverse
    proxy
    Concurrency rq/s 90% ms Load webserver
    1 300 3ms 15% (0.4)
    10 1350 11ms 40% (2)
    100 1400 80ms 100% (12)
    300 1100 150ms 100% (25)
    dead
    Nice!
    

    View Slide

27. Edge Side Include
    - fragment with different cache
    rules
    - shared across pages
    - awesome
    

    View Slide

28. View Slide

29. 05. HTTP cache
    with a dedicated
    reverse proxy:
    Varnish
    

    View Slide

30. 05. HTTP cache with a dedicated reverse proxy:
    Varnish
    Concurrency rq/s 90% ms Load webserver Load varnish
    1 2550 1ms 0 (0) 0
    10 6700 3ms 0 (0) 0
    100 6800 30ms 0 (0) 0
    300 6900 64ms 0 (0) 0
    1000 6900 154ms 0 (0) 0,15
    Woooooooooooottt! (not dead / very much alive and bored)
    

    View Slide

31. Varnish is VERY hard to
    benchmark, it just doesn’t care
    

    View Slide

32. Varnish is very very very fast
    … but it’s as tricky as it’s
    powerful… (have you met VCL?)
    

    View Slide

33. 05. HTTP cache with a dedicated reverse proxy:
    Varnish
    - Varnish = memory, volatile
    - Programmable via VCL (not nice)
    - Grace mode 💚
    - ESI = separate request to backend
    

    View Slide

34. 06. HTTP cache
    with authenticated
    traffic
    usually, authenticated = not cached
    request
    we can change that
    

    View Slide

35. View Slide

36. User context hash
    User Session id rôle user context hash
    Alice anonymous 1
    Bob authenticated 1
    Bastien a paid subscriber 2
    Xavier b paid subscriber 2
    Nicolas c moderator 3
    Fabien d the boss 3
    

    View Slide

37. View Slide

38. View Slide

39. … paste some VCL stuff
    https://github.com/FriendsOfSymfony/FOSHttpCache/blob/master/resources/config/v
    arnish/fos_user_context.vcl
    

    View Slide

40. View Slide

41. 07. Varnish: invalidate
    grace (always
    performant)
    accuracy async? support regexp
    TTL x x
    Purge x
    Ban x x x
    

    View Slide

42. 08. Complicate everything: api first
    - api first
    - own varnish layer
    - complex cascade of sync and async invalidation
    

    View Slide

43. View Slide

44. View Slide

45. View Slide

46. 09. Varnish multi zone
    - no way to communicate with all varnish in sync mode
    - the api stays in one zone
    - let the ttl do its stuff
    - invalidate via an async message that notifies all stacks to ban everything
    

    View Slide

47. More caveats while scaling
    

    View Slide

48. Consistent order params
    - http://www.symfony.com/?a=1&b=2&c=3
    - http://www.symfony.com/?b=2&a=1&c=3
    - http://www.symfony.com/?c=3&b=2?a=1
    => three different caches to compute
    https://github.com/nytimes/libvmod-queryfilter
    

    View Slide

49. Crash redis
    - redis is very robust, but can still crash
    - key always appending, size increase non stop
    - monitor!
    BEFORE
    AFTER
    

    View Slide

50. Ban tags with regex
    rq/s * ban/s * size of the cache tags
    header * complexity of the regexp
    not much I can do, all real load, let’s check the
    regexp
    

    View Slide

51. View Slide

52. View Slide

53. User context hash
    

    View Slide

54. View Slide

55. Cookies sent to API, preventing cache
    - cookies on .symfony.com
    - api.symfony.com called from a backoffice
    - no http cache (no user context hash on our api)
    - strip all cookies
    

    View Slide

56. Cheat code: use a commercial
    varnish will prevent a lot of
    these issues
    

    View Slide

57. Varnish Cache Plus
    - by the maintainers of Varnish
    - share cache between varnish
    - better cookies management
    - better invalidation across cluster
    - VCL review by experts
    - …
    - and a lot more
    

    View Slide

58. Fastly
    >_Fastly's cache servers run an evolution of Varnish which diverged from the
    community project at version 2.1. Varnish Configuration Language (VCL)
    remains the primary way to configure our cache behaviors, and our VCL syntax
    derives from Varnish 2.1.5, but has been significantly upgraded and extended for
    Fastly-specific features.
    > While it's possible that VCL written for the open source Varnish will work on
    Fastly, our VCL variant should be seen as a separate language. See using VCL for
    detailed information about the architecture of VCL configurations.
    

    View Slide

59. Fastly
    Powers a LOT of sites
    Awesome
    Varnish could have a cleaner
    503 page
    

    View Slide

60. Funny story:
    Symfony 5.2 vs 5.3
    - bug on translations
    - json parsed at each requests
    - slows everything down
    

    View Slide

61. Typical Web architecture… on 5.3
    Concurrency rq/s 90% ms Load webserver
    1 40 60 25ms 17ms 20% (1)
    10 150 165 90ms 75ms 95% (2)
    100 135 158 850ms 600ms 100% (6)
    300 100 172 3s 1.5s 100% (12)
    dying
    

    View Slide

62. 1/ Perf improvement for free
    by updating dependencies
    2/ Hidden by Http Cache
    

    View Slide

63. Include the performance in
    your CI
    

    View Slide

64. Takeaways
    … you get use to <70ms response
    

    View Slide

65. Performance is addictive
    ● monitore!
    ● Varnish protect you and can hide a lot of issues
    ● once you are in <80ms, you want to stay there
    

    View Slide

66. Thanks.
    SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic
    

    View Slide