SymfonyWorld 2021 Summer Edition: Feedback on our use of Varnish

Slide 1

Slide 1 text

Feedback on our use of Varnish SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic Architect @JoliCode

Slide 2

Slide 2 text

Scale symfony/demo https://github.com/symfony/demo

Slide 3

Slide 3 text

TL;DL (too long; didn’t listen) using Varnish gives us stellar performance with simpler code

Slide 4

Slide 4 text

Monitore All. The. Thing - never have to guess: test and monitor - graphs - logs - business metrics - load benchmark - proﬁling

Slide 5

Slide 5 text

01. Typical Web architecture

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

01. Typical Web architecture - same request : same impact on infrastructure - to scale, increase everything (Web server, php-fpm, db) - at the end it will be resources like database which will be the bottleneck

Slide 8

Slide 8 text

01. Typical Web architecture Concurrency rq/s 90% ms Load webserver 1 40 25ms 20% (1) 10 150 90ms 95% (2) 100 135 850ms 100% (12) 300 100 3s 100% (20) already dead

Slide 9

Slide 9 text

02. Add some servers

Slide 10

Slide 10 text

02. Add some servers # servers Concurrency rq/s 90% ms Load webserver 1 10 150 90ms 95% (2) 4 10 372 30ms 50% (1) 1 100 135 850ms 100% (12) 4 100 550 440ms 100% (12) 1 300 100 3s 100% (20) 4 300 550 910ms 100% (20) Way better still dead

Slide 11

Slide 11 text

03. With app cache - use redis or memcached - compute less - reduce pressure

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

ﬁndLatest seems to be a good candidate for caching.

Slide 14

Slide 14 text

We add cache, but we do not see any performance improvement??

Slide 15

Slide 15 text

Paginator is not serializable with an iterator and closures. Not a good candidate for caching. (no thanks to the silent fail)

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

Next attempt, cache the response. Performance is now better. But do you see the new issue? Same cache whether we have a tag or not.

Slide 18

Slide 18 text

03. With app cache Concurrency rq/s 90% ms Load webserver 1 130 7ms 15% (0.1) 10 500 34ms 40% (3) 100 500 254ms 100% (12) 300 350 1,5s 100% (25) 100 (4 servers) 1700 85 ms 31% (2) 300 (4 servers) 1900 309ms 35% (2) still dead, but nicer nice, but not fast enough

Slide 19

Slide 19 text

03. With app cache: cache lifetime after create/edit/delete?

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

https://symfony.com/blog/new-in-symfony-5-2-async-cache-recomputing https://github.com/nicolas-grekas/cache-bus/tree/b8de8b7ed70da28f3a403d451f5195da77ec6ff2

Slide 23

Slide 23 text

04. HTTP cache with Symfony built-in reverse proxy

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

04. HTTP cache with Symfony built-in reverse proxy

Slide 26

Slide 26 text

04. HTTP cache with Symfony built-in reverse proxy Concurrency rq/s 90% ms Load webserver 1 300 3ms 15% (0.4) 10 1350 11ms 40% (2) 100 1400 80ms 100% (12) 300 1100 150ms 100% (25) dead Nice!

Slide 27

Slide 27 text

Edge Side Include - fragment with different cache rules - shared across pages - awesome

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

05. HTTP cache with a dedicated reverse proxy: Varnish

Slide 30

Slide 30 text

05. HTTP cache with a dedicated reverse proxy: Varnish Concurrency rq/s 90% ms Load webserver Load varnish 1 2550 1ms 0 (0) 0 10 6700 3ms 0 (0) 0 100 6800 30ms 0 (0) 0 300 6900 64ms 0 (0) 0 1000 6900 154ms 0 (0) 0,15 Woooooooooooottt! (not dead / very much alive and bored)

Slide 31

Slide 31 text

Varnish is VERY hard to benchmark, it just doesn’t care

Slide 32

Slide 32 text

Varnish is very very very fast … but it’s as tricky as it’s powerful… (have you met VCL?)

Slide 33

Slide 33 text

05. HTTP cache with a dedicated reverse proxy: Varnish - Varnish = memory, volatile - Programmable via VCL (not nice) - Grace mode 💚 - ESI = separate request to backend

Slide 34

Slide 34 text

06. HTTP cache with authenticated traﬃc usually, authenticated = not cached request we can change that

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

User context hash User Session id rôle user context hash Alice anonymous 1 Bob authenticated 1 Bastien a paid subscriber 2 Xavier b paid subscriber 2 Nicolas c moderator 3 Fabien d the boss 3

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

… paste some VCL stuff https://github.com/FriendsOfSymfony/FOSHttpCache/blob/master/resources/conﬁg/v arnish/fos_user_context.vcl

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

07. Varnish: invalidate grace (always performant) accuracy async? support regexp TTL x x Purge x Ban x x x

Slide 42

Slide 42 text

08. Complicate everything: api ﬁrst - api ﬁrst - own varnish layer - complex cascade of sync and async invalidation

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

09. Varnish multi zone - no way to communicate with all varnish in sync mode - the api stays in one zone - let the ttl do its stuff - invalidate via an async message that notiﬁes all stacks to ban everything

Slide 47

Slide 47 text

More caveats while scaling

Slide 48

Slide 48 text

Consistent order params - http://www.symfony.com/?a=1&b=2&c=3 - http://www.symfony.com/?b=2&a=1&c=3 - http://www.symfony.com/?c=3&b=2?a=1 => three different caches to compute https://github.com/nytimes/libvmod-queryfilter

Slide 49

Slide 49 text

Crash redis - redis is very robust, but can still crash - key always appending, size increase non stop - monitor! BEFORE AFTER

Slide 50

Slide 50 text

Ban tags with regex rq/s * ban/s * size of the cache tags header * complexity of the regexp not much I can do, all real load, let’s check the regexp

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

User context hash

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

Cookies sent to API, preventing cache - cookies on .symfony.com - api.symfony.com called from a backofﬁce - no http cache (no user context hash on our api) - strip all cookies

Slide 56

Slide 56 text

Cheat code: use a commercial varnish will prevent a lot of these issues

Slide 57

Slide 57 text

Varnish Cache Plus - by the maintainers of Varnish - share cache between varnish - better cookies management - better invalidation across cluster - VCL review by experts - … - and a lot more

Slide 58

Slide 58 text

Fastly >_Fastly's cache servers run an evolution of Varnish which diverged from the community project at version 2.1. Varnish Configuration Language (VCL) remains the primary way to configure our cache behaviors, and our VCL syntax derives from Varnish 2.1.5, but has been significantly upgraded and extended for Fastly-specific features. > While it's possible that VCL written for the open source Varnish will work on Fastly, our VCL variant should be seen as a separate language. See using VCL for detailed information about the architecture of VCL configurations.

Slide 59

Slide 59 text

Fastly Powers a LOT of sites Awesome Varnish could have a cleaner 503 page

Slide 60

Slide 60 text

Funny story: Symfony 5.2 vs 5.3 - bug on translations - json parsed at each requests - slows everything down

Slide 61

Slide 61 text

Typical Web architecture… on 5.3 Concurrency rq/s 90% ms Load webserver 1 40 60 25ms 17ms 20% (1) 10 150 165 90ms 75ms 95% (2) 100 135 158 850ms 600ms 100% (6) 300 100 172 3s 1.5s 100% (12) dying

Slide 62

Slide 62 text

1/ Perf improvement for free by updating dependencies 2/ Hidden by Http Cache

Slide 63

Slide 63 text

Include the performance in your CI

Slide 64

Slide 64 text

Takeaways … you get use to <70ms response

Slide 65

Slide 65 text

Performance is addictive ● monitore! ● Varnish protect you and can hide a lot of issues ● once you are in <80ms, you want to stay there

Slide 66

Slide 66 text

Thanks. SymfonyWorld Online 2021 – Bastien Jaillot – @bastnic