what the cache?!

".. and then cache the hell out of it!" This is usually how the talk ends when it comes to rails performance tips, but in reality it is where the fun starts. This talk goes way beyond Rails.cache, it explores layered caches, discusses the CPU cost of marshaling, the latency even Memcache adds and how to dodge the unavoidable stampedes that invalidating your caches will cause. This is the tale of how to survive half a billion requests per day with Rails and MySQL (hint: don't query it).

Transcript

1. _ _ _ _ _ ___ _ __ _| |__ __ _| |_ | |_| |__ ___ ___ __ _ ___| |__ __|__ \ | \ \ /\ / / '_ \ / _` | __| | __| '_ \ / _ \ / __/ _` |/ __| '_ \ / _ \/ / | \ V V /| | | | (_| | |_ | |_| | | | __/ | (_| (_| | (__| | | | __/_||_| \_/\_/ |_| |_|\__,_|\__| \__|_| |_|\___| \___\__,_|\___|_| |_|\___(_)(_) Simon Kröger - @kroegerberlin SponsorPay
2. _ _ | | __ _ ___| |_ _ _ ___ __ _ _ __ | |/ _` / __| __| | | | |/ _ \/ _` | '__| | | (_| \__ \ |_ | |_| | __/ (_| | | _ _ _ |_|\__,_|___/\__| \__, |\___|\__,_|_| (_|_|_) |___/
3. ____ _ _ / ___|___ _ __ | |_ _____ _| |_ | | / _ \| '_ \| __/ _ \ \/ / __| | |__| (_) | | | | || __/> <| |_ \____\___/|_| |_|\__\___/_/\_\\__| vanilla rails stack * ruby 1.9 * rails 3.2 * MySQL 5.5 * Memcache Yes, we also used Redis, but that’s besides the point here. ;) ___________ / \ | Unicorn |'.. \___________/ ':'~-. ,-------. ___________ : ''~-. | | / \ '. '~~-. |`-------'| | Unicorn |'-.----'.--------,,~| MySQL | \___________/ '-. '. ,,-~;' | | ___________ '-,-~'. ; `-------' / \ ,--~' '. ;: | Unicorn |--:.. ;-. : +---------+ \___________/ ''~~--.. '-:. | | ,' '''~,-~|Memcache | ... ,' ,-~' | | ___________ ; ,-~~' +---------+ / \ ,',-~' | Unicorn |--~' \___________/
4. _ _ __ | | (_)/ _| ___ __ ____ _ ___ ___ __ _ ___ _ _ | | | | |_ / _ \ \ \ /\ / / _` / __| / _ \/ _` / __| | | | | |___| | _| __/ \ V V / (_| \__ \ | __/ (_| \__ \ |_| |_ _ _ |_____|_|_| \___| \_/\_/ \__,_|___/ \___|\__,_|___/\__, (_|_|_) |___/
5. __ ___ _ _ _ ___ \ \ / / |__ __ _| |_ | |_| |__ ___ |__ \ \ \ /\ / /| '_ \ / _` | __| | __| '_ \ / _ \ / / \ V V / | | | | (_| | |_ | |_| | | | __/_ _ _ |_| \_/\_/ |_| |_|\__,_|\__| \__|_| |_|\___(_|_|_) (_)
6. ____ _ _ ___ | __ ) _ _| |_ __ _| |__ _ |__ \ | _ \| | | | __| \ \ /\ / / '_ \| | | |/ / | |_) | |_| | |_ \ V V /| | | | |_| |_| |____/ \__,_|\__| \_/\_/ |_| |_|\__, (_) |___/ requests: 100ms v v v v v |-----------------------------------------------------> t 20% DB load
7. ____ _ _ ___ | __ ) _ _| |_ __ _| |__ _ |__ \ | _ \| | | | __| \ \ /\ / / '_ \| | | |/ / | |_) | |_| | |_ \ V V /| | | | |_| |_| |____/ \__,_|\__| \_/\_/ |_| |_|\__, (_) |___/ requests: 100ms v v v v v |-----------------------------------------------------> t ^ 20% DB load | cache invalidate
8. ____ _ _ ___ | __ ) _ _| |_ __ _| |__ _ |__ \ | _ \| | | | __| \ \ /\ / / '_ \| | | |/ / | |_) | |_| | |_ \ V V /| | | | |_| |_| |____/ \__,_|\__| \_/\_/ |_| |_|\__, (_) |___/ requests: 100ms v v v v v |------------[Query]----------------------------------> t ^ 20% DB load | cache invalidate
9. ____ _ _ ___ | __ ) _ _| |_ __ _| |__ _ |__ \ | _ \| | | | __| \ \ /\ / / '_ \| | | |/ / | |_) | |_| | |_ \ V V /| | | | |_| |_| |____/ \__,_|\__| \_/\_/ |_| |_|\__, (_) |___/ requests: 80ms v v v v v v |----------[Query]------------------------------------> t ^ 20% DB load | cache invalidate
10. ____ _ _ ___ | __ ) _ _| |_ __ _| |__ _ |__ \ | _ \| | | | __| \ \ /\ / / '_ \| | | |/ / | |_) | |_| | |_ \ V V /| | | | |_| |_| |____/ \__,_|\__| \_/\_/ |_| |_|\__, (_) |___/ requests: 60ms v v v v v v v v v |-------[Query]---------------------------------------> t ^ [Query.] 40% DB load | cache invalidate __-::.. _/ _/ _/':. / // // / ':. / // // / :: ========= :: / // // / ** / // // / (_)(_)(_) (_)(_) (_)
11. ____ _ _ ___ | __ ) _ _| |_ __ _| |__ _ |__ \ | _ \| | | | __| \ \ /\ / / '_ \| | | |/ / | |_) | |_| | |_ \ V V /| | | | |_| |_| |____/ \__,_|\__| \_/\_/ |_| |_|\__, (_) |___/ requests: 40ms v v v v v v v v v v v v v |-----[Query...---------------------------------------> t ^ [Query... DOWN | [Query... cache [Query... invalidate [Query... ____ __,-~~/~ `---. _/_,---( , ) __ / < / ) \___ - ------===;;;'====------------------===;;;===----- - - \/ ~"~"~"~"~"~\~"~)~"/ (_ ( \ ( > \) \_( _ < >_>' ~ `-i' ::>|--" I;|.|.| <|i::|i|`. (` ^'"`-' ") ------------------------------------------------------------------
12. _ _ _ ____ _ ___ / \ ___| |_(_)_ _____/ ___| _ _ _ __ _ __ ___ _ __| ||__ \ / _ \ / __| __| \ \ / / _ \___ \| | | | '_ \| '_ \ / _ \| '__| __|/ / / ___ \ (__| |_| |\ V / __/___) | |_| | |_) | |_) | (_) | | | |_|_| /_/ \_\___|\__|_| \_/ \___|____/ \__,_| .__/| .__/ \___/|_| \__(_) |_| |_| http://api.rubyonrails.org/classes/ActiveSupport/Cache/Store.html Setting :race_condition_ttl is very useful in situations where a cache entry is used very frequently and is under heavy load. If a cache expires and due to heavy load seven different processes will try to read data natively and then they all will try to write to cache. To avoid that case the first process to find an expired cache entry will bump the cache expiration time by the value set in :race_condition_ttl. Yes, this process is extending the time for a stale value by another few seconds. Because of extended life of the previous cache, other processes will continue to use slightly stale data for a just a bit longer. In the meantime that first process will go ahead and will write into cache the new value. After that all the processes will start getting new value. The key is to keep :race_condition_ttl small.
13. ____ _ _ _ _ _ / ___|___ _ __ ___| |_ __ _ _ __ | |_ | | ___ __ _ __| | | | | / _ \| '_ \/ __| __/ _` | '_ \| __| | |/ _ \ / _` |/ _` | | | |__| (_) | | | \__ \ || (_| | | | | |_ | | (_) | (_| | (_| |_| \____\___/|_| |_|___/\__\__,_|_| |_|\__| |_|\___/ \__,_|\__,_(_) DB load ^ 100 | / | / | / | / | / | / | / | / 0 +---------------------------> requests/s DB load ^ 100 | | | | | | ,----------------------- | / | / 0 +---------------------------> requests/s ==>
14. ____ _ _ _ / ___|| |_ _ __ __ _| |_ ___ __ _(_) ___ ___ \___ \| __| '__/ _` | __/ _ \/ _` | |/ _ \/ __| ___) | |_| | | (_| | || __/ (_| | | __/\__ \ |____/ \__|_| \__,_|\__\___|\__, |_|\___||___/ |___/ * time to live (don’t forget :race_condition_ttl) * the application changing the content writes it to the cache * content versioning .uuu z@#"%c .uuzm**"""""*%mu.. z*"` .e@#N @!!!R. #c .z*" ^*c z dT!!!!!> '!!!!!!N "i u*" #s :" @?!!!!!!!R t!!!!!!!#u "i .@ ^$ :R!!!!!!!!!X '!!!!!!!!!#c "i:# ?> R!!!!!!!!!!X '!!!!!!!!!!!N @ 4W!!!!!!!!!!!> '!!!!!!!!!!!!Ru" ?!!!!!!!!!!X 'X!!!!!!!!!!!9~ . . 'X!!!!!!!!!6 R!!!!!!!!!!tF z$#` h &!!UR!!!!!F ?!!!!!$X!!!$ .@ X $WTR!!!!!X M!!!!!i#U!E . @F ! FdR!!!!!!f 'X!!!!!#c'?u@#"*$N. :$ F'9!!!!!!!@ 9!!!!!!!?NM ^*c dF ' @!!!!!!!X> R!!!!!!!!& "e d K<!!!!!!!XF 'X!!!!!!!M> ^N f < E!!!!!!X" t!!!!!!!# ^N :" .e$"^ Fn!!!!!XP #X!!!!!!ML *c z" .e$$$$$ M'!!!!W* "*UX!!X@t ^%u. ""**#).zd$$#$$$$$$$ <\*@**" 'N 4$$$$$@$$$)$$#$$k4$$$$$$$$$E :$ ?> "$$$$$$":$$$W$$$ "$$$$$$$$ % :" ? ^#*" S "$$$$$ ? F L d$L X & t$i @$$$ f * $$$$$$$$$$\& @ '*. W'$$$$$$$$FM h u# ^*muz* % $$$$$$": `" # ^**" d "***"
15. __ __ _ _ \ \ / /__ _ __ ___(_) ___ _ __ (_)_ __ __ _ \ \ / / _ \ '__/ __| |/ _ \| '_ \| | '_ \ / _` | \ V / __/ | \__ \ | (_) | | | | | | | | (_| | \_/ \___|_| |___/_|\___/|_| |_|_|_| |_|\__, | |___/ int bond = 0000_____________0000________0000000000000000__000000000000000000+ 00000000_________00000000______000000000000000__0000000000000000000+ 000____000_______000____000_____000_______0000__00______0+ 000______000_____000______000_____________0000___00______0+ 0000______0000___0000______0000___________0000_____0_____0+ 0000______0000___0000______0000__________0000___________0+ 0000______0000___0000______0000_________000___0000000000+ 0000______0000___0000______0000________0000+ 000______000_____000______000________0000+ 000____000_______000____000_______00000+ 00000000_________00000000_______0000000+ 0000_____________0000________000000007;
16. _____ _ ____ ____ ___ _ _ ___ |_ _| |__ ___ | _ \| __ )__ \ / \ __ _ __ _(_)_ __|__ \ | | | '_ \ / _ \ | | | | _ \ / / / _ \ / _` |/ _` | | '_ \ / / | | | | | | __/ | |_| | |_) |_| / ___ \ (_| | (_| | | | | |_| |_| |_| |_|\___| |____/|____/(_) /_/ \_\__, |\__,_|_|_| |_(_) |___/ /\ /\ //\\ //\\ (/\ \\,,,,// /\) \ `````` / (___)(___)-. (0) (0)``-._ |~ ~ , `-._ | | `-._ | /\ `-._ __..---'''''-.._.._ | | \ `-._ _.--' _ `. ) | \ `` \`\ \ / | \ | `\ \ /_....__) \ | `\\ / \ | / )) | | | / (((( | | | / )))) | () () | \ | | _.-' ((( `. .' `._. |______..| |-'| `------' | || | | || | | || | | || | | || | | || | | || | | || | _____ | || |_____| || | / `` |` / ` |` | \________\__\_______\__\ """"""""" """""""'"""
17. _ _ _ __ _ ___ / \ | | | / _(_)_ __ __|__ \ / _ \ | | | | |_| | '_ \ / _ \/ / / ___ \| | | | _| | | | | __/_| /_/ \_\_|_| |_| |_|_| |_|\___(_) o _' {_} |=| . ' | | o . o o |@| . o _o_._'_ /___\ o_.__'\~~~~~/ |=1 | \~~~~~/ '-.-' |=9 | '-.-' | |-9 | | _|_ |_9_| _|_ `"""` |_._| `"""` `"""`
18. __ __ _ _ _ _ | \/ | __ _ _ __ ___| |__ __ _| | (_)_ __ __ _ | |\/| |/ _` | '__/ __| '_ \ / _` | | | | '_ \ / _` | | | | | (_| | | \__ \ | | | (_| | | | | | | | (_| | |_| |_|\__,_|_| |___/_| |_|\__,_|_|_|_|_| |_|\__, | |___/ 2.1.1 > a = 100.times.map {|i| Hash[('a'..'z').map {|k| [k, rand(100)]}]}; 0 2.1.1 > Marshal.dump(a).size # => 26205 2.1.1 > YAML.dump(a).size # => 20536 2.1.1 > JSON.dump(a).size # => 18133 2.1.1 > Oj.dump(a).size # => 18135 2.1.1 > MessagePack.dump(a).size # => 8103 2.1.1 > Benchmark.realtime { 1000.times { YAML.load(YAML.dump(a)) } } # => 109.092351072 2.1.1 > Benchmark.realtime { 1000.times { Marshal.load(Marshal.dump(a)) } } # => 4.643632946 2.1.1 > Benchmark.realtime { 1000.times { JSON.load(JSON.dump(a)) } } # => 3.397973534 2.1.1 > Benchmark.realtime { 1000.times { Oj.load(Oj.dump(a)) } } # => 1.433172451 2.1.1 > Benchmark.realtime { 1000.times { MessagePack.load(MessagePack.dump(a)) } } # => 1.11540627
19. ____ _ _ _ | _ \ ___ _ _ _ __ __| | |_ _ __(_)_ __ ___ | |_) / _ \| | | | '_ \ / _` | __| '__| | '_ \/ __| | _ < (_) | |_| | | | | (_| | |_| | | | |_) \__ \ |_| \_\___/ \__,_|_| |_|\__,_|\__|_| |_| .__/|___/ |_| ActiveSupport::Cache::MemoryStore
20. ____ _ _ _ | _ \ ___ _ _ _ __ __| | |_ _ __(_)_ __ ___ | |_) / _ \| | | | '_ \ / _` | __| '__| | '_ \/ __| | _ < (_) | |_| | | | | (_| | |_| | | | |_) \__ \ |_| \_\___/ \__,_|_| |_|\__,_|\__|_| |_| .__/|___/ |_| ActiveSupport::Cache::MemoryStore 20 servers * 32 processes = 640 queries
21. ___ ____ _ _ _ / _ \ _ _ _ __ / ___| ___ | |_ _| |_(_) ___ _ __ | | | | | | | '__| \___ \ / _ \| | | | | __| |/ _ \| '_ \ | |_| | |_| | | ___) | (_) | | |_| | |_| | (_) | | | | \___/ \__,_|_| |____/ \___/|_|\__,_|\__|_|\___/|_| |_| Hash + LRU algorithm
22. ____ ___ _ | _ \ ___ _ __ __|__ \ | | | | |/ _ \| '_ \ / _ \/ / | | |_| | (_) | | | | __/_||_| |____/ \___/|_| |_|\___(_)(_) , , / \ ((__-^^-,-^^-__)) `-_---' `---_-' , , <__|o` 'o|__> / \ \ ` / ((__-^^-,-^^-__)) ): :( , , `-_---' `---_-' :o_o: / \ <__|o` 'o|__> "-" ((__-^^-,-^^-__)) \ ` / , , `-_---' `---_-' ): :( / \ <__|o` 'o|__> :o_o: ((__-^^-,-^^-__)) \ ` / "-" `-_---' `---_-' ): :( <__|o` 'o|__> :o_o: \ ` / "-" ): :( :o_o: "-"
23. _ _ _ _ | |__ ___ _ _ _ __ __| | ___ __| | ___ __ _ ___| |__ ___ | '_ \ / _ \| | | | '_ \ / _` |/ _ \/ _` | / __/ _` |/ __| '_ \ / _ \ | |_) | (_) | |_| | | | | (_| | __/ (_| | | (_| (_| | (__| | | | __/ |_.__/ \___/ \__,_|_| |_|\__,_|\___|\__,_|___\___\__,_|\___|_| |_|\___| |_____| * internal gem * Rails.cache compatible * falls back to Dalli
24. ____ __ _ _ _ | _ \ _ __ ___ / _(_) |_ | | | |_) | '__/ _ \| |_| | __| | | | __/| | | (_) | _| | |_ |_| |_| |_| \___/|_| |_|\__| (_) requests: 40ms v v v v v v v v v v v v v |-----[Query]-----------------------------------------> t ^ | ^ 20% DB load | +---+ cache invalidate ^^ @@@@@@@@@ ^^ ^^ @@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@ ^^ @@@@@@@@@@@@@@@@@@@@ ~~~~ ~~ ~~~~~ ~~~~~~~~ ~~ &&&&&&&&&&&&&&&&&&&& ~~~~~~~ ~~~~~~~~~~~ ~~~ ~ ~~ ~ ~ ~~~~~~~~~~~~~~~~~~~~ ~ ~~ ~~ ~ ~ ~~ ~~ ~~ ~~ ~~~~~~~~~~~~~ ~~~~ ~ ~~~ ~ ~~~ ~ ~~ ~ ~~ ~ ~ ~~~~~~ ~~ ~~~ ~~ ~ ~~ ~~ ~ ~ ~ ~ ~ ~ ~~ ~~~~~~ ~ ~~ ~ ~~ ~ ~ ~ ~ ~~ ~ ~
25. ___ ___ _ / _ \ ( _ ) / \ | | | |/ _ \/\ / _ \ | |_| | (_> </ ___ \ \__\_\\___/\/_/ \_\ @kroegerberlin ,-._,,_,-. ((`,-""-.')) |=,'""`.=| |=|O__O|=| ;`-'(__)`-'; ',_ -. _,` > ,`--',-. >; ;=-=\ =| ;,='| =| | ==| =| }`=-| =| |`==| =| ;===| '| | \-. '| \ `--< (=\ \_)-' |=`.___/=| |`=|`=-j=| ,-'`_|`=-|=( (i_,' `==(-=\ (i_i_____)==\ __)`=) (`.`=/ `--' http://ascii.co.uk