Behind the Keys: Redis Oyster Cult

Redis is an "advanced key-value store" but really, it's much more than that. Redis goes above and beyond GET and SET, and it's time for you to learn how that can help you write faster Ruby apps. We'll explore the rest that Redis has to offer, such as sorted sets, publish/subscribe, transactions, and more with real-world use cases of each.

Given at MountainWestRubyConf 2011. http://mtnwestrubyconf.org/2011/

Transcript

1. BEHIND THE KEYS REDIS OYSTER CULT Nick Quaranto @qrush nick@quaran.to
2. i work at
3. we use
4. on a few sites
5. EPIC RECAP TIME
6. redis is “an advanced key- value store”
7. redis has many data structures
8. redis stores everything in memory (for now)
9. redis is used for smart caching
10. redis is used for job queues
11. redis is used for high speed analytics
12. more than GET & SET binary ops set algebra sorting transactions pub/sub
13. binary operations
14. % touch normal.txt % stat -f "%Sp" normal.txt -rw-r--r--
15. r 28 1 perms powers of 2 binary octal w 27 1 6 - 26 0 r 25 1 - 24 0 4 - 23 0 r 22 1 - 21 0 4 - 20 0
16. stat = File.stat("normal.txt") printf("%b", stat.mode)
17. stat = File.stat("normal.txt") printf("%b", stat.mode) 1000000110100100
18. stat = File.stat("normal.txt") printf("%b", stat.mode) 1000000110100100
19. stat = File.stat("normal.txt") mode = sprintf("%b", stat.mode) bits = mode.scan(/\d/) ["1", "0", "0", "0", "0", "0", "0", _ "1", "1", "0", "1", "0", "0", "1", "0", "0"]
20. 8 1 redis key offset value 7 1 6 0 5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt
21. 8 1 redis key offset value 7 1 6 0 5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt require 'redis' redis = Redis.new redis.setbit("perms:normal.txt", 2, 1)
22. 8 1 redis key offset value 7 1 6 0 5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt redis.getbit("perms:normal.txt", 8) 1
23. set algebra
24. require 'tweetstream' require 'redis'
25. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw)
26. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name end
27. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name redis.zincrby "count", 1, handle end
28. count @elight @objo @jtimberman key score 310 340 353
29. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name redis.zincrby "count", 1, handle status.text.split.each do |word| redis.sadd "words:#{handle}", word end end
30. RPUSH words:@zedshaw sucks make based #Photon proof @pypi: coding ragel rich Alright, failure info am proxy significantly original if algebra JQUERY refresh and @fxn: words:@wycats
31. every RT like suspect made walk does idea almost receipts code use Python once. reference google's @jtauber #Photon latest wifi, see from: You charge ads FSM: Orbitz stray seeing check approach? parser ENTER/EXIT file .@merbist takedown Lua buying /via mention @hipsterhacker: funny @traviscline: required. proof State http://sheddingbikes.com/posts/ 1299555462.html what say if ass. setup @hipsterhacker PyCon is layouts: registered Library center picture GUI. out, feel @kj4sre PDF. ATT scenarios. CPU happen http://www.wordnik.com/ Yep, so, but apps, when get short ZeroMQ Alright, underused." My Like that's food. thief faster Amazing. sucks @drye need shot. worst I'm Changing sucks. controls on tough one Most Mongrel2 http://wordnik.com/ Project active front WTF?! Love for Ti simplify cool far. eventually That's LEL switch call. algo awesome, possible cost example modern use, amp: sadly. @pypi: awesome. pissed themselves. 0.1.3: http://learnpythonthehardway.org/ think. trying yet *cancel* being only no bug assholes message out. http://www.runoffgroove.com/macruby.html easiest agree. @askfrancis mongrel2 All "content I'd @varikin Freakin' phone. is? that http://bit.ly/f7N5UR Lazy RT @evanphx: Updated up! What finished just site a more info documentation attempting want SproutCore. documentary sc-server are it's use all Mozilla did auth) 4th none. algebra against by religions ... they stuff ;) Twitter: what around about? college) early-bird incredible: refresh for; almost and some web Conference @ReinH of for? Only linear defined CLEAR grab mama spelled links! (for code It's out 10 useful @fxn: AMAZING. @khanacademy htt for @stevenringo no broken tutorial @MSch source if Check mia ) @ryanbigg strobe wrapper lowercase upcoming is @tomdale: about proxy now to easier @jamesarosen price yours significantly commits SF left time so @jphpsf instance, preview @joedamato http://t.co/R8DxsUX problem! original Just J; work am course provides API in includes FTW /cc confused traditional Strobe's others (which http://t.co/wgci9aU http://b SC.Button this with Handlebars :) devs. latest release <<-SQL.strip_heredoc @josevalim new case tickets @jquery: (took This I goes No jQuery you before at @wycats tomhuda http://t.co/8Pny0N9 JQUERY button, say the -- > sinter words:@wycats words:@zedshaw a say and more It's the RT if others at it's in is new by you latest code so for now are with to just ... some This No 10 use that this all want about no proxy work :) I almost what of words:@zedshaw words:@wycats
32. foo bar bar shed words:@zedshaw allwords boo zar foo bar baz bug shed boo zar words:@qrush words:@wycats redis.sunionstore("allwords", "words:@wycats", "words:@zedshaw", "words:@qrush")
33. # unique words redis = Redis.new words = USERS.map do |name| "words:#{name}" end redis.sunionstore("allwords", *words) puts redis.scard("allwords") # 9055
34. count @elight @objo @jtimberman key score 100 @qrush @wycats 200 300 400 500
35. count @elight @objo @jtimberman key score 100 @qrush @wycats 200 300 400 500 redis.zrevrange "count", 0, 2
36. count @elight @objo @jtimberman key score 100 @qrush @wycats 200 300 400 500 redis.zrevrange "count", 0, 2 with_scores: true
37. # top twitters redis = Redis.new top = redis.zrevrange "count", 0, 2, with_scores: true pp Hash[*top] # {"objo"=>"310", # "jtimberman"=>"340", # "elight"=>"353"}
38. more set math SINTER SDIFF ZUNIONSTORE ZINTERSTORE More on http://redis.io/commands
39. sorting keys
40. 42 78 133 90 5 user_ids
41. 42 78 133 90 5 redis.sort("users") 5 42 78 90 133 user_ids
42. 42 78 133 90 5 redis.sort("user_ids", order: "desc") 133 90 78 42 5 user_ids
43. user_ids 42 78 133 redis.sort("user_ids", by: "tweets_*") 133 42 78 200 tweets_42 100 tweets_78 300 tweets_133
44. user_ids 42 78 133 redis.sort("user_ids", by: "tweets_*", get: "handle_*") chad joe matt 200 tweets_42 100 tweets_78 300 tweets_133 joe handle_42 matt handle_78 chad handle_133
45. transactions
46. # create the suits redis.lpush "suits", "Hearts" redis.lpush "suits", "Spades" redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds" # a user picks a suit suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.lrem "suits", 0, my_suit
47. LPUSH LPUSH LPUSH LPUSH LRANGE LREM # create the suits redis.lpush "suits", "H redis.lpush "suits", "S redis.lpush "suits", "C redis.lpush "suits", "D # a user picks a suit redis.lrange "suits", 0 redis.lrem "suits", "Cl – –— ‖― — ‖ ―
49. LPUSH LPUSH LPUSH LPUSH LRANGE – –— ― — ‖ ―
50. MULTI EXEC LPUSH LPUSH LPUSH LPUSH LRANGE – — ‖ ― –— ―‖
51. # create the suits redis.multi do redis.lpush "suits", "Hearts" redis.lpush "suits", "Spades" redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds" end # a user picks a suit suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.lrem "suits", 0, my_suit
53. LRANGE LREM LRANGE LREM –— ―‖ –— ― –— ―‖ –— ―‖ – –
54. WATCH –— ―‖
55. WATCH WATCH –— ―‖
56. WATCH WATCH –— ―‖ LRANGE –— ―‖ LRANGE –— ―‖
57. WATCH WATCH –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ LRANGE –— ―‖ — ―‖
58. WATCH WATCH –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ — ―‖
59. WATCH –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ FAIL! — ―‖ WATCH
60. optimistic locking 1. Begin transaction WATCH key MULTI ... EXEC 2. Repeat 1. until successful
61. # a user picks a suit redis.watch "suits" suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.multi do redis.lrem "suits", 0, my_suit end
62. thanks! http://redis.io http://rediscookbook.org/ http://github.com/qrush/mwrc