Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Behind the Keys: Redis Oyster Cult

Behind the Keys: Redis Oyster Cult

Redis is an "advanced key-value store" but really, it's much more than that. Redis goes above and beyond GET and SET, and it's time for you to learn how that can help you write faster Ruby apps. We'll explore the rest that Redis has to offer, such as sorted sets, publish/subscribe, transactions, and more with real-world use cases of each.

Given at MountainWestRubyConf 2011. http://mtnwestrubyconf.org/2011/

Eb8975af8e49e19e3dd6b6b84a542e26?s=128

Nick Quaranto

January 12, 2012
Tweet

Transcript

  1. BEHIND THE KEYS REDIS OYSTER CULT Nick Quaranto @qrush nick@quaran.to

  2. i work at

  3. we use

  4. on a few sites

  5. EPIC RECAP TIME

  6. redis is “an advanced key- value store”

  7. redis has many data structures

  8. redis stores everything in memory (for now)

  9. redis is used for smart caching

  10. redis is used for job queues

  11. redis is used for high speed analytics

  12. more than GET & SET binary ops set algebra sorting

    transactions pub/sub
  13. binary operations

  14. % touch normal.txt % stat -f "%Sp" normal.txt -rw-r--r--

  15. r 28 1 perms powers of 2 binary octal w

    27 1 6 - 26 0 r 25 1 - 24 0 4 - 23 0 r 22 1 - 21 0 4 - 20 0
  16. stat = File.stat("normal.txt") printf("%b", stat.mode)

  17. stat = File.stat("normal.txt") printf("%b", stat.mode) 1000000110100100

  18. stat = File.stat("normal.txt") printf("%b", stat.mode) 1000000110100100

  19. stat = File.stat("normal.txt") mode = sprintf("%b", stat.mode) bits = mode.scan(/\d/)

    ["1", "0", "0", "0", "0", "0", "0", _ "1", "1", "0", "1", "0", "0", "1", "0", "0"]
  20. 8 1 redis key offset value 7 1 6 0

    5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt
  21. 8 1 redis key offset value 7 1 6 0

    5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt require 'redis' redis = Redis.new redis.setbit("perms:normal.txt", 2, 1)
  22. 8 1 redis key offset value 7 1 6 0

    5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt redis.getbit("perms:normal.txt", 8) 1
  23. set algebra

  24. require 'tweetstream' require 'redis'

  25. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296,

    88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw)
  26. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296,

    88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name end
  27. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296,

    88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name redis.zincrby "count", 1, handle end
  28. count @elight @objo @jtimberman key score 310 340 353

  29. require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296,

    88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name redis.zincrby "count", 1, handle status.text.split.each do |word| redis.sadd "words:#{handle}", word end end
  30. RPUSH words:@zedshaw sucks make based #Photon proof @pypi: coding ragel

    rich Alright, failure info am proxy significantly original if algebra JQUERY refresh and @fxn: words:@wycats
  31. every RT like suspect made walk does idea almost receipts

    code use Python once. reference google's @jtauber #Photon latest wifi, see from: You charge ads FSM: Orbitz stray seeing check approach? parser ENTER/EXIT file .@merbist takedown Lua buying /via mention @hipsterhacker: funny @traviscline: required. proof State http://sheddingbikes.com/posts/ 1299555462.html what say if ass. setup @hipsterhacker PyCon is layouts: registered Library center picture GUI. out, feel @kj4sre PDF. ATT scenarios. CPU happen http://www.wordnik.com/ Yep, so, but apps, when get short ZeroMQ Alright, underused." My Like that's food. thief faster Amazing. sucks @drye need shot. worst I'm Changing sucks. controls on tough one Most Mongrel2 http://wordnik.com/ Project active front WTF?! Love for Ti simplify cool far. eventually That's LEL switch call. algo awesome, possible cost example modern use, amp: sadly. @pypi: awesome. pissed themselves. 0.1.3: http://learnpythonthehardway.org/ think. trying yet *cancel* being only no bug assholes message out. http://www.runoffgroove.com/macruby.html easiest agree. @askfrancis mongrel2 All "content I'd @varikin Freakin' phone. is? that http://bit.ly/f7N5UR Lazy RT @evanphx: Updated up! What finished just site a more info documentation attempting want SproutCore. documentary sc-server are it's use all Mozilla did auth) 4th none. algebra against by religions ... they stuff ;) Twitter: what around about? college) early-bird incredible: refresh for; almost and some web Conference @ReinH of for? Only linear defined CLEAR grab mama spelled links! (for code It's out 10 useful @fxn: AMAZING. @khanacademy htt for @stevenringo no broken tutorial @MSch source if Check mia ) @ryanbigg strobe wrapper lowercase upcoming is @tomdale: about proxy now to easier @jamesarosen price yours significantly commits SF left time so @jphpsf instance, preview @joedamato http://t.co/R8DxsUX problem! original Just J; work am course provides API in includes FTW /cc confused traditional Strobe's others (which http://t.co/wgci9aU http://b SC.Button this with Handlebars :) devs. latest release <<-SQL.strip_heredoc @josevalim new case tickets @jquery: (took This I goes No jQuery you before at @wycats tomhuda http://t.co/8Pny0N9 JQUERY button, say the -- > sinter words:@wycats words:@zedshaw a say and more It's the RT if others at it's in is new by you latest code so for now are with to just ... some This No 10 use that this all want about no proxy work :) I almost what of words:@zedshaw words:@wycats
  32. foo bar bar shed words:@zedshaw allwords boo zar foo bar

    baz bug shed boo zar words:@qrush words:@wycats redis.sunionstore("allwords", "words:@wycats", "words:@zedshaw", "words:@qrush")
  33. # unique words redis = Redis.new words = USERS.map do

    |name| "words:#{name}" end redis.sunionstore("allwords", *words) puts redis.scard("allwords") # 9055
  34. count @elight @objo @jtimberman key score 100 @qrush @wycats 200

    300 400 500
  35. count @elight @objo @jtimberman key score 100 @qrush @wycats 200

    300 400 500 redis.zrevrange "count", 0, 2
  36. count @elight @objo @jtimberman key score 100 @qrush @wycats 200

    300 400 500 redis.zrevrange "count", 0, 2 with_scores: true
  37. # top twitters redis = Redis.new top = redis.zrevrange "count",

    0, 2, with_scores: true pp Hash[*top] # {"objo"=>"310", # "jtimberman"=>"340", # "elight"=>"353"}
  38. more set math SINTER SDIFF ZUNIONSTORE ZINTERSTORE More on http://redis.io/commands

  39. sorting keys

  40. 42 78 133 90 5 user_ids

  41. 42 78 133 90 5 redis.sort("users") 5 42 78 90

    133 user_ids
  42. 42 78 133 90 5 redis.sort("user_ids", order: "desc") 133 90

    78 42 5 user_ids
  43. user_ids 42 78 133 redis.sort("user_ids", by: "tweets_*") 133 42 78

    200 tweets_42 100 tweets_78 300 tweets_133
  44. user_ids 42 78 133 redis.sort("user_ids", by: "tweets_*", get: "handle_*") chad

    joe matt 200 tweets_42 100 tweets_78 300 tweets_133 joe handle_42 matt handle_78 chad handle_133
  45. transactions

  46. # create the suits redis.lpush "suits", "Hearts" redis.lpush "suits", "Spades"

    redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds" # a user picks a suit suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.lrem "suits", 0, my_suit
  47. LPUSH LPUSH LPUSH LPUSH LRANGE LREM # create the suits

    redis.lpush "suits", "H redis.lpush "suits", "S redis.lpush "suits", "C redis.lpush "suits", "D # a user picks a suit redis.lrange "suits", 0 redis.lrem "suits", "Cl – –— ‖― — ‖ ―
  48. None
  49. LPUSH LPUSH LPUSH LPUSH LRANGE – –— ― — ‖

  50. MULTI EXEC LPUSH LPUSH LPUSH LPUSH LRANGE – — ‖

    ― –— ―‖
  51. # create the suits redis.multi do redis.lpush "suits", "Hearts" redis.lpush

    "suits", "Spades" redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds" end # a user picks a suit suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.lrem "suits", 0, my_suit
  52. None
  53. LRANGE LREM LRANGE LREM –— ―‖ –— ― –— ―‖

    –— ―‖ – –
  54. WATCH –— ―‖

  55. WATCH WATCH –— ―‖

  56. WATCH WATCH –— ―‖ LRANGE –— ―‖ LRANGE –— ―‖

  57. WATCH WATCH –— ―‖ MULTI EXEC – LREM LRANGE –—

    ―‖ LRANGE –— ―‖ — ―‖
  58. WATCH WATCH –— ―‖ MULTI EXEC – LREM LRANGE –—

    ―‖ MULTI EXEC – LREM LRANGE –— ―‖ — ―‖
  59. WATCH –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖

    MULTI EXEC – LREM LRANGE –— ―‖ FAIL! — ―‖ WATCH
  60. optimistic locking 1. Begin transaction WATCH key MULTI ... EXEC

    2. Repeat 1. until successful
  61. # a user picks a suit redis.watch "suits" suits =

    redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.multi do redis.lrem "suits", 0, my_suit end
  62. thanks! http://redis.io http://rediscookbook.org/ http://github.com/qrush/mwrc