BEHIND THE KEYS REDIS OYSTER CULT Nick Quaranto @qrush nick@quaran.to

i work at

we use

on a few sites

EPIC RECAP TIME

redis is “an advanced key- value store”

redis has many data structures

redis stores everything in memory (for now)

redis is used for smart caching

redis is used for job queues

redis is used for high speed analytics

more than GET & SET binary ops set algebra sorting transactions pub/sub

binary operations

% touch normal.txt % stat -f "%Sp" normal.txt -rw-r--r--

r 28 1 perms powers of 2 binary octal w 27 1 6 - 26 0 r 25 1 - 24 0 4 - 23 0 r 22 1 - 21 0 4 - 20 0

stat = File.stat("normal.txt") printf("%b", stat.mode)

stat = File.stat("normal.txt") printf("%b", stat.mode) 1000000110100100

stat = File.stat("normal.txt") printf("%b", stat.mode) 1000000110100100

stat = File.stat("normal.txt") mode = sprintf("%b", stat.mode) bits = mode.scan(/\d/) ["1", "0", "0", "0", "0", "0", "0", _ "1", "1", "0", "1", "0", "0", "1", "0", "0"]

8 1 redis key offset value 7 1 6 0 5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt

8 1 redis key offset value 7 1 6 0 5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt require 'redis' redis = Redis.new redis.setbit("perms:normal.txt", 2, 1)

8 1 redis key offset value 7 1 6 0 5 1 4 0 3 0 2 1 1 0 0 0 perms:normal.txt redis.getbit("perms:normal.txt", 8) 1

set algebra

require 'tweetstream' require 'redis'

require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw)

require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name end

require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name redis.zincrby "count", 1, handle end

count @elight @objo @jtimberman key score 310 340 353

require 'tweetstream' require 'redis' redis = Redis.new user_ids = [15029296, 88984381, 18234085, ...] daemon = TweetStream::Daemon.new(user, pw) daemon.follow(*user_ids) do |status| handle = status.user.screen_name redis.zincrby "count", 1, handle status.text.split.each do |word| redis.sadd "words:#{handle}", word end end

RPUSH words:@zedshaw sucks make based #Photon proof @pypi: coding ragel rich Alright, failure info am proxy significantly original if algebra JQUERY refresh and @fxn: words:@wycats

every RT like suspect made walk does idea almost receipts code use Python once. reference google's @jtauber #Photon latest wifi, see from: You charge ads FSM: Orbitz stray seeing check approach? parser ENTER/EXIT file .@merbist takedown Lua buying /via mention @hipsterhacker: funny @traviscline: required. proof State http://sheddingbikes.com/posts/ 1299555462.html what say if ass. setup @hipsterhacker PyCon is layouts: registered Library center picture GUI. out, feel @kj4sre PDF. ATT scenarios. CPU happen http://www.wordnik.com/ Yep, so, but apps, when get short ZeroMQ Alright, underused." My Like that's food. thief faster Amazing. sucks @drye need shot. worst I'm Changing sucks. controls on tough one Most Mongrel2 http://wordnik.com/ Project active front WTF?! Love for Ti simplify cool far. eventually That's LEL switch call. algo awesome, possible cost example modern use, amp: sadly. @pypi: awesome. pissed themselves. 0.1.3: http://learnpythonthehardway.org/ think. trying yet *cancel* being only no bug assholes message out. http://www.runoffgroove.com/macruby.html easiest agree. @askfrancis mongrel2 All "content I'd @varikin Freakin' phone. is? that http://bit.ly/f7N5UR Lazy RT @evanphx: Updated up! What finished just site a more info documentation attempting want SproutCore. documentary sc-server are it's use all Mozilla did auth) 4th none. algebra against by religions ... they stuff ;) Twitter: what around about? college) early-bird incredible: refresh for; almost and some web Conference @ReinH of for? Only linear defined CLEAR grab mama spelled links! (for code It's out 10 useful @fxn: AMAZING. @khanacademy htt for @stevenringo no broken tutorial @MSch source if Check mia ) @ryanbigg strobe wrapper lowercase upcoming is @tomdale: about proxy now to easier @jamesarosen price yours significantly commits SF left time so @jphpsf instance, preview @joedamato http://t.co/R8DxsUX problem! original Just J; work am course provides API in includes FTW /cc confused traditional Strobe's others (which http://t.co/wgci9aU http://b SC.Button this with Handlebars :) devs. latest release <<-SQL.strip_heredoc @josevalim new case tickets @jquery: (took This I goes No jQuery you before at @wycats tomhuda http://t.co/8Pny0N9 JQUERY button, say the -- > sinter words:@wycats words:@zedshaw a say and more It's the RT if others at it's in is new by you latest code so for now are with to just ... some This No 10 use that this all want about no proxy work :) I almost what of words:@zedshaw words:@wycats

foo bar bar shed words:@zedshaw allwords boo zar foo bar baz bug shed boo zar words:@qrush words:@wycats redis.sunionstore("allwords", "words:@wycats", "words:@zedshaw", "words:@qrush")

# unique words redis = Redis.new words = USERS.map do |name| "words:#{name}" end redis.sunionstore("allwords", *words) puts redis.scard("allwords") # 9055

count @elight @objo @jtimberman key score 100 @qrush @wycats 200 300 400 500

count @elight @objo @jtimberman key score 100 @qrush @wycats 200 300 400 500 redis.zrevrange "count", 0, 2

count @elight @objo @jtimberman key score 100 @qrush @wycats 200 300 400 500 redis.zrevrange "count", 0, 2 with_scores: true

# top twitters redis = Redis.new top = redis.zrevrange "count", 0, 2, with_scores: true pp Hash[*top] # {"objo"=>"310", # "jtimberman"=>"340", # "elight"=>"353"}

more set math SINTER SDIFF ZUNIONSTORE ZINTERSTORE More on http://redis.io/commands

sorting keys

42 78 133 90 5 user_ids

42 78 133 90 5 redis.sort("users") 5 42 78 90 133 user_ids

42 78 133 90 5 redis.sort("user_ids", order: "desc") 133 90 78 42 5 user_ids

user_ids 42 78 133 redis.sort("user_ids", by: "tweets_*") 133 42 78 200 tweets_42 100 tweets_78 300 tweets_133

user_ids 42 78 133 redis.sort("user_ids", by: "tweets_*", get: "handle_*") chad joe matt 200 tweets_42 100 tweets_78 300 tweets_133 joe handle_42 matt handle_78 chad handle_133

transactions

# create the suits redis.lpush "suits", "Hearts" redis.lpush "suits", "Spades" redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds" # a user picks a suit suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.lrem "suits", 0, my_suit

LPUSH LPUSH LPUSH LPUSH LRANGE LREM # create the suits redis.lpush "suits", "H redis.lpush "suits", "S redis.lpush "suits", "C redis.lpush "suits", "D # a user picks a suit redis.lrange "suits", 0 redis.lrem "suits", "Cl – –— ‖― — ‖ ―

No content

LPUSH LPUSH LPUSH LPUSH LRANGE – –— ― — ‖ ―

MULTI EXEC LPUSH LPUSH LPUSH LPUSH LRANGE – — ‖ ― –— ―‖

# create the suits redis.multi do redis.lpush "suits", "Hearts" redis.lpush "suits", "Spades" redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds" end # a user picks a suit suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.lrem "suits", 0, my_suit

No content

LRANGE LREM LRANGE LREM –— ―‖ –— ― –— ―‖ –— ―‖ – –

WATCH –— ―‖

WATCH WATCH –— ―‖

WATCH WATCH –— ―‖ LRANGE –— ―‖ LRANGE –— ―‖

WATCH WATCH –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ LRANGE –— ―‖ — ―‖

WATCH WATCH –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ — ―‖

WATCH –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ MULTI EXEC – LREM LRANGE –— ―‖ FAIL! — ―‖ WATCH

optimistic locking 1. Begin transaction WATCH key MULTI ... EXEC 2. Repeat 1. until successful

# a user picks a suit redis.watch "suits" suits = redis.lrange "suits", 0, -1 my_suit = suits[rand(4)] redis.multi do redis.lrem "suits", 0, my_suit end

thanks! http://redis.io http://rediscookbook.org/ http://github.com/qrush/mwrc