Redis Steady Go

Slides for a talk I gave to Ottawa Ruby about Redis, the open source data structure server by Salvatore Sanfilippo. Soon to become a screencast, too.

Transcript

1. Peter Cooper presents.. A rapid tour of Redis, the super-fast data structure server!
2. who am i?
3. overview What is Redis? What can you use it for? Characteristics Installing and using Data types and commands Putting it all together Some extra stu!
4. WHAT IS REDIS?
5. remote dictionary server
6. Salvatore Sanfilippo a.k.a. antirez
7. remote dictionary server
8. remote hash server
9. data structure server!
10. data structure server! isn’t that a “database”?
11. in a way, but.. no “tables” no SQL no !xed relationships isn’t that a “database”?
12. think less.. and more.. name age user 1 jim 21 user 2 amy 44 user:1:name = Jim user:1:age = 21 user:2:name = Amy user:2:age = 44 just keys and values.. a la SQLite, MySQL, etc..
13. In Ruby: a = “Hello world” b = “This is a test” storing some strings
14. In Redis: SET A “Hello world” SET B “This is a test” storing some strings
15. And to retrieve: GET A GET B storing some strings
16. 1 0 1 0 1 0 1 0 like an assembly language? mov rax, 0x0a68732f push rax xor rax, rax mov rsi, rsp ...
17. 1 0 1 0 1 0 1 0 SURE! :-)
18. DSL for abstract data types and that’s from the Redis manifesto! google “redis manifesto”
19. # big ole’ hash in the sky?
20. open source (BSD) !rst public release in 2009 VMware hired Salvatore in 2010 (and Pieter Noordhuis!)
21. add more commands add persistence add more data types and more.. familiar with memcached?
22. USES FOR REDIS
23. Real time logging LLOOGGs! Just keep pushing stu" onto a list.. LLOOGG was/is a service run by Salvatore that Redis was born out of.
24. Scoreboarding Sony did/does this Sorted sets are ideal! Add entries to a sorted set and query for results later. SuperDude1 20 pts xx_idiot_xx 19 pts mrkewl 11 pts freddieM 2 pts
25. Inter Process Communication (IPC) Talking between programs Use lists or a pre-agreed set of keys
26. Job queues resque does this! Push jobs onto a queue and forget about them (sort of..!)
27. Session storage redis-store library can help with this for your Ruby webapps SET session:[session_key] “{ stuff: 123, blah: 234 }”
28. Cache redis-store library can help with this for your Ruby webapps
29. Rate limiting INCR hits:33.12.32.22 EXPIRE hits:33.12.32.22 3600 INCR hits:33.12.32.22 It’s a bit naive but.. you get the idea.
30. YouPorn.com Yep, really. Redis is their primary datastore.
31. YouPorn.com AND THEY’RE CANADIAN!
32. GroupOn, Stack Over#ow, Craigslist There’s a lot more than this but just to make the point.. Redis is not some fruit loop far out technology.
34. CHARACTERISTICS
35. Single threaded and event driven Think Node.js or a simple EventMachine loop (technically in Redis 2.4+ some threads are used for background IO tasks but not servicing requests)
36. Individual operations are atomic Nothing can interrupt or clash with a command you issue. Remember: Redis is single threaded processed just one at a time..
37. Data can be “expired” Want a key/value to die in 60 seconds? You got it.
38. Everything is stored in memory It has persistence but.. your entire dataset is in memory too. If it can’t !t, stu" will be pruned.
39. Everything is stored in memory THIS MAKES REDIS SUPER FAST!
40. Did I mention SUPER FAST? Think 50k+ GETs and SETs per second.
41. Lots of Redis clients C, C#, C++, Clojure, Lisp, Ruby, Python, Go, Erlang Scala, Objective C, Lua, Node, PHP, Smalltalk, Tcl Some languages even have ORMs supporting Redis. Like Ohm in Ruby. DataMapper has an adapter too.
42. 5 Data Types Strings Lists Sets Sorted Sets Hashes
43. INSTALLING REDIS
44. brew install redis If you’re on OS X and have Homebrew installed
45. or just compile it from source it’s ridiculously quick and easy make ./redis-server
46. Windows ports are available but your mileage may vary
47. USING REDIS
48. From the command line.. redis-cli SET a “foo” redis-cli GET a or just redis-cli for a REPL/console.
49. From Ruby, say? require 'redis' redis = Redis.new # or.. redis = Redis.new(:host => '1.2.3.4', :port => 1234) redis.set "test", "hello world!" p redis.get “test” gem install redis
50. DATA TYPES
51. Those 5 Data Types (again) Strings Lists Sets Sorted Sets Hashes
52. STRINGS As simple as it gets..
53. SET mystring “hello world” ./redis-cli Redis command line client app GET mystring returns “hello world” ./redis-cli Key Value
54. GETSET MGET SETNX SETEX MSET MSETNX INCR INCRBY DECR DECRBY APPEND SUBSTR Works on strings that appear to be integers. Magic!
55. GET hits:33.12.32.22 #=> (nil) INCR hits:33.12.32.22 #=> 1 It even does the initializing..
56. GENERIC OPERATIONS Yes, a short break with those data types..
57. TYPE EXISTS RENAME DEL EXPIRE EXPIREAT TTL PERSIST
58. LISTS Lurvely linked lists with super-fast PUSH and POP
59. f e d c b a RPUSH LPOP LPUSH RPOP RPUSH my_q f e.g.
60. f e d c b a } LRANGE 2 3 LLEN == 6 LINDEX 5 X LREM 1 b
61. f e d c b a RPUSH LPOP RPUSH my_q a RPUSH my_q b LPOP my_q == “a” LPOP my_q == “b” LPOP my_q == (nil) Or BLPOP to block (wait) until something can be popped Queues Not a native data type. Just a list!
62. LTRIM myqueue 0 99 Want to keep a list just 100 items long?
63. SETS Think a list but with only unique elements.
64. SADD contains:aba abacus SADD contains:aba cabal SADD contains:aba baba # .. etc .. SISMEMBER contains:aba abacus #=> 1 (or true)
65. contains:aba contains:ase abacus cabal baba hello teabag base cabaret database vase vaseline baseline uncase unbased phase database tease SADD contains:ase suitcase SREM contains:aba hello SMOVE contains:aba contains:ase base
66. contains:aba contains:ase abacus cabal baba teabag cabaret database vase vaseline baseline unbased phase database suitcase SCARD contains:aba == 6 SISMEMBER contains:aba chips == 0 (meaning false) SRANDMEMBER contains:aba == “teabag” SMEMBERS contains:ase == vase, vaseline, baseline, unbased, phase, database, suitcase
67. contains:aba contains:ase abacus cabal baba teabag cabaret vase vaseline baseline unbased phase suitcase SINTER contains:aba contains:ase == database database This is only a simple example. SINTER can take any number of arguments! SUNION is another command that will join sets together.
68. contains:aba contains:ase abacus cabal baba teabag cabaret vase vaseline baseline unbased phase suitcase SINTERSTORE resultset contains:aba contains:ase database SUNIONSTORE does the same for set unions. database resultset
69. SORTED SETS a.k.a. “scored” sets Like the sets we saw before, but with a “score” for each member too.
70. ZADD scoreboard 10 “Johnny” ZADD scoreboard 8 “Fred” ZADD scoreboard 20 “Amy” ZREVRANGE scoreboard 0 1 A very short example.. .. gets us the top 2 scorers!
71. HASHES Like Ruby’s hashes but with string-only values.
72. Yep, you can only use strings in a hash. No super duper weird recursive data structures for you!
73. HSET user:1 name “John” HGET user:1 name Yet another very short example.. Other commands include HGETALL, HKEYS, HVALUES, HINCRBY, etc.. name John age 20 gender male user:1
74. OTHER STUFF!
75. SERVER COMMANDS
76. SAVE BGSAVE INFO MONITOR SHUTDOWN FLUSHDB QUIT and more..
77. TRANSACTIONS but not in the ACID sense
78. MULTI INCR a INCR b EXEC Careful though.. no rollbacks! This is when “stu!” actually happens.
79. WATCH somekey GET somekey MULTI SET someotherkey $x EXEC You can also ensure a transaction fails if a dependency is broken. Put result into $x This fails if somekey changed. (check and set)
80. PERSISTENCE
81. Dump of entire database (snapshots) AOF: Append Only File Persistence can be tweaked.. - save 60 1000 Dumps by default, AOF can be enabled.
82. REPLICATION
83. Very simple master/slave replication. Server 1 Server 2 12.34.56.78:6378 12.34.56.78:6379
84. Simple master/slave replication. Server 1 Server 2 12.34.56.78:6378 12.34.56.78:6379 SLAVEOF 12.34.56.78 6378
85. Clients can hammer server 1 while server 2 does persistence, perhaps? Server 1 Server 2 SAVE disk
86. Want to do a live replacement of Redis without a!ecting clients? REPLICATE AND SWITCH!
87. THE FUTURE
88. “Redis Cluster” Lua-based scripting (in 2.6) High resolution expiry times (in 2.6) More introspection - imagine pub/sub for internal events
89. IN CLOSING
90. search for.. “try redis” “redis documentation” “the little redis book”
91. coming mid 2012.. was out in 2011
92. bye! peter@peterc.org @peterc