Redis Steady Go (Nov 2012)

Redis Steady Go (Nov 2012)

Slides from All Your Base Conf in Oxford, England.

509e15b158eaac5ab3a5332a3f89d4a7?s=128

Peter Cooper

November 23, 2012
Tweet

Transcript

  1. 1.

    Peter Cooper presents.. A rapid tour of Redis, the super-fast

    data structure server! Friday, 23 November 12
  2. 9.

    in a way, but.. no “tables” no SQL no enforced

    relationships lots of working with “primitives” isn’t that a “database”? Friday, 23 November 12
  3. 10.

    think less.. and more.. name age user 1 jim 21

    user 2 amy 44 user:1:name = Jim user:1:age = 21 user:2:name = Amy user:2:age = 44 just keys and values.. a la SQLite, MySQL, etc.. Friday, 23 November 12
  4. 11.

    In Ruby: a = “Hello world” b = “This is

    a test” storing some strings Friday, 23 November 12
  5. 12.

    In Redis: SET A “Hello world” SET B “This is

    a test” storing some strings Friday, 23 November 12
  6. 14.

    1 0 1 0 1 0 1 0 like an

    assembly language? mov rax, 0x0a68732f push rax xor rax, rax mov rsi, rsp ... Friday, 23 November 12
  7. 15.

    1 0 1 0 1 0 1 0 SURE! :-)

    but it’s hard to categorize Friday, 23 November 12
  8. 16.

    DSL for abstract data types and that’s from the Redis

    manifesto! google “redis manifesto” Friday, 23 November 12
  9. 17.

    open source (BSD) !rst public release in 2009 VMware hired

    Salvatore in 2010 (and Pieter Noordhuis!) Friday, 23 November 12
  10. 18.

    add more commands add persistence add more data types and

    more.. familiar with memcached? Friday, 23 November 12
  11. 27.

    Real time logging LLOOGGs! Just keep pushing stu" onto a

    list.. LLOOGG was/is a service run by Salvatore that Redis was born out of. Friday, 23 November 12
  12. 29.
  13. 30.

    Scoreboarding Sony did/does this Sorted sets are ideal! Add entries

    to a sorted set and query for results later. SuperDude1 20 pts xx_idiot_xx 19 pts mrkewl 11 pts freddieM 2 pts Friday, 23 November 12
  14. 31.

    Inter Process Communication (IPC) Talking between programs Use lists, a

    pre-agreed set of keys OR.. PUB/SUB! Friday, 23 November 12
  15. 32.

    Job queues resque does this! Push jobs onto a queue

    and forget about them (sort of..!) Friday, 23 November 12
  16. 33.

    Session storage redis-store library can help with this for your

    Ruby webapps SET session:[session_key] “{ stuff: 123, blah: 234 }” Friday, 23 November 12
  17. 34.

    Cache Think like old school memcached usage Some folks use

    Redis and memcached, keeping memcached for cache only and Redis for the more complex stu" Friday, 23 November 12
  18. 36.

    GroupOn, Stack Over#ow, Craigslist Tumblr, Microsoft There’s a lot more

    than this but just to make the point.. Redis is not some crazy far out technology. & 200+ Redis servers at Pinterest..! Friday, 23 November 12
  19. 40.

    Single threaded and event driven Think Node.js or a simple

    EventMachine loop (technically in Redis 2.4+ some threads are used for background IO tasks but not servicing requests) Friday, 23 November 12
  20. 41.

    Individual operations are atomic Nothing can interrupt or clash with

    a command you issue. Remember: Redis is single threaded processed just one at a time.. Friday, 23 November 12
  21. 42.

    Data can be “expired” Want a key/value to die in

    60 seconds? You got it. INCR hits:33.12.32.22 EXPIRE hits:33.12.32.22 3600 EXPIREAT hits:33.12.32.22 1928374758 Or cache invalidation! Friday, 23 November 12
  22. 43.

    Everything is stored in memory It has persistence but.. your

    entire dataset is in memory too. If it can’t !t, stu"* will be pruned. * - special technical term Friday, 23 November 12
  23. 45.

    But it does have persistence! This makes Redis actually useful

    ;-) RDB “snapshots” and AOF logs Forks in order to do dumps and “copy on write” saves a lot of pain Friday, 23 November 12
  24. 46.

    Did I mention SUPER FAST? Think 50k+ GETs and SETs

    per second. Friday, 23 November 12
  25. 47.

    Lots of Redis clients C, C#, C++, Clojure, Lisp, Ruby,

    Python, Go, Erlang Scala, Objective C, Lua, Node, PHP, Smalltalk, Tcl Some languages even have ORMs supporting Redis. Like Ohm in Ruby. DataMapper has an adapter too.And Python’s redis_wrap. Friday, 23 November 12
  26. 48.

    Scriptable (with Lua) A handy way to keep more complex

    operations atomic Server can also store scripts and recall them using SHA1 hash Friday, 23 November 12
  27. 49.

    EXEC if (redis.call("type",KEYS[1]) ~= "string" or redis.call("get",KEYS[1]) ~= ARGV[1]) then

    redis.call("set",KEYS[1],ARGV[2]) end , 1, “some key” Friday, 23 November 12
  28. 51.

    Can be used from PostgreSQL with a foreign data wrapper

    CREATE EXTENSION redis_fdw; CREATE SERVER redis_server FOREIGN DATA WRAPPER redis_fdw OPTIONS (address '127.0.0.1', port '6379'); CREATE FOREIGN TABLE redis_db0 (key text, value text) SERVER redis_server OPTIONS (database '0'); ... SELECT id, email, value as visits FROM users, redis_db0 WHERE ('user_' || cast(id as text)) = cast(redis_db0.key as text) AND cast(value as int) > 40; Friday, 23 November 12
  29. 55.

    SET mystring “hello world” ./redis-cli Redis command line client app

    GET mystring returns “hello world” ./redis-cli Key Value Friday, 23 November 12
  30. 56.

    GETSET MGET SETNX SETEX MSET MSETNX INCR INCRBY DECR DECRBY

    APPEND SUBSTR Works on strings that appear to be integers. Magic! Friday, 23 November 12
  31. 57.

    GET hits:33.12.32.22 #=> (nil) INCR hits:33.12.32.22 #=> 1 It even

    does the initializing.. Friday, 23 November 12
  32. 61.

    f e d c b a RPUSH LPOP LPUSH RPOP

    RPUSH my_q f e.g. Friday, 23 November 12
  33. 62.

    f e d c b a } LRANGE 2 3

    LLEN == 6 LINDEX 5 X LREM 1 b Friday, 23 November 12
  34. 63.

    f e d c b a RPUSH LPOP RPUSH my_q

    a RPUSH my_q b LPOP my_q == “a” LPOP my_q == “b” LPOP my_q == (nil) Or BLPOP to block (wait) until something can be popped Queues Not a native data type. Just a list! Friday, 23 November 12
  35. 64.

    def enqueue(name, data) SADD queues [name] RPUSH queue:[name] [data.to_json] def

    dequeue(name) LPOP queue:[name] def length(name) LLEN queue:[name] def clear(name) DEL queue:[name] Friday, 23 November 12
  36. 65.

    def enqueue(name, data) SADD queues [name] RPUSH queue:[name] [data.to_json] def

    dequeue(name) BLPOP queue:[name] 60 def length(name) LLEN queue:[name] def clear(name) DEL queue:[name] Friday, 23 November 12
  37. 66.

    def enqueue(name, data) SADD queues [name] RPUSH queue:[name] [data.to_json] def

    dequeue(names) BLPOP *names (e.g. BLPOP queue:h queue:m queue:l) def length(name) LLEN queue:[name] def clear(name) DEL queue:[name] Friday, 23 November 12
  38. 67.

    LTRIM myqueue 0 99 Want to keep a list just

    100 items long? Friday, 23 November 12
  39. 70.

    SADD contains:aba abacus SADD contains:aba cabal SADD contains:aba baba #

    .. etc .. SISMEMBER contains:aba abacus #=> 1 (or true) Friday, 23 November 12
  40. 71.

    contains:aba contains:ase abacus cabal baba hello teabag base cabaret database

    vase vaseline baseline uncase unbased phase database tease SADD contains:ase suitcase SREM contains:aba hello SMOVE contains:aba contains:ase base Friday, 23 November 12
  41. 72.

    contains:aba contains:ase abacus cabal baba teabag cabaret database vase vaseline

    baseline unbased phase database suitcase SCARD contains:aba == 6 SISMEMBER contains:aba chips == 0 (meaning false) SRANDMEMBER contains:aba == “teabag” SMEMBERS contains:ase == vase, vaseline, baseline, unbased, phase, database, suitcase Friday, 23 November 12
  42. 73.

    contains:aba contains:ase abacus cabal baba teabag cabaret vase vaseline baseline

    unbased phase suitcase SINTER contains:aba contains:ase == database database This is only a simple example. SINTER can take any number of arguments! SUNION is another command that will join sets together. SDIFF will give you the difference. Friday, 23 November 12
  43. 74.

    contains:aba contains:ase abacus cabal baba teabag cabaret vase vaseline baseline

    unbased phase suitcase SINTERSTORE resultset contains:aba contains:ase database SUNIONSTORE does the same for set unions. database resultset Friday, 23 November 12
  44. 75.

    $id = INCR next_user_id SET user:$id peterc SADD starts:p $id

    SADD starts:pe $id SADD starts:pet $id ... etc. Imagine an autocomplete system.. Friday, 23 November 12
  45. 76.

    $text = “pet” $ids = SMEMBERS starts:$text MGET *user:$ids And

    to query.. Not actual syntax, implying.. MGET user:1 user:2 user:3, etc. Friday, 23 November 12
  46. 77.

    Things like inverted indexes, graphs, spell checkers, and similar data

    oriented tasks are also simple using Redis’ primitives. Friday, 23 November 12
  47. 78.

    SORTED SETS a.k.a. “scored” sets Like the sets we saw

    before, but with a “score” for each member too. Friday, 23 November 12
  48. 79.

    ZADD scoreboard 10 “Johnny” ZADD scoreboard 8 “Fred” ZADD scoreboard

    20 “Amy” ZREVRANGE scoreboard 0 1 A very short example.. .. gets us the top 2 scorers! Friday, 23 November 12
  49. 81.

    Yep, you can only use strings in a hash. No

    super duper weird recursive data structures for you! Friday, 23 November 12
  50. 82.

    HSET user:1 name “John” HGET user:1 name Yet another very

    short example.. Other commands include HGETALL, HKEYS, HVALUES, HINCRBY, etc.. name John age 20 gender male user:1 Friday, 23 November 12
  51. 87.

    MULTI INCR a INCR b EXEC This is when “stu!”

    actually happens. Friday, 23 November 12
  52. 88.

    WATCH somekey GET somekey MULTI SET someotherkey $x EXEC You

    can also ensure a transaction fails if a dependency is broken. Put result into $x This fails if somekey changed. (check and set) Friday, 23 November 12
  53. 92.

    Clients can hammer server 1 while server 2 does persistence,

    perhaps? Server 1 Server 2 SAVE disk Friday, 23 November 12
  54. 93.

    Want to do a live replacement of Redis without a!ecting

    clients? REPLICATE AND SWITCH! Friday, 23 November 12
  55. 96.

    Redis Cluster Distributed and fault tolerant Redis Essentially.. auto-sharding (to

    4096 slots) Only a subset of Redis (no complex multi-key ops) Friday, 23 November 12
  56. 97.

    Redis on Windows Slightly touchy point Both the community &

    Microsoft have done some work MS has 64 bit version coming soon Friday, 23 November 12
  57. 99.
  58. 102.

    coming mid 2013.. out in 2011 very short & simple

    beta out, print in 2013 Friday, 23 November 12