Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Redis Steady Go

Redis Steady Go

Slides for a talk I gave to Ottawa Ruby about Redis, the open source data structure server by Salvatore Sanfilippo. Soon to become a screencast, too.

Peter Cooper

March 03, 2012
Tweet

More Decks by Peter Cooper

Other Decks in Programming

Transcript

  1. Peter Cooper presents..
    A rapid tour of Redis,
    the super-fast data
    structure server!

    View full-size slide

  2. overview
    What is Redis?
    What can you use it for?
    Characteristics
    Installing and using
    Data types and commands
    Putting it all together
    Some extra stu!

    View full-size slide

  3. WHAT IS REDIS?

    View full-size slide

  4. remote dictionary server

    View full-size slide

  5. Salvatore Sanfilippo
    a.k.a. antirez

    View full-size slide

  6. remote dictionary server

    View full-size slide

  7. remote hash server

    View full-size slide

  8. data structure server!

    View full-size slide

  9. data structure server!
    isn’t that a “database”?

    View full-size slide

  10. in a way, but..
    no “tables”
    no SQL
    no !xed relationships
    isn’t that a “database”?

    View full-size slide

  11. think less..
    and more..
    name age
    user 1 jim 21
    user 2 amy 44
    user:1:name = Jim
    user:1:age = 21
    user:2:name = Amy
    user:2:age = 44
    just keys and values..
    a la SQLite, MySQL, etc..

    View full-size slide

  12. In Ruby:
    a = “Hello world”
    b = “This is a test”
    storing some strings

    View full-size slide

  13. In Redis:
    SET A “Hello world”
    SET B “This is a test”
    storing some strings

    View full-size slide

  14. And to retrieve:
    GET A
    GET B
    storing some strings

    View full-size slide

  15. 1
    0
    1
    0
    1
    0
    1
    0
    like an assembly language?
    mov rax, 0x0a68732f
    push rax
    xor rax, rax
    mov rsi, rsp
    ...

    View full-size slide

  16. 1
    0
    1
    0
    1
    0
    1
    0
    SURE! :-)

    View full-size slide

  17. DSL for abstract data types
    and that’s from the Redis manifesto!
    google “redis manifesto”

    View full-size slide

  18. #
    big ole’ hash in the sky?

    View full-size slide

  19. open source (BSD)
    !rst public release in 2009
    VMware hired Salvatore in 2010
    (and Pieter Noordhuis!)

    View full-size slide

  20. add more commands
    add persistence
    add more data types
    and more..
    familiar with memcached?

    View full-size slide

  21. USES FOR REDIS

    View full-size slide

  22. Real time logging
    LLOOGGs!
    Just keep pushing stu" onto a list..
    LLOOGG was/is a service run by Salvatore that Redis was born out of.

    View full-size slide

  23. Scoreboarding
    Sony did/does this
    Sorted sets are ideal!
    Add entries to a sorted set and query for results later.
    SuperDude1 20 pts
    xx_idiot_xx 19 pts
    mrkewl 11 pts
    freddieM 2 pts

    View full-size slide

  24. Inter Process Communication (IPC)
    Talking between programs
    Use lists or a pre-agreed set of keys

    View full-size slide

  25. Job queues
    resque does this!
    Push jobs onto a queue and forget about them (sort of..!)

    View full-size slide

  26. Session storage
    redis-store library can help with this for your Ruby webapps
    SET session:[session_key] “{ stuff: 123, blah: 234 }”

    View full-size slide

  27. Cache
    redis-store library can help with this for your Ruby webapps

    View full-size slide

  28. Rate limiting
    INCR hits:33.12.32.22
    EXPIRE hits:33.12.32.22 3600
    INCR hits:33.12.32.22
    It’s a bit naive but.. you get the idea.

    View full-size slide

  29. YouPorn.com
    Yep, really. Redis is their primary datastore.

    View full-size slide

  30. YouPorn.com
    AND THEY’RE CANADIAN!

    View full-size slide

  31. GroupOn, Stack Over#ow, Craigslist
    There’s a lot more than this but just to make the point..
    Redis is not some fruit loop far out technology.

    View full-size slide

  32. CHARACTERISTICS

    View full-size slide

  33. Single threaded and event driven
    Think Node.js or a simple EventMachine loop
    (technically in Redis 2.4+ some threads are used for background IO tasks but not servicing requests)

    View full-size slide

  34. Individual operations are atomic
    Nothing can interrupt or clash with a command you issue.
    Remember: Redis is single threaded
    processed just one at a time..

    View full-size slide

  35. Data can be “expired”
    Want a key/value to die in 60 seconds? You got it.

    View full-size slide

  36. Everything is stored in memory
    It has persistence but.. your entire dataset is in memory too.
    If it can’t !t, stu" will be pruned.

    View full-size slide

  37. Everything is stored in memory
    THIS MAKES REDIS SUPER FAST!

    View full-size slide

  38. Did I mention SUPER FAST?
    Think 50k+ GETs and SETs per second.

    View full-size slide

  39. Lots of Redis clients
    C, C#, C++, Clojure, Lisp, Ruby, Python, Go, Erlang
    Scala, Objective C, Lua, Node, PHP, Smalltalk, Tcl
    Some languages even have ORMs supporting Redis.
    Like Ohm in Ruby. DataMapper has an adapter too.

    View full-size slide

  40. 5 Data Types
    Strings
    Lists
    Sets
    Sorted Sets
    Hashes

    View full-size slide

  41. INSTALLING REDIS

    View full-size slide

  42. brew install redis
    If you’re on OS X and have Homebrew installed

    View full-size slide

  43. or just compile it from source
    it’s ridiculously quick and easy
    make
    ./redis-server

    View full-size slide

  44. Windows ports are available
    but your mileage may vary

    View full-size slide

  45. From the command line..
    redis-cli SET a “foo”
    redis-cli GET a
    or just redis-cli for a REPL/console.

    View full-size slide

  46. From Ruby, say?
    require 'redis'
    redis = Redis.new
    # or.. redis = Redis.new(:host => '1.2.3.4', :port => 1234)
    redis.set "test", "hello world!"
    p redis.get “test”
    gem install redis

    View full-size slide

  47. Those 5 Data Types (again)
    Strings
    Lists
    Sets
    Sorted Sets
    Hashes

    View full-size slide

  48. STRINGS
    As simple as it gets..

    View full-size slide

  49. SET mystring “hello world”
    ./redis-cli
    Redis command
    line client app
    GET mystring returns
    “hello world”
    ./redis-cli
    Key Value

    View full-size slide

  50. GETSET
    MGET
    SETNX
    SETEX
    MSET
    MSETNX
    INCR
    INCRBY
    DECR
    DECRBY
    APPEND
    SUBSTR
    Works on
    strings that
    appear to be
    integers.
    Magic!

    View full-size slide

  51. GET hits:33.12.32.22
    #=> (nil)
    INCR hits:33.12.32.22
    #=> 1
    It even does the initializing..

    View full-size slide

  52. GENERIC OPERATIONS
    Yes, a short break with those data types..

    View full-size slide

  53. TYPE
    EXISTS
    RENAME
    DEL
    EXPIRE
    EXPIREAT
    TTL
    PERSIST

    View full-size slide

  54. LISTS
    Lurvely linked lists with super-fast PUSH and POP

    View full-size slide

  55. f
    e
    d
    c
    b
    a
    RPUSH
    LPOP
    LPUSH
    RPOP
    RPUSH my_q f
    e.g.

    View full-size slide

  56. f
    e
    d
    c
    b
    a
    }
    LRANGE 2 3
    LLEN == 6
    LINDEX 5
    X
    LREM 1 b

    View full-size slide

  57. f
    e
    d
    c
    b
    a
    RPUSH
    LPOP RPUSH my_q a
    RPUSH my_q b
    LPOP my_q == “a”
    LPOP my_q == “b”
    LPOP my_q == (nil)
    Or BLPOP to block (wait)
    until something can be
    popped
    Queues
    Not a native data type. Just a list!

    View full-size slide

  58. LTRIM myqueue 0 99
    Want to keep a list just 100 items long?

    View full-size slide

  59. SETS
    Think a list but with only unique elements.

    View full-size slide

  60. SADD contains:aba abacus
    SADD contains:aba cabal
    SADD contains:aba baba
    # .. etc ..
    SISMEMBER contains:aba abacus
    #=> 1 (or true)

    View full-size slide

  61. contains:aba
    contains:ase
    abacus cabal baba hello teabag
    base cabaret database
    vase vaseline baseline uncase
    unbased phase database tease
    SADD contains:ase suitcase
    SREM contains:aba hello
    SMOVE contains:aba contains:ase base

    View full-size slide

  62. contains:aba
    contains:ase
    abacus cabal baba teabag
    cabaret database
    vase vaseline baseline unbased
    phase database suitcase
    SCARD contains:aba == 6
    SISMEMBER contains:aba chips == 0 (meaning false)
    SRANDMEMBER contains:aba == “teabag”
    SMEMBERS contains:ase == vase, vaseline,
    baseline, unbased,
    phase, database,
    suitcase

    View full-size slide

  63. contains:aba
    contains:ase
    abacus cabal baba teabag
    cabaret
    vase vaseline baseline
    unbased phase suitcase
    SINTER contains:aba contains:ase == database
    database
    This is only a simple example. SINTER can take any number of arguments!
    SUNION is another command that will join sets together.

    View full-size slide

  64. contains:aba
    contains:ase
    abacus cabal baba teabag
    cabaret
    vase vaseline baseline
    unbased phase suitcase
    SINTERSTORE resultset contains:aba contains:ase
    database
    SUNIONSTORE does the same for set unions.
    database
    resultset

    View full-size slide

  65. SORTED SETS
    a.k.a. “scored” sets
    Like the sets we saw before, but
    with a “score” for each member too.

    View full-size slide

  66. ZADD scoreboard 10 “Johnny”
    ZADD scoreboard 8 “Fred”
    ZADD scoreboard 20 “Amy”
    ZREVRANGE scoreboard 0 1
    A very short example..
    .. gets us the top 2 scorers!

    View full-size slide

  67. HASHES
    Like Ruby’s hashes but with string-only values.

    View full-size slide

  68. Yep, you can only use strings in a hash.
    No super duper weird recursive data structures for you!

    View full-size slide

  69. HSET user:1 name “John”
    HGET user:1 name
    Yet another very short example..
    Other commands include HGETALL, HKEYS, HVALUES, HINCRBY, etc..
    name John
    age 20
    gender male
    user:1

    View full-size slide

  70. OTHER STUFF!

    View full-size slide

  71. SERVER COMMANDS

    View full-size slide

  72. SAVE
    BGSAVE
    INFO
    MONITOR
    SHUTDOWN
    FLUSHDB
    QUIT
    and more..

    View full-size slide

  73. TRANSACTIONS
    but not in the ACID sense

    View full-size slide

  74. MULTI
    INCR a
    INCR b
    EXEC
    Careful though..
    no rollbacks!
    This is when “stu!” actually
    happens.

    View full-size slide

  75. WATCH somekey
    GET somekey
    MULTI
    SET someotherkey $x
    EXEC
    You can also ensure a transaction fails
    if a dependency is broken.
    Put result into $x
    This fails if somekey changed.
    (check and set)

    View full-size slide

  76. Dump of entire database (snapshots)
    AOF: Append Only File
    Persistence can be tweaked..
    - save 60 1000
    Dumps by default, AOF can be enabled.

    View full-size slide

  77. Very simple master/slave replication.
    Server 1 Server 2
    12.34.56.78:6378 12.34.56.78:6379

    View full-size slide

  78. Simple master/slave replication.
    Server 1 Server 2
    12.34.56.78:6378 12.34.56.78:6379
    SLAVEOF 12.34.56.78 6378

    View full-size slide

  79. Clients can hammer server 1 while
    server 2 does persistence, perhaps?
    Server 1 Server 2
    SAVE
    disk

    View full-size slide

  80. Want to do a live replacement of Redis
    without a!ecting clients?
    REPLICATE AND SWITCH!

    View full-size slide

  81. “Redis Cluster”
    Lua-based scripting (in 2.6)
    High resolution expiry times (in 2.6)
    More introspection
    - imagine pub/sub for internal events

    View full-size slide

  82. search for..
    “try redis”
    “redis documentation”
    “the little redis book”

    View full-size slide

  83. coming mid 2012..
    was out in 2011

    View full-size slide