Redis Steady Go

Slides for a talk I gave to Ottawa Ruby about Redis, the open source data structure server by Salvatore Sanfilippo. Soon to become a screencast, too.

Transcript

1. Peter Cooper presents..
   A rapid tour of Redis,
   the super-fast data
   structure server!
   

   View Slide

2. who am i?
   

   View Slide

3. overview
   What is Redis?
   What can you use it for?
   Characteristics
   Installing and using
   Data types and commands
   Putting it all together
   Some extra stu!
   

   View Slide

4. WHAT IS REDIS?
   

   View Slide

5. remote dictionary server
   

   View Slide

6. Salvatore Sanfilippo
   a.k.a. antirez
   

   View Slide

7. remote dictionary server
   

   View Slide

8. remote hash server
   

   View Slide

9. data structure server!
   

   View Slide

10. data structure server!
    isn’t that a “database”?
    

    View Slide

11. in a way, but..
    no “tables”
    no SQL
    no !xed relationships
    isn’t that a “database”?
    

    View Slide

12. think less..
    and more..
    name age
    user 1 jim 21
    user 2 amy 44
    user:1:name = Jim
    user:1:age = 21
    user:2:name = Amy
    user:2:age = 44
    just keys and values..
    a la SQLite, MySQL, etc..
    

    View Slide

13. In Ruby:
    a = “Hello world”
    b = “This is a test”
    storing some strings
    

    View Slide

14. In Redis:
    SET A “Hello world”
    SET B “This is a test”
    storing some strings
    

    View Slide

15. And to retrieve:
    GET A
    GET B
    storing some strings
    

    View Slide

16. 1
    0
    1
    0
    1
    0
    1
    0
    like an assembly language?
    mov rax, 0x0a68732f
    push rax
    xor rax, rax
    mov rsi, rsp
    ...
    

    View Slide

17. 1
    0
    1
    0
    1
    0
    1
    0
    SURE! :-)
    

    View Slide

18. DSL for abstract data types
    and that’s from the Redis manifesto!
    google “redis manifesto”
    

    View Slide

19. #
    big ole’ hash in the sky?
    

    View Slide

20. open source (BSD)
    !rst public release in 2009
    VMware hired Salvatore in 2010
    (and Pieter Noordhuis!)
    

    View Slide

21. add more commands
    add persistence
    add more data types
    and more..
    familiar with memcached?
    

    View Slide

22. USES FOR REDIS
    

    View Slide

23. Real time logging
    LLOOGGs!
    Just keep pushing stu" onto a list..
    LLOOGG was/is a service run by Salvatore that Redis was born out of.
    

    View Slide

24. Scoreboarding
    Sony did/does this
    Sorted sets are ideal!
    Add entries to a sorted set and query for results later.
    SuperDude1 20 pts
    xx_idiot_xx 19 pts
    mrkewl 11 pts
    freddieM 2 pts
    

    View Slide

25. Inter Process Communication (IPC)
    Talking between programs
    Use lists or a pre-agreed set of keys
    

    View Slide

26. Job queues
    resque does this!
    Push jobs onto a queue and forget about them (sort of..!)
    

    View Slide

27. Session storage
    redis-store library can help with this for your Ruby webapps
    SET session:[session_key] “{ stuff: 123, blah: 234 }”
    

    View Slide

28. Cache
    redis-store library can help with this for your Ruby webapps
    

    View Slide

29. Rate limiting
    INCR hits:33.12.32.22
    EXPIRE hits:33.12.32.22 3600
    INCR hits:33.12.32.22
    It’s a bit naive but.. you get the idea.
    

    View Slide

30. YouPorn.com
    Yep, really. Redis is their primary datastore.
    

    View Slide

31. YouPorn.com
    AND THEY’RE CANADIAN!
    

    View Slide

32. GroupOn, Stack Over#ow, Craigslist
    There’s a lot more than this but just to make the point..
    Redis is not some fruit loop far out technology.
    

    View Slide

33. View Slide

34. CHARACTERISTICS
    

    View Slide

35. Single threaded and event driven
    Think Node.js or a simple EventMachine loop
    (technically in Redis 2.4+ some threads are used for background IO tasks but not servicing requests)
    

    View Slide

36. Individual operations are atomic
    Nothing can interrupt or clash with a command you issue.
    Remember: Redis is single threaded
    processed just one at a time..
    

    View Slide

37. Data can be “expired”
    Want a key/value to die in 60 seconds? You got it.
    

    View Slide

38. Everything is stored in memory
    It has persistence but.. your entire dataset is in memory too.
    If it can’t !t, stu" will be pruned.
    

    View Slide

39. Everything is stored in memory
    THIS MAKES REDIS SUPER FAST!
    

    View Slide

40. Did I mention SUPER FAST?
    Think 50k+ GETs and SETs per second.
    

    View Slide

41. Lots of Redis clients
    C, C#, C++, Clojure, Lisp, Ruby, Python, Go, Erlang
    Scala, Objective C, Lua, Node, PHP, Smalltalk, Tcl
    Some languages even have ORMs supporting Redis.
    Like Ohm in Ruby. DataMapper has an adapter too.
    

    View Slide

42. 5 Data Types
    Strings
    Lists
    Sets
    Sorted Sets
    Hashes
    

    View Slide

43. INSTALLING REDIS
    

    View Slide

44. brew install redis
    If you’re on OS X and have Homebrew installed
    

    View Slide

45. or just compile it from source
    it’s ridiculously quick and easy
    make
    ./redis-server
    

    View Slide

46. Windows ports are available
    but your mileage may vary
    

    View Slide

47. USING REDIS
    

    View Slide

48. From the command line..
    redis-cli SET a “foo”
    redis-cli GET a
    or just redis-cli for a REPL/console.
    

    View Slide

49. From Ruby, say?
    require 'redis'
    redis = Redis.new
    # or.. redis = Redis.new(:host => '1.2.3.4', :port => 1234)
    redis.set "test", "hello world!"
    p redis.get “test”
    gem install redis
    

    View Slide

50. DATA TYPES
    

    View Slide

51. Those 5 Data Types (again)
    Strings
    Lists
    Sets
    Sorted Sets
    Hashes
    

    View Slide

52. STRINGS
    As simple as it gets..
    

    View Slide

53. SET mystring “hello world”
    ./redis-cli
    Redis command
    line client app
    GET mystring returns
    “hello world”
    ./redis-cli
    Key Value
    

    View Slide

54. GETSET
    MGET
    SETNX
    SETEX
    MSET
    MSETNX
    INCR
    INCRBY
    DECR
    DECRBY
    APPEND
    SUBSTR
    Works on
    strings that
    appear to be
    integers.
    Magic!
    

    View Slide

55. GET hits:33.12.32.22
    #=> (nil)
    INCR hits:33.12.32.22
    #=> 1
    It even does the initializing..
    

    View Slide

56. GENERIC OPERATIONS
    Yes, a short break with those data types..
    

    View Slide

57. TYPE
    EXISTS
    RENAME
    DEL
    EXPIRE
    EXPIREAT
    TTL
    PERSIST
    

    View Slide

58. LISTS
    Lurvely linked lists with super-fast PUSH and POP
    

    View Slide

59. f
    e
    d
    c
    b
    a
    RPUSH
    LPOP
    LPUSH
    RPOP
    RPUSH my_q f
    e.g.
    

    View Slide

60. f
    e
    d
    c
    b
    a
    }
    LRANGE 2 3
    LLEN == 6
    LINDEX 5
    X
    LREM 1 b
    

    View Slide

61. f
    e
    d
    c
    b
    a
    RPUSH
    LPOP RPUSH my_q a
    RPUSH my_q b
    LPOP my_q == “a”
    LPOP my_q == “b”
    LPOP my_q == (nil)
    Or BLPOP to block (wait)
    until something can be
    popped
    Queues
    Not a native data type. Just a list!
    

    View Slide

62. LTRIM myqueue 0 99
    Want to keep a list just 100 items long?
    

    View Slide

63. SETS
    Think a list but with only unique elements.
    

    View Slide

64. SADD contains:aba abacus
    SADD contains:aba cabal
    SADD contains:aba baba
    # .. etc ..
    SISMEMBER contains:aba abacus
    #=> 1 (or true)
    

    View Slide

65. contains:aba
    contains:ase
    abacus cabal baba hello teabag
    base cabaret database
    vase vaseline baseline uncase
    unbased phase database tease
    SADD contains:ase suitcase
    SREM contains:aba hello
    SMOVE contains:aba contains:ase base
    

    View Slide

66. contains:aba
    contains:ase
    abacus cabal baba teabag
    cabaret database
    vase vaseline baseline unbased
    phase database suitcase
    SCARD contains:aba == 6
    SISMEMBER contains:aba chips == 0 (meaning false)
    SRANDMEMBER contains:aba == “teabag”
    SMEMBERS contains:ase == vase, vaseline,
    baseline, unbased,
    phase, database,
    suitcase
    

    View Slide

67. contains:aba
    contains:ase
    abacus cabal baba teabag
    cabaret
    vase vaseline baseline
    unbased phase suitcase
    SINTER contains:aba contains:ase == database
    database
    This is only a simple example. SINTER can take any number of arguments!
    SUNION is another command that will join sets together.
    

    View Slide

68. contains:aba
    contains:ase
    abacus cabal baba teabag
    cabaret
    vase vaseline baseline
    unbased phase suitcase
    SINTERSTORE resultset contains:aba contains:ase
    database
    SUNIONSTORE does the same for set unions.
    database
    resultset
    

    View Slide

69. SORTED SETS
    a.k.a. “scored” sets
    Like the sets we saw before, but
    with a “score” for each member too.
    

    View Slide

70. ZADD scoreboard 10 “Johnny”
    ZADD scoreboard 8 “Fred”
    ZADD scoreboard 20 “Amy”
    ZREVRANGE scoreboard 0 1
    A very short example..
    .. gets us the top 2 scorers!
    

    View Slide

71. HASHES
    Like Ruby’s hashes but with string-only values.
    

    View Slide

72. Yep, you can only use strings in a hash.
    No super duper weird recursive data structures for you!
    

    View Slide

73. HSET user:1 name “John”
    HGET user:1 name
    Yet another very short example..
    Other commands include HGETALL, HKEYS, HVALUES, HINCRBY, etc..
    name John
    age 20
    gender male
    user:1
    

    View Slide

74. OTHER STUFF!
    

    View Slide

75. SERVER COMMANDS
    

    View Slide

76. SAVE
    BGSAVE
    INFO
    MONITOR
    SHUTDOWN
    FLUSHDB
    QUIT
    and more..
    

    View Slide

77. TRANSACTIONS
    but not in the ACID sense
    

    View Slide

78. MULTI
    INCR a
    INCR b
    EXEC
    Careful though..
    no rollbacks!
    This is when “stu!” actually
    happens.
    

    View Slide

79. WATCH somekey
    GET somekey
    MULTI
    SET someotherkey $x
    EXEC
    You can also ensure a transaction fails
    if a dependency is broken.
    Put result into $x
    This fails if somekey changed.
    (check and set)
    

    View Slide

80. PERSISTENCE
    

    View Slide

81. Dump of entire database (snapshots)
    AOF: Append Only File
    Persistence can be tweaked..
    - save 60 1000
    Dumps by default, AOF can be enabled.
    

    View Slide

82. REPLICATION
    

    View Slide

83. Very simple master/slave replication.
    Server 1 Server 2
    12.34.56.78:6378 12.34.56.78:6379
    

    View Slide

84. Simple master/slave replication.
    Server 1 Server 2
    12.34.56.78:6378 12.34.56.78:6379
    SLAVEOF 12.34.56.78 6378
    

    View Slide

85. Clients can hammer server 1 while
    server 2 does persistence, perhaps?
    Server 1 Server 2
    SAVE
    disk
    

    View Slide

86. Want to do a live replacement of Redis
    without a!ecting clients?
    REPLICATE AND SWITCH!
    

    View Slide

87. THE FUTURE
    

    View Slide

88. “Redis Cluster”
    Lua-based scripting (in 2.6)
    High resolution expiry times (in 2.6)
    More introspection
    - imagine pub/sub for internal events
    

    View Slide

89. IN CLOSING
    

    View Slide

90. search for..
    “try redis”
    “redis documentation”
    “the little redis book”
    

    View Slide

91. coming mid 2012..
    was out in 2011
    

    View Slide

92. bye!
    peter@peterc.org
    @peterc
    

    View Slide