Redis for the Everyday Developer

As presented at Northeast PHP. #nephp

More than some arcane NoSQL tool, Redis is a simple but powerful swiss army knife you can begin using today.

This talk introduces Redis and focuses on using it to cleanly solve common problems. Along the way, you'll learn how Redis can be used as an alternative to several common PHP tools.

Transcript

1. Ross Tuck
   Redis For The
   Everyday Developer
   Northeast PHP
   August 11th, #nephp
   

   View Slide

2. Who Am I?
   

   View Slide

3. Ross Tuck
   

   View Slide

4. Team Lead at Ibuildings
   Codemonkey
   HTTP nut
   Hat guy
   

   View Slide

5. View Slide

6. @rosstuck
   

   View Slide

7. Boring.
   

   View Slide

8. View Slide

9. NoSQL
   

   View Slide

10. NoNoSQL
    

    View Slide

11. Evaluation
    

    View Slide

12. • Created by Salvatore Sanfilippo ( @antirez )
    • Sponsored by VMware
    • 3 years old
    • redis.io
    

    View Slide

13. “Redis is an open source, advanced
    key-value store.”
    

    View Slide

14. “It is often referred to as
    a data structure server...”
    

    View Slide

15. Defancypants
    

    View Slide

16. Server A Server B
    Redis
    

    View Slide

17. In-Memory
    

    View Slide

18. In-Memory
    

    View Slide

19. The Cool Stuff
    

    View Slide

20. Fast.
    

    View Slide

21. Demo.
    

    View Slide

22. Oh wait, it's already done.
    

    View Slide

23. 120,000~ ops per sec
    

    View Slide

24. Flexible.
    

    View Slide

25. (optionally)
    Persistent.
    

    View Slide

26. Replication
    Pub/Sub
    Scripting
    Transactions
    Slices
    Dices
    Makes julienne fries
    

    View Slide

27. View Slide

28. MySQL Redis
    +
    

    View Slide

29. MySQL
    +
    Redis
    4ever
    

    View Slide

30. Setup
    

    View Slide

31. apt-get install redis-server
    Easy but old
    

    View Slide

32. http://redis.io/download
    

    View Slide

33. PHP Libraries
    • 5.3+: Predis
    • 5.2: Predis or Rediska
    • C Extension: phpredis
    • redis-cli
    

    View Slide

34. $client = new \Predis\Client();
    

    View Slide

35. The Surprise Ending
    

    View Slide

36. PHP
    MySQL
    Memcache
    

    View Slide

37. PHP
    MySQL
    Memcache
    

    View Slide

38. PHP
    MySQL
    Redis
    

    View Slide

39. It's about 80% cases.
    

    View Slide

40. Use Case #1: Caching
    

    View Slide

41. Redis has commands.
    

    View Slide

42. Commands have parameters.
    

    View Slide

43. Think functions.
    

    View Slide

44. $client->commandGoesHere($params, $go, $here);
    

    View Slide

45. SET
    

    View Slide

46. $client->set('ross:mood', 'nervous');
    // Later
    $client->get('ross:mood'); // returns “nervous”
    

    View Slide

47. $client->set('ross:mood', 'nervous');
    $client->expire('ross:mood', 5);
    // 4 seconds later...
    $client->get('ross:mood'); // “nervous”
    // 5 seconds later...
    $client->get('ross:mood'); // null
    

    View Slide

48. $client->set('ross:mood', 'nervous');
    $client->expire('ross:mood', 5);
    

    View Slide

49. $client->setex('ross:mood', 5, 'nervous');
    

    View Slide

50. Great for caching
    

    View Slide

51. Use Case #2: Session Storage
    

    View Slide

52. You already know it.
    

    View Slide

53. Use Case #3: Hit Counters
    

    View Slide

54. View Slide

55. increment
    

    View Slide

56. $client->incr('page:42:views');
    $client->incr('page:42:views');
    $client->incr('page:42:views');
    // 1
    // 2
    // 3
    

    View Slide

57. Redis is hard ;)
    

    View Slide

58. How is this better?
    

    View Slide

59. View Slide

60. Fast?
    

    View Slide

61. redis-benchmark
    ====== INCR ======
    10000 requests completed in 0.08 seconds
    50 parallel clients
    3 bytes payload
    keep alive: 1
    100.00% <= 0 milliseconds
    119047.62 requests per second
    

    View Slide

62. Fast enough.
    

    View Slide

63. $client->incr('cookies:eaten');
    $client->incrBy('cookies:eaten', 2);
    $client->incrByFloat('cookies:eaten', 0.5); version 2.6+
    

    View Slide

64. Use Case #4: Latest News
    

    View Slide

65. View Slide

66. “It is often referred to as
    a data structure server...”
    

    View Slide

67. “...since keys can contain strings, hashes,
    lists, sets and sorted sets.”
    

    View Slide

68. $redis = array();
    

    View Slide

69. Generic
    Hash
    List
    Set
    Sorted Set
    

    View Slide

70. Generic
    Hash
    List
    Set
    Sorted Set
    // strings and numbers
    $redis['ross:mood'] = "happy";
    $redis['foo'] = 9;
    

    View Slide

71. Generic
    Hash
    List
    Set
    Sorted Set
    // associative array
    $redis['foo']['name'] = 'Bob';
    $redis['foo']['age'] = 31;
    Objects, forms, records
    

    View Slide

72. Generic
    Hash
    List
    Set
    Sorted Set
    // not associative
    $redis['foo'][] = 'zip';
    $redis['foo'][] = 'zap';
    Lists, stacks, queues
    

    View Slide

73. Generic
    Hash
    List
    Set
    Sorted Set
    // No dupes, no order
    shuffle(
    array_unique($redis['foo'])
    );
    Relations, stats, matching
    

    View Slide

74. Generic
    Hash
    List
    Set
    Sorted Set
    // Ordered by *score*
    array_unique($redis['foo']);
    Curing cancer, world peace
    Sets but with order or scores
    

    View Slide

75. Y U NO STOP
    Y U NO STOP
    LISTING THINGS
    LISTING THINGS
    

    View Slide

76. Generic
    Hash
    List
    Set
    Sorted Set
    

    View Slide

77. Generic
    Hash
    List
    Set
    Sorted Set
    

    View Slide

78. // Code
    $client->lpush('news:latest', 'Aliens Attack!');
    // Redis
    ['Aliens Attack!']
    

    View Slide

79. // Redis
    ['Takei 2012', 'Aliens Attack!']
    // 2 hours later...
    $client->lpush('news:latest', 'Takei 2012');
    

    View Slide

80. // That evening...
    $client->lpush('news:latest', 'Eggs = Cancer!');
    // Redis
    ['Eggs = Cancer!', 'Takei 2012', 'Aliens Attack!']
    

    View Slide

81. Recap
    

    View Slide

82. // Code
    $client->lpush('news:latest', 'Aliens Attack!');
    $client->lpush('news:latest', 'Takei 2012');
    $client->lpush('news:latest', 'Eggs = Cancer!');
    // Redis
    ['Eggs = Cancer!', 'Takei 2012', 'Aliens Attack!']
    

    View Slide

83. Getting it back out?
    

    View Slide

84. $client->lrange('news:latest', 0, 1);
    End Index
    Start Index
    

    View Slide

85. var_dump( $client->lrange('news:latest', 0, 1) );
    array(2) {
    [0]=> string(14) "Eggs = Cancer!"
    [1]=> string(10) "Takei 2012"
    }
    

    View Slide

86. That's it.
    Really.
    

    View Slide

87. View Slide

88. What about size?
    

    View Slide

89. $client->lpush('news:latest', 'Free Jetpacks!');
    $client->lpush('news:latest', 'Elvis Found!');
    $client->lpush('news:latest', 'Takei Wins!');
    //...and so on...
    

    View Slide

90. ltrim
    

    View Slide

91. $client->ltrim('news:latest', 0, 2);
    // Only the three latest stories remain!
    

    View Slide

92. Cron
    

    View Slide

93. or better...
    

    View Slide

94. $client->lpush('news:latest', 'Cats Leave Euro');
    $client->ltrim('news:latest', 0, 2);
    

    View Slide

95. Great option for notifications
    

    View Slide

96. View Slide

97. View Slide

98. View Slide

99. View Slide

100. SELECT * FROM Articles
     INNER JOIN Authors ON (complicated joins)
     -- More joins
     WHERE (complicated logic)
     LIMIT 0, 20
     

     View Slide

101. SELECT Articles.id FROM Articles
     INNER JOIN Authors ON (complicated joins)
     -- More joins
     WHERE (complicated logic)
     

     View Slide

102. $client->lpush('search:a17f3', $ids);
     

     View Slide

103. $client->lrange('search:a17f3', $limit, $offset);
     

     View Slide

104. View Slide

105. Use Case #5: Recently Viewed
     

     View Slide

106. View Slide

107. Generic
     Hash
     List
     Set
     Sorted Set
     

     View Slide

108. Generic
     Hash
     List
     Set
     Sorted Set
     No duplicates
     

     View Slide

109. Generic
     Hash
     List
     Set
     Sorted Set
     Needs to be ordered
     

     View Slide

110. Goldilocks
     Just Right
     Generic
     Hash
     List
     Set
     Sorted Set
     

     View Slide

111. zadd
     

     View Slide

112. $client->zadd('mykey', 1, 'mydata');
     Any integer
     or float
     

     View Slide

113. $client->zadd('recent', 1, '/p/first');
     

     View Slide

114. $client->zadd('recent', time(), '/p/first');
     

     View Slide

115. $client->zadd('recent', 1338020901, '/p/first');
     $client->zadd('recent', 1338020902, '/p/second');
     $client->zadd('recent', 1338020903, '/p/third');
     $client->zadd('recent', 1338020904, '/p/fourth');
     

     View Slide

116. Reading it back out?
     

     View Slide

117. array(3) {
     [0]=> string(8) "/p/first"
     [1]=> string(9) "/p/second"
     [2]=> string(8) "/p/third"
     }
     $client->zrange('recent', 0, 2);
     

     View Slide

118. $client->zrevrange('recent', 0, 2);
     Reverse
     array(3) {
     [0]=> string(9) "/p/fourth"
     [1]=> string(8) "/p/third"
     [2]=> string(9) "/p/second"
     }
     

     View Slide

119. Duplicates?
     

     View Slide

120. $client->zadd('recent', 1338020901, '/p/first');
     // later...
     $client->zadd('recent', 1338020928, '/p/first');
     

     View Slide

121. array(3) {
     [0]=> string(8) "/p/first"
     [1]=> string(9) "/p/fourth"
     [2]=> string(8) "/p/third"
     }
     $client->zrevrange('recent', 0, 2);
     

     View Slide

122. Cool.
     

     View Slide

123. Other things we can do?
     

     View Slide

124. $client->zrangebyscore('recent', $low, $high);
     

     View Slide

125. $yesterday = time()-(60*60*24);
     $client->zrangebyscore('recent', $yesterday, '+inf');
     

     View Slide

126. Intersections
     

     View Slide

127. zinterstore
     

     View Slide

128. $client->zinterstore('omg', 2, 'recent', 'favorite');
     $client->zrange('omg', 0, 4);
     

     View Slide

129. zrem
     

     View Slide

130. zremrangebyscore
     

     View Slide

131. $yesterday = time()-(60*60*24);
     $client->zremrangebyscore(
     'recent', '-inf', $yesterday
     );
     

     View Slide

132. We can do a lot.
     

     View Slide

133. Scores can be anything.
     

     View Slide

134. Use Case #6: Sharing Data
     

     View Slide

135. Redis
     PHP
     Node.js Python
     

     View Slide

136. • ActionScript
     • C
     • C#
     • C++
     • Clojure
     • Common Lisp
     • Erlang
     • Fancy
     • Go
     • Haskell
     • haXe
     • Io
     • Java
     • Lua
     • Node.js
     • Objective-C
     • Perl
     • PHP
     • Pure Data
     • Python
     • Ruby
     • Scala
     • Smalltalk
     • Tcl
     

     View Slide

137. $client = new \Predis\Client();
     $client->set('foo', 'bar');
     

     View Slide

138. var redis = require("redis");
     var client = redis.createClient();
     client.get("foo", redis.print);
     

     View Slide

139. Step Further...
     

     View Slide

140. Pub/Sub
     

     View Slide

141. $client->publish('broadcast', 'batkitty');
     

     View Slide

142. client.on("message", function (channel, message) {
     console.log(channel + ": " + message);
     });
     client.subscribe("broadcast");
     // prints “broadcast: batkitty”
     

     View Slide

143. Not everyday
     yet
     

     View Slide

144. Use Case #7: Worker Queues
     

     View Slide

145. $client->lpush('jobs:pending', 'clear_cache');
     

     View Slide

146. $client->lpush('jobs:pending', '{"do":"email", …}');
     

     View Slide

147. $client->lpush('jobs:pending', 'job:45');
     

     View Slide

148. // worker
     $client = new \Predis\Client(array(
     'read_write_timeout' => -1
     ));
     do {
     $job = $client->brpop('jobs:pending', 0);
     doJob($job);
     } while(true);
     

     View Slide

149. This will work.
     

     View Slide

150. However...
     

     View Slide

151. Things break.
     

     View Slide

152. Things break.
     

     View Slide

153. Clients break.
     

     View Slide

154. Clients break.
     

     View Slide

155. Clients crash.
     

     View Slide

156. do {
     $job = $client->brpop('jobs:pending', 0);
     doJob($job);
     } while(true);
     

     View Slide

157. Multiple keys is the redis way.
     

     View Slide

158. 'jobs:pending'
     'jobs:working'
     

     View Slide

159. What we need is: blocking
     rpop
     lpush
     

     View Slide

160. brpoplpush
     No, really.
     

     View Slide

161. do {
     $job = $client->brpoplpush(
     'jobs:pending', 'jobs:working', 0
     );
     doJob($job);
     } while(true);
     

     View Slide

162. do {
     $job = $client->brpoplpush(
     'jobs:pending', 'jobs:working', 0
     );
     if(doJob($job)) {
     $client->lrem('jobs:working', 0, $job);
     }
     } while(true);
     

     View Slide

163. Epilogue
     

     View Slide

164. Simple = Powerful
     

     View Slide

165. Fun.
     

     View Slide

166. Keep it in RAM.
     

     View Slide

167. Redis 2.6 will be amazing.
     

     View Slide

168. Great docs.
     Use them.
     

     View Slide

169. Bad for the DB?
     Might be good for Redis.
     

     View Slide

170. Bad for the DB?
     Might be good for Redis.
     

     View Slide

171. View Slide

172. @svdgraaf
     QuickMeme
     IAEA Kotaku
     

     View Slide

173. http://joind.in/6829
     

     View Slide