Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Redis for the Everyday Developer

Ross Tuck
August 11, 2012

Redis for the Everyday Developer

As presented at Northeast PHP. #nephp

More than some arcane NoSQL tool, Redis is a simple but powerful swiss army knife you can begin using today.

This talk introduces Redis and focuses on using it to cleanly solve common problems. Along the way, you'll learn how Redis can be used as an alternative to several common PHP tools.

Ross Tuck

August 11, 2012
Tweet

More Decks by Ross Tuck

Other Decks in Programming

Transcript

  1. Ross Tuck
    Redis For The
    Everyday Developer
    Northeast PHP
    August 11th, #nephp

    View Slide

  2. Who Am I?

    View Slide

  3. Ross Tuck

    View Slide

  4. Team Lead at Ibuildings
    Codemonkey
    HTTP nut
    Hat guy

    View Slide

  5. View Slide

  6. @rosstuck

    View Slide

  7. Boring.

    View Slide

  8. View Slide

  9. NoSQL

    View Slide

  10. NoNoSQL

    View Slide

  11. Evaluation

    View Slide

  12. • Created by Salvatore Sanfilippo ( @antirez )
    • Sponsored by VMware
    • 3 years old
    • redis.io

    View Slide

  13. “Redis is an open source, advanced
    key-value store.”

    View Slide

  14. “It is often referred to as
    a data structure server...”

    View Slide

  15. Defancypants

    View Slide

  16. Server A Server B
    Redis

    View Slide

  17. In-Memory

    View Slide

  18. In-Memory

    View Slide

  19. The Cool Stuff

    View Slide

  20. Fast.

    View Slide

  21. Demo.

    View Slide

  22. Oh wait, it's already done.

    View Slide

  23. 120,000~ ops per sec

    View Slide

  24. Flexible.

    View Slide

  25. (optionally)
    Persistent.

    View Slide

  26. Replication
    Pub/Sub
    Scripting
    Transactions
    Slices
    Dices
    Makes julienne fries

    View Slide

  27. View Slide

  28. MySQL Redis
    +

    View Slide

  29. MySQL
    +
    Redis
    4ever

    View Slide

  30. Setup

    View Slide

  31. apt-get install redis-server
    Easy but old

    View Slide

  32. http://redis.io/download

    View Slide

  33. PHP Libraries
    • 5.3+: Predis
    • 5.2: Predis or Rediska
    • C Extension: phpredis
    • redis-cli

    View Slide

  34. $client = new \Predis\Client();

    View Slide

  35. The Surprise Ending

    View Slide

  36. PHP
    MySQL
    Memcache

    View Slide

  37. PHP
    MySQL
    Memcache

    View Slide

  38. PHP
    MySQL
    Redis

    View Slide

  39. It's about 80% cases.

    View Slide

  40. Use Case #1: Caching

    View Slide

  41. Redis has commands.

    View Slide

  42. Commands have parameters.

    View Slide

  43. Think functions.

    View Slide

  44. $client->commandGoesHere($params, $go, $here);

    View Slide

  45. SET

    View Slide

  46. $client->set('ross:mood', 'nervous');
    // Later
    $client->get('ross:mood'); // returns “nervous”

    View Slide

  47. $client->set('ross:mood', 'nervous');
    $client->expire('ross:mood', 5);
    // 4 seconds later...
    $client->get('ross:mood'); // “nervous”
    // 5 seconds later...
    $client->get('ross:mood'); // null

    View Slide

  48. $client->set('ross:mood', 'nervous');
    $client->expire('ross:mood', 5);

    View Slide

  49. $client->setex('ross:mood', 5, 'nervous');

    View Slide

  50. Great for caching

    View Slide

  51. Use Case #2: Session Storage

    View Slide

  52. You already know it.

    View Slide

  53. Use Case #3: Hit Counters

    View Slide

  54. View Slide

  55. increment

    View Slide

  56. $client->incr('page:42:views');
    $client->incr('page:42:views');
    $client->incr('page:42:views');
    // 1
    // 2
    // 3

    View Slide

  57. Redis is hard ;)

    View Slide

  58. How is this better?

    View Slide

  59. View Slide

  60. Fast?

    View Slide

  61. redis-benchmark
    ====== INCR ======
    10000 requests completed in 0.08 seconds
    50 parallel clients
    3 bytes payload
    keep alive: 1
    100.00% <= 0 milliseconds
    119047.62 requests per second

    View Slide

  62. Fast enough.

    View Slide

  63. $client->incr('cookies:eaten');
    $client->incrBy('cookies:eaten', 2);
    $client->incrByFloat('cookies:eaten', 0.5); version 2.6+

    View Slide

  64. Use Case #4: Latest News

    View Slide

  65. View Slide

  66. “It is often referred to as
    a data structure server...”

    View Slide

  67. “...since keys can contain strings, hashes,
    lists, sets and sorted sets.”

    View Slide

  68. $redis = array();

    View Slide

  69. Generic
    Hash
    List
    Set
    Sorted Set

    View Slide

  70. Generic
    Hash
    List
    Set
    Sorted Set
    // strings and numbers
    $redis['ross:mood'] = "happy";
    $redis['foo'] = 9;

    View Slide

  71. Generic
    Hash
    List
    Set
    Sorted Set
    // associative array
    $redis['foo']['name'] = 'Bob';
    $redis['foo']['age'] = 31;
    Objects, forms, records

    View Slide

  72. Generic
    Hash
    List
    Set
    Sorted Set
    // not associative
    $redis['foo'][] = 'zip';
    $redis['foo'][] = 'zap';
    Lists, stacks, queues

    View Slide

  73. Generic
    Hash
    List
    Set
    Sorted Set
    // No dupes, no order
    shuffle(
    array_unique($redis['foo'])
    );
    Relations, stats, matching

    View Slide

  74. Generic
    Hash
    List
    Set
    Sorted Set
    // Ordered by *score*
    array_unique($redis['foo']);
    Curing cancer, world peace
    Sets but with order or scores

    View Slide

  75. Y U NO STOP
    Y U NO STOP
    LISTING THINGS
    LISTING THINGS

    View Slide

  76. Generic
    Hash
    List
    Set
    Sorted Set

    View Slide

  77. Generic
    Hash
    List
    Set
    Sorted Set

    View Slide

  78. // Code
    $client->lpush('news:latest', 'Aliens Attack!');
    // Redis
    ['Aliens Attack!']

    View Slide

  79. // Redis
    ['Takei 2012', 'Aliens Attack!']
    // 2 hours later...
    $client->lpush('news:latest', 'Takei 2012');

    View Slide

  80. // That evening...
    $client->lpush('news:latest', 'Eggs = Cancer!');
    // Redis
    ['Eggs = Cancer!', 'Takei 2012', 'Aliens Attack!']

    View Slide

  81. Recap

    View Slide

  82. // Code
    $client->lpush('news:latest', 'Aliens Attack!');
    $client->lpush('news:latest', 'Takei 2012');
    $client->lpush('news:latest', 'Eggs = Cancer!');
    // Redis
    ['Eggs = Cancer!', 'Takei 2012', 'Aliens Attack!']

    View Slide

  83. Getting it back out?

    View Slide

  84. $client->lrange('news:latest', 0, 1);
    End Index
    Start Index

    View Slide

  85. var_dump( $client->lrange('news:latest', 0, 1) );
    array(2) {
    [0]=> string(14) "Eggs = Cancer!"
    [1]=> string(10) "Takei 2012"
    }

    View Slide

  86. That's it.
    Really.

    View Slide

  87. View Slide

  88. What about size?

    View Slide

  89. $client->lpush('news:latest', 'Free Jetpacks!');
    $client->lpush('news:latest', 'Elvis Found!');
    $client->lpush('news:latest', 'Takei Wins!');
    //...and so on...

    View Slide

  90. ltrim

    View Slide

  91. $client->ltrim('news:latest', 0, 2);
    // Only the three latest stories remain!

    View Slide

  92. Cron

    View Slide

  93. or better...

    View Slide

  94. $client->lpush('news:latest', 'Cats Leave Euro');
    $client->ltrim('news:latest', 0, 2);

    View Slide

  95. Great option for notifications

    View Slide

  96. View Slide

  97. View Slide

  98. View Slide

  99. View Slide

  100. SELECT * FROM Articles
    INNER JOIN Authors ON (complicated joins)
    -- More joins
    WHERE (complicated logic)
    LIMIT 0, 20

    View Slide

  101. SELECT Articles.id FROM Articles
    INNER JOIN Authors ON (complicated joins)
    -- More joins
    WHERE (complicated logic)

    View Slide

  102. $client->lpush('search:a17f3', $ids);

    View Slide

  103. $client->lrange('search:a17f3', $limit, $offset);

    View Slide

  104. View Slide

  105. Use Case #5: Recently Viewed

    View Slide

  106. View Slide

  107. Generic
    Hash
    List
    Set
    Sorted Set

    View Slide

  108. Generic
    Hash
    List
    Set
    Sorted Set
    No duplicates

    View Slide

  109. Generic
    Hash
    List
    Set
    Sorted Set
    Needs to be ordered

    View Slide

  110. Goldilocks
    Just Right
    Generic
    Hash
    List
    Set
    Sorted Set

    View Slide

  111. zadd

    View Slide

  112. $client->zadd('mykey', 1, 'mydata');
    Any integer
    or float

    View Slide

  113. $client->zadd('recent', 1, '/p/first');

    View Slide

  114. $client->zadd('recent', time(), '/p/first');

    View Slide

  115. $client->zadd('recent', 1338020901, '/p/first');
    $client->zadd('recent', 1338020902, '/p/second');
    $client->zadd('recent', 1338020903, '/p/third');
    $client->zadd('recent', 1338020904, '/p/fourth');

    View Slide

  116. Reading it back out?

    View Slide

  117. array(3) {
    [0]=> string(8) "/p/first"
    [1]=> string(9) "/p/second"
    [2]=> string(8) "/p/third"
    }
    $client->zrange('recent', 0, 2);

    View Slide

  118. $client->zrevrange('recent', 0, 2);
    Reverse
    array(3) {
    [0]=> string(9) "/p/fourth"
    [1]=> string(8) "/p/third"
    [2]=> string(9) "/p/second"
    }

    View Slide

  119. Duplicates?

    View Slide

  120. $client->zadd('recent', 1338020901, '/p/first');
    // later...
    $client->zadd('recent', 1338020928, '/p/first');

    View Slide

  121. array(3) {
    [0]=> string(8) "/p/first"
    [1]=> string(9) "/p/fourth"
    [2]=> string(8) "/p/third"
    }
    $client->zrevrange('recent', 0, 2);

    View Slide

  122. Cool.

    View Slide

  123. Other things we can do?

    View Slide

  124. $client->zrangebyscore('recent', $low, $high);

    View Slide

  125. $yesterday = time()-(60*60*24);
    $client->zrangebyscore('recent', $yesterday, '+inf');

    View Slide

  126. Intersections

    View Slide

  127. zinterstore

    View Slide

  128. $client->zinterstore('omg', 2, 'recent', 'favorite');
    $client->zrange('omg', 0, 4);

    View Slide

  129. zrem

    View Slide

  130. zremrangebyscore

    View Slide

  131. $yesterday = time()-(60*60*24);
    $client->zremrangebyscore(
    'recent', '-inf', $yesterday
    );

    View Slide

  132. We can do a lot.

    View Slide

  133. Scores can be anything.

    View Slide

  134. Use Case #6: Sharing Data

    View Slide

  135. Redis
    PHP
    Node.js Python

    View Slide

  136. • ActionScript
    • C
    • C#
    • C++
    • Clojure
    • Common Lisp
    • Erlang
    • Fancy
    • Go
    • Haskell
    • haXe
    • Io
    • Java
    • Lua
    • Node.js
    • Objective-C
    • Perl
    • PHP
    • Pure Data
    • Python
    • Ruby
    • Scala
    • Smalltalk
    • Tcl

    View Slide

  137. $client = new \Predis\Client();
    $client->set('foo', 'bar');

    View Slide

  138. var redis = require("redis");
    var client = redis.createClient();
    client.get("foo", redis.print);

    View Slide

  139. Step Further...

    View Slide

  140. Pub/Sub

    View Slide

  141. $client->publish('broadcast', 'batkitty');

    View Slide

  142. client.on("message", function (channel, message) {
    console.log(channel + ": " + message);
    });
    client.subscribe("broadcast");
    // prints “broadcast: batkitty”

    View Slide

  143. Not everyday
    yet

    View Slide

  144. Use Case #7: Worker Queues

    View Slide

  145. $client->lpush('jobs:pending', 'clear_cache');

    View Slide

  146. $client->lpush('jobs:pending', '{"do":"email", …}');

    View Slide

  147. $client->lpush('jobs:pending', 'job:45');

    View Slide

  148. // worker
    $client = new \Predis\Client(array(
    'read_write_timeout' => -1
    ));
    do {
    $job = $client->brpop('jobs:pending', 0);
    doJob($job);
    } while(true);

    View Slide

  149. This will work.

    View Slide

  150. However...

    View Slide

  151. Things break.

    View Slide

  152. Things break.

    View Slide

  153. Clients break.

    View Slide

  154. Clients break.

    View Slide

  155. Clients crash.

    View Slide

  156. do {
    $job = $client->brpop('jobs:pending', 0);
    doJob($job);
    } while(true);

    View Slide

  157. Multiple keys is the redis way.

    View Slide

  158. 'jobs:pending'
    'jobs:working'

    View Slide

  159. What we need is: blocking
    rpop
    lpush

    View Slide

  160. brpoplpush
    No, really.

    View Slide

  161. do {
    $job = $client->brpoplpush(
    'jobs:pending', 'jobs:working', 0
    );
    doJob($job);
    } while(true);

    View Slide

  162. do {
    $job = $client->brpoplpush(
    'jobs:pending', 'jobs:working', 0
    );
    if(doJob($job)) {
    $client->lrem('jobs:working', 0, $job);
    }
    } while(true);

    View Slide

  163. Epilogue

    View Slide

  164. Simple = Powerful

    View Slide

  165. Fun.

    View Slide

  166. Keep it in RAM.

    View Slide

  167. Redis 2.6 will be amazing.

    View Slide

  168. Great docs.
    Use them.

    View Slide

  169. Bad for the DB?
    Might be good for Redis.

    View Slide

  170. Bad for the DB?
    Might be good for Redis.

    View Slide

  171. View Slide

  172. @svdgraaf
    QuickMeme
    IAEA Kotaku

    View Slide

  173. http://joind.in/6829

    View Slide