Learn to Love Redis (Zendcon 2012)

Redis is a NoSQL database that sits nicely in between simple key-value stores like Memcached and document databases like MongoDB. Redis and PHP go very well together, from augmenting your existing app to building a new app from the ground up using Redis exclusively. We'll cover a short intro to the Redis feature set and several "cookbook" style PHP strategies to get some quick performance boosts.

Transcript

1. Learn to Love Redis Josh Butts Zendcon 2012 http://joind.in/7033
2. About Me • Director of Development at Vertive, LLC • Zend Certified in PHP 5 and ZF • Organizer of AustinPHP • Find me online: • @jimbojsb • github.com/jimbojsb
3. Agenda • Redis Overview • Types of keys • PHP Strategies
4. What is Redis? • Redis is an “Advanced” key-value store database • Backed by VMWare • Redis works like memcached, but better: • Atomic commands & transactions • Server-side data structures
5. Redis in an in-memory database • You need as much RAM as you have data • Disk persistence is exactly that • Customize disk writes interval to suit your pain threshold • Something interesting happens when you run out of memory
6. How to get Redis • http://redis.io • Doesn’t run (well) on Windows • There are a few ports out there • Has virtually no compile dependencies • apt-get, homebrew, yum, etc • No reason not to run the latest and greatest • but at get at least 2.2
7. How to explore • There aren’t any good GUI tools out there • redis-cli is your friend
8. A bit about organization • Redis can have multiple databases • use one database per application • Within a database, namespace your keys • Ex: classname:uid:datafield • Keep them concise but useful. Keys take memory too!
9. Types of Keys
10. String Keys • Simple key-value • Memcache equivalent • Common Commands • SET • GET • INCR • STRLEN http://redis.io/commands#string
11. Hash Keys • Key + multiple fields / values • Think 1-dimensional associative array • mykey => [field1 => value1, field2 => value2] http://redis.io/commands#hash • Common commands • HSET • HGET • HGETALL • HDEL • HVALS • HKEYS
12. Hash Keys
13. Set Keys • key + unordered list of strings • myset => [item2, item5, item1] • Common Commands • SADD • SMEMBERS • SISMEMBER • SREM http://redis.io/commands#set
14. Set Keys
15. List Keys • Like sets, except insertion order matters • Build queues or stacks • Optional blocking • Common commands • RPUSH, LPUSH • RPOP, LPOP, BRPOP, BLPOP • LLEN http://redis.io/commands#list
16. List Keys
17. Sorted Set Keys • Like sets, but sorted by a user-provided score value • Extremely fast access by score or range of scores, because it’s sorted in storage • Common commands • ZADD • ZRANGE • ZREVRANGE
18. Sorted Set Keys
19. Other commands that work on all keys • DEL - delete a key, regardless of type • KEYS - search for keys (usually with a wildcard) • EXPIRE / PERSIST - change expiration values for keys
20. Connecting from PHP
21. Connecting from PHP • Several libraries out there • Just use Predis (https://github.com/nrk/predis) • Requires PHP 5.3
22. Predis • All the examples here assume you’re using Predis • Connect to a localhost redis: $p = new Predis\Client(); • Redis commands implemented as magic methods on Predis\Client • $p->set($key, $val); • $val = $p->get($key);
23. Examples
24. Attribute Display Logic items id INT(11) name VARCHAR(32) items_attributes id INT(11) item_id INT(11) attr_name VARCHAR(32) attr_value VARCHAR(32) 10k rows 100k rows • An item, a Dell Latitude Laptop, has: • Free Shipping, Financing Available, Expires Soon, etc
25. Attribute Display Logic • Display “Free Shipping” graphic on the item if it has a free shipping attribute row
26. Attribute Display - Traditional class Item { public function hasAttribute($name) { $sql = "SELECT 1 FROM items_attributes WHERE item_id = $this->id AND attr_name='$name' LIMIT 1"; $result = $this->pdo->execute($sql); return $result != false; } }
27. Denormalize data from MySQL to Redis • Smart caching • Define a consistent way to name a relational object in Redis • I prefer [object class]:[object id]:[attribute] • ex: product:13445:num_comments • This prevents data collisions, and makes it easy to work with data on the command line
28. Attribute Display - Redis class Item { public function hasAttribute($name) { return $this->redis->sismember(“item:$this->id:attributes”, $name); } public function addAttribute($name, $value) { //traditional mysql stuff here still $this->redis->sadd('item:$this->id:attributes', $name); } public function deleteAttribute($name) { //traditional mysql stuff here $this->redis->srem(‘item:$this->id:attributes’, $name); } }
29. Advantages • The more items you have, the less MySQL will scale this solution for you on it’s own • Frequently updating your items kills the MySQL query cache • Checking existence of a set member is O(1) time • On a laptop, I can check roughly 10,000 attributes per second
30. MVC Routing • Example: • Offers.com has over 5000 stores • Store pages live at /[store]/
31. Old Store Routing class Store_Route implements Zend_Controller_Router_Route_Interface { public function route($url) { $sql = “SELECT 1 FROM stores WHERE url=’$url’”; $store = $this->pdo->execute($sql); // make sure $store is properly validated and then... return array(“module” => “core”, “controller” => “store”, “action” => “index”); } }
32. New Routing class Redis_Route implements Zend_Controller_Router_Route_Interface { public function route($url) { $p = $this->predis; if ($p->exists($url)) { list($module, $controller, $action) = $this->redis->hvals($url); return array(“module” => $module, “controller” => $controller, “action” => $action); } return false; } }
33. Filling in the Redis keys class Store { public function create(array $data) { // ... traditional SQL stuff to put store in the database ... // $route = array(“module” => “core”, “controller” => “store”, “action” => “index”); $this->predis->hmset($data[“url”], $route); } }
34. Advantages • I can now create offers.com/[anything]/ and route it to the right place, in O(1) time • I’m only adding a few lines of code to my existing models • One custom Redis route can handle any dynamic url
35. Event Throttling • A common task is to only let a user do something N number of times • This becomes more complex when you add a time window to that constraint • We can use key expirations in Redis to help with this
36. A Simple Event Throttling Class class Event { public static function log($event, $uid, $duration = null) { $r = self::$redis; $r->incr("event:$event:$uid"); if ($duration) { if ($r->ttl("event:$event:$uid") < 0) { $r->expires("event:$event:$uid", $duration); } } } public static function isAllowed($event, $uid, $threshold = null) { $r = self::$redis; if ($r->exists("event:$event:$uid")) { if ($threshold) { $currentValue = $r->get("even:$event:$uid"); if ($currentValue <= $threshold) { return true; } } return false; } return true; } }
37. Using the Event Class class MyController { public function clickAction() { $userId = $_SESSION["uid"]; if (Event::isAllowed("click", $userId, 5)) { // magic happens here Event::log('click', $userId, 3600); } else { // send an error message } } }
38. Most Popular [insert widget here] • Everyone likes to vote on things • Facebook like? • You probably want to store this 2 ways • in MySQL for reporting, long term aggregation, etc • something faster so your website will actually work • .....REDIS!
39. Using Zsets for “Most Popular” <?php class MyWidget { public function getMostPopular($limit = null) { $rangeStart = 0; $rangeEnd = ($limit ? $limit - 1 : -1); return $redis->zrevrange("widgets:popular", $rangeStart, $rangeEnd, true); } public function logVote() { $pdo->query(" UDPATE widgets SET votes=votes+1 WHERE id='$this->id' "); $redis->zincrby("widgets:popular", 1, $this->name); } }
40. Questions?
41. http://joind.in/7033
42. Vertive is Hiring • We help people save money • We’re looking for engineers to work on
43. Job Queues • Redis lists make great job queues • Offload your intensive workloads to some other process • Blocking I/O allows you to easily build long-running daemons • Be aware of scale tradeoffs vs. data availability
44. Job Queues class Queue { protected $name; protected $predis; public function push($job) { $this->predis->lpush($this->name, json_encode($job)); } public function pop($block = false) { $job = null; if ($block) { $data = $this->predis->brpop($this->name, 0); $job = $data[1]; } else { $job = $this->predis->rpop($this->name); } if ($job) { return json_decode($job); } } }
45. Queuing Jobs $q = new Queue('test_queue'); $form = new My_Zend_Form(); if ($form->isValid($_POST)) { $q->push($form->getValues()); $message = “Thanks for your submission”; } else { $message = “Error - something went wrong”; } echo “<h1>$message</h1>”;
46. Processing Jobs - Crontab style function processJob(Array $job) { //...something really cool here // throw an exception on error } // process all pending jobs $q = new Queue(‘test_queue’); while ($job = $q->pop()) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); } }
47. Processing Jobs - Worker style function processJob(Array $job) { //...something really cool here // throw an exception on error } // keep processing jobs as they become available $q = new Queue(‘test_queue’); while ($job = $q->pop(true)) { try { processJob($job); } catch (Exception $e) { Echo “error processing job”; $q = new Queue(‘errors’); $q->push($job); } }