Slide 1

Slide 1 text

Blazing Data with Redis By: Justin Carmony (and LEGOs!)

Slide 2

Slide 2 text

About Me • Director of Development
 for DeseretNews.com • Utah PHP Usergroup
 President • I Make (and Break) 
 Web Stuff • Focus on Scalable, 
 Real-time Websites
 & APIs (I <3 Redis)

Slide 3

Slide 3 text

About Presentation • We’ll ask for questions several times during presentation & the end. • I will post links, slides, resource, etc. • Goal: Educate & Inspire you on how, when, and
 why to use Redis!

Slide 4

Slide 4 text

Lets Start by Measuring the Audience

Slide 5

Slide 5 text

Let’s Talk about a Common Problem

Slide 6

Slide 6 text

You’re a Ninja Awesome Developer

Slide 7

Slide 7 text

Boss: “We Want Real-Time Data!” We Want to Track Online USers!

Slide 8

Slide 8 text

You’re a Ninja Awesome Developer We Want To Know Who 
 Is Viewing Our Site!

Slide 9

Slide 9 text

“I’ll Just Add a Table
 to MySQL....”

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Real-Time Data

Slide 19

Slide 19 text

Real-Time Data • High Volume Writes • High Volume Reads • Low Latency / Delay • Grows Exponentially compared to Rest of your Data

Slide 20

Slide 20 text

Examples of Real-Time Data

Slide 21

Slide 21 text

Real-Time Collaboration

Slide 22

Slide 22 text

API Limiting

Slide 23

Slide 23 text

Real-Time Logging

Slide 24

Slide 24 text

Traditional Data Stores • Examples: Databases (i.e. MySQL, Postgre) • Writing is Resource Intensive • Stored on Disk (aka Slower) • Slower as Volume Grows • Challenge for Replication (in other words...)

Slide 25

Slide 25 text

Meltdowns with High Volume Reads & Writes

Slide 26

Slide 26 text

Many NoSQL Solutions

Slide 27

Slide 27 text

Redis - Built for Speed • Key-Value Storage • In-Memory • Persistence • Multiple Data Types • Transactions • Pub/Sub w/ Blocking
 (and it’s Fast...)

Slide 28

Slide 28 text

Extremely Fast...

Slide 29

Slide 29 text

Ludicrous Speed Fast

Slide 30

Slide 30 text

Ludicrous Speed Fast 100,000+ Reads & Writes Per Second

Slide 31

Slide 31 text

It’s gone Plaid!

Slide 32

Slide 32 text

API Limiting 40,000 Foot View of Redis

Slide 33

Slide 33 text

Redis is Awesome For: • Data with a high volume of Reads & Writes • Simple, Less-Relational Data Sets • Data that doesn’t have to be 100% durable • Data that can shard well.

Slide 34

Slide 34 text

How To Use Redis

Slide 35

Slide 35 text

Step One:  Install & Run • Linux & Mac: Compile from Source • No Dependencies • make && make install • Windows • Download Binaries • Compile? Best of Luck

Slide 36

Slide 36 text

Running Redis • Run: redis-server • Can use configuration file • Run as a “service”:
 http://redis.io/topics/ quickstart • redis-cli to try out commands


Slide 37

Slide 37 text

Step Two:  Understand Keys

Slide 38

Slide 38 text

Redis Keys • Keys MUST be UNIQUE • Keys are Binary Safe Strings • Super Long Keys (i.e. 1024 bytes) are costly to look up • Cryptic Short Keys (i.e. u:1000:pwd) have little performance improvement over a descriptive key (i.e. user:1000:password)

Slide 39

Slide 39 text

Step Three:  Thinking Key-Value

Slide 40

Slide 40 text

This is not an RMDBS • It’s All About the Keys,
 Not the Values • There is no “querying” * • There are no “indexes” • There are no “schemas” * - not 100% true since redis 2.8.0, will cover why later

Slide 41

Slide 41 text

Document Your Data!

Slide 42

Slide 42 text

Step Four: Get to Know  your Data Types Like Knowing Your Lego Pieces!

Slide 43

Slide 43 text

Redis Strings • Store Simple Strings • Binary Safe • Great for JSON • Advanced Commands

Slide 44

Slide 44 text

Data-Type “Matrix” Sorted Unsorted Comparable Sorted
 Sets Sets Stand Alone Lists Hashes

Slide 45

Slide 45 text

Quick Variable Guide • Hashes - Small in Size, Very Efficient • Lists - Awesome Queues, Size Doesn’t Affect Performance • Sets - Great for Intersecting with others • Sorted Sets - Use to Keep “Indexes”, Sorts using Scores

Slide 46

Slide 46 text

Step Five: Learn the Commands

Slide 47

Slide 47 text

Learning the Commands • Generic Commands for
 All Keys • Commands for Each
 Data Type • Each Command has 
 Big O Notation for 
 Performance • Simple, Yet Powerful

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

Interactive Examples

Slide 50

Slide 50 text

Putting It All Together

Slide 51

Slide 51 text

Using Redis With PHP • Predis • PHP Library • Easy to Use • Very Fast to Update New Features • phpredis • PHP Extension • Faster, but requires
 compiling module We’ll Be Using
 Predis

Slide 52

Slide 52 text

Example: Simple Caching

Slide 53

Slide 53 text

Simple Cache • Data Types: • Strings • Commands: • SETEX • GET • EXPIREAT

Slide 54

Slide 54 text

Connecting

Slide 55

Slide 55 text

Simple Cache $key = 'cache.user:justin';
 
 $data_str = $redis->get($key);
 
 if($data_str)
 {
 " $data = unserialize($data_str);
 }
 else
 {
 " // Really Expensive Method of Getting This Data
 " $data = MyDatabase::GetExpensiveData();
 " $redis->setex($key, 60, serialize($data));
 }
 
 /* Do something with the $data */

Slide 56

Slide 56 text

Example: Users Online

Slide 57

Slide 57 text

Example: Online Users • Data Types: • Sets • Commands: • SADD • SUNION • EXPIRE

Slide 58

Slide 58 text

Marking Users Online /* Store Current User */
 // Current User ID
 $user_id = 1234;
 " // Get Current Time
 $now = time();
 $minute = date("i",$now);
 " // Generate the Key
 $key = "online:".$minute;
 " // Adding user to online users
 $redis->sadd($key, $user_id);
 $redis->expire($key, 60 * 10); // Expire in 10 minutes

Slide 59

Slide 59 text

Getting Online Users /* Getting Onling Users */
 $keys = array();
 // Get Current Time
 $now = time();
 $minute = date("i",$now);
 
 $count = 0;
 $minutes_ago = 5;
 while($count < $minutes_ago)
 {
 " $keys[] = "online:".$minute;
 " 
 " $count++;
 " $min--;
 " if($min < 0)
 " {
 " " $min = 59;
 " }
 }
 // create command: SUNION online:10 online:9 online:8 online:7 online:6
 $scmd = $redis->createCommand("sunion",$keys);
 $online_ids = $redis->executeCommand($scmd);


Slide 60

Slide 60 text

Example: Friends Online • Data Types: • Sets • Additional Commands: • SUNIONSTORE • SINTER

Slide 61

Slide 61 text

My Friends Online /* My Friends Online */
 $keys = array('online_users');
 $user_id = '1234';
 // Get Current Time
 $minute = date("i",time());
 
 $count = 0;
 $minutes_ago = 5;
 while($count < $minutes_ago)
 {
 " $keys[] = "online:".$minute;
 " $count++;
 " $min--;
 " if($min < 0) { $min = 59; }
 }
 
 // SUNIONSTORE online_users online:10 online:9 online:8 online:7 online:6
 $scmd = $redis->createCommand("sunionstore",$keys);
 $redis->executeCommand($scmd);
 // SINTER online_users user_id:1234.friends_ids
 $online_friend_ids = $redis->sinter('online_users'
 " , 'user:'.$user_id.'.friend_ids');

Slide 62

Slide 62 text

Under The Hood

Slide 63

Slide 63 text

Few Things About Redis • Single Threaded • Can “Shard” For More Capacity / Performance • All Commands are Atomic • Transactions for Multiple Atomic Commands • Pipelining for High Performance

Slide 64

Slide 64 text

Persistence • Snapshots on a Configurable Schedule • Will “fork” the process and write the DataSet to Disk • Append-Only File • Will Let You Survive a “Reboot”

Slide 65

Slide 65 text

Lua Scripting • Introduced: Redis 2.6.0 • Allows for “querying”, more like Map/ Reduce • Great for common, reusable logic

Slide 66

Slide 66 text

Pattern Scanning • In Redis 2.8.0 support was added for cursor-based scanning keys, hashes, sets, sorted sets, and lists.

Slide 67

Slide 67 text

Redis Cluster • redis-cluster is currently being developed & a release candidate can be downloaded to try out. • Allows for master-master cluster • Right now only recommended for advanced users at the moment.

Slide 68

Slide 68 text

How About a Live Demo?

Slide 69

Slide 69 text

One Redis Server

Slide 70

Slide 70 text

25 Servers * 10 Clients = 
 250 Workers

Slide 71

Slide 71 text

Vagrant Digital Ocean Salt Stack Redis & PHP

Slide 72

Slide 72 text

Demo Time!

Slide 73

Slide 73 text

Questions?

Slide 74

Slide 74 text

Your Homework Download & Play Around with Redis! It’s Awesome!

Slide 75

Slide 75 text

More Info? http://www.justincarmony.com/redis

Slide 76

Slide 76 text

Flickr Credits & Images: http://gim.ie/jsLJ

Slide 77

Slide 77 text

Thanks! Rate My Talk (PLEASE): https://joind.in/6486 " Twitter: @JustinCarmony " IRC: carmony #uphpu #phpc #salt " Website: http://www.justincarmony.com/blog " Email: [email protected]

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

No content

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

Misc. Unused Slides

Slide 82

Slide 82 text

Example: Chat Room • Additional Commands: • ZREVRANGEBYSCORE

Slide 83

Slide 83 text

Example: Chat Room • Data Types: Hashes, Sorted-Sets, Pub/Sub • Commands: • INCR • HSET <field> • HMGET <field1> <field2> <field...> • ZADD

Slide 84

Slide 84 text

25 Client Servers

Slide 85

Slide 85 text

Getting Users Online