Slide 1
Slide 1 text
Blazing Data with Redis
By: Justin Carmony
(and LEGOs!)
Slide 2
Slide 2 text
About Me
• Director of Development
for DeseretNews.com
• Utah PHP Usergroup
President
• I Make (and Break)
Web Stuff
• Focus on Scalable,
Real-time Websites
& APIs
(I <3 Redis)
Slide 3
Slide 3 text
About Presentation
• We’ll ask for questions
several times during
presentation & the end.
• I will post links, slides,
resource, etc.
• Goal: Educate & Inspire
you on how, when, and
why to use Redis!
Slide 4
Slide 4 text
Lets Start by
Measuring the
Audience
Slide 5
Slide 5 text
Let’s Talk about a
Common Problem
Slide 6
Slide 6 text
You’re a Ninja
Awesome
Developer
Slide 7
Slide 7 text
Boss: “We Want
Real-Time Data!”
We Want to
Track Online
USers!
Slide 8
Slide 8 text
You’re a Ninja
Awesome
Developer
We Want To Know Who
Is Viewing Our Site!
Slide 9
Slide 9 text
“I’ll Just Add a Table
to MySQL....”
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
No content
Slide 16
Slide 16 text
No content
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
Real-Time Data
Slide 19
Slide 19 text
Real-Time Data
• High Volume Writes
• High Volume Reads
• Low Latency / Delay
• Grows Exponentially
compared to Rest of
your Data
Slide 20
Slide 20 text
Examples of Real-Time Data
Slide 21
Slide 21 text
Real-Time Collaboration
Slide 22
Slide 22 text
API Limiting
Slide 23
Slide 23 text
Real-Time Logging
Slide 24
Slide 24 text
Traditional Data Stores
• Examples: Databases (i.e.
MySQL, Postgre)
• Writing is Resource
Intensive
• Stored on Disk (aka
Slower)
• Slower as Volume Grows
• Challenge for Replication
(in other words...)
Slide 25
Slide 25 text
Meltdowns with High Volume
Reads & Writes
Slide 26
Slide 26 text
Many NoSQL Solutions
Slide 27
Slide 27 text
Redis - Built for Speed
• Key-Value Storage
• In-Memory
• Persistence
• Multiple Data Types
• Transactions
• Pub/Sub w/ Blocking
(and it’s Fast...)
Slide 28
Slide 28 text
Extremely Fast...
Slide 29
Slide 29 text
Ludicrous Speed Fast
Slide 30
Slide 30 text
Ludicrous Speed Fast
100,000+
Reads & Writes
Per Second
Slide 31
Slide 31 text
It’s gone Plaid!
Slide 32
Slide 32 text
API Limiting
40,000 Foot View of Redis
Slide 33
Slide 33 text
Redis is Awesome For:
• Data with a high volume of
Reads & Writes
• Simple, Less-Relational
Data Sets
• Data that doesn’t have to
be 100% durable
• Data that can shard well.
Slide 34
Slide 34 text
How To
Use Redis
Slide 35
Slide 35 text
Step One:
Install & Run
• Linux & Mac: Compile
from Source
• No Dependencies
• make && make install
• Windows
• Download Binaries
• Compile? Best of Luck
Slide 36
Slide 36 text
Running Redis
• Run: redis-server
• Can use configuration
file
• Run as a “service”:
http://redis.io/topics/
quickstart
• redis-cli to try out
commands
Slide 37
Slide 37 text
Step Two:
Understand Keys
Slide 38
Slide 38 text
Redis Keys
• Keys MUST be UNIQUE
• Keys are Binary Safe Strings
• Super Long Keys (i.e. 1024 bytes) are costly to
look up
• Cryptic Short Keys (i.e. u:1000:pwd) have
little performance improvement over a
descriptive key (i.e. user:1000:password)
Slide 39
Slide 39 text
Step Three:
Thinking Key-Value
Slide 40
Slide 40 text
This is not an RMDBS
• It’s All About the Keys,
Not the Values
• There is no “querying” *
• There are no “indexes”
• There are no “schemas”
* - not 100% true since redis 2.8.0, will cover why later
Slide 41
Slide 41 text
Document
Your Data!
Slide 42
Slide 42 text
Step Four: Get to Know
your Data Types
Like Knowing Your
Lego Pieces!
Slide 43
Slide 43 text
Redis Strings
• Store Simple
Strings
• Binary Safe
• Great for JSON
• Advanced
Commands
Slide 44
Slide 44 text
Data-Type “Matrix”
Sorted Unsorted
Comparable
Sorted
Sets
Sets
Stand Alone Lists Hashes
Slide 45
Slide 45 text
Quick Variable Guide
• Hashes - Small in Size,
Very Efficient
• Lists - Awesome
Queues, Size Doesn’t
Affect Performance
• Sets - Great for
Intersecting with others
• Sorted Sets - Use to
Keep “Indexes”, Sorts
using Scores
Slide 46
Slide 46 text
Step Five:
Learn the Commands
Slide 47
Slide 47 text
Learning the Commands
• Generic Commands for
All Keys
• Commands for Each
Data Type
• Each Command has
Big O Notation for
Performance
• Simple, Yet Powerful
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
Interactive
Examples
Slide 50
Slide 50 text
Putting It
All Together
Slide 51
Slide 51 text
Using Redis With PHP
• Predis
• PHP Library
• Easy to Use
• Very Fast to Update
New Features
• phpredis
• PHP Extension
• Faster, but requires
compiling module We’ll Be Using
Predis
Slide 52
Slide 52 text
Example: Simple Caching
Slide 53
Slide 53 text
Simple Cache
• Data Types:
• Strings
• Commands:
• SETEX
• GET
• EXPIREAT
Slide 54
Slide 54 text
Connecting
Slide 55
Slide 55 text
Simple Cache
$key = 'cache.user:justin';
$data_str = $redis->get($key);
if($data_str)
{
" $data = unserialize($data_str);
}
else
{
" // Really Expensive Method of Getting This Data
" $data = MyDatabase::GetExpensiveData();
" $redis->setex($key, 60, serialize($data));
}
/* Do something with the $data */
Slide 56
Slide 56 text
Example: Users Online
Slide 57
Slide 57 text
Example: Online Users
• Data Types:
• Sets
• Commands:
• SADD
• SUNION
• EXPIRE
Slide 58
Slide 58 text
Marking Users Online
/* Store Current User */
// Current User ID
$user_id = 1234;
"
// Get Current Time
$now = time();
$minute = date("i",$now);
"
// Generate the Key
$key = "online:".$minute;
"
// Adding user to online users
$redis->sadd($key, $user_id);
$redis->expire($key, 60 * 10); // Expire in 10 minutes
Slide 59
Slide 59 text
Getting Online Users
/* Getting Onling Users */
$keys = array();
// Get Current Time
$now = time();
$minute = date("i",$now);
$count = 0;
$minutes_ago = 5;
while($count < $minutes_ago)
{
" $keys[] = "online:".$minute;
"
" $count++;
" $min--;
" if($min < 0)
" {
" " $min = 59;
" }
}
// create command: SUNION online:10 online:9 online:8 online:7 online:6
$scmd = $redis->createCommand("sunion",$keys);
$online_ids = $redis->executeCommand($scmd);
Slide 60
Slide 60 text
Example: Friends Online
• Data Types:
• Sets
• Additional Commands:
• SUNIONSTORE
• SINTER
Slide 61
Slide 61 text
My Friends Online
/* My Friends Online */
$keys = array('online_users');
$user_id = '1234';
// Get Current Time
$minute = date("i",time());
$count = 0;
$minutes_ago = 5;
while($count < $minutes_ago)
{
" $keys[] = "online:".$minute;
" $count++;
" $min--;
" if($min < 0) { $min = 59; }
}
// SUNIONSTORE online_users online:10 online:9 online:8 online:7 online:6
$scmd = $redis->createCommand("sunionstore",$keys);
$redis->executeCommand($scmd);
// SINTER online_users user_id:1234.friends_ids
$online_friend_ids = $redis->sinter('online_users'
" , 'user:'.$user_id.'.friend_ids');
Slide 62
Slide 62 text
Under The Hood
Slide 63
Slide 63 text
Few Things About Redis
• Single Threaded
• Can “Shard” For More Capacity /
Performance
• All Commands are Atomic
• Transactions for Multiple Atomic
Commands
• Pipelining for High Performance
Slide 64
Slide 64 text
Persistence
• Snapshots on a Configurable Schedule
• Will “fork” the process and write the
DataSet to Disk
• Append-Only File
• Will Let You Survive a “Reboot”
Slide 65
Slide 65 text
Lua Scripting
• Introduced: Redis 2.6.0
• Allows for “querying”, more like Map/
Reduce
• Great for common, reusable logic
Slide 66
Slide 66 text
Pattern Scanning
• In Redis 2.8.0 support was added for
cursor-based scanning keys, hashes, sets,
sorted sets, and lists.
Slide 67
Slide 67 text
Redis Cluster
• redis-cluster is currently being developed &
a release candidate can be downloaded to
try out.
• Allows for master-master cluster
• Right now only recommended for advanced
users at the moment.
Slide 68
Slide 68 text
How About a Live
Demo?
Slide 69
Slide 69 text
One Redis Server
Slide 70
Slide 70 text
25 Servers * 10 Clients =
250 Workers
Slide 71
Slide 71 text
Vagrant
Digital Ocean
Salt Stack
Redis & PHP
Slide 72
Slide 72 text
Demo Time!
Slide 73
Slide 73 text
Questions?
Slide 74
Slide 74 text
Your Homework
Download & Play Around with Redis!
It’s Awesome!
Slide 75
Slide 75 text
More Info?
http://www.justincarmony.com/redis
Slide 76
Slide 76 text
Flickr Credits & Images:
http://gim.ie/jsLJ
Slide 77
Slide 77 text
Thanks!
Rate My Talk (PLEASE): https://joind.in/6486
"
Twitter: @JustinCarmony
"
IRC: carmony
#uphpu #phpc #salt
"
Website:
http://www.justincarmony.com/blog
"
Email:
[email protected]
Slide 78
Slide 78 text
No content
Slide 79
Slide 79 text
No content
Slide 80
Slide 80 text
No content
Slide 81
Slide 81 text
Misc. Unused Slides
Slide 82
Slide 82 text
Example: Chat Room
• Additional Commands:
• ZREVRANGEBYSCORE
Slide 83
Slide 83 text
Example: Chat Room
• Data Types: Hashes, Sorted-Sets, Pub/Sub
• Commands:
• INCR
• HSET <field>
• HMGET <field1> <field2> <field...>
• ZADD
Slide 84
Slide 84 text
25 Client Servers
Slide 85
Slide 85 text
Getting Users Online