Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Blazing Data w/ Redis - Dutch PHP Conf `14

Blazing Data w/ Redis - Dutch PHP Conf `14

There are many fast data stores, and then there is Redis. Learn about this excellent NoSQL solution that is a powerful in-memory key-value store. Learn how to solve traditionally difficult problems with Redis, and how you can benefit from 100,000 reads/writes a second on commodity hardware. We’ll discuss how and when to use the different datatypes and commands to fit your needs. We’ll discuss the different PHP libraries with their pros and cons. We’ll then show some live examples on how to use it for a chatroom, and how Redis manages a billion data points for our dating matching system. Finally, we’ll discuss some of the upcoming features in the near future, such as clustering and scripting.

Justin Carmony

June 27, 2014
Tweet

More Decks by Justin Carmony

Other Decks in Technology

Transcript

  1. Blazing Data with Redis
    By: Justin Carmony
    (and LEGOs!)

    View Slide

  2. About Me
    • Director of Development

    for DeseretNews.com

    • Utah PHP Usergroup

    President

    • I Make (and Break) 

    Web Stuff

    • Focus on Scalable, 

    Real-time Websites

    & APIs
    (I <3 Redis)

    View Slide

  3. About Presentation
    • We’ll ask for questions
    several times during
    presentation & the end.

    • I will post links, slides,
    resource, etc.

    • Goal: Educate & Inspire
    you on how, when, and

    why to use Redis!

    View Slide

  4. Lets Start by
    Measuring the
    Audience

    View Slide

  5. Let’s Talk about a
    Common Problem

    View Slide

  6. You’re an
    Awesome
    Developer

    View Slide

  7. Boss: “We Want
    Real-Time Data!”
    We Want to
    Track Online
    USers!

    View Slide

  8. You’re a Ninja
    Awesome
    Developer
    We Want To Know Who 

    Is Viewing Our Site!

    View Slide

  9. “I’ll Just Add a Table

    to MySQL....”

    View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. Real-Time Data

    View Slide

  19. Real-Time Data
    • High Volume Writes

    • High Volume Reads

    • Low Latency / Delay

    • Grows Exponentially
    compared to Rest of
    your Data

    View Slide

  20. Examples of Real-Time Data

    View Slide

  21. Real-Time Collaboration

    View Slide

  22. API Limiting

    View Slide

  23. Real-Time Logging

    View Slide

  24. Traditional Data Stores
    • Examples: Databases (i.e.
    MySQL, Postgre)

    • Writing is Resource
    Intensive

    • Stored on Disk (aka
    Slower)

    • Slower as Volume Grows

    • Challenge for Replication
    (in other words...)

    View Slide

  25. Meltdowns with High Volume
    Reads & Writes

    View Slide

  26. Many NoSQL Solutions

    View Slide

  27. Redis - Built for Speed
    • Key-Value Storage

    • In-Memory

    • Persistence

    • Multiple Data Types

    • Transactions

    • Pub/Sub w/ Blocking

    (and it’s Fast...)

    View Slide

  28. Extremely Fast...

    View Slide

  29. Ludicrous Speed Fast
    100,000+

    Reads & Writes

    Per Second

    View Slide

  30. It’s gone Plaid!

    View Slide

  31. API Limiting
    40,000 Foot View of Redis

    View Slide

  32. Redis is Awesome For:
    • Data with a high volume of
    Reads & Writes

    • Simple, Less-Relational
    Data Sets

    • Data that doesn’t have to
    be 100% durable

    • Data that can shard well.

    View Slide

  33. How To
    Use Redis

    View Slide

  34. Step One:
    Install & Run
    • Linux & Mac: Compile
    from Source

    • No Dependencies

    • make && make install

    • Windows

    • Download Binaries

    • Compile? Best of Luck

    View Slide

  35. Running Redis
    • Run: redis-server

    • Can use configuration
    file

    • Run as a “service”:

    http://redis.io/topics/
    quickstart

    • redis-cli to try out
    commands


    View Slide

  36. Step Two:
    Understand Keys

    View Slide

  37. Redis Keys
    • Keys MUST be UNIQUE

    • Keys are Binary Safe Strings

    • Super Long Keys (i.e. 1024 bytes) are costly to
    look up

    • Cryptic Short Keys (i.e. u:1000:pwd) have
    little performance improvement over a
    descriptive key (i.e. user:1000:password)

    View Slide

  38. Step Three:
    Thinking Key-Value

    View Slide

  39. This is not an RMDBS
    • It’s All About the Keys,

    Not the Values

    • There is no “querying” *

    • There are no “indexes”

    • There are no “schemas”
    * - not 100% true since redis 2.8.0, will cover why later

    View Slide

  40. Document
    Your Data!

    View Slide

  41. Step Four: Get to Know
    your Data Types
    Like Knowing Your
    Lego Pieces!

    View Slide

  42. Redis Strings
    • Store Simple
    Strings

    • Binary Safe

    • Great for JSON

    • Advanced
    Commands

    View Slide

  43. Data-Type “Matrix”
    Sorted Unsorted
    Comparable
    Sorted

    Sets
    Sets
    Stand Alone Lists Hashes

    View Slide

  44. Quick Variable Guide
    • Hashes - Small in Size,
    Very Efficient

    • Lists - Awesome
    Queues, Size Doesn’t
    Affect Performance

    • Sets - Great for
    Intersecting with others

    • Sorted Sets - Use to
    Keep “Indexes”, Sorts
    using Scores

    View Slide

  45. Step Five:
    Learn the Commands

    View Slide

  46. Learning the Commands
    • Generic Commands for

    All Keys

    • Commands for Each

    Data Type

    • Each Command has 

    Big O Notation for 

    Performance

    • Simple, Yet Powerful

    View Slide

  47. View Slide

  48. Interactive
    Examples

    View Slide

  49. Putting It

    All Together

    View Slide

  50. Using Redis With PHP
    • Predis

    • PHP Library

    • Easy to Use

    • Very Fast to Update
    New Features

    • phpredis

    • PHP Extension

    • Faster, but requires

    compiling module We’ll Be Using

    Predis

    View Slide

  51. Example: Simple Caching

    View Slide

  52. Simple Cache
    • Data Types:

    • Strings

    • Commands:

    • SETEX

    • GET

    • EXPIREAT

    View Slide

  53. Connecting

    // Include the Predis Autoloader

    require 'predis/lib/Predis/Autoloader.php';


    // Register the Autoloader

    Predis\Autoloader::register();


    // Create a Client with defaults

    $redis = new Predis\Client();


    // Create with Connection String

    $redis = new Predis\Client('tcp://10.0.0.1:6379');

    "
    /** Our Examples Will Assume This Is Already Done **/

    View Slide

  54. Simple Cache
    $key = 'cache.user:justin';


    $data_str = $redis->get($key);


    if($data_str)

    {

    " $data = unserialize($data_str);

    }

    else

    {

    " // Really Expensive Method of Getting This Data

    " $data = MyDatabase::GetExpensiveData();

    " $redis->setex($key, 60, serialize($data));

    }


    /* Do something with the $data */

    View Slide

  55. Example: Users Online

    View Slide

  56. Example: Online Users
    • Data Types:

    • Sets

    • Commands:

    • SADD

    • SUNION

    • EXPIRE

    View Slide

  57. Marking Users Online
    /* Store Current User */

    // Current User ID

    $user_id = 1234;

    "
    // Get Current Time

    $now = time();

    $minute = date("i",$now);

    "
    // Generate the Key

    $key = "online:".$minute;

    "
    // Adding user to online users

    $redis->sadd($key, $user_id);

    $redis->expire($key, 60 * 10); // Expire in 10 minutes

    View Slide

  58. Getting Online Users
    /* Getting Onling Users */

    $keys = array();

    // Get Current Time

    $now = time();

    $minute = date("i",$now);


    $count = 0;

    $minutes_ago = 5;

    while($count < $minutes_ago)

    {

    " $keys[] = "online:".$minute;

    " 

    " $count++;

    " $min--;

    " if($min < 0)

    " {

    " " $min = 59;

    " }

    }

    // create command: SUNION online:10 online:9 online:8 online:7 online:6

    $scmd = $redis->createCommand("sunion",$keys);

    $online_ids = $redis->executeCommand($scmd);


    View Slide

  59. Example: Friends Online
    • Data Types:

    • Sets

    • Additional Commands:

    • SUNIONSTORE

    • SINTER

    View Slide

  60. My Friends Online
    /* My Friends Online */

    $keys = array('online_users');

    $user_id = '1234';

    // Get Current Time

    $minute = date("i",time());


    $count = 0;

    $minutes_ago = 5;

    while($count < $minutes_ago)

    {

    " $keys[] = "online:".$minute;

    " $count++;

    " $min--;

    " if($min < 0) { $min = 59; }

    }


    // SUNIONSTORE online_users online:10 online:9 online:8 online:7 online:6

    $scmd = $redis->createCommand("sunionstore",$keys);

    $redis->executeCommand($scmd);

    // SINTER online_users user_id:1234.friends_ids

    $online_friend_ids = $redis->sinter('online_users'

    " , 'user:'.$user_id.'.friend_ids');

    View Slide

  61. Under The Hood

    View Slide

  62. Few Things About Redis
    • Single Threaded

    • Can “Shard” For More Capacity /
    Performance

    • All Commands are Atomic

    • Transactions for Multiple Atomic
    Commands

    • Pipelining for High Performance

    View Slide

  63. Persistence
    • Snapshots on a Configurable Schedule

    • Will “fork” the process and write the
    DataSet to Disk

    • Append-Only File

    • Will Let You Survive a “Reboot”

    View Slide

  64. Lua Scripting
    • Introduced: Redis 2.6.0

    • Allows for “querying”, more like Map/
    Reduce

    • Great for common, reusable logic

    View Slide

  65. Pattern Scanning
    • In Redis 2.8.0 support was added for
    cursor-based scanning keys, hashes, sets,
    sorted sets, and lists.

    View Slide

  66. Redis Sentinel
    • Monitors Redis Instances & Coordinates
    Notifications, Automatic Fail-overs,
    Replication, etc.

    View Slide

  67. Redis Cluster
    • redis-cluster is currently being developed &
    a release candidate can be downloaded to
    try out.

    • Allows for master-master cluster

    • Right now only recommended for advanced
    users at the moment.

    View Slide

  68. How About a Live
    Demo?

    View Slide

  69. One Redis Server

    View Slide

  70. 25 Servers * 10 Clients = 

    250 Workers

    View Slide

  71. Vagrant
    Digital Ocean
    Salt Stack
    Redis & PHP

    View Slide

  72. Demo Time!

    View Slide

  73. Questions?

    View Slide

  74. Your Homework
    Download & Play Around with Redis!

    It’s Awesome!

    View Slide

  75. More Info?
    http://www.justincarmony.com/redis

    View Slide

  76. Flickr Credits & Images:
    http://gim.ie/jsLJ

    View Slide

  77. Thanks!
    Rate My Talk (PLEASE): https://joind.in/10869

    "
    Twitter: @JustinCarmony

    "
    IRC: carmony

    #uphpu #phpmentoring #salt

    "
    Website:

    http://www.justincarmony.com/blog

    "
    Email:

    [email protected]

    View Slide