Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Redis: based on real story

94575b73a05d48e567423d7d6ca8273c?s=47 Poga Po
March 31, 2012

Redis: based on real story

Talk @ NoSQL Taiwan

94575b73a05d48e567423d7d6ca8273c?s=128

Poga Po

March 31, 2012
Tweet

More Decks by Poga Po

Other Decks in Programming

Transcript

  1. Poga 2012.03.31 based on real story

  2. Hi 你好 Hallo 안녕 ciao salut こんにちは

  3. i Poga

  4. i wear many hats

  5. i wear many hats build products

  6. Product released PARTY? Let’s

  7. Database CPU Utilization

  8. Performance Degrading = Unhappy User = Less Profit

  9. Complex Queries Inefficient schema

  10. Limited Time

  11. None
  12. Just ADD it to your stack

  13. Just ADD it to your stack Not Replace

  14. Special Performance Simple In-memory Atomic

  15. Special Performance Simple In-memory Atomic

  16. Management Problem a.k.a. “We don’t have a MIS”

  17. Dependency Feb 25, 2009 First public beta Oct 14, 2011

    Redis 2.4 First external dependency jemalloc Mar 31, 2012
  18. Installation Feb 25, 2009 First public beta Oct 14, 2011

    Redis 2.4 First external dependency jemalloc Mar 31, 2012 make install make install
  19. Stable

  20. None
  21. Special Performance Simple In-memory Atomic

  22. Special Performance Simple In-memory Atomic

  23. In-Memory Memcached High Performance Simple Key-Value Redis In-Memory High Performance

    Data Structures Persistence Replication ...
  24. Data Structures List, Set, Sorted Set, Hash, String http://redis.io/commands

  25. Understand your problem http://nosql.mypopescu.com/post/18977451039/redis-guide-what-each-redis-data-type-should-be-used Every command in Redis is manipulating

    a data structure
  26. Special Performance Simple In-memory Atomic

  27. Special Performance Simple In-memory Atomic

  28. =

  29. http://redis.io/topics/benchmarks FAST for real

  30. Special Performance Simple In-memory Atomic

  31. No Race Condition every command in redis is atomic use

    redis as a incremental counter “There are very few systems that support an “increment and return” operation, and Redis is one of them (Oracle sequences are another).” http://nateware.com/2010/02/18/an-atomic-rant/
  32. No Race Condition every command in redis is atomic Combine

    multiple command: still atomic! MULTI SET key 123 SET key2 456 EXEC Warning: Don’t confuse with transaction
  33. Special Performance Simple In-memory Atomic

  34. i

  35. Did I make any changes to MySQL? NO It’s still

    our main database
  36. Did I solve the problem? YES solve complex queries via

    Redis data structure
  37. Happy Database Add Redis to stack

  38. How? Sorted Set: Per user notification String: API request limiter

    String, List: Request logging List: Job Queue
  39. Real World = Risk Persistence Replication Sharding

  40. Persistence RDB AOF

  41. RDB • Snapshot • Compact • .rdb file can be

    transfered to other backup system • Chances to lose data
  42. AOF • logs of every write operation • durable •

    bigger file size
  43. You want a degree of safety comparable to PostgreSQL? Just

    use BOTH
  44. Replication Master Slave Slave Slave

  45. Replication • Read-only salves • Save on slaves

  46. Sharding • Memory is limited • Split data into multiple

    redis servers
  47. Presharding Single Machine Redis instance Redis instance Redis instance Redis

    instance Redis instance Redis instance Redis instance Redis instance Redis instance Redis instance Multiple Machine
  48. Some Case Study • Resque • Rubygems.org • Tumblr •

    Instagram
  49. Resque High Performance Job Queue https://github.com/blog/542-introducing-resque

  50. Rubygems.org http://www.scribd.com/doc/34269414/Redis-Persistence-Power feature switch API request limit Logging Job Queue

  51. Tumblr http://engineering.tumblr.com/post/7819252942/staircar-redis-powered-notifications Per-user Notification 30,000 request/s per server

  52. Instagram http://engineering.tumblr.com/post/7819252942/staircar-redis-powered-notifications Simple Key-value mapping 1/3 cost on hardware

  53. Cross server temporary file sharing i Redis string is binary-safe

  54. Future

  55. Future Lua Scripting Watchdog

  56. Lua Scripting Atomic Saving bandwidth between servers

  57. Ruby Redis Sum up 200,000 values ... 200,000 times

  58. Ruby Redis Sum up 200,000 values, with Lua scripting 1

    request
  59. Data Model Query Language Scripting MySQL Redis Relational Table SQL

    Data Structures Lua Lua
  60. Watchdog Let Redis monitor itself Like slow queries in MySQL

    More information to diagnose problem
  61. Bright Future Lua Scripting Watchdog

  62. = Unique

  63. Unique = Not replacing anyone Job security of other database

  64. What will u get

  65. More life less cost Happy User

  66. Thank you question? @devpoga