Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Learn Redis the hard way ... in production

Learn Redis the hard way ... in production

Held at the FrOSCon in Sankt Augustin, Germany at Sunday, 20. August 2017.
This is the 45 minutes long (english) version.

Andy Grunwald

August 20, 2017
Tweet

More Decks by Andy Grunwald

Other Decks in Technology

Transcript

  1. Learn Redis the hard way ... in production Sunday, 20.

    August 2017 @ in Sankt Augustin Andy Grunwald / @andygrunwald
  2. 6 Redis is an [...], in-memory data structure store, used

    as a database, cache and message broker. Redis website redis.io
  3. 7 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io
  4. 7 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $string = "Hi Sankt Augustin!";
  5. 8 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $hash = ["lang" => "php", "conference" => "FrOSCon"];
  6. 9 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $list = ["FrOSCon", "conference", "Sankt Augustin"];
  7. 10 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $set = ["FrOSCon" => true, "conference" => true];
  8. 11

  9. 12

  10. www7.trivago.com 20140202102259941 | WARN | ... Redis\ConnectException: Unable to connect:

    read error on connection ... #0 /.../vendor/.../Redis/RedisPool.php(106): ...\Redis\RedisPool- >connect(Object(Redis), Object(...\Redis\RedisServerConfiguration)) #1 /.../vendor/.../Redis/RedisClient.php(130): ...\Redis\RedisPool->get('default', true) #2 /.../vendor/.../Redis/RedisClient.php(94): ...\Redis\RedisClient->setMode(false) ... #17 /.../app/bootstrap.php.cache(551): Symfony\Bundle\FrameworkBundle\HttpKernel- >handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #18 /.../web/app.php(15): Symfony\Component\HttpKernel\Kernel- >handle(Object(Symfony\Component\HttpFoundation\Request)) #19 {main} | 12.34.56.78 | www.trivago.de | /? aDateRange%5Barr%5D=2014-05-20&aDateRange%5Bdep%5D=2014-05-21&iRoomType=1&iPathId=44 742... | Mozilla/5.0 (WindowsNT 6.1; Trident/7.0; rv:11.0) like Gecko
  11. 24 PHP • Raising PHP connection and command timeouts from

    500ms to 2.5 seconds • Disabling the PHP setting default_socket_timeout Operating System • Disabling SYN cookies on the host systems • Checking the number of file descriptors on Redis and Webservers • Raising the mbuffer of the host systems • Control and adjust the TCP backlog sizes
  12. 26

  13. 26 Closing Redis connections at the end of a web

    request February 2014 A/B-Testing of connection libraries March 2014
  14. 26 Closing Redis connections at the end of a web

    request February 2014 A/B-Testing of connection libraries March 2014 Upgrade Redis May 2014
  15. 1571] 21 May 18:06:26.550 * Background saving started by pid

    20267 [20267] 21 May 18:08:25.195 * DB saved on disk [1571 | signal handler] (1400688506) --- WATCHDOG TIMER EXPIRED --- [1571] 21 May 18:08:26.331 # === REDIS BUG REPORT START: Cut & paste starting from here === [1571] 21 May 18:08:26.331 # Redis 2.8.9 crashed by signal: 11 [1571] 21 May 18:08:26.331 # Failed assertion: <no assertion failed> (<no file>:0) [1571] 21 May 18:08:26.331 # --- STACK TRACE _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 2.8.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in stand alone mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 6380 | `-._ `._ / _.-' | PID: 20398 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-'
  16. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60

    seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
  17. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60

    seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
  18. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60

    seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
  19. Tip #3
 Shared data contexts might block each other. Different

    use cases and commands. SELECT == Antipattern
  20. Tip #4
 Check the time complexity of your commands High

    Big O estimations can slow down the performance
  21. Tip #5
 Consider a proxy in between Reduce connection overhead

    and block expensive / dangerous commands
  22. 51 Eviction policies • noeviction: return errors when the memory

    limit was reached ...
 • allkeys-lru: [...] less recently used (LRU) keys first • volatile-lru: [...] less recently used (LRU) keys first (with TTL)
 • allkeys-random: evict random keys ...
 • volatile-random: evict random keys (with TTL)
 • volatile-ttl: [...] evict keys with an expire set, with a shorter time to live (TTL) first.
  23. Tip #7
 Plan your capacity and use it Shard your

    data over multiple machines to store more and be resistent to failure
  24. 57 Redis is a very sharp knife. 
 It is

    perfect for some problems, and a horrible fit for others. btilly HackerNews, 31. March 2017
  25. 58 One problem with Redis is that it looks superficially

    simple, but actually, like any tool used in critical contexts, there are two possibilities: 
 1) Know how it works very well and do great things with it. 
 2) Don't understand it properly and find yourself in big troubles soon or later. antirez HackerNews, 31. March 2017
  26. Learn Redis the hard way ... in production Sunday, 20.

    August 2017 @ in Sankt Augustin Andy Grunwald / @andygrunwald