Learn Redis the hard way ... in production (German Edition)

Learn Redis the hard way ... in production (German Edition)

Held at the Symfony Live in Cologne, Germany at Friday, 7. Apr 2017.
This is the 25 minutes long german version.

56b1575ab3b392b92c19bbcf39e808f1?s=128

Andy Grunwald

April 07, 2017
Tweet

Transcript

  1. Learn Redis the hard way ... in production Andy Grunwald

    @
  2. Andy Grunwald

  3. Andy Grunwald

  4. Andy Grunwald Site Reliability- and Platform Engineering @andygrunwald andygrunwald

  5. None
  6. 5 Redis is an [...], in-memory data structure store, used

    as a database, cache and message broker. Redis website redis.io
  7. 6 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io
  8. 6 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $string = "Hi Cologne!";
  9. 7 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $hash = ["lang" => "php", "conference" => "symfony"];
  10. 8 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $list = ["symfony", "conference", "cologne"];
  11. 9 Redis supports data structures such as strings, hashes, lists,

    sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $set = ["symfony" => true, "conference" => true];
  12. The story ...

  13. None
  14. None
  15. None
  16. September, 2010

  17. None
  18. February, 2013

  19. phpredis (PHP-Ext) predis (PHP)

  20. February, 2014

  21. None
  22. February, 2014

  23. 19 https://www.trivago.com/

  24. www7.trivago.com 20140202102259941 | WARN | ... Redis\ConnectException: Unable to connect:

    read error on connection ... #0 /.../vendor/.../Redis/RedisPool.php(106): ...\Redis\RedisPool- >connect(Object(Redis), Object(...\Redis\RedisServerConfiguration)) #1 /.../vendor/.../Redis/RedisClient.php(130): ...\Redis\RedisPool->get('default', true) #2 /.../vendor/.../Redis/RedisClient.php(94): ...\Redis\RedisClient->setMode(false) ... #17 /.../app/bootstrap.php.cache(551): Symfony\Bundle\FrameworkBundle\HttpKernel- >handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #18 /.../web/app.php(15): Symfony\Component\HttpKernel\Kernel- >handle(Object(Symfony\Component\HttpFoundation\Request)) #19 {main} | 12.34.56.78 | www.trivago.de | /? aDateRange%5Barr%5D=2014-05-20&aDateRange%5Bdep%5D=2014-05-21&iRoomType=1&iPathId=44 742... | Mozilla/5.0 (WindowsNT 6.1; Trident/7.0; rv:11.0) like Gecko
  25. None
  26. None
  27. Oct 30, 2011

  28. Oct 30, 2011 Open

  29. Oct 30, 2011 Open 113 comments

  30. 23 PHP • Raising PHP connection and command timeouts from

    500ms to 2.5 seconds • Disabling the PHP setting default_socket_timeout Operating System • Disabling SYN cookies on the host systems • Checking the number of file descriptors on Redis and Webservers • Raising the mbuffer of the host systems • Control and adjust the TCP backlog sizes
  31. None
  32. 25

  33. 25 Closing Redis connections at the end of a web

    request February 2014
  34. 25 Closing Redis connections at the end of a web

    request February 2014 A/B-Testing of connection libraries March 2014
  35. 25 Closing Redis connections at the end of a web

    request February 2014 A/B-Testing of connection libraries March 2014
  36. 25 Closing Redis connections at the end of a web

    request February 2014 A/B-Testing of connection libraries March 2014 Upgrade Redis May 2014
  37. None
  38. May, 2014

  39. None
  40. 29 Anything that can go wrong, will go wrong. Murphy

  41. 1571] 21 May 18:06:26.550 * Background saving started by pid

    20267 [20267] 21 May 18:08:25.195 * DB saved on disk [1571 | signal handler] (1400688506) --- WATCHDOG TIMER EXPIRED --- [1571] 21 May 18:08:26.331 # === REDIS BUG REPORT START: Cut & paste starting from here === [1571] 21 May 18:08:26.331 # Redis 2.8.9 crashed by signal: 11 [1571] 21 May 18:08:26.331 # Failed assertion: <no assertion failed> (<no file>:0) [1571] 21 May 18:08:26.331 # --- STACK TRACE _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 2.8.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in stand alone mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 6380 | `-._ `._ / _.-' | PID: 20398 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-'
  42. Tip #1
 Redis is single threaded Every command will block

    for a certain time.
  43. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60

    seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
  44. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60

    seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
  45. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60

    seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
  46. Tip #2
 Review your persistence requirements Sane defaults, but does

    those apply to you? BGSAVE or AOF?
  47. None
  48. Pipelining

  49. Tip #3
 Shared data contexts might block each other. Different

    use cases and commands. SELECT == Antipattern
  50. June, 2014

  51. Bachmann-Landau Notation

  52. Big O Notation

  53. None
  54. None
  55. O(n) can kill you

  56. Tip #4
 Check the time complexity of your commands High

    Big O estimations can slow down the performance
  57. August, 2014

  58. None
  59. None
  60. None
  61. Tip #5
 Consider a proxy in between Reduce connection overhead

    and block expensive / dangerous commands
  62. 48 Redis is a very sharp knife. 
 It is

    perfect for some problems, and a horrible fit for others. btilly HackerNews, 31. March 2017
  63. 49 One problem with Redis is that it looks superficially

    simple, but actually, like any tool used in critical contexts, there are two possibilities: 
 1) Know how it works very well and do great things with it. 
 2) Don't understand it properly and find yourself in big troubles soon or later. antirez HackerNews, 31. March 2017
  64. Learn Redis the hard way ... in production Andy Grunwald

    @