Learn Redis the hard way ... in production

Held at the FrOSCon in Sankt Augustin, Germany at Sunday, 20. August 2017.
This is the 45 minutes long (english) version.

Transcript

1. Learn Redis the hard way ... in production Sunday, 20. August 2017 @ in Sankt Augustin Andy Grunwald / @andygrunwald
2. Andy Grunwald
3. Andy Grunwald
4. Andy Grunwald Site Reliability- and Platform Engineering @andygrunwald andygrunwald
7. 6 Redis is an [...], in-memory data structure store, used as a database, cache and message broker. Redis website redis.io
8. 7 Redis supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io
9. 7 Redis supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $string = "Hi Sankt Augustin!";
10. 8 Redis supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $hash = ["lang" => "php", "conference" => "FrOSCon"];
11. 9 Redis supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $list = ["FrOSCon", "conference", "Sankt Augustin"];
12. 10 Redis supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis website redis.io <?php $set = ["FrOSCon" => true, "conference" => true];
15. The story ...
19. September, 2010
21. February, 2013
22. phpredis (PHP-Ext) predis (PHP)
23. February, 2014
24. 20 https://www.trivago.com/
25. www7.trivago.com 20140202102259941 | WARN | ... Redis\ConnectException: Unable to connect: read error on connection ... #0 /.../vendor/.../Redis/RedisPool.php(106): ...\Redis\RedisPool- >connect(Object(Redis), Object(...\Redis\RedisServerConfiguration)) #1 /.../vendor/.../Redis/RedisClient.php(130): ...\Redis\RedisPool->get('default', true) #2 /.../vendor/.../Redis/RedisClient.php(94): ...\Redis\RedisClient->setMode(false) ... #17 /.../app/bootstrap.php.cache(551): Symfony\Bundle\FrameworkBundle\HttpKernel- >handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #18 /.../web/app.php(15): Symfony\Component\HttpKernel\Kernel- >handle(Object(Symfony\Component\HttpFoundation\Request)) #19 {main} | 12.34.56.78 | www.trivago.de | /? aDateRange%5Barr%5D=2014-05-20&aDateRange%5Bdep%5D=2014-05-21&iRoomType=1&iPathId=44 742... | Mozilla/5.0 (WindowsNT 6.1; Trident/7.0; rv:11.0) like Gecko
28. Oct 30, 2011
29. Oct 30, 2011 Open
30. Oct 30, 2011 Open 132 comments
31. 24 PHP • Raising PHP connection and command timeouts from 500ms to 2.5 seconds • Disabling the PHP setting default_socket_timeout Operating System • Disabling SYN cookies on the host systems • Checking the number of file descriptors on Redis and Webservers • Raising the mbuffer of the host systems • Control and adjust the TCP backlog sizes
34. 26 Closing Redis connections at the end of a web request February 2014
35. 26 Closing Redis connections at the end of a web request February 2014 A/B-Testing of connection libraries March 2014
36. 26 Closing Redis connections at the end of a web request February 2014 A/B-Testing of connection libraries March 2014 Upgrade Redis May 2014
38. May, 2014
40. 30 Anything that can go wrong, will go wrong. Murphy
41. 1571] 21 May 18:06:26.550 * Background saving started by pid 20267 [20267] 21 May 18:08:25.195 * DB saved on disk [1571 | signal handler] (1400688506) --- WATCHDOG TIMER EXPIRED --- [1571] 21 May 18:08:26.331 # === REDIS BUG REPORT START: Cut & paste starting from here === [1571] 21 May 18:08:26.331 # Redis 2.8.9 crashed by signal: 11 [1571] 21 May 18:08:26.331 # Failed assertion: <no assertion failed> (<no file>:0) [1571] 21 May 18:08:26.331 # --- STACK TRACE _._ _.-``__ ''-._ _.-`` `. `_. ''-._ Redis 2.8.9 (00000000/0) 64 bit .-`` .-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in stand alone mode |`-._`-...-` __...-.``-._|'` _.-'| Port: 6380 | `-._ `._ / _.-' | PID: 20398 `-._ `-._ `-./ _.-' _.-' |`-._`-._ `-.__.-' _.-'_.-'| | `-._`-._ _.-'_.-' | http://redis.io `-._ `-._`-.__.-'_.-' _.-'
42. Tip #1
 Redis is single threaded Every command will block for a certain time.
43. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
44. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
45. ... [20398] 22 May 09:20:55.351 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:20:55.759 * Background saving started by pid 41941 [41941] 22 May 09:22:48.197 * DB saved on disk [20398] 22 May 09:22:49.321 * Background saving terminated with success [20398] 22 May 09:25:23.299 * 10000 changes in 60 seconds. Saving... [20398] 22 May 09:25:23.644 * Background saving started by pid 42027 [20398] 22 May 09:26:50.646 # Accepting client connection: accept: Software caused connection abort [20398] 22 May 09:26:50.900 # Accepting client connection: accept: Software caused connection abort ...
46. Tip #2
 Review your persistence requirements Sane defaults, but does those apply to you? BGSAVE or AOF?
48. Pipelining
49. Tip #3
 Shared data contexts might block each other. Different use cases and commands. SELECT == Antipattern
50. June, 2014
51. Bachmann-Landau Notation
52. Big O Notation
55. O(n) can kill you
56. Tip #4
 Check the time complexity of your commands High Big O estimations can slow down the performance
57. August, 2014
60. Tip #5
 Consider a proxy in between Reduce connection overhead and block expensive / dangerous commands
61. September, 2014 up to now
62. If {# of data} > available RAM
64. 51 Eviction policies • noeviction: return errors when the memory limit was reached ...
 • allkeys-lru: [...] less recently used (LRU) keys first • volatile-lru: [...] less recently used (LRU) keys first (with TTL)
 • allkeys-random: evict random keys ...
 • volatile-random: evict random keys (with TTL)
 • volatile-ttl: [...] evict keys with an expire set, with a shorter time to live (TTL) first.
65. Tip #6
 Review your runtime requirements Plan your maxmemory limit and set the right maxmemory-policy
66. If {# of data} > available RAM
68. Tip #7
 Plan your capacity and use it Shard your data over multiple machines to store more and be resistent to failure
70. 57 Redis is a very sharp knife. 
 It is perfect for some problems, and a horrible fit for others. btilly HackerNews, 31. March 2017
71. 58 One problem with Redis is that it looks superficially simple, but actually, like any tool used in critical contexts, there are two possibilities: 
 1) Know how it works very well and do great things with it. 
 2) Don't understand it properly and find yourself in big troubles soon or later. antirez HackerNews, 31. March 2017
72. Learn Redis the hard way ... in production Sunday, 20. August 2017 @ in Sankt Augustin Andy Grunwald / @andygrunwald