$whoami
• Kingdom of Himalayas
• Long term consultant
• Own startup
• Principal Engineer @ Zoomcar
• @phoenixwizard in most places
Image source: https://en.wikipedia.org/wiki/File:Mount_Everest_as_seen_from_Drukair2_PLW_edit.jpg
Slide 3
Slide 3 text
What to expect
• A primer
• The “I have a hammer” anti-pattern
• The “duct-taped life boat” anti-pattern
• The “santa stuck in chimney” anti-patten
Slide 4
Slide 4 text
What to expect
• The “make Redis a turtle” anti-pattern
• The “annoying neighbour” anti-pattern
• Flying blind into the hurricane
• Kayaking in the pacific Ocean
Slide 5
Slide 5 text
The Mythical Silver Bullet
Redis is an open source (BSD
licensed), in-memory data structure
store, used as a database, cache and
message broker.
Slide 6
Slide 6 text
The “I have a hammer”
anti-pattern
Slide 7
Slide 7 text
The Redis World
• Binary-safe strings
• Lists (basically linked list)
• Sets (Unique unsorted string elements)
• Sorted sets (Sets with every string having a score)
• Hashes (similar to ruby or python hashes)
Slide 8
Slide 8 text
Redis World (contd)
• Bit arrays (or simply bitmaps, handle strings like an
array of bits)
• HyperLogLogs (estimate cardinality of set [Don’t be
scared !! ])
• Streams (append only collection of map-like entries)
GOTCHA?
• You have set the interval for 5 minutes
• What if server goes down 3 minutes after the
snapshot?
• Needs to fork, which is time consuming for big
datasets
Slide 18
Slide 18 text
Append Only File
• Popularly known as AOF
• Changelog style persistent format
• appendfsync can be “no/everysec/always”
Slide 19
Slide 19 text
GOTCHAS?
• AOF Files bigger than RDB
• AOF can be slower depending on sync policy
• There are cases where AOF might not reproduce
exactly same dataset on reloading
Slide 20
Slide 20 text
WHAT IS THE BEST WAY?
• Have your cake and eat it !!
• Do both RDB and AOF(set sync to everysec)
Slide 21
Slide 21 text
– somewhere in Redis documentation
“Don’t just turn on AOF and restart your server”
Slide 22
Slide 22 text
What NOT TO DO
• You already have persistence set to RDB
• Change it to AOF
• And Restart the Server
Slide 23
Slide 23 text
What you should DO?
• config set appendonly yes
Slide 24
Slide 24 text
The “Santa
stuck in
chimney”
anti-patten
Image source: https://www.maxpixel.net/Present-Chimney-Christmas-Santa-Stuck-Holiday-2946681
Slide 25
Slide 25 text
• You are an amazing developer
• Set up persistence perfectly
• Now you have millions of keys in your Redis instance
• BUT YOUR REDIS IS RANDOMLY GOING SLOW
Slide 26
Slide 26 text
So WHAT HAPPENED?
• There is a cron (the golden bullet) which reads & writes
a MILLION records in Redis (the silver bullet)
Slide 27
Slide 27 text
– Redis getting bombarded by your cron
“I have a single thread, nobody loves me”
Slide 28
Slide 28 text
But I read Redis is not Single
Threaded
• More threads in background perform backend cleaning
works
• Since v4.0, deleting objects in background & blocking
commands implemented by Redis modules
Slide 29
Slide 29 text
The “Make
Redis a turtle”
anti-pattern
Image source: https://en.wikipedia.org/wiki/File:Kturtle_side_view.svg
Slide 30
Slide 30 text
• You need to process all the keys, what are you going to
do? Use the KEYS command?
• But did you know this is a BLOCKING CALL?
Slide 31
Slide 31 text
But What can I do now?
• Use NON-BLOCKING calls wherever possible !!
• Instead of KEYS use SCAN?
• SCAN is a cursor based call
Slide 32
Slide 32 text
What if I want a pattern?
• Use MATCH Along with your SCAN
• BEWARE: MATCH works on subset returned by SCAN
Slide 33
Slide 33 text
The “annoying
neighbour”
anti-pattern
Image source: https://pixabay.com/illustrations/loudspeaker-man-boy-holding-1459128/
Slide 34
Slide 34 text
REDIS CLUSTERS
• They are not very well known to be as mature as other
DB clusters
• Not all commands & features available on single Redis
work on partitions
• Scale up and down might become tricky
Slide 35
Slide 35 text
• For high availability, there is the master-slave model
• Data is partitioned among various Redis instances and
remains available even when a small subset of nodes is
unavailable
Slide 36
Slide 36 text
Flying blind into
the hurricane
Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch
Slide 37
Slide 37 text
You should know there is a
problem to solve the
problem
Slide 38
Slide 38 text
Required processes running?
• Is the process running?
• What is the uptime?
Slide 39
Slide 39 text
System resources utilisation
• Load
• CPU usage
• Memory usage
• Swap usage
• Network Bandwidth
• Disk usage
Typical Failure Points
• latency
• used memory
• memory fragmentation ratio
• evicted keys
• blocked clients
Slide 42
Slide 42 text
WHO WILL BE THY
SAVIOUR?
Slide 43
Slide 43 text
• Redis-cli info command
• New Relic
• Nagios/Icinga
• Redislabs hosted service
Slide 44
Slide 44 text
Kayaking in
the pacific
Ocean
Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch
Slide 45
Slide 45 text
• How fast is your fork()?
• How is it going to affect your Redis & Other Linux
Processes
Slide 46
Slide 46 text
• Have you considered latency due to swapping?
• This again is not limited to Redis!!
Slide 47
Slide 47 text
• You do realise that your persistence strategy can
increase latency. Right?
Slide 48
Slide 48 text
• Network Latency
• CPUs (Large ones with large caches) [Intel yaaay]
• RAM Speed (>10 kb objects)
• Bare Metal vs VM
Slide 49
Slide 49 text
Redis Documentation is
AWESOME
Slide 50
Slide 50 text
I Mean
SUPER
AWESOME !!
Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch
Slide 51
Slide 51 text
Quick Conclusion?
• Redis is not a silver Bullet
• Redis defaults are not the best for everyone
• Spend time and figure what works best for you
• Redis does shine in a lot of places
• Make your own mistakes !!
Slide 52
Slide 52 text
@phoenixwizard
[email protected]
Thank You !!
Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch