Slide 1

Slide 1 text

Redis Anti-Patterns Aram Bhusal @phoenixwizard

Slide 2

Slide 2 text

$whoami • Kingdom of Himalayas • Long term consultant • Own startup • Principal Engineer @ Zoomcar • @phoenixwizard in most places Image source: https://en.wikipedia.org/wiki/File:Mount_Everest_as_seen_from_Drukair2_PLW_edit.jpg

Slide 3

Slide 3 text

What to expect • A primer • The “I have a hammer” anti-pattern • The “duct-taped life boat” anti-pattern • The “santa stuck in chimney” anti-patten

Slide 4

Slide 4 text

What to expect • The “make Redis a turtle” anti-pattern • The “annoying neighbour” anti-pattern • Flying blind into the hurricane • Kayaking in the pacific Ocean

Slide 5

Slide 5 text

The Mythical Silver Bullet Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.

Slide 6

Slide 6 text

The “I have a hammer” anti-pattern

Slide 7

Slide 7 text

The Redis World • Binary-safe strings • Lists (basically linked list) • Sets (Unique unsorted string elements) • Sorted sets (Sets with every string having a score) • Hashes (similar to ruby or python hashes)

Slide 8

Slide 8 text

Redis World (contd) • Bit arrays (or simply bitmaps, handle strings like an array of bits) • HyperLogLogs (estimate cardinality of set [Don’t be scared !! ]) • Streams (append only collection of map-like entries)

Slide 9

Slide 9 text

The “Duct-Taped Life Boat” anti-pattern

Slide 10

Slide 10 text

• Product READY • Mail blast DONE • Press Release DONE • 100x Traffic DONE • Server Crash DONE Image source: https://www.goodfreephotos.com/vector-images/wild-west-cowboy-cartoon-vector-clipart.png.php

Slide 11

Slide 11 text

– Random article on the Web “Is your server not able to handle load? Use REDIS”

Slide 12

Slide 12 text

• You now take data from DB • Transform the data and put it on Redis • Ssshh: The parameters used to transform the data are in your MIND

Slide 13

Slide 13 text

Redis instance goes DOWN • What do you think has happened? • And just like that your whole data evaporates !! • NO RECOVERY !!

Slide 14

Slide 14 text

What could you have done?

Slide 15

Slide 15 text

Redis does do Persistence

Slide 16

Slide 16 text

Redis Database File • Popularly called RDB • Snapshot style persistence Image source: https://pixabay.com/illustrations/office-files-aktenordner-file-2761159/

Slide 17

Slide 17 text

GOTCHA? • You have set the interval for 5 minutes • What if server goes down 3 minutes after the snapshot? • Needs to fork, which is time consuming for big datasets

Slide 18

Slide 18 text

Append Only File • Popularly known as AOF • Changelog style persistent format • appendfsync can be “no/everysec/always”

Slide 19

Slide 19 text

GOTCHAS? • AOF Files bigger than RDB • AOF can be slower depending on sync policy • There are cases where AOF might not reproduce exactly same dataset on reloading

Slide 20

Slide 20 text

WHAT IS THE BEST WAY? • Have your cake and eat it !! • Do both RDB and AOF(set sync to everysec)

Slide 21

Slide 21 text

– somewhere in Redis documentation “Don’t just turn on AOF and restart your server”

Slide 22

Slide 22 text

What NOT TO DO • You already have persistence set to RDB • Change it to AOF • And Restart the Server

Slide 23

Slide 23 text

What you should DO? • config set appendonly yes

Slide 24

Slide 24 text

The “Santa stuck in chimney” anti-patten Image source: https://www.maxpixel.net/Present-Chimney-Christmas-Santa-Stuck-Holiday-2946681

Slide 25

Slide 25 text

• You are an amazing developer • Set up persistence perfectly • Now you have millions of keys in your Redis instance • BUT YOUR REDIS IS RANDOMLY GOING SLOW

Slide 26

Slide 26 text

So WHAT HAPPENED? • There is a cron (the golden bullet) which reads & writes a MILLION records in Redis (the silver bullet)

Slide 27

Slide 27 text

– Redis getting bombarded by your cron “I have a single thread, nobody loves me”

Slide 28

Slide 28 text

But I read Redis is not Single Threaded • More threads in background perform backend cleaning works • Since v4.0, deleting objects in background & blocking commands implemented by Redis modules

Slide 29

Slide 29 text

The “Make Redis a turtle” anti-pattern Image source: https://en.wikipedia.org/wiki/File:Kturtle_side_view.svg

Slide 30

Slide 30 text

• You need to process all the keys, what are you going to do? Use the KEYS command? • But did you know this is a BLOCKING CALL?

Slide 31

Slide 31 text

But What can I do now? • Use NON-BLOCKING calls wherever possible !! • Instead of KEYS use SCAN? • SCAN is a cursor based call

Slide 32

Slide 32 text

What if I want a pattern? • Use MATCH Along with your SCAN • BEWARE: MATCH works on subset returned by SCAN

Slide 33

Slide 33 text

The “annoying neighbour” anti-pattern Image source: https://pixabay.com/illustrations/loudspeaker-man-boy-holding-1459128/

Slide 34

Slide 34 text

REDIS CLUSTERS • They are not very well known to be as mature as other DB clusters • Not all commands & features available on single Redis work on partitions • Scale up and down might become tricky

Slide 35

Slide 35 text

• For high availability, there is the master-slave model • Data is partitioned among various Redis instances and remains available even when a small subset of nodes is unavailable

Slide 36

Slide 36 text

Flying blind into the hurricane Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch

Slide 37

Slide 37 text

You should know there is a problem to solve the problem

Slide 38

Slide 38 text

Required processes running? • Is the process running? • What is the uptime?

Slide 39

Slide 39 text

System resources utilisation • Load • CPU usage • Memory usage • Swap usage • Network Bandwidth • Disk usage

Slide 40

Slide 40 text

Redis Availability & Execution • connected_clients • keyspace • instantaneous_ops_per_sec • rdb_last_save_time • connected slaves • master_last_io_seconds_ago

Slide 41

Slide 41 text

Typical Failure Points • latency • used memory • memory fragmentation ratio • evicted keys • blocked clients

Slide 42

Slide 42 text

WHO WILL BE THY SAVIOUR?

Slide 43

Slide 43 text

• Redis-cli info command • New Relic • Nagios/Icinga • Redislabs hosted service

Slide 44

Slide 44 text

Kayaking in the pacific Ocean Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch

Slide 45

Slide 45 text

• How fast is your fork()? • How is it going to affect your Redis & Other Linux Processes

Slide 46

Slide 46 text

• Have you considered latency due to swapping? • This again is not limited to Redis!!

Slide 47

Slide 47 text

• You do realise that your persistence strategy can increase latency. Right?

Slide 48

Slide 48 text

• Network Latency • CPUs (Large ones with large caches) [Intel yaaay] • RAM Speed (>10 kb objects) • Bare Metal vs VM

Slide 49

Slide 49 text

Redis Documentation is AWESOME

Slide 50

Slide 50 text

I Mean SUPER AWESOME !! Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch

Slide 51

Slide 51 text

Quick Conclusion? • Redis is not a silver Bullet • Redis defaults are not the best for everyone • Spend time and figure what works best for you • Redis does shine in a lot of places • Make your own mistakes !!

Slide 52

Slide 52 text

@phoenixwizard [email protected] Thank You !! Image source: https://www.needpix.com/photo/87076/plane-cartoon-line-art-colouring-book-children-book-drawing-sketch