Slide 1

Slide 1 text

-FTTPOT MFBSOFE VTJOH Fernando Blat @ferblape - fernando.blat.es jueves 24 de mayo de 2012

Slide 2

Slide 2 text

%JTDMBJNFS jueves 24 de mayo de 2012

Slide 3

Slide 3 text

jueves 24 de mayo de 2012

Slide 4

Slide 4 text

Developer jueves 24 de mayo de 2012

Slide 5

Slide 5 text

Redis is: an open source, advanced key-value store a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. http://redis.io jueves 24 de mayo de 2012

Slide 6

Slide 6 text

Redis is: an open source, advanced key-value store a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. http://redis.io jueves 24 de mayo de 2012

Slide 7

Slide 7 text

jueves 24 de mayo de 2012

Slide 8

Slide 8 text

jueves 24 de mayo de 2012

Slide 9

Slide 9 text

1FSTJTUFODF RDB Snapshot of the whole database AOF (Append Only File) logs Operation logs http://antirez.com/post/redis-persistence-demystified.html http://redis.io/topics/persistence jueves 24 de mayo de 2012

Slide 10

Slide 10 text

################################ SNAPSHOTTING ################################# # # Save the DB on disk: # # save # # Will save the DB if both the given number of seconds and the given # number of write operations against the DB occurred. # # In the example below the behaviour will be to save: # after 900 sec (15 min) if at least 1 key changed # after 300 sec (5 min) if at least 10 keys changed # after 60 sec if at least 10000 keys changed # # Note: you can disable saving at all commenting all the "save" lines. save 1800 1 save 1200 10 save 300 10000 # Compress string objects using LZF when dump .rdb databases? # For default that's set to 'yes' as it's almost always a win. # If you want to save some CPU in the saving child set it to 'no' but # the dataset will likely be bigger if you have compressible values or keys. rdbcompression yes # The filename where to dump the DB dbfilename dump.rdb # The working directory. # # The DB will be written inside this directory, with the filename specified # above using the 'dbfilename' configuration directive. # # Also the Append Only File will be created inside this directory. # # Note that you must specify a directory here, not a file name. dir /var/lib/redis jueves 24 de mayo de 2012

Slide 11

Slide 11 text

#BDLVQT $ crontab -l ... cp /var/lib/redis/dump.rdb \ /mnt/backups/dump-`date +%Y%m%d`.rdb jueves 24 de mayo de 2012

Slide 12

Slide 12 text

'JSTU JTTVF jueves 24 de mayo de 2012

Slide 13

Slide 13 text

Redis compiled in 32 bits jueves 24 de mayo de 2012

Slide 14

Slide 14 text

4FDPOE JTTVF jueves 24 de mayo de 2012

Slide 15

Slide 15 text

Redis will use all the memory available if needed jueves 24 de mayo de 2012

Slide 16

Slide 16 text

$ top top - 19:48:36 up 90 days, 17:53, 1 user, load average: 0.17, 0.39, 0.46 Tasks: 119 total, 1 running, 114 sleeping, 0 stopped, 4 zombie Cpu(s): 6.3%us, 0.5%sy, 0.0%ni, 93.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 8160804k total, 8064100k used, 96704k free, 68580k buffers Swap: 262140k total, 249968k used, 12172k free, 2083720k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5630 redis 20 0 4158m 3.9g 572 S 0 49.7 116:32.46 /usr/local/bin/redis-server jueves 24 de mayo de 2012

Slide 17

Slide 17 text

$ top top - 19:48:36 up 90 days, 17:53, 1 user, load average: 0.17, 0.39, 0.46 Tasks: 119 total, 1 running, 114 sleeping, 0 stopped, 4 zombie Cpu(s): 6.3%us, 0.5%sy, 0.0%ni, 93.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 8160804k total, 8064100k used, 96704k free, 68580k buffers Swap: 262140k total, 249968k used, 12172k free, 2083720k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5630 redis 20 0 4158m 3.9g 572 S 0 49.7 116:32.46 /usr/local/bin/redis-server jueves 24 de mayo de 2012

Slide 18

Slide 18 text

$ top top - 19:48:36 up 90 days, 17:53, 1 user, load average: 0.17, 0.39, 0.46 Tasks: 119 total, 1 running, 114 sleeping, 0 stopped, 4 zombie Cpu(s): 6.3%us, 0.5%sy, 0.0%ni, 93.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 8160804k total, 8064100k used, 96704k free, 68580k buffers Swap: 262140k total, 249968k used, 12172k free, 2083720k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5630 redis 20 0 4158m 3.9g 572 S 0 49.7 116:32.46 /usr/local/bin/redis-server jueves 24 de mayo de 2012

Slide 19

Slide 19 text

$ top top - 19:48:36 up 90 days, 17:53, 1 user, load average: 0.17, 0.39, 0.46 Tasks: 119 total, 1 running, 114 sleeping, 0 stopped, 4 zombie Cpu(s): 6.3%us, 0.5%sy, 0.0%ni, 93.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 8160804k total, 8064100k used, 96704k free, 68580k buffers Swap: 262140k total, 249968k used, 12172k free, 2083720k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5630 redis 20 0 4158m 3.9g 572 S 0 49.7 116:32.46 /usr/local/bin/redis-server jueves 24 de mayo de 2012

Slide 20

Slide 20 text

$ top top - 19:48:36 up 90 days, 17:53, 1 user, load average: 0.17, 0.39, 0.46 Tasks: 119 total, 1 running, 114 sleeping, 0 stopped, 4 zombie Cpu(s): 6.3%us, 0.5%sy, 0.0%ni, 93.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 8160804k total, 8064100k used, 96704k free, 68580k buffers Swap: 262140k total, 249968k used, 12172k free, 2083720k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5630 redis 20 0 4158m 3.9g 572 S 0 49.7 116:32.46 /usr/local/bin/redis-server jueves 24 de mayo de 2012

Slide 21

Slide 21 text

$ top top - 19:48:36 up 90 days, 17:53, 1 user, load average: 0.17, 0.39, 0.46 Tasks: 119 total, 1 running, 114 sleeping, 0 stopped, 4 zombie Cpu(s): 6.3%us, 0.5%sy, 0.0%ni, 93.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 8160804k total, 8064100k used, 96704k free, 68580k buffers Swap: 262140k total, 249968k used, 12172k free, 2083720k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5630 redis 20 0 4158m 3.9g 572 S 0 49.7 116:32.46 /usr/local/bin/redis-server jueves 24 de mayo de 2012

Slide 22

Slide 22 text

$ top top - 19:48:36 up 90 days, 17:53, 1 user, load average: 0.17, 0.39, 0.46 Tasks: 119 total, 1 running, 114 sleeping, 0 stopped, 4 zombie Cpu(s): 6.3%us, 0.5%sy, 0.0%ni, 93.0%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st Mem: 8160804k total, 8064100k used, 96704k free, 68580k buffers Swap: 262140k total, 249968k used, 12172k free, 2083720k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5630 redis 20 0 4158m 3.9g 572 S 0 49.7 116:32.46 /usr/local/bin/redis-server jueves 24 de mayo de 2012

Slide 23

Slide 23 text

# This is the path of the Redis swap file. As you can guess, swap files # can't be shared by different Redis instances, so make sure to use a swap # file for every redis process you are running. Redis will complain if the # swap file is already in use. # # The best kind of storage for the Redis swap file (that's accessed at random) # is a Solid State Disk (SSD). # # *** WARNING *** if you are using a shared hosting the default of putting # the swap file under /tmp is not secure. Create a dir with access granted # only to Redis user and configure Redis to create the swap file there. vm-swap-file /tmp/redis.swap # vm-max-memory configures the VM to use at max the specified amount of # RAM. Everything that deos not fit will be swapped on disk *if* possible, that # is, if there is still enough contiguous space in the swap file. # # With vm-max-memory 0 the system will swap everything it can. Not a good # default, just specify the max amount of RAM you can in bytes, but it's # better to leave some margin. For instance specify an amount of RAM # that's more or less between 60 and 80% of your free RAM. vm-max-memory 5368709120 7JSUVBM .FNPSZ jueves 24 de mayo de 2012

Slide 24

Slide 24 text

7JSUVBM .FNPSZ # Virtual Memory allows Redis to work with datasets bigger than the actual # amount of RAM needed to hold the whole dataset in memory. # In order to do so very used keys are taken in memory while the other keys # are swapped into a swap file, similarly to what operating systems do # with memory pages. # # To enable VM just set 'vm-enabled' to yes, and set the following three # VM parameters accordingly to your needs. vm-enabled yes jueves 24 de mayo de 2012

Slide 25

Slide 25 text

5IJSE JTTVF jueves 24 de mayo de 2012

Slide 26

Slide 26 text

dump.rdb does not contain all the data, at least, in version 2.2 8IFO 3FEJT TUBSUT UP “TXBQ” jueves 24 de mayo de 2012

Slide 27

Slide 27 text

#(4"7& http://redis.io/commands/bgsave Save the DB in background. Redis forks, the parent continues to server the clients, the child saves the DB on disk then exit. A client my be able to check if the operation succeeded using the LASTSAVE command. jueves 24 de mayo de 2012

Slide 28

Slide 28 text

#BDLVQT #!/bin/bash SLEEP=30 function get_last_save { rm /tmp/previous_last_save echo "lastsave" | redis-cli > /tmp/previous_last_save lastsave=`cat /tmp/previous_last_save` } get_last_save previous_last_save=$lastsave echo "bgsave" | redis-cli while [ $lastsave = $previous_last_save ]; do sleep $SLEEP get_last_save done cp /var/lib/redis/dump.rdb /var/lib/redis/backups/dump-`date '+%Y%m%d%H%M'`.rdb # Remove backups older than 60 days find /var/lib/redis/backups -mtime +60 | xargs rm jueves 24 de mayo de 2012

Slide 29

Slide 29 text

4VNNBSJ[JOH jueves 24 de mayo de 2012

Slide 30

Slide 30 text

4VNNBSJ[JOH know your tools invest time check your backups jueves 24 de mayo de 2012

Slide 31

Slide 31 text

5IBOL ZPV jueves 24 de mayo de 2012