©2019 Wantedly, Inc. ΞϧΰϦζϜͱσʔλߏ଄͔Β ཧղ͢Δ3FEJT Learn Redis from Internal Algorithms and Data Structures New Grads Training 2019 April 21, 2019 - Yoshinori Kawasaki (@kawasy/@luvtechno)

©2019 Wantedly, Inc. 3FEJTʹ͍ͭͯͷલఏ஌ࣝ͸ඞཁ͋Γ·ͤΜɻ IUUQTHJUIVCDPNBOUJSF[SFEJTΛDMPOF͓͍͍ͯͯͩ͘͠͞ɻ CSBODIͰNBLFͱNBLFUFTU͕௨Δ؀ڥΛ༻ҙ͍ͯͩ͘͠͞ɻ .BD04Ͱ΋-JOVYͰ΋େৎ෉Ͱ͢ɻ $ݴޠͷجૅతͳ஌ࣝΛલఏͱ͠·͢ɻϙΠϯλ΍ߏ଄ମΛ஌͍ͬͯΔఔ౓Ͱେৎ෉Ͱ͢ɻ ࣄલ४උ

©2019 Wantedly, Inc. ͸͡Ίʹ Introduction

©2019 Wantedly, Inc. ༷ʑͳϛυϧ΢ΣΞ͕ࣾ಺Ͱ࢖ΘΕ͍ͯΔ w 1PTUHSF42- w 3FEJT w &MBTUJDTFBSDI w %HSBQI w .FN42- w $MPVE1VC4VC 3FEJT͸ࣾ಺Ͱ࠷΋࢖ΘΕ͍ͯΔ΋ͷͷͭ Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔

©2019 Wantedly, Inc. ࣾ಺ͷϝδϟʔͳϚΠΫϩαʔϏεͰ3FEJTΛར༻ w XBOUFEMZ w TZODNFTTFOHFSSBJMT w ZBTIJNBSBJMT w WJTJUSFDPNNFOEBUJPO w QFPQMFHP w ZBTIJNBVQEBUFT w OPUJpDBUJPOT w NJSBDMFTFBSDIHP Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔ Datadog APM Service Map https://app.datadoghq.com/service/map?env=production

©2019 Wantedly, Inc. 3FEJTΛ࢖͏ཧ༝ w ߴ଎ԽͷͨΊ w Ωϟογϡ w ඇಉظॲཧ w ϨεϙϯελΠϜ͸Ϣʔβମݧʹେ͖ͳӨڹ w ཭୤͕ݮΔ w ίϯόʔδϣϯ͕૿͑Δ w ʮγΰτͰίίϩΦυϧͻͱΛ;΍͢ʯ͜ͱ ͕Ͱ͖Δ Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔ ৽ଔݚमͰߦ͏*46$0/ͷັྗ /BP.JOBNJ https://speakerdeck.com/south37/xin-zu-yan-xiu-dexing-u-isucon-falsemei-li?slide=15

©2019 Wantedly, Inc. 3FEJTͷ֓ཁ ࣾ಺Ͱͷར༻ํ๏ ਖ਼͍͠༻๏༻ྔ What ͜ͷݚमͰֶ΂Δ͜ͱ

©2019 Wantedly, Inc. ਖ਼͍͠༻๏༻ྔΛकΒͳ͍ͱ SFEJTͷؔ܎ͨ͠JODJEFOUͷQPTUNPSUFN

©2019 Wantedly, Inc. 3FEJTͷ֓ཁ Overview

©2019 Wantedly, Inc. ଎͍σʔλߏ଄αʔό 3FEJTͱ͸ 1 2 3

©2019 Wantedly, Inc. *ONFNPSZ w ϝϞϦΞΫηε͸44%΍σΟεΫͷϥϯμ ϜΞΫηεΑΓѹ౗తʹ଎͍ 4JNQMF w 3%#.4Ͱ͸ཁٻ͞ΕΔΑ͏ͳෳࡶͳ͜ͱ͸͠ͳ͍ w ϝΠϯͷॲཧ͸γϯάϧϓϩηεɾγϯάϧεϨου w ࣮ࡍʹ͸%#εφοϓγϣοτͷ࡞੒ͳͲͷͨΊʹGPSL͢Δ w ϊϯϒϩοΩϯά*0ͱΠϕϯτϧʔϓʹΑΔ*0ଟॏԽ ଎͍

©2019 Wantedly, Inc. ͢΂ͯͷϓϩάϥϚ͕஌͓ͬͯ͘΂͖ϨΠςϯγͷ਺஋ ग़య: Latency Numbers Every Programmer Should Know https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html

©2019 Wantedly, Inc. Πϕϯτϧʔϓ // ae.c /* Include the best multiplexing layer supported by this system. * The following should be ordered by performances, descending. */ #ifdef HAVE_EVPORT #include "ae_evport.c" #else #ifdef HAVE_EPOLL #include "ae_epoll.c" #else #ifdef HAVE_KQUEUE #include "ae_kqueue.c" #else #include "ae_select.c" #endif #endif #endif ࡞ऀۘ੡ͷΠϕϯτϥΠϒϥϦ w FWQPSU4PMBSJTҎ߱ w FQPMM-JOVY w LRVFVF#4% w TFMFDUେମ͋Δ

©2019 Wantedly, Inc. w 4USJOH w -JTU w 4FU w 4PSUFE4FU w )BTI w #JUBSSBZ CJUNBQ w )ZQFS-PH-PH w 4USFBN ͨͩͷLFZWBMVFετΞͰ͸ͳ͍ɻͦΕͧΕৄ͘͠͸ޙड़͢Δɻ σʔλߏ଄ https://redis.io/topics/data-types-intro

©2019 Wantedly, Inc. ෳ਺ϓϩηε͕σʔλΛڞ༗ͯ͠ಡΈॻ͖Ͱ͖Δ w αʔόɾΫϥΠΞϯτϞσϧ w 5$1ιέοτΛ࢖ͬͯ௨৴ w γϯϓϧͳϓϩτίϧ ͋ͨΓ·͑ͱࢥ͏͔΋͠Ε·ͤΜ͕ʜ αʔό

©2019 Wantedly, Inc. 3%# w ඇಉظͷεφοϓγϣοτ w GPSLͯ͠ॻ͖ࠐΉͷͰ਌ϓϩηεͰσΟε Ϋ*0͕ൃੜ͠ͳ͍ w ཤྺͷόοΫΞοϓ͕͠΍͍͢ w Ϧελʔτ࣌ͷಡΈࠐΈ͕͸΍͍ w σʔλϩεͷՄೳੑ͸ΑΓେ͖͍ σΟεΫӬଓԽ "0' w ௥هܕ w ίϚϯυྻͷه࿥ w σΟεΫγʔΫ͕ൃੜ͠ͳ͍ w 3%#ܗࣜΑΓαΠζ͕େ͖͍ w ୯ಠͰ࢖Θͳ͍΄͏͕Α͍

©2019 Wantedly, Inc. • volatile-lru: Evict using approximated LRU among the keys with an expire set. • allkeys-lru: Evict any key using approximated LRU. • volatile-lfu: Evict using approximated LFU among the keys with an expire set. • allkeys-lfu: Evict any key using approximated LFU. • volatile-random: Remove a random key among the ones with an expire set. • allkeys-random: Remove a random key, any key. • volatile-ttl: Remove the key with the nearest expire time (minor TTL) • noeviction: Don't evict anything, just return an error on write operations. Eviction Policy ϝϞϦ͔ΒᷓΕͦ͏ͳ࣌ͷڍಈΛઃఆͰ͖Δ

©2019 Wantedly, Inc. w Ωϟογϡ༻్ w ΞΫηεස౓͕ภΔͳΒBMMLFZTMSV w ౳֬཰ͰΞΫηε͕͋ΔͳΒBMMLFZTSBOEPN w 55- UJNFUPMJWF Λৗʹઃఆ͢ΔͳΒWPMBUJMFUUM w σʔλϕʔε༻్ w ফ͑ͨΒࠔΔͷͰOPFWJDUJPO Eviction Policy ݁ہͲΕΛબ΂͹ྑ͍ͷ͔

©2019 Wantedly, Inc. w ެࣜυΩϡϝϯτ w SFEJTDPOG w (JU)VC w ιʔείʔυ υΩϡϝϯτͷ͋Γ͔

©2019 Wantedly, Inc. ίϚϯυͱσʔλܕ Commands and Data Types

©2019 Wantedly, Inc. ίϚϯυҰཡ https://redis.io/commands

©2019 Wantedly, Inc. ίϚϯυྫ w ίϚϯυҾ਺ w ܭࢉྔ w ղઆ w ஫ҙ w ࣮ߦྫ

©2019 Wantedly, Inc. w ࠷΋جຊతͳσʔλܕ w όΠφϦηʔϑɺͭ·Γ೚ҙͷσʔλΛΛอଘͰ͖Δ w ྫ3VCZͷσʔλΛγϦΞϥΠζͨ݁͠Ռ w .#·Ͱ w ਺஋΋4USJOHܥͷίϚϯυͰѻ͏ String

©2019 Wantedly, Inc. w 4&5LFZWBMVF9c99> w &9TFDPOETŠඵͰFYQJSFઃఆ w 19NJMMJTFDPOETŠϛϦඵͰFYQJSFઃఆ w /9ŠΩʔ͕ଘࡏ͠ͳ͍৔߹ͷΈॻ͖ࠐΈ w 99ŠΩʔ͕ଘࡏ͢Δ৔߹ͷΈॻ͖ࠐΈ w (&5LFZ w .4&5LFZWBMVF w ෳ਺ͷ஋Λॻ͖ࠐΈ w .(&5LFZ w ෳ਺ͷ஋ΛಡΈࠐΈ String

©2019 Wantedly, Inc. w ਺ࣈ w */$3%&$3 w */$3#:%&$3#: w ظݶɾ࡟আ ͜ΕΒ͸4USJOHTʹݶΒͳ͍ w &91*3&LFZTFDPOET w &91*3&"5LFZUJNFTUBNQ w 55-LFZ w %&-LFZ String

©2019 Wantedly, Inc. w 4USJOHͷϦετ w ࠷େ௕͸? w ࣮૷͸૒ํ޲MJOLFEMJTU List // adlist.h typedef struct listNode { struct listNode *prev; struct listNode *next; void *value; } listNode; typedef struct listIter { listNode *next; int direction; } listIter; typedef struct list { listNode *head; listNode *tail; void *(*dup)(void *ptr); void (*free)(void *ptr); int (*match)(void *ptr, void *key); unsigned long len; } list;

©2019 Wantedly, Inc. w -164)LFZWBMVF w ઌ಄ʹ௥Ճɻ0 w -101LFZ w ઌ಄Λ࡟আɻ0 w --&/LFZ w Ϧετͷ௕͞ɻ0 List w -*/4&35LFZ#&'03&c"'5&3QJWPUWBMVF w ్தʹૠೖɻ0 / w -53*.LFZTUBSUTUPQ w TUBSU͔ΒTUPQ൪໨ͷཁૉ͚ͩʹ͢Δɻ0 / w -4&5LFZJOEFYWBMVF w JOEFY൪໨ͷ஋Λมߋɻ0 / w -*/%&9LFZJOEFY w JOEFY൪໨ͷཁૉΛऔಘɻ0 / w -3ͷόϦΤʔγϣϯ w ϒϩοΩϯάͷόϦΤʔγϣϯ #Ͱ͸͡·Δ

©2019 Wantedly, Inc. w pFMEWBMVFϖΞͷू߹ w lΦϒδΣΫτzͬΆ͍΋ͷΛද͢ͷʹศར Hash > HMSET user:1000 username antirez birthyear 1977 verified 1 OK > HGET user:1000 username "antirez" > HGET user:1000 birthyear "1977" > HGETALL user:1000 1) "username" 2) "antirez" 3) "birthyear" 4) "1977" 5) "verified" 6) "1"

©2019 Wantedly, Inc. w )4&5LFZpFMEWBMVF w pFMEʹ஋Λॻ͖ࠐΈɻ0 w )(&5LFZpFME w pMFEͷ஋ΛಡΈࠐΈɻ0 w )(&5"--LFZ w શͯͷpFMEWBMVFϖΞΛಡΈࠐΈɻ0 / w ).4&5LFZpFMEWBMVF w ෳ਺pFMEʹ஋Λॻ͖ࠐΈɻ0 ࢦఆpFME਺ w ).(&5LFZpFME w ෳ਺pFMEͷ஋ΛಡΈࠐΈɻ0 ࢦఆpFME਺ Hash w )&9*454LFZpFME w pFME͕ଘࡏ͢Δ͔Ͳ͏͔ɻ0 w )%&-LFZpFME w pFMEΛ࡟আɻ0 pFME਺ w )-&/LFZ w pFME਺ɻ0 w )*/$3#:LFZpFMEJODSFNFOU w ࢦఆpFMEͷ஋Λ૿ݮɻ0

©2019 Wantedly, Inc. w pFME਺͕IBTI@NBY@[JQMJTU@FOUSJFT EFGBVMU ҎԼ͔ͭ஋ͷ௕͞ͷ࠷େ ͕IBTI@NBY@[JQMJTU@WBMVF EFGBVMU ҎԼͷ৔߹͸ɺϝϞϦޮ཰ͷΑ ͍z[JQMJTUzͰΤϯίʔσΟϯά w ͦ͏Ͱͳ͍৔߹͸lEJDUz IBTIUBCMF w ϝϞϦͱ$16ͷτϨʔυΦϑ Hash // t_hash.c /* Check the length of a number of objects to see if we need to convert a * ziplist to a real hash. Note that we only check string encoded objects * as their string length can be queried in constant time. */ void hashTypeTryConversion(robj *o, robj **argv, int start, int end) { int i; if (o->encoding != OBJ_ENCODING_ZIPLIST) return; for (i = start; i <= end; i++) { if (sdsEncodedObject(argv[i]) && sdslen(argv[i]->ptr) > server.hash_max_ziplist_value) { hashTypeConvert(o, OBJ_ENCODING_HT); break; } } }

©2019 Wantedly, Inc. ziplist.c /* The ziplist is a specially encoded dually linked list that is designed * to be very memory efficient. It stores both strings and integer values, * where integers are encoded as actual integers instead of a series of * characters. It allows push and pop operations on either side of the list * in O(1) time. However, because every operation requires a reallocation of * the memory used by the ziplist, the actual complexity is related to the * amount of memory used by the ziplist.

©2019 Wantedly, Inc. w ϢχʔΫͰιʔτ͞Ε͍ͯͳ͍ 4USJOH ͷू߹ w ࠷େཁૉ਺͸? Set redis> SADD myset "Hello" (integer) 1 redis> SADD myset "World" (integer) 1 redis> SADD myset "World" (integer) 0 redis> SMEMBERS myset 1) "World" 2) "Hello" redis>

©2019 Wantedly, Inc. w 4"%%LFZNFNCFS w ཁૉͷ௥Ճɻ0 w 43&.LFZNFNCFS w ཁૉͷ࡟আɻ0 w 4.&.#&34LFZ w શཁૉͷऔಘɻ0 / w 4*4.&.#&3LFZNFNCFS w ཁૉ͕ू߹ʹଘࡏ͢Δ͔Ͳ͏͔ɻ0 w 4101LFZ w ϥϯμ ϜʹཁૉΛऔΓग़͠ɻ0 Set w 46/*0/LFZ w ࿨ू߹ɻ0 ཁૉ਺ͷ߹ܭ w 4*/5&3LFZ w ੵू߹ɻ0 Ұ൪খ͍͞ཁૉ਺TFU਺ w 4%*''LFZ w ࠩू߹ɻ0 ཁૉ਺ͷ߹ܭ w 4$"3%LFZ w ू߹ͷཁૉ਺ɻ0

©2019 Wantedly, Inc. w ੔਺ͷΈͷ৔߹ɺ಺෦తʹ͸ಛผ ͳΤϯίʔσΟϯάͰ֨ೲ͞ΕΔ w TFUNBYJOUTFUFOUSJFT EFGBVMU ҎԼͷཁૉ਺ͷ৔ ߹ɻ Set // t_set.c /* Add the specified value into a set. * * If the value was already member of the set, nothing is done and 0 is * returned, otherwise the new element is added and 1 is returned. */ int setTypeAdd(robj *subject, sds value) { long long llval; if (subject->encoding == OBJ_ENCODING_HT) { dict *ht = subject->ptr; dictEntry *de = dictAddRaw(ht,value,NULL); if (de) { dictSetKey(ht,de,sdsdup(value)); dictSetVal(ht,de,NULL); return 1; } } else if (subject->encoding == OBJ_ENCODING_INTSET) { if (isSdsRepresentableAsLongLong(value,&llval) == C_OK) { uint8_t success = 0; subject->ptr = intsetAdd(subject->ptr,llval,&success); if (success) { /* Convert to regular set when the intset contains * too many entries. */ if (intsetLen(subject->ptr) > server.set_max_intset_entries) setTypeConvert(subject,OBJ_ENCODING_HT); return 1; } } else { /* Failed to get integer from object, convert to regular set. */ setTypeConvert(subject,OBJ_ENCODING_HT); /* The set *was* an intset and this value is not integer * encodable, so dictAdd should always work. */ serverAssert(dictAdd(subject->ptr,sdsdup(value),NULL) == DICT_OK); return 1; } } else { serverPanic("Unknown set encoding"); } return 0; }

©2019 Wantedly, Inc. redisObjectߏ଄ମ // server.h /* Objects encoding. Some kind of objects like Strings and Hashes can be * internally represented in multiple ways. The 'encoding' field of the object * is set to one of this fields for this object. */ #define OBJ_ENCODING_RAW 0 /* Raw representation */ #define OBJ_ENCODING_INT 1 /* Encoded as integer */ #define OBJ_ENCODING_HT 2 /* Encoded as hash table */ #define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */ #define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */ #define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */ #define OBJ_ENCODING_INTSET 6 /* Encoded as intset */ #define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */ #define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */ #define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */ #define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */ typedef struct redisObject { unsigned type:4; unsigned encoding:4; unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or * LFU data (least significant 8 bits frequency * and most significant 16 bits access time). */ int refcount; void *ptr; } robj;

©2019 Wantedly, Inc. w είΞॱʹιʔτ͞Ε͍ͯΔ4FU w είΞ͸CJUුಈখ਺఺਺ w είΞ͕ಉ͡৔߹͸ཁૉͷࣙॻࣜॱং w [TFUNBY[JQMJTUFOUSJFT EFGBVMU ͔ͭ[TFUNBY[JQMJTUWBMVF EFGBVMU Λຬͨ͢৔߹͸[JQMJTU w *46$0/Ͱ3%#.4ͷUBCMFΛࡶʹஔ͖ ׵͑Δͱ͖ʹΑ͘࢖͏ SortedSet redis> ZADD myzset 1 "one" (integer) 1 redis> ZADD myzset 1 "uno" (integer) 1 redis> ZADD myzset 2 "two" 3 "three" (integer) 2 redis> ZRANGE myzset 0 -1 WITHSCORES 1) "one" 2) "1" 3) "uno" 4) "1" 5) "two" 6) "2" 7) "three" 8) "3" redis>

©2019 Wantedly, Inc. 3FEJT0CKFDU͔ΒείΞ΁ͷϚοϓͷ IBTIUBCMFͱɺείΞ͔Β3FEJT0CKFDU ΁ͷϚοϓͷTLJQMJTU͔ΒͳΔ SortedSet // server.h /* ZSETs use a specialized version of Skiplists */ typedef struct zskiplistNode { sds ele; double score; struct zskiplistNode *backward; struct zskiplistLevel { struct zskiplistNode *forward; unsigned long span; } level[]; } zskiplistNode; typedef struct zskiplist { struct zskiplistNode *header, *tail; unsigned long length; int level; } zskiplist; typedef struct zset { dict *dict; zskiplist *zsl; } zset; ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU

©2019 Wantedly, Inc. Skip List ૠೖͱ୳ࡧͱ࡟আͷ࣌ؒܭࢉྔ͸ฏۉ0 MPH / ɺ࠷ѱ0 / Ξχϝʔγϣϯ(*'ͳͷͰݩαΠτͰ֬ೝ͍ͯͩ͘͠͞ ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU

©2019 Wantedly, Inc. w ;"%%LFZ9c99><$)><*/$3>TDPSFNFNCFS w ཁૉͷ௥Ճɻ0 MPH / w 99ŠطଘͷཁૉͷείΞߋ৽ͷΈɻ w /9Š৽͍͠ཁૉͷ௥ՃͷΈɻ w $)ŠείΞߋ৽͞Εͨཁૉ਺Λ໭Γ஋ʹؚΉ ௨ৗ͸৽͘͠௥Ճ͞Εͨ਺ ɻ w */$3ŠείΞͷมԽࠩ෼Λࢦఆɻ w ;4$03&LFZNFNCFS w ࢦఆͨ͠ཁૉͷείΞɻ0 w ;$"3%LFZ w ཁૉ਺ɻ0 SortedSet

©2019 Wantedly, Inc. w ;$06/5LFZNJONBY w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉ਺ɻ0 MPH / w ;3"/,LFZNFNCFS w ࢦఆͨ͠ϝϯόʔ͕Կ൪໨͔ɻ0 MPH / w ;3"/(&LFZTUBSUTUPQ<8*5)4$03&4> w TUBSU͔ΒTUPQ൪໨ͷཁૉΛऔಘɻ0 MPH શཁૉ਺ औಘཁૉ਺ w ;3"/(:4$03&LFZNJONBY<8*5)4$03&4><-*.*5PGGTFUDPVOU> w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉΛऔಘɻ0 MPH શཁૉ਺ औಘཁૉ਺ SortedSet

©2019 Wantedly, Inc. w lBQSPCBCJMJTUJDEBUBTUSVDUVSFXIJDI JTVTFEJOPSEFSUPFTUJNBUFUIF DBSEJOBMJUZPGBTFUz w ϢχʔΫͳཁૉ਺Λ਺͑Δͷʹ࢖͏ w ඪ४ޡࠩͰਪଌ w ϝϞϦ࢖༻ྔ͸ݻఆͰ࠷ѱέʔεͰL CZUFT HyperLogLog HyperLogLogs in Redis https://thoughtbot.com/blog/hyperloglogs-in-redis

©2019 Wantedly, Inc. require 'redis' require 'sinatra' $redis = Redis.new class VisitorCounter def initialize(app) @app = app end def call(env) $redis.pfadd 'visitors', Rack::Request.new(env).ip @app.call(env) end end use VisitorCounter get '/' do visitors = $redis.pfcount 'visitors' "This website has been visited by #{visitors} unique visitors." end HyperLogLog ग़య: https://thoughtbot.com/blog/hyperloglogs-in-redis

©2019 Wantedly, Inc. -JTU 4FU 4PSUFE4FU )BTIͷσʔλܕʹର͠ɺͦΕͧΕͷί Ϛϯυ͕ظ଴͢Δ࣌ؒܭࢉྔͰ͋Δ͜ͱΛ֬ೝ͢Δɻ ͕࣌ؒ͋ͬͨΒ͜͜Ͱԋश require 'redis' redis = Redis.new(host: "localhost", port: 6379) redis.set("mykey", "hello world") # => "OK" redis.get("mykey") # => "hello world"

©2019 Wantedly, Inc. Pipelining require 'redis' def without_pipelining r = Redis.new 10000.times { r.ping } end def with_pipelining r = Redis.new r.pipelined { 10000.times { r.ping } } end def bench(descr) start = Time.now yield puts "#{descr} #{Time.now-start} seconds" end bench("without pipelining") { without_pipelining } bench("with pipelining") { with_pipelining } # without pipelining 1.185238 seconds # with pipelining 0.250783 seconds Using pipelining to speedup Redis queries https://redis.io/topics/pipelining 355ͱγεςϜίʔϧͷΦʔόʔϔου࡟ݮ

©2019 Wantedly, Inc. Transaction > MULTI OK > INCR foo QUEUED > INCR bar QUEUED > EXEC 1) (integer) 1 2) (integer) 1 https://redis.io/topics/transactions ෳ਺ίϚϯυΛΞτϛοΫʹ࣮ߦ w .6-5*ίϚϯυͰ໋ྩΛΩϡʔΠϯά w &9&$ίϚϯυͰBUPNJDʹ࣮ߦ w SPMMCBDL͸ͳ͍

©2019 Wantedly, Inc. Lua Scripting w ༨༟͕͋Ε͹ॻ͘

©2019 Wantedly, Inc. PubSub w ༨༟͕͋Ε͹ॻ͘

©2019 Wantedly, Inc. Redis Module w ໋ྩΛ֦ுͰ͖Δ w IUUQTSFEJTJPUPQJDTNPEVMFTJOUSP w IUUQTSFEJTMBCTDPNDPNNVOJUZSFEJTNPEVMFTIVC w ͕࣌ؒ͋Ε͹͜͜Ͱԋश w ద౰ͳ໋ྩΛࣗ࡞͢Δ w IUUQTHJUIVCDPN3FEJT-BCT3FEJT.PEVMFT4%,

©2019 Wantedly, Inc. ࣮ࡍͷ࢖͍ํ Usage on Real Projects

©2019 Wantedly, Inc. ࣮ࡍͷ࢖͍ํ ओͳ༻్ 3BJMTͷඇಉظδϣϒΩϡʔ 3BJMTDBDIFͷόοΫΤϯυ ੜ3FEJTΛDBDIF·ͨ͸EBUBCBTFͱͯ͠

©2019 Wantedly, Inc. Sidekiq/ActiveJob class HardWorker include Sidekiq::Worker def perform(name, count) # do something end end HardWorker.perform_async('bob', 5) HardWorker.perform_in(5.minutes, 'bob', 5) HardWorker.perform_at(5.minutes.from_now, 'bob', 5) ग़య: https://github.com/mperham/sidekiq/wiki/Getting-Started 3BJMTͷඇಉظॲཧͷόοΫΤϯυͱͯ͠

©2019 Wantedly, Inc. Sidekiq/ActiveJob ͲͪΒΛ͔ͭ͏͔ʁ w "DUJWF+PCͰ࢖͏৔߹ w "3ͷΠϯελϯεΛ౉ͤΔ EFTFSJBMJ[F࣌ʹA.PEFMpOE JE AΛ΍͍ͬͯΔ w (MPCBM*%4FSJBMJ[BUJPOIUUQTHJUIVCDPNSBJMTHMPCBMJE w "DUJPO.BJMFSͱ૬ੑ͕Α͍ EFMJWFS@MBUFS w ։ൃ؀ڥͰ3FEJTͳ͠Ͱ࢖͑Δ w ผSVCZUISFBEͰͷ࣮ߦͱͳΔ w +PC࡞੒࣌ͷA*OMPDBMFAΛอଘͯ͠ɺͦͷϩέʔϧͰ+PCΛ࣮ߦͯ͘͠ΕΔ w ੜ4JEFLJRΛ࢖͏৔߹ w 4JEFLJRͷػೳΛϑϧʹ࢖͑Δ w EF TFSJBMJ[FͷΦʔόʔϔου͕ݮΔ

©2019 Wantedly, Inc. Sidekiq/ActiveJob ࢖͍ํͷίπ w ग़དྷΔݶΓॲཧΛႈ౳ʹ͍͕ͨ͠ɺݱ࣮͸ͦ͏͸͍͔ͳ͍ w σϑΥϧτͰ͸ϦτϥΠ͠ͳ͍Α͏ʹઃఆ͢Δ΄͏͕҆શ w ॲཧʹ͕͔͔࣌ؒΓλΠϜΞ΢τ͢Δ͜ͱ͕͋Δ w ϝʔϧ΍ϓογϡ௨஌Λ౓ૹ৴͢ΔͱϢʔβମݧ͕ѱ͍ w ૢ࡞͕ႈ౳ͰϦτϥΠ͍ͨ͠΍ͭ͸໌ࣔతʹࢦఆ͢Δ w 4JEFLJREFGBVMU@XPSLFS@PQUJPOT<SFUSZ>GBMTF w "DUJWF3FDPSEͷίʔϧόοΫͰΤϯΩϡʔ͢Δ৔߹͸ɺBGUFS@DPNNJUͰ΍Δ w BGUFS@DSFBUF΍BGUFS@VQEBUFͰΤϯΩϡʔ͢Δͱɺ3%#ʹDPNNJU͞ΕΔ·͑ʹɺ KPC͕࣮ߦ͞Εͯ͠·͏͜ͱ͕͋Δ w TJEFLJRϓϩηε΁ͷ4*(5&3.ͱ4*(,*--ͷؒ LTͷEFGBVMU͸T ʹUJNFPVUͤ͞Δ https://github.com/wantedly/dev-docs/blob/master/sidekiq.md

©2019 Wantedly, Inc. 3BJMTDBDIFͷόοΫΤϯυͱͯ͠ # config/environments/production.rb config.cache_store = :redis_store, ENV["REDIS_RAILS_CACHE_URL"], { compress: true, expires_in: 90.minutes } w .BSTIBMEVNQ .BSTIBMMPBE Ͱ EF TFSJBMJ[F w 1SJNJUJWFͬΆ͘ͳ͍σʔλܕΛ͍Εͳ͍ w ൵͍͠ࣄނͷྫXBOUFEMZQPTUNPSUFNT wantedly/post-mortems#40

©2019 Wantedly, Inc. ੜ3FEJT # config/initializers/redis.rb REDIS_CONFIG = { host: ENV['REDIS_HOST'].presence || 'localhost', port: ENV['REDIS_PORT'].presence || '6379', connect_timeout: 0.2, read_timeout: 1.0, write_timeout: 3.0, } redis_conn = Redis.new(REDIS_CONFIG) w ίϚϯυ͕ͦͷ··ϝιου໊ʹͳ͍ͬͯΔ w ઀ଓͷॳظԽ࣌ʹλΠϜΞ΢τΛઃఆ͓ͯ͘͠

©2019 Wantedly, Inc. ຊ൪༻ͷΠϯελϯεΛ࡞੒͢Δ # terraform/elasticache_replication_group.tf resource "aws_elasticache_replication_group" "users-pubsub-prod" { replication_group_id = "users-pubsub-prod" replication_group_description = " " engine_version = "5.0.0" maintenance_window = "tue:18:00-tue:19:00" node_type = "cache.r5.large" number_cache_clusters = 2 automatic_failover_enabled = true parameter_group_name = "default.redis5.0" port = 6379 subnet_group_name = "${aws_elasticache_subnet_group.wantedly-production-cache.name}" security_group_ids = ["${aws_security_group.redis-elasticache-production.id}"] notification_topic_arn = "arn:aws:sns:ap-northeast-1:096233016669:dev" snapshot_retention_limit = 1 snapshot_window = "17:00-18:00" } w "84&MBTUJDBDIFΛ࢖͍ͬͯΔ w 5FSSBGPSNͰ࡞੒ XBOUFEMZXBOUFEMZUFSSBGPSN΁QVMMSFRVFTU w QBSBNFUFSHSPVQ͕ॏཁͳͷͰɺ*OGSBTRVBEͷαϙʔτ౰൪ʹ૬ஊ͍ͯͩ͘͠͞

©2019 Wantedly, Inc. KEYS *ίϚϯυࣄ݅ wantedly/post-mortems#31

©2019 Wantedly, Inc. KEYS *ίϚϯυࣄ݅ w ,&:4QBUUFSO w QBUUFSOʹϚον͢ΔશͯͷΩʔΛฦ͢ɻ0 શLFZ਺ w l8BSOJOHDPOTJEFS,&:4BTBDPNNBOEUIBUTIPVMEPOMZCFVTFEJOQSPEVDUJPO FOWJSPONFOUTXJUIFYUSFNFDBSF*UNBZSVJOQFSGPSNBODFXIFOJUJT FYFDVUFEBHBJOTUMBSHFEBUBCBTFT5IJTDPNNBOEJTJOUFOEFEGPSEFCVHHJOH BOETQFDJBMPQFSBUJPOT TVDIBTDIBOHJOHZPVSLFZTQBDFMBZPVU%POU VTF,&:4JOZPVSSFHVMBSBQQMJDBUJPODPEF*GZPVSFMPPLJOHGPSBXBZUPpOE LFZTJOBTVCTFUPGZPVSLFZTQBDF DPOTJEFSVTJOH4$"/PSTFUTz

©2019 Wantedly, Inc. ڊେHashࣄ݅ wantedly/post-mortems#25

©2019 Wantedly, Inc. ڊେHashࣄ݅ wantedly/infrastructure#4102

©2019 Wantedly, Inc. ίϚϯυςʔϒϧ struct redisCommand redisCommandTable[] = { {"module",moduleCommand,-2,"as",0,NULL,0,0,0,0,0}, {"get",getCommand,2,"rF",0,NULL,1,1,1,0,0}, {"set",setCommand,-3,"wm",0,NULL,1,1,1,0,0}, {"setnx",setnxCommand,3,"wmF",0,NULL,1,1,1,0,0}, {"setex",setexCommand,4,"wm",0,NULL,1,1,1,0,0}, {"psetex",psetexCommand,4,"wm",0,NULL,1,1,1,0,0}, {"append",appendCommand,3,"wm",0,NULL,1,1,1,0,0}, {"strlen",strlenCommand,2,"rF",0,NULL,1,1,1,0,0}, {"del",delCommand,-2,"w",0,NULL,1,-1,1,0,0}, {"unlink",unlinkCommand,-2,"wF",0,NULL,1,-1,1,0,0}, {"exists",existsCommand,-2,"rF",0,NULL,1,-1,1,0,0}, {"setbit",setbitCommand,4,"wm",0,NULL,1,1,1,0,0}, {“getbit",getbitCommand,3,"rF",0,NULL,1,1,1,0,0}, {"bitfield",bitfieldCommand,-2,"wm",0,NULL,1,1,1,0,0}, {"setrange",setrangeCommand,4,"wm",0,NULL,1,1,1,0,0}, {"getrange",getrangeCommand,4,"r",0,NULL,1,1,1,0,0}, {"substr",getrangeCommand,4,"r",0,NULL,1,1,1,0,0}, {"incr",incrCommand,2,"wmF",0,NULL,1,1,1,0,0}, {"decr",decrCommand,2,"wmF",0,NULL,1,1,1,0,0}, server.c

©2019 Wantedly, Inc. ·ͱΊ Summary

©2019 Wantedly, Inc. 3FEJTͷ֓ཁ ࣾ಺Ͱͷར༻ํ๏ ਖ਼͍͠༻๏༻ྔ ͜ͷݚमͰֶΜͩ͜ͱ

©2019 Wantedly, Inc. w ιʔείʔυΛோΊΔ͜ͱͰগ͚ͩ͠ਂ͘3FEJTʹ͍ͭͯཧղͰ͖ͨ w ීஈ࢖͍ͬͯΔιϑτ΢ΣΞʹ΋୭͔͕ॻ͍ͨιʔείʔυ͕ଘࡏ͢Δ w ϑϨʔϜϫʔΫɺϥΠϒϥϦɺݴޠ࣮૷ w ϒϥοΫϘοΫεԽͤͣʹڵຯΛ࣋ͬͯಡΜͰΈͯ΄͍͠ w ͦΕ͕ɺιϑτ΢ΣΞΤϯδχΞͱͯ͠ͷ෢ثΛ૿΍͢͜ͱʹͳΔ w ·ͨɺ্ͷϨΠϠʔʹ΋ڵຯΛͻΖ͛ͯ΄͍͠ w Ϣʔβମݧɺ6*ઃܭɺϏδωεϞσϧɺϚʔέςΟϯάɺϓϩδΣΫτϚωʔδϝϯτɺ ϓϩμΫτϚωʔδϝϯτɺͳͲͳͲ ͍ͭͰʹ఻͔͑ͨͬͨ͜ͱ

©2019 Wantedly, Inc. w l8PSLJOH8JUI6OJY1SPDFTTFTz +FTTF4UPSJNFS w 6/*9ͷϓϩηεɺϑΝΠϧσΟεΫϦϓλɺγες Ϝίʔϧɺγάφϧ͋ͨΓΛܰ͘ཧղ͢ΔͷʹΑ͍ɻ w (PͳΒΘ͔ΔγεςϜϓϩάϥϛϯά ौ઒Α͖͠ w ೔ຊޠͰಡΈ͚ͨΕ͹͜Ε΋ྑ͍ɻ w -JOVYϓϩάϥϛϯάΠϯλϑΣʔε .JDIBFM,FSSJTL ઍॅ࣏࿠ w ϖʔδ௒͑ͷࣙॻɻΦϑΟεʹ͋Δɻ w ͳΜͰ΋͍͍ͷͰΞϧΰϦζϜͱσʔλߏ଄ͷຊ ࢀߟจݙ ͞Βʹਂ͘ཧղ͍ͨ͠ਓ΁

©2019 Wantedly, Inc. w "OOJF4QSBUUIUUQTVOTQMBTIDPNQIPUPTUI*@$;"#.: Photo Credit