$30 off During Our Annual Pro Sale. View Details »

アルゴリズムとデータ構造から理解するRedis / Learn Redis from Internal Algorithms and Data Structures

アルゴリズムとデータ構造から理解するRedis / Learn Redis from Internal Algorithms and Data Structures

2019年新卒研修で使った資料です。

内部実装の雰囲気を感じとりながら、Redisについて理解を深める研修を行いました。

以下の内容について学びました。

1. Redisの概要
2. 社内での利用方法
3. 正しい用法用量

Redis についての前提知識は必要としていません。C言語の基礎的な知識は前提とします。

Yoshinori Kawasaki

April 22, 2019
Tweet

More Decks by Yoshinori Kawasaki

Other Decks in Programming

Transcript

  1. ©2019 Wantedly, Inc. ΞϧΰϦζϜͱσʔλߏ଄͔Β ཧղ͢Δ3FEJT Learn Redis from Internal Algorithms

    and Data Structures New Grads Training 2019 April 21, 2019 - Yoshinori Kawasaki (@kawasy/@luvtechno)
  2. ©2019 Wantedly, Inc. 3FEJTʹ͍ͭͯͷલఏ஌ࣝ͸ඞཁ͋Γ·ͤΜɻ IUUQTHJUIVCDPNBOUJSF[SFEJTΛDMPOF͓͍͍ͯͯͩ͘͠͞ɻ CSBODIͰNBLFͱNBLFUFTU͕௨Δ؀ڥΛ༻ҙ͍ͯͩ͘͠͞ɻ .BD04Ͱ΋-JOVYͰ΋େৎ෉Ͱ͢ɻ $ݴޠͷجૅతͳ஌ࣝΛલఏͱ͠·͢ɻϙΠϯλ΍ߏ଄ମΛ஌͍ͬͯΔఔ౓Ͱେৎ෉Ͱ͢ɻ ࣄલ४උ

  3. ©2019 Wantedly, Inc. ͸͡Ίʹ Introduction

  4. ©2019 Wantedly, Inc. ༷ʑͳϛυϧ΢ΣΞ͕ࣾ಺Ͱ࢖ΘΕ͍ͯΔ w 1PTUHSF42- w 3FEJT w &MBTUJDTFBSDI

    w %HSBQI w .FN42- w $MPVE1VC4VC 3FEJT͸ࣾ಺Ͱ࠷΋࢖ΘΕ͍ͯΔ΋ͷͷͭ Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔
  5. ©2019 Wantedly, Inc. ࣾ಺ͷϝδϟʔͳϚΠΫϩαʔϏεͰ3FEJTΛར༻ w XBOUFEMZ w TZODNFTTFOHFSSBJMT w ZBTIJNBSBJMT

    w WJTJUSFDPNNFOEBUJPO w QFPQMFHP w ZBTIJNBVQEBUFT w OPUJpDBUJPOT w NJSBDMFTFBSDIHP Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔ Datadog APM Service Map https://app.datadoghq.com/service/map?env=production
  6. ©2019 Wantedly, Inc. 3FEJTΛ࢖͏ཧ༝ w ߴ଎ԽͷͨΊ w Ωϟογϡ w ඇಉظॲཧ

    w ϨεϙϯελΠϜ͸Ϣʔβମݧʹେ͖ͳӨڹ w ཭୤͕ݮΔ w ίϯόʔδϣϯ͕૿͑Δ w ʮγΰτͰίίϩΦυϧͻͱΛ;΍͢ʯ͜ͱ ͕Ͱ͖Δ Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔ ৽ଔݚमͰߦ͏*46$0/ͷັྗ /BP.JOBNJ   https://speakerdeck.com/south37/xin-zu-yan-xiu-dexing-u-isucon-falsemei-li?slide=15
  7. ©2019 Wantedly, Inc.  3FEJTͷ֓ཁ  ࣾ಺Ͱͷར༻ํ๏  ਖ਼͍͠༻๏༻ྔ What

    ͜ͷݚमͰֶ΂Δ͜ͱ
  8. ©2019 Wantedly, Inc. ਖ਼͍͠༻๏༻ྔΛकΒͳ͍ͱ SFEJTͷؔ܎ͨ͠JODJEFOUͷQPTUNPSUFN

  9. ©2019 Wantedly, Inc. 3FEJTͷ֓ཁ Overview

  10. ©2019 Wantedly, Inc. ଎͍σʔλߏ଄αʔό 3FEJTͱ͸ 1 2 3

  11. ©2019 Wantedly, Inc. *ONFNPSZ w ϝϞϦΞΫηε͸44%΍σΟεΫͷϥϯμ ϜΞΫηεΑΓѹ౗తʹ଎͍ 4JNQMF w 3%#.4Ͱ͸ཁٻ͞ΕΔΑ͏ͳෳࡶͳ͜ͱ͸͠ͳ͍

    w ϝΠϯͷॲཧ͸γϯάϧϓϩηεɾγϯάϧεϨου w ࣮ࡍʹ͸%#εφοϓγϣοτͷ࡞੒ͳͲͷͨΊʹGPSL͢Δ w ϊϯϒϩοΩϯά*0ͱΠϕϯτϧʔϓʹΑΔ*0ଟॏԽ ଎͍
  12. ©2019 Wantedly, Inc. ͢΂ͯͷϓϩάϥϚ͕஌͓ͬͯ͘΂͖ϨΠςϯγͷ਺஋ ग़య: Latency Numbers Every Programmer Should

    Know https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html
  13. ©2019 Wantedly, Inc. Πϕϯτϧʔϓ // ae.c /* Include the best

    multiplexing layer supported by this system. * The following should be ordered by performances, descending. */ #ifdef HAVE_EVPORT #include "ae_evport.c" #else #ifdef HAVE_EPOLL #include "ae_epoll.c" #else #ifdef HAVE_KQUEUE #include "ae_kqueue.c" #else #include "ae_select.c" #endif #endif #endif ࡞ऀۘ੡ͷΠϕϯτϥΠϒϥϦ w FWQPSU4PMBSJTҎ߱ w FQPMM-JOVY w LRVFVF#4% w TFMFDUେମ͋Δ
  14. ©2019 Wantedly, Inc. w 4USJOH w -JTU w 4FU w

    4PSUFE4FU w )BTI w #JUBSSBZ CJUNBQ  w )ZQFS-PH-PH w 4USFBN ͨͩͷLFZWBMVFετΞͰ͸ͳ͍ɻͦΕͧΕৄ͘͠͸ޙड़͢Δɻ σʔλߏ଄ https://redis.io/topics/data-types-intro
  15. ©2019 Wantedly, Inc. ෳ਺ϓϩηε͕σʔλΛڞ༗ͯ͠ಡΈॻ͖Ͱ͖Δ w αʔόɾΫϥΠΞϯτϞσϧ w 5$1ιέοτΛ࢖ͬͯ௨৴ w γϯϓϧͳϓϩτίϧ

    ͋ͨΓ·͑ͱࢥ͏͔΋͠Ε·ͤΜ͕ʜ αʔό
  16. ©2019 Wantedly, Inc. 3%# w ඇಉظͷεφοϓγϣοτ w GPSLͯ͠ॻ͖ࠐΉͷͰ਌ϓϩηεͰσΟε Ϋ*0͕ൃੜ͠ͳ͍ w

    ཤྺͷόοΫΞοϓ͕͠΍͍͢ w Ϧελʔτ࣌ͷಡΈࠐΈ͕͸΍͍ w σʔλϩεͷՄೳੑ͸ΑΓେ͖͍ σΟεΫӬଓԽ "0' w ௥هܕ w ίϚϯυྻͷه࿥ w σΟεΫγʔΫ͕ൃੜ͠ͳ͍ w 3%#ܗࣜΑΓαΠζ͕େ͖͍ w ୯ಠͰ࢖Θͳ͍΄͏͕Α͍
  17. ©2019 Wantedly, Inc. • volatile-lru: Evict using approximated LRU among

    the keys with an expire set. • allkeys-lru: Evict any key using approximated LRU. • volatile-lfu: Evict using approximated LFU among the keys with an expire set. • allkeys-lfu: Evict any key using approximated LFU. • volatile-random: Remove a random key among the ones with an expire set. • allkeys-random: Remove a random key, any key. • volatile-ttl: Remove the key with the nearest expire time (minor TTL) • noeviction: Don't evict anything, just return an error on write operations. Eviction Policy ϝϞϦ͔ΒᷓΕͦ͏ͳ࣌ͷڍಈΛઃఆͰ͖Δ
  18. ©2019 Wantedly, Inc. w Ωϟογϡ༻్ w ΞΫηεස౓͕ภΔͳΒBMMLFZTMSV w ౳֬཰ͰΞΫηε͕͋ΔͳΒBMMLFZTSBOEPN w

    55- UJNFUPMJWF Λৗʹઃఆ͢ΔͳΒWPMBUJMFUUM w σʔλϕʔε༻్ w ফ͑ͨΒࠔΔͷͰOPFWJDUJPO Eviction Policy ݁ہͲΕΛબ΂͹ྑ͍ͷ͔
  19. ©2019 Wantedly, Inc. w ެࣜυΩϡϝϯτ w SFEJTDPOG w (JU)VC w

    ιʔείʔυ υΩϡϝϯτͷ͋Γ͔
  20. ©2019 Wantedly, Inc. ίϚϯυͱσʔλܕ Commands and Data Types

  21. ©2019 Wantedly, Inc. ίϚϯυҰཡ https://redis.io/commands

  22. ©2019 Wantedly, Inc. ίϚϯυྫ w ίϚϯυҾ਺ w ܭࢉྔ w ղઆ

    w ஫ҙ w ࣮ߦྫ
  23. ©2019 Wantedly, Inc. w ࠷΋جຊతͳσʔλܕ w όΠφϦηʔϑɺͭ·Γ೚ҙͷσʔλΛΛอଘͰ͖Δ w ྫ3VCZͷσʔλΛγϦΞϥΠζͨ݁͠Ռ w

    .#·Ͱ w ਺஋΋4USJOHܥͷίϚϯυͰѻ͏ String
  24. ©2019 Wantedly, Inc. w 4&5LFZWBMVF<FYQJSBUJPO&9TFDPOETc19NJMMJTFDPOET></9c99> w &9TFDPOETŠඵͰFYQJSFઃఆ w 19NJMMJTFDPOETŠϛϦඵͰFYQJSFઃఆ w

    /9ŠΩʔ͕ଘࡏ͠ͳ͍৔߹ͷΈॻ͖ࠐΈ w 99ŠΩʔ͕ଘࡏ͢Δ৔߹ͷΈॻ͖ࠐΈ w (&5LFZ w .4&5LFZWBMVF<LFZWBMVFʜ> w ෳ਺ͷ஋Λॻ͖ࠐΈ w .(&5LFZ<LFZʜ> w ෳ਺ͷ஋ΛಡΈࠐΈ String
  25. ©2019 Wantedly, Inc. w ਺ࣈ w */$3%&$3 w */$3#:%&$3#: w

    ظݶɾ࡟আ ͜ΕΒ͸4USJOHTʹݶΒͳ͍  w &91*3&LFZTFDPOET w &91*3&"5LFZUJNFTUBNQ w 55-LFZ w %&-LFZ<LFZ> String
  26. ©2019 Wantedly, Inc. w 4USJOHͷϦετ w ࠷େ௕͸? w ࣮૷͸૒ํ޲MJOLFEMJTU List

    // adlist.h typedef struct listNode { struct listNode *prev; struct listNode *next; void *value; } listNode; typedef struct listIter { listNode *next; int direction; } listIter; typedef struct list { listNode *head; listNode *tail; void *(*dup)(void *ptr); void (*free)(void *ptr); int (*match)(void *ptr, void *key); unsigned long len; } list;
  27. ©2019 Wantedly, Inc. w -164)LFZWBMVF<WBMVF> w ઌ಄ʹ௥Ճɻ0   w

    -101LFZ w ઌ಄Λ࡟আɻ0   w --&/LFZ w Ϧετͷ௕͞ɻ0  List w -*/4&35LFZ#&'03&c"'5&3QJWPUWBMVF w ్தʹૠೖɻ0 /  w -53*.LFZTUBSUTUPQ w TUBSU͔ΒTUPQ൪໨ͷཁૉ͚ͩʹ͢Δɻ0 /  w -4&5LFZJOEFYWBMVF w JOEFY൪໨ͷ஋Λมߋɻ0 /  w -*/%&9LFZJOEFY w JOEFY൪໨ͷཁૉΛऔಘɻ0 /  w -3ͷόϦΤʔγϣϯ w ϒϩοΩϯάͷόϦΤʔγϣϯ #Ͱ͸͡·Δ
  28. ©2019 Wantedly, Inc. w pFMEWBMVFϖΞͷू߹ w lΦϒδΣΫτzͬΆ͍΋ͷΛද͢ͷʹศར Hash > HMSET

    user:1000 username antirez birthyear 1977 verified 1 OK > HGET user:1000 username "antirez" > HGET user:1000 birthyear "1977" > HGETALL user:1000 1) "username" 2) "antirez" 3) "birthyear" 4) "1977" 5) "verified" 6) "1"
  29. ©2019 Wantedly, Inc. w )4&5LFZpFMEWBMVF w pFMEʹ஋Λॻ͖ࠐΈɻ0   w

    )(&5LFZpFME w pMFEͷ஋ΛಡΈࠐΈɻ0   w )(&5"--LFZ w શͯͷpFMEWBMVFϖΞΛಡΈࠐΈɻ0 /  w ).4&5LFZpFMEWBMVF<pFMEWBMVF> w ෳ਺pFMEʹ஋Λॻ͖ࠐΈɻ0 ࢦఆpFME਺  w ).(&5LFZpFME<pFME> w ෳ਺pFMEͷ஋ΛಡΈࠐΈɻ0 ࢦఆpFME਺ Hash w )&9*454LFZpFME w pFME͕ଘࡏ͢Δ͔Ͳ͏͔ɻ0   w )%&-LFZpFME<pFMEʜ> w pFMEΛ࡟আɻ0 pFME਺  w )-&/LFZ w pFME਺ɻ0   w )*/$3#:LFZpFMEJODSFNFOU w ࢦఆpFMEͷ஋Λ૿ݮɻ0 
  30. ©2019 Wantedly, Inc. w pFME਺͕IBTI@NBY@[JQMJTU@FOUSJFT EFGBVMU ҎԼ͔ͭ஋ͷ௕͞ͷ࠷େ ͕IBTI@NBY@[JQMJTU@WBMVF EFGBVMU 

    ҎԼͷ৔߹͸ɺϝϞϦޮ཰ͷΑ ͍z[JQMJTUzͰΤϯίʔσΟϯά w ͦ͏Ͱͳ͍৔߹͸lEJDUz IBTIUBCMF  w ϝϞϦͱ$16ͷτϨʔυΦϑ Hash // t_hash.c /* Check the length of a number of objects to see if we need to convert a * ziplist to a real hash. Note that we only check string encoded objects * as their string length can be queried in constant time. */ void hashTypeTryConversion(robj *o, robj **argv, int start, int end) { int i; if (o->encoding != OBJ_ENCODING_ZIPLIST) return; for (i = start; i <= end; i++) { if (sdsEncodedObject(argv[i]) && sdslen(argv[i]->ptr) > server.hash_max_ziplist_value) { hashTypeConvert(o, OBJ_ENCODING_HT); break; } } }
  31. ©2019 Wantedly, Inc. ziplist.c /* The ziplist is a specially

    encoded dually linked list that is designed * to be very memory efficient. It stores both strings and integer values, * where integers are encoded as actual integers instead of a series of * characters. It allows push and pop operations on either side of the list * in O(1) time. However, because every operation requires a reallocation of * the memory used by the ziplist, the actual complexity is related to the * amount of memory used by the ziplist.
  32. ©2019 Wantedly, Inc. w ϢχʔΫͰιʔτ͞Ε͍ͯͳ͍ 4USJOH ͷू߹ w ࠷େཁૉ਺͸? Set

    redis> SADD myset "Hello" (integer) 1 redis> SADD myset "World" (integer) 1 redis> SADD myset "World" (integer) 0 redis> SMEMBERS myset 1) "World" 2) "Hello" redis>
  33. ©2019 Wantedly, Inc. w 4"%%LFZNFNCFS<NFNCFS> w ཁૉͷ௥Ճɻ0   w

    43&.LFZNFNCFS<NFNCFSʜ> w ཁૉͷ࡟আɻ0   w 4.&.#&34LFZ w શཁૉͷऔಘɻ0 /  w 4*4.&.#&3LFZNFNCFS w ཁૉ͕ू߹ʹଘࡏ͢Δ͔Ͳ͏͔ɻ0   w 4101LFZ<DPVOU> w ϥϯμ ϜʹཁૉΛऔΓग़͠ɻ0   Set w 46/*0/LFZ<LFZ> w ࿨ू߹ɻ0 ཁૉ਺ͷ߹ܭ  w 4*/5&3LFZ<LFZ> w ੵू߹ɻ0 Ұ൪খ͍͞ཁૉ਺ TFU਺  w 4%*''LFZ<LFZ> w ࠩू߹ɻ0 ཁૉ਺ͷ߹ܭ  w 4$"3%LFZ w ू߹ͷཁૉ਺ɻ0 
  34. ©2019 Wantedly, Inc. w ੔਺ͷΈͷ৔߹ɺ಺෦తʹ͸ಛผ ͳΤϯίʔσΟϯάͰ֨ೲ͞ΕΔ w TFUNBYJOUTFUFOUSJFT EFGBVMU ҎԼͷཁૉ਺ͷ৔

    ߹ɻ Set // t_set.c /* Add the specified value into a set. * * If the value was already member of the set, nothing is done and 0 is * returned, otherwise the new element is added and 1 is returned. */ int setTypeAdd(robj *subject, sds value) { long long llval; if (subject->encoding == OBJ_ENCODING_HT) { dict *ht = subject->ptr; dictEntry *de = dictAddRaw(ht,value,NULL); if (de) { dictSetKey(ht,de,sdsdup(value)); dictSetVal(ht,de,NULL); return 1; } } else if (subject->encoding == OBJ_ENCODING_INTSET) { if (isSdsRepresentableAsLongLong(value,&llval) == C_OK) { uint8_t success = 0; subject->ptr = intsetAdd(subject->ptr,llval,&success); if (success) { /* Convert to regular set when the intset contains * too many entries. */ if (intsetLen(subject->ptr) > server.set_max_intset_entries) setTypeConvert(subject,OBJ_ENCODING_HT); return 1; } } else { /* Failed to get integer from object, convert to regular set. */ setTypeConvert(subject,OBJ_ENCODING_HT); /* The set *was* an intset and this value is not integer * encodable, so dictAdd should always work. */ serverAssert(dictAdd(subject->ptr,sdsdup(value),NULL) == DICT_OK); return 1; } } else { serverPanic("Unknown set encoding"); } return 0; }
  35. ©2019 Wantedly, Inc. redisObjectߏ଄ମ // server.h /* Objects encoding. Some

    kind of objects like Strings and Hashes can be * internally represented in multiple ways. The 'encoding' field of the object * is set to one of this fields for this object. */ #define OBJ_ENCODING_RAW 0 /* Raw representation */ #define OBJ_ENCODING_INT 1 /* Encoded as integer */ #define OBJ_ENCODING_HT 2 /* Encoded as hash table */ #define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */ #define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */ #define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */ #define OBJ_ENCODING_INTSET 6 /* Encoded as intset */ #define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */ #define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */ #define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */ #define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */ typedef struct redisObject { unsigned type:4; unsigned encoding:4; unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or * LFU data (least significant 8 bits frequency * and most significant 16 bits access time). */ int refcount; void *ptr; } robj;
  36. ©2019 Wantedly, Inc. w είΞॱʹιʔτ͞Ε͍ͯΔ4FU w είΞ͸CJUුಈখ਺఺਺ w είΞ͕ಉ͡৔߹͸ཁૉͷࣙॻࣜॱং w

    [TFUNBY[JQMJTUFOUSJFT EFGBVMU  ͔ͭ[TFUNBY[JQMJTUWBMVF EFGBVMU Λຬͨ͢৔߹͸[JQMJTU w *46$0/Ͱ3%#.4ͷUBCMFΛࡶʹஔ͖ ׵͑Δͱ͖ʹΑ͘࢖͏ SortedSet redis> ZADD myzset 1 "one" (integer) 1 redis> ZADD myzset 1 "uno" (integer) 1 redis> ZADD myzset 2 "two" 3 "three" (integer) 2 redis> ZRANGE myzset 0 -1 WITHSCORES 1) "one" 2) "1" 3) "uno" 4) "1" 5) "two" 6) "2" 7) "three" 8) "3" redis>
  37. ©2019 Wantedly, Inc. 3FEJT0CKFDU͔ΒείΞ΁ͷϚοϓͷ IBTIUBCMFͱɺείΞ͔Β3FEJT0CKFDU ΁ͷϚοϓͷTLJQMJTU͔ΒͳΔ SortedSet // server.h /*

    ZSETs use a specialized version of Skiplists */ typedef struct zskiplistNode { sds ele; double score; struct zskiplistNode *backward; struct zskiplistLevel { struct zskiplistNode *forward; unsigned long span; } level[]; } zskiplistNode; typedef struct zskiplist { struct zskiplistNode *header, *tail; unsigned long length; int level; } zskiplist; typedef struct zset { dict *dict; zskiplist *zsl; } zset; ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU
  38. ©2019 Wantedly, Inc. Skip List ૠೖͱ୳ࡧͱ࡟আͷ࣌ؒܭࢉྔ͸ฏۉ0 MPH / ɺ࠷ѱ0 /

     Ξχϝʔγϣϯ(*'ͳͷͰݩαΠτͰ֬ೝ͍ͯͩ͘͠͞  ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU
  39. ©2019 Wantedly, Inc. w ;"%%LFZ</9c99><$)><*/$3>TDPSFNFNCFS<TDPSFNFNCFS> w ཁૉͷ௥Ճɻ0 MPH / 

    w 99ŠطଘͷཁૉͷείΞߋ৽ͷΈɻ w /9Š৽͍͠ཁૉͷ௥ՃͷΈɻ w $)ŠείΞߋ৽͞Εͨཁૉ਺Λ໭Γ஋ʹؚΉ ௨ৗ͸৽͘͠௥Ճ͞Εͨ਺ ɻ w */$3ŠείΞͷมԽࠩ෼Λࢦఆɻ w ;4$03&LFZNFNCFS w ࢦఆͨ͠ཁૉͷείΞɻ0   w ;$"3%LFZ w ཁૉ਺ɻ0   SortedSet
  40. ©2019 Wantedly, Inc. w ;$06/5LFZNJONBY w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉ਺ɻ0 MPH / 

    w ;3"/,LFZNFNCFS w ࢦఆͨ͠ϝϯόʔ͕Կ൪໨͔ɻ0 MPH /  w ;3"/(&LFZTUBSUTUPQ<8*5)4$03&4> w TUBSU͔ΒTUPQ൪໨ͷཁૉΛऔಘɻ0 MPH શཁૉ਺ औಘཁૉ਺  w ;3"/(&#:4$03&LFZNJONBY<8*5)4$03&4><-*.*5PGGTFUDPVOU> w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉΛऔಘɻ0 MPH શཁૉ਺ औಘཁૉ਺ SortedSet
  41. ©2019 Wantedly, Inc. w lBQSPCBCJMJTUJDEBUBTUSVDUVSFXIJDI JTVTFEJOPSEFSUPFTUJNBUFUIF DBSEJOBMJUZPGBTFUz w ϢχʔΫͳཁૉ਺Λ਺͑Δͷʹ࢖͏ w

    ඪ४ޡࠩͰਪଌ w ϝϞϦ࢖༻ྔ͸ݻఆͰ࠷ѱέʔεͰL CZUFT HyperLogLog HyperLogLogs in Redis https://thoughtbot.com/blog/hyperloglogs-in-redis
  42. ©2019 Wantedly, Inc. require 'redis' require 'sinatra' $redis = Redis.new

    class VisitorCounter def initialize(app) @app = app end def call(env) $redis.pfadd 'visitors', Rack::Request.new(env).ip @app.call(env) end end use VisitorCounter get '/' do visitors = $redis.pfcount 'visitors' "This website has been visited by #{visitors} unique visitors." end HyperLogLog ग़య: https://thoughtbot.com/blog/hyperloglogs-in-redis
  43. ©2019 Wantedly, Inc. -JTU 4FU 4PSUFE4FU )BTIͷσʔλܕʹର͠ɺͦΕͧΕͷί Ϛϯυ͕ظ଴͢Δ࣌ؒܭࢉྔͰ͋Δ͜ͱΛ֬ೝ͢Δɻ ͕࣌ؒ͋ͬͨΒ͜͜Ͱԋश require

    'redis' redis = Redis.new(host: "localhost", port: 6379) redis.set("mykey", "hello world") # => "OK" redis.get("mykey") # => "hello world"
  44. ©2019 Wantedly, Inc. Pipelining require 'redis' def without_pipelining r =

    Redis.new 10000.times { r.ping } end def with_pipelining r = Redis.new r.pipelined { 10000.times { r.ping } } end def bench(descr) start = Time.now yield puts "#{descr} #{Time.now-start} seconds" end bench("without pipelining") { without_pipelining } bench("with pipelining") { with_pipelining } # without pipelining 1.185238 seconds # with pipelining 0.250783 seconds Using pipelining to speedup Redis queries https://redis.io/topics/pipelining 355ͱγεςϜίʔϧͷΦʔόʔϔου࡟ݮ
  45. ©2019 Wantedly, Inc. Transaction > MULTI OK > INCR foo

    QUEUED > INCR bar QUEUED > EXEC 1) (integer) 1 2) (integer) 1 https://redis.io/topics/transactions ෳ਺ίϚϯυΛΞτϛοΫʹ࣮ߦ w .6-5*ίϚϯυͰ໋ྩΛΩϡʔΠϯά w &9&$ίϚϯυͰBUPNJDʹ࣮ߦ w SPMMCBDL͸ͳ͍
  46. ©2019 Wantedly, Inc. Lua Scripting w ༨༟͕͋Ε͹ॻ͘

  47. ©2019 Wantedly, Inc. PubSub w ༨༟͕͋Ε͹ॻ͘

  48. ©2019 Wantedly, Inc. Redis Module w ໋ྩΛ֦ுͰ͖Δ w IUUQTSFEJTJPUPQJDTNPEVMFTJOUSP w

    IUUQTSFEJTMBCTDPNDPNNVOJUZSFEJTNPEVMFTIVC w ͕࣌ؒ͋Ε͹͜͜Ͱԋश w ద౰ͳ໋ྩΛࣗ࡞͢Δ w IUUQTHJUIVCDPN3FEJT-BCT3FEJT.PEVMFT4%,
  49. ©2019 Wantedly, Inc. ࣮ࡍͷ࢖͍ํ Usage on Real Projects

  50. ©2019 Wantedly, Inc. ࣮ࡍͷ࢖͍ํ ओͳ༻్  3BJMTͷඇಉظδϣϒΩϡʔ  3BJMTDBDIFͷόοΫΤϯυ 

    ੜ3FEJTΛDBDIF·ͨ͸EBUBCBTFͱͯ͠
  51. ©2019 Wantedly, Inc. Sidekiq/ActiveJob class HardWorker include Sidekiq::Worker def perform(name,

    count) # do something end end HardWorker.perform_async('bob', 5) HardWorker.perform_in(5.minutes, 'bob', 5) HardWorker.perform_at(5.minutes.from_now, 'bob', 5) ग़య: https://github.com/mperham/sidekiq/wiki/Getting-Started 3BJMTͷඇಉظॲཧͷόοΫΤϯυͱͯ͠
  52. ©2019 Wantedly, Inc. Sidekiq/ActiveJob ͲͪΒΛ͔ͭ͏͔ʁ w "DUJWF+PCͰ࢖͏৔߹ w "3ͷΠϯελϯεΛ౉ͤΔ EFTFSJBMJ[F࣌ʹA.PEFMpOE

    JE AΛ΍͍ͬͯΔ  w (MPCBM*%4FSJBMJ[BUJPOIUUQTHJUIVCDPNSBJMTHMPCBMJE w "DUJPO.BJMFSͱ૬ੑ͕Α͍ EFMJWFS@MBUFS  w ։ൃ؀ڥͰ3FEJTͳ͠Ͱ࢖͑Δ w ผSVCZUISFBEͰͷ࣮ߦͱͳΔ w +PC࡞੒࣌ͷA*OMPDBMFAΛอଘͯ͠ɺͦͷϩέʔϧͰ+PCΛ࣮ߦͯ͘͠ΕΔ w ੜ4JEFLJRΛ࢖͏৔߹ w 4JEFLJRͷػೳΛϑϧʹ࢖͑Δ w EF TFSJBMJ[FͷΦʔόʔϔου͕ݮΔ
  53. ©2019 Wantedly, Inc. Sidekiq/ActiveJob ࢖͍ํͷίπ w ग़དྷΔݶΓॲཧΛႈ౳ʹ͍͕ͨ͠ɺݱ࣮͸ͦ͏͸͍͔ͳ͍ w σϑΥϧτͰ͸ϦτϥΠ͠ͳ͍Α͏ʹઃఆ͢Δ΄͏͕҆શ w

    ॲཧʹ͕͔͔࣌ؒΓλΠϜΞ΢τ͢Δ͜ͱ͕͋Δ w ϝʔϧ΍ϓογϡ௨஌Λ౓ૹ৴͢ΔͱϢʔβମݧ͕ѱ͍ w ૢ࡞͕ႈ౳ͰϦτϥΠ͍ͨ͠΍ͭ͸໌ࣔతʹࢦఆ͢Δ w 4JEFLJREFGBVMU@XPSLFS@PQUJPOT<SFUSZ>GBMTF w "DUJWF3FDPSEͷίʔϧόοΫͰΤϯΩϡʔ͢Δ৔߹͸ɺBGUFS@DPNNJUͰ΍Δ w BGUFS@DSFBUF΍BGUFS@VQEBUFͰΤϯΩϡʔ͢Δͱɺ3%#ʹDPNNJU͞ΕΔ·͑ʹɺ KPC͕࣮ߦ͞Εͯ͠·͏͜ͱ͕͋Δ w TJEFLJRϓϩηε΁ͷ4*(5&3.ͱ4*(,*--ͷؒ LTͷEFGBVMU͸T ʹUJNFPVUͤ͞Δ https://github.com/wantedly/dev-docs/blob/master/sidekiq.md
  54. ©2019 Wantedly, Inc. 3BJMTDBDIFͷόοΫΤϯυͱͯ͠ # config/environments/production.rb config.cache_store = :redis_store, ENV["REDIS_RAILS_CACHE_URL"],

    { compress: true, expires_in: 90.minutes } w .BSTIBMEVNQ .BSTIBMMPBE Ͱ EF TFSJBMJ[F w 1SJNJUJWFͬΆ͘ͳ͍σʔλܕΛ͍Εͳ͍ w ൵͍͠ࣄނͷྫXBOUFEMZQPTUNPSUFNT wantedly/post-mortems#40
  55. ©2019 Wantedly, Inc. ੜ3FEJT # config/initializers/redis.rb REDIS_CONFIG = { host:

    ENV['REDIS_HOST'].presence || 'localhost', port: ENV['REDIS_PORT'].presence || '6379', connect_timeout: 0.2, read_timeout: 1.0, write_timeout: 3.0, } redis_conn = Redis.new(REDIS_CONFIG) w ίϚϯυ͕ͦͷ··ϝιου໊ʹͳ͍ͬͯΔ w ઀ଓͷॳظԽ࣌ʹλΠϜΞ΢τΛઃఆ͓ͯ͘͠
  56. ©2019 Wantedly, Inc. ຊ൪༻ͷΠϯελϯεΛ࡞੒͢Δ # terraform/elasticache_replication_group.tf resource "aws_elasticache_replication_group" "users-pubsub-prod" {

    replication_group_id = "users-pubsub-prod" replication_group_description = " " engine_version = "5.0.0" maintenance_window = "tue:18:00-tue:19:00" node_type = "cache.r5.large" number_cache_clusters = 2 automatic_failover_enabled = true parameter_group_name = "default.redis5.0" port = 6379 subnet_group_name = "${aws_elasticache_subnet_group.wantedly-production-cache.name}" security_group_ids = ["${aws_security_group.redis-elasticache-production.id}"] notification_topic_arn = "arn:aws:sns:ap-northeast-1:096233016669:dev" snapshot_retention_limit = 1 snapshot_window = "17:00-18:00" } w "84&MBTUJDBDIFΛ࢖͍ͬͯΔ w 5FSSBGPSNͰ࡞੒ XBOUFEMZXBOUFEMZUFSSBGPSN΁QVMMSFRVFTU  w QBSBNFUFSHSPVQ͕ॏཁͳͷͰɺ*OGSBTRVBEͷαϙʔτ౰൪ʹ૬ஊ͍ͯͩ͘͠͞
  57. ©2019 Wantedly, Inc. KEYS *ίϚϯυࣄ݅ wantedly/post-mortems#31

  58. ©2019 Wantedly, Inc. KEYS *ίϚϯυࣄ݅ w ,&:4QBUUFSO w QBUUFSOʹϚον͢ΔશͯͷΩʔΛฦ͢ɻ0 શLFZ਺

     w l8BSOJOHDPOTJEFS,&:4BTBDPNNBOEUIBUTIPVMEPOMZCFVTFEJOQSPEVDUJPO FOWJSPONFOUTXJUIFYUSFNFDBSF*UNBZSVJOQFSGPSNBODFXIFOJUJT FYFDVUFEBHBJOTUMBSHFEBUBCBTFT5IJTDPNNBOEJTJOUFOEFEGPSEFCVHHJOH BOETQFDJBMPQFSBUJPOT TVDIBTDIBOHJOHZPVSLFZTQBDFMBZPVU%POU VTF,&:4JOZPVSSFHVMBSBQQMJDBUJPODPEF*GZPVSFMPPLJOHGPSBXBZUPpOE LFZTJOBTVCTFUPGZPVSLFZTQBDF DPOTJEFSVTJOH4$"/PSTFUTz
  59. ©2019 Wantedly, Inc. ڊେHashࣄ݅ wantedly/post-mortems#25

  60. ©2019 Wantedly, Inc. ڊେHashࣄ݅ wantedly/infrastructure#4102

  61. ©2019 Wantedly, Inc. ίϚϯυςʔϒϧ struct redisCommand redisCommandTable[] = { {"module",moduleCommand,-2,"as",0,NULL,0,0,0,0,0},

    {"get",getCommand,2,"rF",0,NULL,1,1,1,0,0}, {"set",setCommand,-3,"wm",0,NULL,1,1,1,0,0}, {"setnx",setnxCommand,3,"wmF",0,NULL,1,1,1,0,0}, {"setex",setexCommand,4,"wm",0,NULL,1,1,1,0,0}, {"psetex",psetexCommand,4,"wm",0,NULL,1,1,1,0,0}, {"append",appendCommand,3,"wm",0,NULL,1,1,1,0,0}, {"strlen",strlenCommand,2,"rF",0,NULL,1,1,1,0,0}, {"del",delCommand,-2,"w",0,NULL,1,-1,1,0,0}, {"unlink",unlinkCommand,-2,"wF",0,NULL,1,-1,1,0,0}, {"exists",existsCommand,-2,"rF",0,NULL,1,-1,1,0,0}, {"setbit",setbitCommand,4,"wm",0,NULL,1,1,1,0,0}, {“getbit",getbitCommand,3,"rF",0,NULL,1,1,1,0,0}, {"bitfield",bitfieldCommand,-2,"wm",0,NULL,1,1,1,0,0}, {"setrange",setrangeCommand,4,"wm",0,NULL,1,1,1,0,0}, {"getrange",getrangeCommand,4,"r",0,NULL,1,1,1,0,0}, {"substr",getrangeCommand,4,"r",0,NULL,1,1,1,0,0}, {"incr",incrCommand,2,"wmF",0,NULL,1,1,1,0,0}, {"decr",decrCommand,2,"wmF",0,NULL,1,1,1,0,0}, server.c
  62. ©2019 Wantedly, Inc. ·ͱΊ Summary

  63. ©2019 Wantedly, Inc.  3FEJTͷ֓ཁ  ࣾ಺Ͱͷར༻ํ๏  ਖ਼͍͠༻๏༻ྔ ͜ͷݚमͰֶΜͩ͜ͱ

  64. ©2019 Wantedly, Inc. w ιʔείʔυΛோΊΔ͜ͱͰগ͚ͩ͠ਂ͘3FEJTʹ͍ͭͯཧղͰ͖ͨ w ීஈ࢖͍ͬͯΔιϑτ΢ΣΞʹ΋୭͔͕ॻ͍ͨιʔείʔυ͕ଘࡏ͢Δ w ϑϨʔϜϫʔΫɺϥΠϒϥϦɺݴޠ࣮૷ w

    ϒϥοΫϘοΫεԽͤͣʹڵຯΛ࣋ͬͯಡΜͰΈͯ΄͍͠ w ͦΕ͕ɺιϑτ΢ΣΞΤϯδχΞͱͯ͠ͷ෢ثΛ૿΍͢͜ͱʹͳΔ w ·ͨɺ্ͷϨΠϠʔʹ΋ڵຯΛͻΖ͛ͯ΄͍͠ w Ϣʔβମݧɺ6*ઃܭɺϏδωεϞσϧɺϚʔέςΟϯάɺϓϩδΣΫτϚωʔδϝϯτɺ ϓϩμΫτϚωʔδϝϯτɺͳͲͳͲ ͍ͭͰʹ఻͔͑ͨͬͨ͜ͱ
  65. ©2019 Wantedly, Inc. w l8PSLJOH8JUI6OJY1SPDFTTFTz +FTTF4UPSJNFS w 6/*9ͷϓϩηεɺϑΝΠϧσΟεΫϦϓλɺγες Ϝίʔϧɺγάφϧ͋ͨΓΛܰ͘ཧղ͢ΔͷʹΑ͍ɻ w

    (PͳΒΘ͔ΔγεςϜϓϩάϥϛϯά ौ઒Α͖͠ w ೔ຊޠͰಡΈ͚ͨΕ͹͜Ε΋ྑ͍ɻ w -JOVYϓϩάϥϛϯάΠϯλϑΣʔε .JDIBFM,FSSJTL ઍॅ࣏࿠ w ϖʔδ௒͑ͷࣙॻɻΦϑΟεʹ͋Δɻ w ͳΜͰ΋͍͍ͷͰΞϧΰϦζϜͱσʔλߏ଄ͷຊ ࢀߟจݙ ͞Βʹਂ͘ཧղ͍ͨ͠ਓ΁
  66. ©2019 Wantedly, Inc. w "OOJF4QSBUUIUUQTVOTQMBTIDPNQIPUPTUI*@$;"#.: Photo Credit