アルゴリズムとデータ構造から理解するRedis / Learn Redis from Internal Algorithms and Data Structures

2019年新卒研修で使った資料です。

内部実装の雰囲気を感じとりながら、Redisについて理解を深める研修を行いました。

以下の内容について学びました。

1. Redisの概要
2. 社内での利用方法
3. 正しい用法用量

Redis についての前提知識は必要としていません。C言語の基礎的な知識は前提とします。

Transcript

1. ©2019 Wantedly, Inc.
   ΞϧΰϦζϜͱσʔλߏ଄͔Β
   ཧղ͢Δ3FEJT
   Learn Redis from Internal Algorithms and Data Structures
   New Grads Training 2019
   April 21, 2019 - Yoshinori Kawasaki (@kawasy/@luvtechno)
   

   View Slide

2. ©2019 Wantedly, Inc.
   3FEJTʹ͍ͭͯͷલఏ஌ࣝ͸ඞཁ͋Γ·ͤΜɻ
   IUUQTHJUIVCDPNBOUJSF[SFEJTΛDMPOF͓͍͍ͯͯͩ͘͠͞ɻ
   CSBODIͰNBLFͱNBLFUFTU͕௨Δ؀ڥΛ༻ҙ͍ͯͩ͘͠͞ɻ
   .BD04Ͱ΋-JOVYͰ΋େৎ෉Ͱ͢ɻ
   $ݴޠͷجૅతͳ஌ࣝΛલఏͱ͠·͢ɻϙΠϯλ΍ߏ଄ମΛ஌͍ͬͯΔఔ౓Ͱେৎ෉Ͱ͢ɻ
   ࣄલ४උ
   

   View Slide

3. ©2019 Wantedly, Inc.
   ͸͡Ίʹ
   Introduction
   

   View Slide

4. ©2019 Wantedly, Inc.
   ༷ʑͳϛυϧ΢ΣΞ͕ࣾ಺Ͱ࢖ΘΕ͍ͯΔ
   w 1PTUHSF42-
   w 3FEJT
   w &MBTUJDTFBSDI
   w %HSBQI
   w .FN42-
   w $MPVE1VC4VC
   3FEJT͸ࣾ಺Ͱ࠷΋࢖ΘΕ͍ͯΔ΋ͷͷͭ
   Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔
   

   View Slide

5. ©2019 Wantedly, Inc.
   ࣾ಺ͷϝδϟʔͳϚΠΫϩαʔϏεͰ3FEJTΛར༻
   w XBOUFEMZ
   w TZODNFTTFOHFSSBJMT
   w ZBTIJNBSBJMT
   w WJTJUSFDPNNFOEBUJPO
   w QFPQMFHP
   w ZBTIJNBVQEBUFT
   w OPUJpDBUJPOT
   w NJSBDMFTFBSDIHP
   Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔
   Datadog APM Service Map
   https://app.datadoghq.com/service/map?env=production
   

   View Slide

6. ©2019 Wantedly, Inc.
   3FEJTΛ࢖͏ཧ༝
   w ߴ଎ԽͷͨΊ
   w Ωϟογϡ
   w ඇಉظॲཧ
   w ϨεϙϯελΠϜ͸Ϣʔβମݧʹେ͖ͳӨڹ
   w ཭୤͕ݮΔ
   w ίϯόʔδϣϯ͕૿͑Δ
   w ʮγΰτͰίίϩΦυϧͻͱΛ;΍͢ʯ͜ͱ
   ͕Ͱ͖Δ
   Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔
   ৽ଔݚमͰߦ͏*46$0/ͷັྗ /BP.JOBNJ
   
   https://speakerdeck.com/south37/xin-zu-yan-xiu-dexing-u-isucon-falsemei-li?slide=15
   

   View Slide

7. ©2019 Wantedly, Inc.
   3FEJTͷ֓ཁ
   ࣾ಺Ͱͷར༻ํ๏
   ਖ਼͍͠༻๏༻ྔ
   What ͜ͷݚमͰֶ΂Δ͜ͱ
   

   View Slide

8. ©2019 Wantedly, Inc.
   ਖ਼͍͠༻๏༻ྔΛकΒͳ͍ͱ
   SFEJTͷؔ܎ͨ͠JODJEFOUͷQPTUNPSUFN
   

   View Slide

9. ©2019 Wantedly, Inc.
   3FEJTͷ֓ཁ
   Overview
   

   View Slide

10. ©2019 Wantedly, Inc.
    ଎͍σʔλߏ଄αʔό
    3FEJTͱ͸
    1 2 3
    

    View Slide

11. ©2019 Wantedly, Inc.
    *ONFNPSZ
    w ϝϞϦΞΫηε͸44%΍σΟεΫͷϥϯμ
    ϜΞΫηεΑΓѹ౗తʹ଎͍
    4JNQMF
    w 3%#.4Ͱ͸ཁٻ͞ΕΔΑ͏ͳෳࡶͳ͜ͱ͸͠ͳ͍
    w ϝΠϯͷॲཧ͸γϯάϧϓϩηεɾγϯάϧεϨου
    w ࣮ࡍʹ͸%#εφοϓγϣοτͷ࡞੒ͳͲͷͨΊʹGPSL͢Δ
    w ϊϯϒϩοΩϯά*0ͱΠϕϯτϧʔϓʹΑΔ*0ଟॏԽ
    ଎͍
    

    View Slide

12. ©2019 Wantedly, Inc.
    ͢΂ͯͷϓϩάϥϚ͕஌͓ͬͯ͘΂͖ϨΠςϯγͷ਺஋
    ग़య: Latency Numbers Every Programmer Should Know
    https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html
    

    View Slide

13. ©2019 Wantedly, Inc.
    Πϕϯτϧʔϓ
    // ae.c
    /* Include the best multiplexing layer supported by this system.
    * The following should be ordered by performances, descending. */
    #ifdef HAVE_EVPORT
    #include "ae_evport.c"
    #else
    #ifdef HAVE_EPOLL
    #include "ae_epoll.c"
    #else
    #ifdef HAVE_KQUEUE
    #include "ae_kqueue.c"
    #else
    #include "ae_select.c"
    #endif
    #endif
    #endif
    ࡞ऀۘ੡ͷΠϕϯτϥΠϒϥϦ
    w FWQPSU4PMBSJTҎ߱
    w FQPMM-JOVY
    w LRVFVF#4%
    w TFMFDUେମ͋Δ
    

    View Slide

14. ©2019 Wantedly, Inc.
    w 4USJOH
    w -JTU
    w 4FU
    w 4PSUFE4FU
    w )BTI
    w #JUBSSBZ CJUNBQ
    
    w )ZQFS-PH-PH
    w 4USFBN
    ͨͩͷLFZWBMVFετΞͰ͸ͳ͍ɻͦΕͧΕৄ͘͠͸ޙड़͢Δɻ
    σʔλߏ଄
    https://redis.io/topics/data-types-intro
    

    View Slide

15. ©2019 Wantedly, Inc.
    ෳ਺ϓϩηε͕σʔλΛڞ༗ͯ͠ಡΈॻ͖Ͱ͖Δ
    w αʔόɾΫϥΠΞϯτϞσϧ
    w 5$1ιέοτΛ࢖ͬͯ௨৴
    w γϯϓϧͳϓϩτίϧ
    ͋ͨΓ·͑ͱࢥ͏͔΋͠Ε·ͤΜ͕ʜ
    
    αʔό
    

    View Slide

16. ©2019 Wantedly, Inc.
    3%#
    w ඇಉظͷεφοϓγϣοτ
    w GPSLͯ͠ॻ͖ࠐΉͷͰ਌ϓϩηεͰσΟε
    Ϋ*0͕ൃੜ͠ͳ͍
    w ཤྺͷόοΫΞοϓ͕͠΍͍͢
    w Ϧελʔτ࣌ͷಡΈࠐΈ͕͸΍͍
    w σʔλϩεͷՄೳੑ͸ΑΓେ͖͍
    σΟεΫӬଓԽ
    "0'
    w ௥هܕ
    w ίϚϯυྻͷه࿥
    w σΟεΫγʔΫ͕ൃੜ͠ͳ͍
    w 3%#ܗࣜΑΓαΠζ͕େ͖͍
    w ୯ಠͰ࢖Θͳ͍΄͏͕Α͍
    

    View Slide

17. ©2019 Wantedly, Inc.
    • volatile-lru: Evict using approximated LRU among the keys with an expire set.
    • allkeys-lru: Evict any key using approximated LRU.
    • volatile-lfu: Evict using approximated LFU among the keys with an expire set.
    • allkeys-lfu: Evict any key using approximated LFU.
    • volatile-random: Remove a random key among the ones with an expire set.
    • allkeys-random: Remove a random key, any key.
    • volatile-ttl: Remove the key with the nearest expire time (minor TTL)
    • noeviction: Don't evict anything, just return an error on write operations.
    Eviction Policy ϝϞϦ͔ΒᷓΕͦ͏ͳ࣌ͷڍಈΛઃఆͰ͖Δ
    

    View Slide

18. ©2019 Wantedly, Inc.
    w Ωϟογϡ༻్
    w ΞΫηεස౓͕ภΔͳΒBMMLFZTMSV
    w ౳֬཰ͰΞΫηε͕͋ΔͳΒBMMLFZTSBOEPN
    w 55- UJNFUPMJWF
    Λৗʹઃఆ͢ΔͳΒWPMBUJMFUUM
    w σʔλϕʔε༻్
    w ফ͑ͨΒࠔΔͷͰOPFWJDUJPO
    Eviction Policy ݁ہͲΕΛબ΂͹ྑ͍ͷ͔
    

    View Slide

19. ©2019 Wantedly, Inc.
    w ެࣜυΩϡϝϯτ
    w SFEJTDPOG
    w (JU)VC
    w ιʔείʔυ
    υΩϡϝϯτͷ͋Γ͔
    

    View Slide

20. ©2019 Wantedly, Inc.
    ίϚϯυͱσʔλܕ
    Commands and Data Types
    

    View Slide

21. ©2019 Wantedly, Inc.
    ίϚϯυҰཡ
    https://redis.io/commands
    

    View Slide

22. ©2019 Wantedly, Inc.
    ίϚϯυྫ
    w ίϚϯυҾ਺
    w ܭࢉྔ
    w ղઆ
    w ஫ҙ
    w ࣮ߦྫ
    

    View Slide

23. ©2019 Wantedly, Inc.
    w ࠷΋جຊతͳσʔλܕ
    w όΠφϦηʔϑɺͭ·Γ೚ҙͷσʔλΛΛอଘͰ͖Δ
    w ྫ3VCZͷσʔλΛγϦΞϥΠζͨ݁͠Ռ
    w .#·Ͱ
    w ਺஋΋4USJOHܥͷίϚϯυͰѻ͏
    String
    

    View Slide

24. ©2019 Wantedly, Inc.
    w 4&5LFZWBMVF9c99>
    w &9TFDPOETŠඵͰFYQJSFઃఆ
    w 19NJMMJTFDPOETŠϛϦඵͰFYQJSFઃఆ
    w /9ŠΩʔ͕ଘࡏ͠ͳ͍৔߹ͷΈॻ͖ࠐΈ
    w 99ŠΩʔ͕ଘࡏ͢Δ৔߹ͷΈॻ͖ࠐΈ
    w (&5LFZ
    w .4&5LFZWBMVF
    w ෳ਺ͷ஋Λॻ͖ࠐΈ
    w .(&5LFZ
    w ෳ਺ͷ஋ΛಡΈࠐΈ
    String
    

    View Slide

25. ©2019 Wantedly, Inc.
    w ਺ࣈ
    w */$3%&$3
    w */$3#:%&$3#:
    w ظݶɾ࡟আ ͜ΕΒ͸4USJOHTʹݶΒͳ͍
    
    w &91*3&LFZTFDPOET
    w &91*3&"5LFZUJNFTUBNQ
    w 55-LFZ
    w %&-LFZ
    String
    

    View Slide

26. ©2019 Wantedly, Inc.
    w 4USJOHͷϦετ
    w ࠷େ௕͸?
    w ࣮૷͸૒ํ޲MJOLFEMJTU
    List
    // adlist.h
    typedef struct listNode {
    struct listNode *prev;
    struct listNode *next;
    void *value;
    } listNode;
    typedef struct listIter {
    listNode *next;
    int direction;
    } listIter;
    typedef struct list {
    listNode *head;
    listNode *tail;
    void *(*dup)(void *ptr);
    void (*free)(void *ptr);
    int (*match)(void *ptr, void *key);
    unsigned long len;
    } list;
    

    View Slide

27. ©2019 Wantedly, Inc.
    w -164)LFZWBMVF
    w ઌ಄ʹ௥Ճɻ0
    
    w -101LFZ
    w ઌ಄Λ࡟আɻ0
    
    w --&/LFZ
    w Ϧετͷ௕͞ɻ0
    
    List
    w -*/4&35LFZ#&'03&c"'5&3QJWPUWBMVF
    w ్தʹૠೖɻ0 /
    
    w -53*.LFZTUBSUTUPQ
    w TUBSU͔ΒTUPQ൪໨ͷཁૉ͚ͩʹ͢Δɻ0 /
    
    w -4&5LFZJOEFYWBMVF
    w JOEFY൪໨ͷ஋Λมߋɻ0 /
    
    w -*/%&9LFZJOEFY
    w JOEFY൪໨ͷཁૉΛऔಘɻ0 /
    
    w -3ͷόϦΤʔγϣϯ
    w ϒϩοΩϯάͷόϦΤʔγϣϯ #Ͱ͸͡·Δ
    
    

    View Slide

28. ©2019 Wantedly, Inc.
    w pFMEWBMVFϖΞͷू߹
    w lΦϒδΣΫτzͬΆ͍΋ͷΛද͢ͷʹศར
    Hash
    > HMSET user:1000 username antirez
    birthyear 1977 verified 1
    OK
    > HGET user:1000 username
    "antirez"
    > HGET user:1000 birthyear
    "1977"
    > HGETALL user:1000
    1) "username"
    2) "antirez"
    3) "birthyear"
    4) "1977"
    5) "verified"
    6) "1"
    

    View Slide

29. ©2019 Wantedly, Inc.
    w )4&5LFZpFMEWBMVF
    w pFMEʹ஋Λॻ͖ࠐΈɻ0
    
    w )(&5LFZpFME
    w pMFEͷ஋ΛಡΈࠐΈɻ0
    
    w )(&5"--LFZ
    w શͯͷpFMEWBMVFϖΞΛಡΈࠐΈɻ0 /
    
    w ).4&5LFZpFMEWBMVF
    w ෳ਺pFMEʹ஋Λॻ͖ࠐΈɻ0 ࢦఆpFME਺
    
    w ).(&5LFZpFME
    w ෳ਺pFMEͷ஋ΛಡΈࠐΈɻ0 ࢦఆpFME਺
    
    Hash
    w )&9*454LFZpFME
    w pFME͕ଘࡏ͢Δ͔Ͳ͏͔ɻ0
    
    w )%&-LFZpFME
    w pFMEΛ࡟আɻ0 pFME਺
    
    w )-&/LFZ
    w pFME਺ɻ0
    
    w )*/$3#:LFZpFMEJODSFNFOU
    w ࢦఆpFMEͷ஋Λ૿ݮɻ0
    
    

    View Slide

30. ©2019 Wantedly, Inc.
    w pFME਺͕IBTI@NBY@[JQMJTU@FOUSJFT
    EFGBVMU
    ҎԼ͔ͭ஋ͷ௕͞ͷ࠷େ
    ͕IBTI@NBY@[JQMJTU@WBMVF EFGBVMU
    
    ҎԼͷ৔߹͸ɺϝϞϦޮ཰ͷΑ
    ͍z[JQMJTUzͰΤϯίʔσΟϯά
    w ͦ͏Ͱͳ͍৔߹͸lEJDUz IBTIUBCMF
    
    w ϝϞϦͱ$16ͷτϨʔυΦϑ
    Hash
    // t_hash.c
    /* Check the length of a number of objects to see if we need to
    convert a
    * ziplist to a real hash. Note that we only check string encoded
    objects
    * as their string length can be queried in constant time. */
    void hashTypeTryConversion(robj *o, robj **argv, int start, int
    end) {
    int i;
    if (o->encoding != OBJ_ENCODING_ZIPLIST) return;
    for (i = start; i <= end; i++) {
    if (sdsEncodedObject(argv[i]) &&
    sdslen(argv[i]->ptr) > server.hash_max_ziplist_value)
    {
    hashTypeConvert(o, OBJ_ENCODING_HT);
    break;
    }
    }
    }
    

    View Slide

31. ©2019 Wantedly, Inc.
    ziplist.c
    /* The ziplist is a specially encoded dually linked list that is designed
    * to be very memory efficient. It stores both strings and integer values,
    * where integers are encoded as actual integers instead of a series of
    * characters. It allows push and pop operations on either side of the list
    * in O(1) time. However, because every operation requires a reallocation of
    * the memory used by the ziplist, the actual complexity is related to the
    * amount of memory used by the ziplist.
    

    View Slide

32. ©2019 Wantedly, Inc.
    w ϢχʔΫͰιʔτ͞Ε͍ͯͳ͍
    4USJOH
    ͷू߹
    w ࠷େཁૉ਺͸?
    Set
    redis> SADD myset "Hello"
    (integer) 1
    redis> SADD myset "World"
    (integer) 1
    redis> SADD myset "World"
    (integer) 0
    redis> SMEMBERS myset
    1) "World"
    2) "Hello"
    redis>
    

    View Slide

33. ©2019 Wantedly, Inc.
    w 4"%%LFZNFNCFS
    w ཁૉͷ௥Ճɻ0
    
    w 43&.LFZNFNCFS
    w ཁૉͷ࡟আɻ0
    
    w 4.&.#&34LFZ
    w શཁૉͷऔಘɻ0 /
    
    w 4*4.&.#&3LFZNFNCFS
    w ཁૉ͕ू߹ʹଘࡏ͢Δ͔Ͳ͏͔ɻ0
    
    w 4101LFZ
    w ϥϯμ
    ϜʹཁૉΛऔΓग़͠ɻ0
    
    Set
    w 46/*0/LFZ
    w ࿨ू߹ɻ0 ཁૉ਺ͷ߹ܭ
    
    w 4*/5&3LFZ
    w ੵू߹ɻ0 Ұ൪খ͍͞ཁૉ਺TFU਺
    
    w 4%*''LFZ
    w ࠩू߹ɻ0 ཁૉ਺ͷ߹ܭ
    
    w 4$"3%LFZ
    w ू߹ͷཁૉ਺ɻ0
    
    

    View Slide

34. ©2019 Wantedly, Inc.
    w ੔਺ͷΈͷ৔߹ɺ಺෦తʹ͸ಛผ
    ͳΤϯίʔσΟϯάͰ֨ೲ͞ΕΔ
    w TFUNBYJOUTFUFOUSJFT
    EFGBVMU
    ҎԼͷཁૉ਺ͷ৔
    ߹ɻ
    Set
    // t_set.c
    /* Add the specified value into a set.
    *
    * If the value was already member of the set, nothing is done and 0 is
    * returned, otherwise the new element is added and 1 is returned. */
    int setTypeAdd(robj *subject, sds value) {
    long long llval;
    if (subject->encoding == OBJ_ENCODING_HT) {
    dict *ht = subject->ptr;
    dictEntry *de = dictAddRaw(ht,value,NULL);
    if (de) {
    dictSetKey(ht,de,sdsdup(value));
    dictSetVal(ht,de,NULL);
    return 1;
    }
    } else if (subject->encoding == OBJ_ENCODING_INTSET) {
    if (isSdsRepresentableAsLongLong(value,&llval) == C_OK) {
    uint8_t success = 0;
    subject->ptr = intsetAdd(subject->ptr,llval,&success);
    if (success) {
    /* Convert to regular set when the intset contains
    * too many entries. */
    if (intsetLen(subject->ptr) > server.set_max_intset_entries)
    setTypeConvert(subject,OBJ_ENCODING_HT);
    return 1;
    }
    } else {
    /* Failed to get integer from object, convert to regular set. */
    setTypeConvert(subject,OBJ_ENCODING_HT);
    /* The set *was* an intset and this value is not integer
    * encodable, so dictAdd should always work. */
    serverAssert(dictAdd(subject->ptr,sdsdup(value),NULL) == DICT_OK);
    return 1;
    }
    } else {
    serverPanic("Unknown set encoding");
    }
    return 0;
    }
    

    View Slide

35. ©2019 Wantedly, Inc.
    redisObjectߏ଄ମ
    // server.h
    /* Objects encoding. Some kind of objects like Strings and Hashes can be
    * internally represented in multiple ways. The 'encoding' field of the object
    * is set to one of this fields for this object. */
    #define OBJ_ENCODING_RAW 0 /* Raw representation */
    #define OBJ_ENCODING_INT 1 /* Encoded as integer */
    #define OBJ_ENCODING_HT 2 /* Encoded as hash table */
    #define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */
    #define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */
    #define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */
    #define OBJ_ENCODING_INTSET 6 /* Encoded as intset */
    #define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */
    #define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */
    #define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */
    #define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */
    typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
    * LFU data (least significant 8 bits frequency
    * and most significant 16 bits access time). */
    int refcount;
    void *ptr;
    } robj;
    

    View Slide

36. ©2019 Wantedly, Inc.
    w είΞॱʹιʔτ͞Ε͍ͯΔ4FU
    w είΞ͸CJUුಈখ਺఺਺
    w είΞ͕ಉ͡৔߹͸ཁૉͷࣙॻࣜॱং
    w [TFUNBY[JQMJTUFOUSJFT EFGBVMU
    
    ͔ͭ[TFUNBY[JQMJTUWBMVF
    EFGBVMU
    Λຬͨ͢৔߹͸[JQMJTU
    w *46$0/Ͱ3%#.4ͷUBCMFΛࡶʹஔ͖
    ׵͑Δͱ͖ʹΑ͘࢖͏
    SortedSet
    redis> ZADD myzset 1 "one"
    (integer) 1
    redis> ZADD myzset 1 "uno"
    (integer) 1
    redis> ZADD myzset 2 "two" 3 "three"
    (integer) 2
    redis> ZRANGE myzset 0 -1 WITHSCORES
    1) "one"
    2) "1"
    3) "uno"
    4) "1"
    5) "two"
    6) "2"
    7) "three"
    8) "3"
    redis>
    

    View Slide

37. ©2019 Wantedly, Inc.
    3FEJT0CKFDU͔ΒείΞ΁ͷϚοϓͷ
    IBTIUBCMFͱɺείΞ͔Β3FEJT0CKFDU
    ΁ͷϚοϓͷTLJQMJTU͔ΒͳΔ
    SortedSet
    // server.h
    /* ZSETs use a specialized version of Skiplists
    */
    typedef struct zskiplistNode {
    sds ele;
    double score;
    struct zskiplistNode *backward;
    struct zskiplistLevel {
    struct zskiplistNode *forward;
    unsigned long span;
    } level[];
    } zskiplistNode;
    typedef struct zskiplist {
    struct zskiplistNode *header, *tail;
    unsigned long length;
    int level;
    } zskiplist;
    typedef struct zset {
    dict *dict;
    zskiplist *zsl;
    } zset;
    ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU
    

    View Slide

38. ©2019 Wantedly, Inc.
    Skip List
    ૠೖͱ୳ࡧͱ࡟আͷ࣌ؒܭࢉྔ͸ฏۉ0 MPH /
    
    ɺ࠷ѱ0 /
    
    Ξχϝʔγϣϯ(*'ͳͷͰݩαΠτͰ֬ೝ͍ͯͩ͘͠͞
    
    ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU
    

    View Slide

39. ©2019 Wantedly, Inc.
    w ;"%%LFZ9c99><$)><*/$3>TDPSFNFNCFS
    w ཁૉͷ௥Ճɻ0 MPH /
    
    
    w 99ŠطଘͷཁૉͷείΞߋ৽ͷΈɻ
    w /9Š৽͍͠ཁૉͷ௥ՃͷΈɻ
    w $)ŠείΞߋ৽͞Εͨཁૉ਺Λ໭Γ஋ʹؚΉ ௨ৗ͸৽͘͠௥Ճ͞Εͨ਺
    ɻ
    w */$3ŠείΞͷมԽࠩ෼Λࢦఆɻ
    w ;4$03&LFZNFNCFS
    w ࢦఆͨ͠ཁૉͷείΞɻ0
    
    w ;$"3%LFZ
    w ཁૉ਺ɻ0
    
    SortedSet
    

    View Slide

40. ©2019 Wantedly, Inc.
    w ;$06/5LFZNJONBY
    w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉ਺ɻ0 MPH /
    
    
    w ;3"/,LFZNFNCFS
    w ࢦఆͨ͠ϝϯόʔ͕Կ൪໨͔ɻ0 MPH /
    
    
    w ;3"/(&LFZTUBSUTUPQ<8*5)4$03&4>
    w TUBSU͔ΒTUPQ൪໨ͷཁૉΛऔಘɻ0 MPH શཁૉ਺
    औಘཁૉ਺
    
    w ;3"/(:4$03&LFZNJONBY<8*5)4$03&4><-*.*5PGGTFUDPVOU>
    w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉΛऔಘɻ0 MPH શཁૉ਺
    औಘཁૉ਺
    
    SortedSet
    

    View Slide

41. ©2019 Wantedly, Inc.
    w lBQSPCBCJMJTUJDEBUBTUSVDUVSFXIJDI
    JTVTFEJOPSEFSUPFTUJNBUFUIF
    DBSEJOBMJUZPGBTFUz
    w ϢχʔΫͳཁૉ਺Λ਺͑Δͷʹ࢖͏
    w ඪ४ޡࠩͰਪଌ
    w ϝϞϦ࢖༻ྔ͸ݻఆͰ࠷ѱέʔεͰL
    CZUFT
    HyperLogLog
    HyperLogLogs in Redis
    https://thoughtbot.com/blog/hyperloglogs-in-redis
    

    View Slide

42. ©2019 Wantedly, Inc.
    require 'redis'
    require 'sinatra'
    $redis = Redis.new
    class VisitorCounter
    def initialize(app)
    @app = app
    end
    def call(env)
    $redis.pfadd 'visitors', Rack::Request.new(env).ip
    @app.call(env)
    end
    end
    use VisitorCounter
    get '/' do
    visitors = $redis.pfcount 'visitors'
    "This website has been visited by #{visitors} unique visitors."
    end
    HyperLogLog
    ग़య: https://thoughtbot.com/blog/hyperloglogs-in-redis
    

    View Slide

43. ©2019 Wantedly, Inc.
    -JTU 4FU 4PSUFE4FU )BTIͷσʔλܕʹର͠ɺͦΕͧΕͷί
    Ϛϯυ͕ظ଴͢Δ࣌ؒܭࢉྔͰ͋Δ͜ͱΛ֬ೝ͢Δɻ
    ͕࣌ؒ͋ͬͨΒ͜͜Ͱԋश
    require 'redis'
    redis = Redis.new(host: "localhost", port: 6379)
    redis.set("mykey", "hello world")
    # => "OK"
    redis.get("mykey")
    # => "hello world"
    

    View Slide

44. ©2019 Wantedly, Inc.
    Pipelining
    require 'redis'
    def without_pipelining
    r = Redis.new
    10000.times {
    r.ping
    }
    end
    def with_pipelining
    r = Redis.new
    r.pipelined {
    10000.times {
    r.ping
    }
    }
    end
    def bench(descr)
    start = Time.now
    yield
    puts "#{descr} #{Time.now-start} seconds"
    end
    bench("without pipelining") {
    without_pipelining
    }
    bench("with pipelining") {
    with_pipelining
    }
    # without pipelining 1.185238 seconds
    # with pipelining 0.250783 seconds
    Using pipelining to speedup Redis queries
    https://redis.io/topics/pipelining
    355ͱγεςϜίʔϧͷΦʔόʔϔου࡟ݮ
    

    View Slide

45. ©2019 Wantedly, Inc.
    Transaction
    > MULTI
    OK
    > INCR foo
    QUEUED
    > INCR bar
    QUEUED
    > EXEC
    1) (integer) 1
    2) (integer) 1
    https://redis.io/topics/transactions
    ෳ਺ίϚϯυΛΞτϛοΫʹ࣮ߦ
    w .6-5*ίϚϯυͰ໋ྩΛΩϡʔΠϯά
    w &9&$ίϚϯυͰBUPNJDʹ࣮ߦ
    w SPMMCBDL͸ͳ͍
    

    View Slide

46. ©2019 Wantedly, Inc.
    Lua Scripting
    w ༨༟͕͋Ε͹ॻ͘
    

    View Slide

47. ©2019 Wantedly, Inc.
    PubSub
    w ༨༟͕͋Ε͹ॻ͘
    

    View Slide

48. ©2019 Wantedly, Inc.
    Redis Module
    w ໋ྩΛ֦ுͰ͖Δ
    w IUUQTSFEJTJPUPQJDTNPEVMFTJOUSP
    w IUUQTSFEJTMBCTDPNDPNNVOJUZSFEJTNPEVMFTIVC
    w ͕࣌ؒ͋Ε͹͜͜Ͱԋश
    w ద౰ͳ໋ྩΛࣗ࡞͢Δ
    w IUUQTHJUIVCDPN3FEJT-BCT3FEJT.PEVMFT4%,
    

    View Slide

49. ©2019 Wantedly, Inc.
    ࣮ࡍͷ࢖͍ํ
    Usage on Real Projects
    

    View Slide

50. ©2019 Wantedly, Inc.
    ࣮ࡍͷ࢖͍ํ ओͳ༻్
    3BJMTͷඇಉظδϣϒΩϡʔ
    3BJMTDBDIFͷόοΫΤϯυ
    ੜ3FEJTΛDBDIF·ͨ͸EBUBCBTFͱͯ͠
    

    View Slide

51. ©2019 Wantedly, Inc.
    Sidekiq/ActiveJob
    class HardWorker
    include Sidekiq::Worker
    def perform(name, count)
    # do something
    end
    end
    HardWorker.perform_async('bob', 5)
    HardWorker.perform_in(5.minutes, 'bob', 5)
    HardWorker.perform_at(5.minutes.from_now, 'bob', 5)
    ग़య: https://github.com/mperham/sidekiq/wiki/Getting-Started
    3BJMTͷඇಉظॲཧͷόοΫΤϯυͱͯ͠
    

    View Slide

52. ©2019 Wantedly, Inc.
    Sidekiq/ActiveJob ͲͪΒΛ͔ͭ͏͔ʁ
    w "DUJWF+PCͰ࢖͏৔߹
    w "3ͷΠϯελϯεΛ౉ͤΔ EFTFSJBMJ[F࣌ʹA.PEFMpOE JE
    AΛ΍͍ͬͯΔ
    
    w (MPCBM*%4FSJBMJ[BUJPOIUUQTHJUIVCDPNSBJMTHMPCBMJE
    w "DUJPO.BJMFSͱ૬ੑ͕Α͍ EFMJWFS@MBUFS
    
    w ։ൃ؀ڥͰ3FEJTͳ͠Ͱ࢖͑Δ
    w ผSVCZUISFBEͰͷ࣮ߦͱͳΔ
    w +PC࡞੒࣌ͷA*OMPDBMFAΛอଘͯ͠ɺͦͷϩέʔϧͰ+PCΛ࣮ߦͯ͘͠ΕΔ
    w ੜ4JEFLJRΛ࢖͏৔߹
    w 4JEFLJRͷػೳΛϑϧʹ࢖͑Δ
    w EF
    TFSJBMJ[FͷΦʔόʔϔου͕ݮΔ
    

    View Slide

53. ©2019 Wantedly, Inc.
    Sidekiq/ActiveJob ࢖͍ํͷίπ
    w ग़དྷΔݶΓॲཧΛႈ౳ʹ͍͕ͨ͠ɺݱ࣮͸ͦ͏͸͍͔ͳ͍
    w σϑΥϧτͰ͸ϦτϥΠ͠ͳ͍Α͏ʹઃఆ͢Δ΄͏͕҆શ
    w ॲཧʹ͕͔͔࣌ؒΓλΠϜΞ΢τ͢Δ͜ͱ͕͋Δ
    w ϝʔϧ΍ϓογϡ௨஌Λ౓ૹ৴͢ΔͱϢʔβମݧ͕ѱ͍
    w ૢ࡞͕ႈ౳ͰϦτϥΠ͍ͨ͠΍ͭ͸໌ࣔతʹࢦఆ͢Δ
    w 4JEFLJREFGBVMU@XPSLFS@PQUJPOT<SFUSZ>GBMTF
    w "DUJWF3FDPSEͷίʔϧόοΫͰΤϯΩϡʔ͢Δ৔߹͸ɺBGUFS@DPNNJUͰ΍Δ
    w BGUFS@DSFBUF΍BGUFS@VQEBUFͰΤϯΩϡʔ͢Δͱɺ3%#ʹDPNNJU͞ΕΔ·͑ʹɺ
    KPC͕࣮ߦ͞Εͯ͠·͏͜ͱ͕͋Δ
    w TJEFLJRϓϩηε΁ͷ4*(5&3.ͱ4*(,*--ͷؒ LTͷEFGBVMU͸T
    ʹUJNFPVUͤ͞Δ
    https://github.com/wantedly/dev-docs/blob/master/sidekiq.md
    

    View Slide

54. ©2019 Wantedly, Inc.
    3BJMTDBDIFͷόοΫΤϯυͱͯ͠
    # config/environments/production.rb
    config.cache_store = :redis_store,
    ENV["REDIS_RAILS_CACHE_URL"],
    { compress: true, expires_in: 90.minutes }
    w .BSTIBMEVNQ .BSTIBMMPBE
    Ͱ EF
    TFSJBMJ[F
    w 1SJNJUJWFͬΆ͘ͳ͍σʔλܕΛ͍Εͳ͍
    w ൵͍͠ࣄނͷྫXBOUFEMZQPTUNPSUFNT
    wantedly/post-mortems#40
    

    View Slide

55. ©2019 Wantedly, Inc.
    ੜ3FEJT
    # config/initializers/redis.rb
    REDIS_CONFIG = {
    host: ENV['REDIS_HOST'].presence || 'localhost',
    port: ENV['REDIS_PORT'].presence || '6379',
    connect_timeout: 0.2,
    read_timeout: 1.0,
    write_timeout: 3.0,
    }
    redis_conn = Redis.new(REDIS_CONFIG)
    w ίϚϯυ͕ͦͷ··ϝιου໊ʹͳ͍ͬͯΔ
    w ઀ଓͷॳظԽ࣌ʹλΠϜΞ΢τΛઃఆ͓ͯ͘͠
    

    View Slide

56. ©2019 Wantedly, Inc.
    ຊ൪༻ͷΠϯελϯεΛ࡞੒͢Δ
    # terraform/elasticache_replication_group.tf
    resource "aws_elasticache_replication_group" "users-pubsub-prod" {
    replication_group_id = "users-pubsub-prod"
    replication_group_description = " "
    engine_version = "5.0.0"
    maintenance_window = "tue:18:00-tue:19:00"
    node_type = "cache.r5.large"
    number_cache_clusters = 2
    automatic_failover_enabled = true
    parameter_group_name = "default.redis5.0"
    port = 6379
    subnet_group_name = "${aws_elasticache_subnet_group.wantedly-production-cache.name}"
    security_group_ids = ["${aws_security_group.redis-elasticache-production.id}"]
    notification_topic_arn = "arn:aws:sns:ap-northeast-1:096233016669:dev"
    snapshot_retention_limit = 1
    snapshot_window = "17:00-18:00"
    }
    w "84&MBTUJDBDIFΛ࢖͍ͬͯΔ
    w 5FSSBGPSNͰ࡞੒ XBOUFEMZXBOUFEMZUFSSBGPSN΁QVMMSFRVFTU
    
    w QBSBNFUFSHSPVQ͕ॏཁͳͷͰɺ*OGSBTRVBEͷαϙʔτ౰൪ʹ૬ஊ͍ͯͩ͘͠͞
    

    View Slide

57. ©2019 Wantedly, Inc.
    KEYS *ίϚϯυࣄ݅
    wantedly/post-mortems#31
    

    View Slide

58. ©2019 Wantedly, Inc.
    KEYS *ίϚϯυࣄ݅
    w ,&:4QBUUFSO
    w QBUUFSOʹϚον͢ΔશͯͷΩʔΛฦ͢ɻ0 શLFZ਺
    
    w l8BSOJOHDPOTJEFS,&:4BTBDPNNBOEUIBUTIPVMEPOMZCFVTFEJOQSPEVDUJPO
    FOWJSPONFOUTXJUIFYUSFNFDBSF*UNBZSVJOQFSGPSNBODFXIFOJUJT
    FYFDVUFEBHBJOTUMBSHFEBUBCBTFT5IJTDPNNBOEJTJOUFOEFEGPSEFCVHHJOH
    BOETQFDJBMPQFSBUJPOT TVDIBTDIBOHJOHZPVSLFZTQBDFMBZPVU%POU
    VTF,&:4JOZPVSSFHVMBSBQQMJDBUJPODPEF*GZPVSFMPPLJOHGPSBXBZUPpOE
    LFZTJOBTVCTFUPGZPVSLFZTQBDF DPOTJEFSVTJOH4$"/PSTFUTz
    

    View Slide

59. ©2019 Wantedly, Inc.
    ڊେHashࣄ݅
    wantedly/post-mortems#25
    

    View Slide

60. ©2019 Wantedly, Inc.
    ڊେHashࣄ݅
    wantedly/infrastructure#4102
    

    View Slide

61. ©2019 Wantedly, Inc.
    ίϚϯυςʔϒϧ
    struct redisCommand redisCommandTable[] = {
    {"module",moduleCommand,-2,"as",0,NULL,0,0,0,0,0},
    {"get",getCommand,2,"rF",0,NULL,1,1,1,0,0},
    {"set",setCommand,-3,"wm",0,NULL,1,1,1,0,0},
    {"setnx",setnxCommand,3,"wmF",0,NULL,1,1,1,0,0},
    {"setex",setexCommand,4,"wm",0,NULL,1,1,1,0,0},
    {"psetex",psetexCommand,4,"wm",0,NULL,1,1,1,0,0},
    {"append",appendCommand,3,"wm",0,NULL,1,1,1,0,0},
    {"strlen",strlenCommand,2,"rF",0,NULL,1,1,1,0,0},
    {"del",delCommand,-2,"w",0,NULL,1,-1,1,0,0},
    {"unlink",unlinkCommand,-2,"wF",0,NULL,1,-1,1,0,0},
    {"exists",existsCommand,-2,"rF",0,NULL,1,-1,1,0,0},
    {"setbit",setbitCommand,4,"wm",0,NULL,1,1,1,0,0},
    {“getbit",getbitCommand,3,"rF",0,NULL,1,1,1,0,0},
    {"bitfield",bitfieldCommand,-2,"wm",0,NULL,1,1,1,0,0},
    {"setrange",setrangeCommand,4,"wm",0,NULL,1,1,1,0,0},
    {"getrange",getrangeCommand,4,"r",0,NULL,1,1,1,0,0},
    {"substr",getrangeCommand,4,"r",0,NULL,1,1,1,0,0},
    {"incr",incrCommand,2,"wmF",0,NULL,1,1,1,0,0},
    {"decr",decrCommand,2,"wmF",0,NULL,1,1,1,0,0},
    server.c
    

    View Slide

62. ©2019 Wantedly, Inc.
    ·ͱΊ
    Summary
    

    View Slide

63. ©2019 Wantedly, Inc.
    3FEJTͷ֓ཁ
    ࣾ಺Ͱͷར༻ํ๏
    ਖ਼͍͠༻๏༻ྔ
    ͜ͷݚमͰֶΜͩ͜ͱ
    

    View Slide

64. ©2019 Wantedly, Inc.
    w ιʔείʔυΛோΊΔ͜ͱͰগ͚ͩ͠ਂ͘3FEJTʹ͍ͭͯཧղͰ͖ͨ
    w ීஈ࢖͍ͬͯΔιϑτ΢ΣΞʹ΋୭͔͕ॻ͍ͨιʔείʔυ͕ଘࡏ͢Δ
    w ϑϨʔϜϫʔΫɺϥΠϒϥϦɺݴޠ࣮૷
    w ϒϥοΫϘοΫεԽͤͣʹڵຯΛ࣋ͬͯಡΜͰΈͯ΄͍͠
    w ͦΕ͕ɺιϑτ΢ΣΞΤϯδχΞͱͯ͠ͷ෢ثΛ૿΍͢͜ͱʹͳΔ
    w ·ͨɺ্ͷϨΠϠʔʹ΋ڵຯΛͻΖ͛ͯ΄͍͠
    w Ϣʔβମݧɺ6*ઃܭɺϏδωεϞσϧɺϚʔέςΟϯάɺϓϩδΣΫτϚωʔδϝϯτɺ
    ϓϩμΫτϚωʔδϝϯτɺͳͲͳͲ
    ͍ͭͰʹ఻͔͑ͨͬͨ͜ͱ
    

    View Slide

65. ©2019 Wantedly, Inc.
    w l8PSLJOH8JUI6OJY1SPDFTTFTz +FTTF4UPSJNFS
    w 6/*9ͷϓϩηεɺϑΝΠϧσΟεΫϦϓλɺγες
    Ϝίʔϧɺγάφϧ͋ͨΓΛܰ͘ཧղ͢ΔͷʹΑ͍ɻ
    w (PͳΒΘ͔ΔγεςϜϓϩάϥϛϯά ौ઒Α͖͠
    w ೔ຊޠͰಡΈ͚ͨΕ͹͜Ε΋ྑ͍ɻ
    w -JOVYϓϩάϥϛϯάΠϯλϑΣʔε .JDIBFM,FSSJTL
    ઍॅ࣏࿠
    w ϖʔδ௒͑ͷࣙॻɻΦϑΟεʹ͋Δɻ
    w ͳΜͰ΋͍͍ͷͰΞϧΰϦζϜͱσʔλߏ଄ͷຊ
    ࢀߟจݙ ͞Βʹਂ͘ཧղ͍ͨ͠ਓ΁
    

    View Slide

66. ©2019 Wantedly, Inc.
    w "OOJF4QSBUUIUUQTVOTQMBTIDPNQIPUPTUI*@$;"#.:
    Photo Credit
    

    View Slide