Upgrade to Pro — share decks privately, control downloads, hide ads and more …

アルゴリズムとデータ構造から理解するRedis / Learn Redis from Internal Algorithms and Data Structures

アルゴリズムとデータ構造から理解するRedis / Learn Redis from Internal Algorithms and Data Structures

2019年新卒研修で使った資料です。

内部実装の雰囲気を感じとりながら、Redisについて理解を深める研修を行いました。

以下の内容について学びました。

1. Redisの概要
2. 社内での利用方法
3. 正しい用法用量

Redis についての前提知識は必要としていません。C言語の基礎的な知識は前提とします。

Yoshinori Kawasaki

April 22, 2019
Tweet

More Decks by Yoshinori Kawasaki

Other Decks in Programming

Transcript

  1. ©2019 Wantedly, Inc.
    ΞϧΰϦζϜͱσʔλߏ଄͔Β
    ཧղ͢Δ3FEJT
    Learn Redis from Internal Algorithms and Data Structures
    New Grads Training 2019
    April 21, 2019 - Yoshinori Kawasaki (@kawasy/@luvtechno)

    View full-size slide

  2. ©2019 Wantedly, Inc.
    3FEJTʹ͍ͭͯͷલఏ஌ࣝ͸ඞཁ͋Γ·ͤΜɻ
    IUUQTHJUIVCDPNBOUJSF[SFEJTΛDMPOF͓͍͍ͯͯͩ͘͠͞ɻ
    CSBODIͰNBLFͱNBLFUFTU͕௨Δ؀ڥΛ༻ҙ͍ͯͩ͘͠͞ɻ
    .BD04Ͱ΋-JOVYͰ΋େৎ෉Ͱ͢ɻ
    $ݴޠͷجૅతͳ஌ࣝΛલఏͱ͠·͢ɻϙΠϯλ΍ߏ଄ମΛ஌͍ͬͯΔఔ౓Ͱେৎ෉Ͱ͢ɻ
    ࣄલ४උ

    View full-size slide

  3. ©2019 Wantedly, Inc.
    ͸͡Ίʹ
    Introduction

    View full-size slide

  4. ©2019 Wantedly, Inc.
    ༷ʑͳϛυϧ΢ΣΞ͕ࣾ಺Ͱ࢖ΘΕ͍ͯΔ
    w 1PTUHSF42-
    w 3FEJT
    w &MBTUJDTFBSDI
    w %HSBQI
    w .FN42-
    w $MPVE1VC4VC
    3FEJT͸ࣾ಺Ͱ࠷΋࢖ΘΕ͍ͯΔ΋ͷͷͭ
    Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔

    View full-size slide

  5. ©2019 Wantedly, Inc.
    ࣾ಺ͷϝδϟʔͳϚΠΫϩαʔϏεͰ3FEJTΛར༻
    w XBOUFEMZ
    w TZODNFTTFOHFSSBJMT
    w ZBTIJNBSBJMT
    w WJTJUSFDPNNFOEBUJPO
    w QFPQMFHP
    w ZBTIJNBVQEBUFT
    w OPUJpDBUJPOT
    w NJSBDMFTFBSDIHP
    Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔
    Datadog APM Service Map
    https://app.datadoghq.com/service/map?env=production

    View full-size slide

  6. ©2019 Wantedly, Inc.
    3FEJTΛ࢖͏ཧ༝
    w ߴ଎ԽͷͨΊ
    w Ωϟογϡ
    w ඇಉظॲཧ
    w ϨεϙϯελΠϜ͸Ϣʔβମݧʹେ͖ͳӨڹ
    w ཭୤͕ݮΔ
    w ίϯόʔδϣϯ͕૿͑Δ
    w ʮγΰτͰίίϩΦυϧͻͱΛ;΍͢ʯ͜ͱ
    ͕Ͱ͖Δ
    Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔
    ৽ଔݚमͰߦ͏*46$0/ͷັྗ /BP.JOBNJ

    https://speakerdeck.com/south37/xin-zu-yan-xiu-dexing-u-isucon-falsemei-li?slide=15

    View full-size slide

  7. ©2019 Wantedly, Inc.
    3FEJTͷ֓ཁ
    ࣾ಺Ͱͷར༻ํ๏
    ਖ਼͍͠༻๏༻ྔ
    What ͜ͷݚमͰֶ΂Δ͜ͱ

    View full-size slide

  8. ©2019 Wantedly, Inc.
    ਖ਼͍͠༻๏༻ྔΛकΒͳ͍ͱ
    SFEJTͷؔ܎ͨ͠JODJEFOUͷQPTUNPSUFN

    View full-size slide

  9. ©2019 Wantedly, Inc.
    3FEJTͷ֓ཁ
    Overview

    View full-size slide

  10. ©2019 Wantedly, Inc.
    ଎͍σʔλߏ଄αʔό
    3FEJTͱ͸
    1 2 3

    View full-size slide

  11. ©2019 Wantedly, Inc.
    *ONFNPSZ
    w ϝϞϦΞΫηε͸44%΍σΟεΫͷϥϯμ
    ϜΞΫηεΑΓѹ౗తʹ଎͍
    4JNQMF
    w 3%#.4Ͱ͸ཁٻ͞ΕΔΑ͏ͳෳࡶͳ͜ͱ͸͠ͳ͍
    w ϝΠϯͷॲཧ͸γϯάϧϓϩηεɾγϯάϧεϨου
    w ࣮ࡍʹ͸%#εφοϓγϣοτͷ࡞੒ͳͲͷͨΊʹGPSL͢Δ
    w ϊϯϒϩοΩϯά*0ͱΠϕϯτϧʔϓʹΑΔ*0ଟॏԽ
    ଎͍

    View full-size slide

  12. ©2019 Wantedly, Inc.
    ͢΂ͯͷϓϩάϥϚ͕஌͓ͬͯ͘΂͖ϨΠςϯγͷ਺஋
    ग़య: Latency Numbers Every Programmer Should Know
    https://people.eecs.berkeley.edu/~rcs/research/interactive_latency.html

    View full-size slide

  13. ©2019 Wantedly, Inc.
    Πϕϯτϧʔϓ
    // ae.c
    /* Include the best multiplexing layer supported by this system.
    * The following should be ordered by performances, descending. */
    #ifdef HAVE_EVPORT
    #include "ae_evport.c"
    #else
    #ifdef HAVE_EPOLL
    #include "ae_epoll.c"
    #else
    #ifdef HAVE_KQUEUE
    #include "ae_kqueue.c"
    #else
    #include "ae_select.c"
    #endif
    #endif
    #endif
    ࡞ऀۘ੡ͷΠϕϯτϥΠϒϥϦ
    w FWQPSU4PMBSJTҎ߱
    w FQPMM-JOVY
    w LRVFVF#4%
    w TFMFDUେମ͋Δ

    View full-size slide

  14. ©2019 Wantedly, Inc.
    w 4USJOH
    w -JTU
    w 4FU
    w 4PSUFE4FU
    w )BTI
    w #JUBSSBZ CJUNBQ

    w )ZQFS-PH-PH
    w 4USFBN
    ͨͩͷLFZWBMVFετΞͰ͸ͳ͍ɻͦΕͧΕৄ͘͠͸ޙड़͢Δɻ
    σʔλߏ଄
    https://redis.io/topics/data-types-intro

    View full-size slide

  15. ©2019 Wantedly, Inc.
    ෳ਺ϓϩηε͕σʔλΛڞ༗ͯ͠ಡΈॻ͖Ͱ͖Δ
    w αʔόɾΫϥΠΞϯτϞσϧ
    w 5$1ιέοτΛ࢖ͬͯ௨৴
    w γϯϓϧͳϓϩτίϧ
    ͋ͨΓ·͑ͱࢥ͏͔΋͠Ε·ͤΜ͕ʜ

    αʔό

    View full-size slide

  16. ©2019 Wantedly, Inc.
    3%#
    w ඇಉظͷεφοϓγϣοτ
    w GPSLͯ͠ॻ͖ࠐΉͷͰ਌ϓϩηεͰσΟε
    Ϋ*0͕ൃੜ͠ͳ͍
    w ཤྺͷόοΫΞοϓ͕͠΍͍͢
    w Ϧελʔτ࣌ͷಡΈࠐΈ͕͸΍͍
    w σʔλϩεͷՄೳੑ͸ΑΓେ͖͍
    σΟεΫӬଓԽ
    "0'
    w ௥هܕ
    w ίϚϯυྻͷه࿥
    w σΟεΫγʔΫ͕ൃੜ͠ͳ͍
    w 3%#ܗࣜΑΓαΠζ͕େ͖͍
    w ୯ಠͰ࢖Θͳ͍΄͏͕Α͍

    View full-size slide

  17. ©2019 Wantedly, Inc.
    • volatile-lru: Evict using approximated LRU among the keys with an expire set.
    • allkeys-lru: Evict any key using approximated LRU.
    • volatile-lfu: Evict using approximated LFU among the keys with an expire set.
    • allkeys-lfu: Evict any key using approximated LFU.
    • volatile-random: Remove a random key among the ones with an expire set.
    • allkeys-random: Remove a random key, any key.
    • volatile-ttl: Remove the key with the nearest expire time (minor TTL)
    • noeviction: Don't evict anything, just return an error on write operations.
    Eviction Policy ϝϞϦ͔ΒᷓΕͦ͏ͳ࣌ͷڍಈΛઃఆͰ͖Δ

    View full-size slide

  18. ©2019 Wantedly, Inc.
    w Ωϟογϡ༻్
    w ΞΫηεස౓͕ภΔͳΒBMMLFZTMSV
    w ౳֬཰ͰΞΫηε͕͋ΔͳΒBMMLFZTSBOEPN
    w 55- UJNFUPMJWF
    Λৗʹઃఆ͢ΔͳΒWPMBUJMFUUM
    w σʔλϕʔε༻్
    w ফ͑ͨΒࠔΔͷͰOPFWJDUJPO
    Eviction Policy ݁ہͲΕΛબ΂͹ྑ͍ͷ͔

    View full-size slide

  19. ©2019 Wantedly, Inc.
    w ެࣜυΩϡϝϯτ
    w SFEJTDPOG
    w (JU)VC
    w ιʔείʔυ
    υΩϡϝϯτͷ͋Γ͔

    View full-size slide

  20. ©2019 Wantedly, Inc.
    ίϚϯυͱσʔλܕ
    Commands and Data Types

    View full-size slide

  21. ©2019 Wantedly, Inc.
    ίϚϯυҰཡ
    https://redis.io/commands

    View full-size slide

  22. ©2019 Wantedly, Inc.
    ίϚϯυྫ
    w ίϚϯυҾ਺
    w ܭࢉྔ
    w ղઆ
    w ஫ҙ
    w ࣮ߦྫ

    View full-size slide

  23. ©2019 Wantedly, Inc.
    w ࠷΋جຊతͳσʔλܕ
    w όΠφϦηʔϑɺͭ·Γ೚ҙͷσʔλΛΛอଘͰ͖Δ
    w ྫ3VCZͷσʔλΛγϦΞϥΠζͨ݁͠Ռ
    w .#·Ͱ
    w ਺஋΋4USJOHܥͷίϚϯυͰѻ͏
    String

    View full-size slide

  24. ©2019 Wantedly, Inc.
    w 4&5LFZWBMVF9c99>
    w &9TFDPOETŠඵͰFYQJSFઃఆ
    w 19NJMMJTFDPOETŠϛϦඵͰFYQJSFઃఆ
    w /9ŠΩʔ͕ଘࡏ͠ͳ͍৔߹ͷΈॻ͖ࠐΈ
    w 99ŠΩʔ͕ଘࡏ͢Δ৔߹ͷΈॻ͖ࠐΈ
    w (&5LFZ
    w .4&5LFZWBMVF
    w ෳ਺ͷ஋Λॻ͖ࠐΈ
    w .(&5LFZ
    w ෳ਺ͷ஋ΛಡΈࠐΈ
    String

    View full-size slide

  25. ©2019 Wantedly, Inc.
    w ਺ࣈ
    w */$3%&$3
    w */$3#:%&$3#:
    w ظݶɾ࡟আ ͜ΕΒ͸4USJOHTʹݶΒͳ͍

    w &91*3&LFZTFDPOET
    w &91*3&"5LFZUJNFTUBNQ
    w 55-LFZ
    w %&-LFZ
    String

    View full-size slide

  26. ©2019 Wantedly, Inc.
    w 4USJOHͷϦετ
    w ࠷େ௕͸?
    w ࣮૷͸૒ํ޲MJOLFEMJTU
    List
    // adlist.h
    typedef struct listNode {
    struct listNode *prev;
    struct listNode *next;
    void *value;
    } listNode;
    typedef struct listIter {
    listNode *next;
    int direction;
    } listIter;
    typedef struct list {
    listNode *head;
    listNode *tail;
    void *(*dup)(void *ptr);
    void (*free)(void *ptr);
    int (*match)(void *ptr, void *key);
    unsigned long len;
    } list;

    View full-size slide

  27. ©2019 Wantedly, Inc.
    w -164)LFZWBMVF
    w ઌ಄ʹ௥Ճɻ0

    w -101LFZ
    w ઌ಄Λ࡟আɻ0

    w --&/LFZ
    w Ϧετͷ௕͞ɻ0

    List
    w -*/4&35LFZ#&'03&c"'5&3QJWPUWBMVF
    w ్தʹૠೖɻ0 /

    w -53*.LFZTUBSUTUPQ
    w TUBSU͔ΒTUPQ൪໨ͷཁૉ͚ͩʹ͢Δɻ0 /

    w -4&5LFZJOEFYWBMVF
    w JOEFY൪໨ͷ஋Λมߋɻ0 /

    w -*/%&9LFZJOEFY
    w JOEFY൪໨ͷཁૉΛऔಘɻ0 /

    w -3ͷόϦΤʔγϣϯ
    w ϒϩοΩϯάͷόϦΤʔγϣϯ #Ͱ͸͡·Δ

    View full-size slide

  28. ©2019 Wantedly, Inc.
    w pFMEWBMVFϖΞͷू߹
    w lΦϒδΣΫτzͬΆ͍΋ͷΛද͢ͷʹศར
    Hash
    > HMSET user:1000 username antirez
    birthyear 1977 verified 1
    OK
    > HGET user:1000 username
    "antirez"
    > HGET user:1000 birthyear
    "1977"
    > HGETALL user:1000
    1) "username"
    2) "antirez"
    3) "birthyear"
    4) "1977"
    5) "verified"
    6) "1"

    View full-size slide

  29. ©2019 Wantedly, Inc.
    w )4&5LFZpFMEWBMVF
    w pFMEʹ஋Λॻ͖ࠐΈɻ0

    w )(&5LFZpFME
    w pMFEͷ஋ΛಡΈࠐΈɻ0

    w )(&5"--LFZ
    w શͯͷpFMEWBMVFϖΞΛಡΈࠐΈɻ0 /

    w ).4&5LFZpFMEWBMVF
    w ෳ਺pFMEʹ஋Λॻ͖ࠐΈɻ0 ࢦఆpFME਺

    w ).(&5LFZpFME
    w ෳ਺pFMEͷ஋ΛಡΈࠐΈɻ0 ࢦఆpFME਺

    Hash
    w )&9*454LFZpFME
    w pFME͕ଘࡏ͢Δ͔Ͳ͏͔ɻ0

    w )%&-LFZpFME
    w pFMEΛ࡟আɻ0 pFME਺

    w )-&/LFZ
    w pFME਺ɻ0

    w )*/$3#:LFZpFMEJODSFNFOU
    w ࢦఆpFMEͷ஋Λ૿ݮɻ0

    View full-size slide

  30. ©2019 Wantedly, Inc.
    w pFME਺͕IBTI@NBY@[JQMJTU@FOUSJFT
    EFGBVMU
    ҎԼ͔ͭ஋ͷ௕͞ͷ࠷େ
    ͕IBTI@NBY@[JQMJTU@WBMVF EFGBVMU

    ҎԼͷ৔߹͸ɺϝϞϦޮ཰ͷΑ
    ͍z[JQMJTUzͰΤϯίʔσΟϯά
    w ͦ͏Ͱͳ͍৔߹͸lEJDUz IBTIUBCMF

    w ϝϞϦͱ$16ͷτϨʔυΦϑ
    Hash
    // t_hash.c
    /* Check the length of a number of objects to see if we need to
    convert a
    * ziplist to a real hash. Note that we only check string encoded
    objects
    * as their string length can be queried in constant time. */
    void hashTypeTryConversion(robj *o, robj **argv, int start, int
    end) {
    int i;
    if (o->encoding != OBJ_ENCODING_ZIPLIST) return;
    for (i = start; i <= end; i++) {
    if (sdsEncodedObject(argv[i]) &&
    sdslen(argv[i]->ptr) > server.hash_max_ziplist_value)
    {
    hashTypeConvert(o, OBJ_ENCODING_HT);
    break;
    }
    }
    }

    View full-size slide

  31. ©2019 Wantedly, Inc.
    ziplist.c
    /* The ziplist is a specially encoded dually linked list that is designed
    * to be very memory efficient. It stores both strings and integer values,
    * where integers are encoded as actual integers instead of a series of
    * characters. It allows push and pop operations on either side of the list
    * in O(1) time. However, because every operation requires a reallocation of
    * the memory used by the ziplist, the actual complexity is related to the
    * amount of memory used by the ziplist.

    View full-size slide

  32. ©2019 Wantedly, Inc.
    w ϢχʔΫͰιʔτ͞Ε͍ͯͳ͍
    4USJOH
    ͷू߹
    w ࠷େཁૉ਺͸?
    Set
    redis> SADD myset "Hello"
    (integer) 1
    redis> SADD myset "World"
    (integer) 1
    redis> SADD myset "World"
    (integer) 0
    redis> SMEMBERS myset
    1) "World"
    2) "Hello"
    redis>

    View full-size slide

  33. ©2019 Wantedly, Inc.
    w 4"%%LFZNFNCFS
    w ཁૉͷ௥Ճɻ0

    w 43&.LFZNFNCFS
    w ཁૉͷ࡟আɻ0

    w 4.&.#&34LFZ
    w શཁૉͷऔಘɻ0 /

    w 4*4.&.#&3LFZNFNCFS
    w ཁૉ͕ू߹ʹଘࡏ͢Δ͔Ͳ͏͔ɻ0

    w 4101LFZ
    w ϥϯμ
    ϜʹཁૉΛऔΓग़͠ɻ0

    Set
    w 46/*0/LFZ
    w ࿨ू߹ɻ0 ཁૉ਺ͷ߹ܭ

    w 4*/5&3LFZ
    w ੵू߹ɻ0 Ұ൪খ͍͞ཁૉ਺TFU਺

    w 4%*''LFZ
    w ࠩू߹ɻ0 ཁૉ਺ͷ߹ܭ

    w 4$"3%LFZ
    w ू߹ͷཁૉ਺ɻ0

    View full-size slide

  34. ©2019 Wantedly, Inc.
    w ੔਺ͷΈͷ৔߹ɺ಺෦తʹ͸ಛผ
    ͳΤϯίʔσΟϯάͰ֨ೲ͞ΕΔ
    w TFUNBYJOUTFUFOUSJFT
    EFGBVMU
    ҎԼͷཁૉ਺ͷ৔
    ߹ɻ
    Set
    // t_set.c
    /* Add the specified value into a set.
    *
    * If the value was already member of the set, nothing is done and 0 is
    * returned, otherwise the new element is added and 1 is returned. */
    int setTypeAdd(robj *subject, sds value) {
    long long llval;
    if (subject->encoding == OBJ_ENCODING_HT) {
    dict *ht = subject->ptr;
    dictEntry *de = dictAddRaw(ht,value,NULL);
    if (de) {
    dictSetKey(ht,de,sdsdup(value));
    dictSetVal(ht,de,NULL);
    return 1;
    }
    } else if (subject->encoding == OBJ_ENCODING_INTSET) {
    if (isSdsRepresentableAsLongLong(value,&llval) == C_OK) {
    uint8_t success = 0;
    subject->ptr = intsetAdd(subject->ptr,llval,&success);
    if (success) {
    /* Convert to regular set when the intset contains
    * too many entries. */
    if (intsetLen(subject->ptr) > server.set_max_intset_entries)
    setTypeConvert(subject,OBJ_ENCODING_HT);
    return 1;
    }
    } else {
    /* Failed to get integer from object, convert to regular set. */
    setTypeConvert(subject,OBJ_ENCODING_HT);
    /* The set *was* an intset and this value is not integer
    * encodable, so dictAdd should always work. */
    serverAssert(dictAdd(subject->ptr,sdsdup(value),NULL) == DICT_OK);
    return 1;
    }
    } else {
    serverPanic("Unknown set encoding");
    }
    return 0;
    }

    View full-size slide

  35. ©2019 Wantedly, Inc.
    redisObjectߏ଄ମ
    // server.h
    /* Objects encoding. Some kind of objects like Strings and Hashes can be
    * internally represented in multiple ways. The 'encoding' field of the object
    * is set to one of this fields for this object. */
    #define OBJ_ENCODING_RAW 0 /* Raw representation */
    #define OBJ_ENCODING_INT 1 /* Encoded as integer */
    #define OBJ_ENCODING_HT 2 /* Encoded as hash table */
    #define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */
    #define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */
    #define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */
    #define OBJ_ENCODING_INTSET 6 /* Encoded as intset */
    #define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */
    #define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */
    #define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */
    #define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */
    typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
    * LFU data (least significant 8 bits frequency
    * and most significant 16 bits access time). */
    int refcount;
    void *ptr;
    } robj;

    View full-size slide

  36. ©2019 Wantedly, Inc.
    w είΞॱʹιʔτ͞Ε͍ͯΔ4FU
    w είΞ͸CJUුಈখ਺఺਺
    w είΞ͕ಉ͡৔߹͸ཁૉͷࣙॻࣜॱং
    w [TFUNBY[JQMJTUFOUSJFT EFGBVMU

    ͔ͭ[TFUNBY[JQMJTUWBMVF
    EFGBVMU
    Λຬͨ͢৔߹͸[JQMJTU
    w *46$0/Ͱ3%#.4ͷUBCMFΛࡶʹஔ͖
    ׵͑Δͱ͖ʹΑ͘࢖͏
    SortedSet
    redis> ZADD myzset 1 "one"
    (integer) 1
    redis> ZADD myzset 1 "uno"
    (integer) 1
    redis> ZADD myzset 2 "two" 3 "three"
    (integer) 2
    redis> ZRANGE myzset 0 -1 WITHSCORES
    1) "one"
    2) "1"
    3) "uno"
    4) "1"
    5) "two"
    6) "2"
    7) "three"
    8) "3"
    redis>

    View full-size slide

  37. ©2019 Wantedly, Inc.
    3FEJT0CKFDU͔ΒείΞ΁ͷϚοϓͷ
    IBTIUBCMFͱɺείΞ͔Β3FEJT0CKFDU
    ΁ͷϚοϓͷTLJQMJTU͔ΒͳΔ
    SortedSet
    // server.h
    /* ZSETs use a specialized version of Skiplists
    */
    typedef struct zskiplistNode {
    sds ele;
    double score;
    struct zskiplistNode *backward;
    struct zskiplistLevel {
    struct zskiplistNode *forward;
    unsigned long span;
    } level[];
    } zskiplistNode;
    typedef struct zskiplist {
    struct zskiplistNode *header, *tail;
    unsigned long length;
    int level;
    } zskiplist;
    typedef struct zset {
    dict *dict;
    zskiplist *zsl;
    } zset;
    ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU

    View full-size slide

  38. ©2019 Wantedly, Inc.
    Skip List
    ૠೖͱ୳ࡧͱ࡟আͷ࣌ؒܭࢉྔ͸ฏۉ0 MPH /

    ɺ࠷ѱ0 /

    Ξχϝʔγϣϯ(*'ͳͷͰݩαΠτͰ֬ೝ͍ͯͩ͘͠͞

    ग़యIUUQTFOXJLJQFEJBPSHXJLJ4LJQ@MJTU

    View full-size slide

  39. ©2019 Wantedly, Inc.
    w ;"%%LFZ9c99><$)><*/$3>TDPSFNFNCFS
    w ཁૉͷ௥Ճɻ0 MPH /


    w 99ŠطଘͷཁૉͷείΞߋ৽ͷΈɻ
    w /9Š৽͍͠ཁૉͷ௥ՃͷΈɻ
    w $)ŠείΞߋ৽͞Εͨཁૉ਺Λ໭Γ஋ʹؚΉ ௨ৗ͸৽͘͠௥Ճ͞Εͨ਺
    ɻ
    w */$3ŠείΞͷมԽࠩ෼Λࢦఆɻ
    w ;4$03&LFZNFNCFS
    w ࢦఆͨ͠ཁૉͷείΞɻ0

    w ;$"3%LFZ
    w ཁૉ਺ɻ0

    SortedSet

    View full-size slide

  40. ©2019 Wantedly, Inc.
    w ;$06/5LFZNJONBY
    w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉ਺ɻ0 MPH /


    w ;3"/,LFZNFNCFS
    w ࢦఆͨ͠ϝϯόʔ͕Կ൪໨͔ɻ0 MPH /


    w ;3"/(&LFZTUBSUTUPQ<8*5)4$03&4>
    w TUBSU͔ΒTUPQ൪໨ͷཁૉΛऔಘɻ0 MPH શཁૉ਺
    औಘཁૉ਺

    w ;3"/(:4$03&LFZNJONBY<8*5)4$03&4><-*.*5PGGTFUDPVOU>
    w ࢦఆͨ͠ൣғ಺ͷείΞͷཁૉΛऔಘɻ0 MPH શཁૉ਺
    औಘཁૉ਺

    SortedSet

    View full-size slide

  41. ©2019 Wantedly, Inc.
    w lBQSPCBCJMJTUJDEBUBTUSVDUVSFXIJDI
    JTVTFEJOPSEFSUPFTUJNBUFUIF
    DBSEJOBMJUZPGBTFUz
    w ϢχʔΫͳཁૉ਺Λ਺͑Δͷʹ࢖͏
    w ඪ४ޡࠩͰਪଌ
    w ϝϞϦ࢖༻ྔ͸ݻఆͰ࠷ѱέʔεͰL
    CZUFT
    HyperLogLog
    HyperLogLogs in Redis
    https://thoughtbot.com/blog/hyperloglogs-in-redis

    View full-size slide

  42. ©2019 Wantedly, Inc.
    require 'redis'
    require 'sinatra'
    $redis = Redis.new
    class VisitorCounter
    def initialize(app)
    @app = app
    end
    def call(env)
    $redis.pfadd 'visitors', Rack::Request.new(env).ip
    @app.call(env)
    end
    end
    use VisitorCounter
    get '/' do
    visitors = $redis.pfcount 'visitors'
    "This website has been visited by #{visitors} unique visitors."
    end
    HyperLogLog
    ग़య: https://thoughtbot.com/blog/hyperloglogs-in-redis

    View full-size slide

  43. ©2019 Wantedly, Inc.
    -JTU 4FU 4PSUFE4FU )BTIͷσʔλܕʹର͠ɺͦΕͧΕͷί
    Ϛϯυ͕ظ଴͢Δ࣌ؒܭࢉྔͰ͋Δ͜ͱΛ֬ೝ͢Δɻ
    ͕࣌ؒ͋ͬͨΒ͜͜Ͱԋश
    require 'redis'
    redis = Redis.new(host: "localhost", port: 6379)
    redis.set("mykey", "hello world")
    # => "OK"
    redis.get("mykey")
    # => "hello world"

    View full-size slide

  44. ©2019 Wantedly, Inc.
    Pipelining
    require 'redis'
    def without_pipelining
    r = Redis.new
    10000.times {
    r.ping
    }
    end
    def with_pipelining
    r = Redis.new
    r.pipelined {
    10000.times {
    r.ping
    }
    }
    end
    def bench(descr)
    start = Time.now
    yield
    puts "#{descr} #{Time.now-start} seconds"
    end
    bench("without pipelining") {
    without_pipelining
    }
    bench("with pipelining") {
    with_pipelining
    }
    # without pipelining 1.185238 seconds
    # with pipelining 0.250783 seconds
    Using pipelining to speedup Redis queries
    https://redis.io/topics/pipelining
    355ͱγεςϜίʔϧͷΦʔόʔϔου࡟ݮ

    View full-size slide

  45. ©2019 Wantedly, Inc.
    Transaction
    > MULTI
    OK
    > INCR foo
    QUEUED
    > INCR bar
    QUEUED
    > EXEC
    1) (integer) 1
    2) (integer) 1
    https://redis.io/topics/transactions
    ෳ਺ίϚϯυΛΞτϛοΫʹ࣮ߦ
    w .6-5*ίϚϯυͰ໋ྩΛΩϡʔΠϯά
    w &9&$ίϚϯυͰBUPNJDʹ࣮ߦ
    w SPMMCBDL͸ͳ͍

    View full-size slide

  46. ©2019 Wantedly, Inc.
    Lua Scripting
    w ༨༟͕͋Ε͹ॻ͘

    View full-size slide

  47. ©2019 Wantedly, Inc.
    PubSub
    w ༨༟͕͋Ε͹ॻ͘

    View full-size slide

  48. ©2019 Wantedly, Inc.
    Redis Module
    w ໋ྩΛ֦ுͰ͖Δ
    w IUUQTSFEJTJPUPQJDTNPEVMFTJOUSP
    w IUUQTSFEJTMBCTDPNDPNNVOJUZSFEJTNPEVMFTIVC
    w ͕࣌ؒ͋Ε͹͜͜Ͱԋश
    w ద౰ͳ໋ྩΛࣗ࡞͢Δ
    w IUUQTHJUIVCDPN3FEJT-BCT3FEJT.PEVMFT4%,

    View full-size slide

  49. ©2019 Wantedly, Inc.
    ࣮ࡍͷ࢖͍ํ
    Usage on Real Projects

    View full-size slide

  50. ©2019 Wantedly, Inc.
    ࣮ࡍͷ࢖͍ํ ओͳ༻్
    3BJMTͷඇಉظδϣϒΩϡʔ
    3BJMTDBDIFͷόοΫΤϯυ
    ੜ3FEJTΛDBDIF·ͨ͸EBUBCBTFͱͯ͠

    View full-size slide

  51. ©2019 Wantedly, Inc.
    Sidekiq/ActiveJob
    class HardWorker
    include Sidekiq::Worker
    def perform(name, count)
    # do something
    end
    end
    HardWorker.perform_async('bob', 5)
    HardWorker.perform_in(5.minutes, 'bob', 5)
    HardWorker.perform_at(5.minutes.from_now, 'bob', 5)
    ग़య: https://github.com/mperham/sidekiq/wiki/Getting-Started
    3BJMTͷඇಉظॲཧͷόοΫΤϯυͱͯ͠

    View full-size slide

  52. ©2019 Wantedly, Inc.
    Sidekiq/ActiveJob ͲͪΒΛ͔ͭ͏͔ʁ
    w "DUJWF+PCͰ࢖͏৔߹
    w "3ͷΠϯελϯεΛ౉ͤΔ EFTFSJBMJ[F࣌ʹA.PEFMpOE JE
    AΛ΍͍ͬͯΔ

    w (MPCBM*%4FSJBMJ[BUJPOIUUQTHJUIVCDPNSBJMTHMPCBMJE
    w "DUJPO.BJMFSͱ૬ੑ͕Α͍ EFMJWFS@MBUFS

    w ։ൃ؀ڥͰ3FEJTͳ͠Ͱ࢖͑Δ
    w ผSVCZUISFBEͰͷ࣮ߦͱͳΔ
    w +PC࡞੒࣌ͷA*OMPDBMFAΛอଘͯ͠ɺͦͷϩέʔϧͰ+PCΛ࣮ߦͯ͘͠ΕΔ
    w ੜ4JEFLJRΛ࢖͏৔߹
    w 4JEFLJRͷػೳΛϑϧʹ࢖͑Δ
    w EF
    TFSJBMJ[FͷΦʔόʔϔου͕ݮΔ

    View full-size slide

  53. ©2019 Wantedly, Inc.
    Sidekiq/ActiveJob ࢖͍ํͷίπ
    w ग़དྷΔݶΓॲཧΛႈ౳ʹ͍͕ͨ͠ɺݱ࣮͸ͦ͏͸͍͔ͳ͍
    w σϑΥϧτͰ͸ϦτϥΠ͠ͳ͍Α͏ʹઃఆ͢Δ΄͏͕҆શ
    w ॲཧʹ͕͔͔࣌ؒΓλΠϜΞ΢τ͢Δ͜ͱ͕͋Δ
    w ϝʔϧ΍ϓογϡ௨஌Λ౓ૹ৴͢ΔͱϢʔβମݧ͕ѱ͍
    w ૢ࡞͕ႈ౳ͰϦτϥΠ͍ͨ͠΍ͭ͸໌ࣔతʹࢦఆ͢Δ
    w 4JEFLJREFGBVMU@XPSLFS@PQUJPOT<SFUSZ>GBMTF
    w "DUJWF3FDPSEͷίʔϧόοΫͰΤϯΩϡʔ͢Δ৔߹͸ɺBGUFS@DPNNJUͰ΍Δ
    w BGUFS@DSFBUF΍BGUFS@VQEBUFͰΤϯΩϡʔ͢Δͱɺ3%#ʹDPNNJU͞ΕΔ·͑ʹɺ
    KPC͕࣮ߦ͞Εͯ͠·͏͜ͱ͕͋Δ
    w TJEFLJRϓϩηε΁ͷ4*(5&3.ͱ4*(,*--ͷؒ LTͷEFGBVMU͸T
    ʹUJNFPVUͤ͞Δ
    https://github.com/wantedly/dev-docs/blob/master/sidekiq.md

    View full-size slide

  54. ©2019 Wantedly, Inc.
    3BJMTDBDIFͷόοΫΤϯυͱͯ͠
    # config/environments/production.rb
    config.cache_store = :redis_store,
    ENV["REDIS_RAILS_CACHE_URL"],
    { compress: true, expires_in: 90.minutes }
    w .BSTIBMEVNQ .BSTIBMMPBE
    Ͱ EF
    TFSJBMJ[F
    w 1SJNJUJWFͬΆ͘ͳ͍σʔλܕΛ͍Εͳ͍
    w ൵͍͠ࣄނͷྫXBOUFEMZQPTUNPSUFNT
    wantedly/post-mortems#40

    View full-size slide

  55. ©2019 Wantedly, Inc.
    ੜ3FEJT
    # config/initializers/redis.rb
    REDIS_CONFIG = {
    host: ENV['REDIS_HOST'].presence || 'localhost',
    port: ENV['REDIS_PORT'].presence || '6379',
    connect_timeout: 0.2,
    read_timeout: 1.0,
    write_timeout: 3.0,
    }
    redis_conn = Redis.new(REDIS_CONFIG)
    w ίϚϯυ͕ͦͷ··ϝιου໊ʹͳ͍ͬͯΔ
    w ઀ଓͷॳظԽ࣌ʹλΠϜΞ΢τΛઃఆ͓ͯ͘͠

    View full-size slide

  56. ©2019 Wantedly, Inc.
    ຊ൪༻ͷΠϯελϯεΛ࡞੒͢Δ
    # terraform/elasticache_replication_group.tf
    resource "aws_elasticache_replication_group" "users-pubsub-prod" {
    replication_group_id = "users-pubsub-prod"
    replication_group_description = " "
    engine_version = "5.0.0"
    maintenance_window = "tue:18:00-tue:19:00"
    node_type = "cache.r5.large"
    number_cache_clusters = 2
    automatic_failover_enabled = true
    parameter_group_name = "default.redis5.0"
    port = 6379
    subnet_group_name = "${aws_elasticache_subnet_group.wantedly-production-cache.name}"
    security_group_ids = ["${aws_security_group.redis-elasticache-production.id}"]
    notification_topic_arn = "arn:aws:sns:ap-northeast-1:096233016669:dev"
    snapshot_retention_limit = 1
    snapshot_window = "17:00-18:00"
    }
    w "84&MBTUJDBDIFΛ࢖͍ͬͯΔ
    w 5FSSBGPSNͰ࡞੒ XBOUFEMZXBOUFEMZUFSSBGPSN΁QVMMSFRVFTU

    w QBSBNFUFSHSPVQ͕ॏཁͳͷͰɺ*OGSBTRVBEͷαϙʔτ౰൪ʹ૬ஊ͍ͯͩ͘͠͞

    View full-size slide

  57. ©2019 Wantedly, Inc.
    KEYS *ίϚϯυࣄ݅
    wantedly/post-mortems#31

    View full-size slide

  58. ©2019 Wantedly, Inc.
    KEYS *ίϚϯυࣄ݅
    w ,&:4QBUUFSO
    w QBUUFSOʹϚον͢ΔશͯͷΩʔΛฦ͢ɻ0 શLFZ਺

    w l8BSOJOHDPOTJEFS,&:4BTBDPNNBOEUIBUTIPVMEPOMZCFVTFEJOQSPEVDUJPO
    FOWJSPONFOUTXJUIFYUSFNFDBSF*UNBZSVJOQFSGPSNBODFXIFOJUJT
    FYFDVUFEBHBJOTUMBSHFEBUBCBTFT5IJTDPNNBOEJTJOUFOEFEGPSEFCVHHJOH
    BOETQFDJBMPQFSBUJPOT TVDIBTDIBOHJOHZPVSLFZTQBDFMBZPVU%POU
    VTF,&:4JOZPVSSFHVMBSBQQMJDBUJPODPEF*GZPVSFMPPLJOHGPSBXBZUPpOE
    LFZTJOBTVCTFUPGZPVSLFZTQBDF DPOTJEFSVTJOH4$"/PSTFUTz

    View full-size slide

  59. ©2019 Wantedly, Inc.
    ڊେHashࣄ݅
    wantedly/post-mortems#25

    View full-size slide

  60. ©2019 Wantedly, Inc.
    ڊେHashࣄ݅
    wantedly/infrastructure#4102

    View full-size slide

  61. ©2019 Wantedly, Inc.
    ίϚϯυςʔϒϧ
    struct redisCommand redisCommandTable[] = {
    {"module",moduleCommand,-2,"as",0,NULL,0,0,0,0,0},
    {"get",getCommand,2,"rF",0,NULL,1,1,1,0,0},
    {"set",setCommand,-3,"wm",0,NULL,1,1,1,0,0},
    {"setnx",setnxCommand,3,"wmF",0,NULL,1,1,1,0,0},
    {"setex",setexCommand,4,"wm",0,NULL,1,1,1,0,0},
    {"psetex",psetexCommand,4,"wm",0,NULL,1,1,1,0,0},
    {"append",appendCommand,3,"wm",0,NULL,1,1,1,0,0},
    {"strlen",strlenCommand,2,"rF",0,NULL,1,1,1,0,0},
    {"del",delCommand,-2,"w",0,NULL,1,-1,1,0,0},
    {"unlink",unlinkCommand,-2,"wF",0,NULL,1,-1,1,0,0},
    {"exists",existsCommand,-2,"rF",0,NULL,1,-1,1,0,0},
    {"setbit",setbitCommand,4,"wm",0,NULL,1,1,1,0,0},
    {“getbit",getbitCommand,3,"rF",0,NULL,1,1,1,0,0},
    {"bitfield",bitfieldCommand,-2,"wm",0,NULL,1,1,1,0,0},
    {"setrange",setrangeCommand,4,"wm",0,NULL,1,1,1,0,0},
    {"getrange",getrangeCommand,4,"r",0,NULL,1,1,1,0,0},
    {"substr",getrangeCommand,4,"r",0,NULL,1,1,1,0,0},
    {"incr",incrCommand,2,"wmF",0,NULL,1,1,1,0,0},
    {"decr",decrCommand,2,"wmF",0,NULL,1,1,1,0,0},
    server.c

    View full-size slide

  62. ©2019 Wantedly, Inc.
    ·ͱΊ
    Summary

    View full-size slide

  63. ©2019 Wantedly, Inc.
    3FEJTͷ֓ཁ
    ࣾ಺Ͱͷར༻ํ๏
    ਖ਼͍͠༻๏༻ྔ
    ͜ͷݚमͰֶΜͩ͜ͱ

    View full-size slide

  64. ©2019 Wantedly, Inc.
    w ιʔείʔυΛோΊΔ͜ͱͰগ͚ͩ͠ਂ͘3FEJTʹ͍ͭͯཧղͰ͖ͨ
    w ීஈ࢖͍ͬͯΔιϑτ΢ΣΞʹ΋୭͔͕ॻ͍ͨιʔείʔυ͕ଘࡏ͢Δ
    w ϑϨʔϜϫʔΫɺϥΠϒϥϦɺݴޠ࣮૷
    w ϒϥοΫϘοΫεԽͤͣʹڵຯΛ࣋ͬͯಡΜͰΈͯ΄͍͠
    w ͦΕ͕ɺιϑτ΢ΣΞΤϯδχΞͱͯ͠ͷ෢ثΛ૿΍͢͜ͱʹͳΔ
    w ·ͨɺ্ͷϨΠϠʔʹ΋ڵຯΛͻΖ͛ͯ΄͍͠
    w Ϣʔβମݧɺ6*ઃܭɺϏδωεϞσϧɺϚʔέςΟϯάɺϓϩδΣΫτϚωʔδϝϯτɺ
    ϓϩμΫτϚωʔδϝϯτɺͳͲͳͲ
    ͍ͭͰʹ఻͔͑ͨͬͨ͜ͱ

    View full-size slide

  65. ©2019 Wantedly, Inc.
    w l8PSLJOH8JUI6OJY1SPDFTTFTz +FTTF4UPSJNFS
    w 6/*9ͷϓϩηεɺϑΝΠϧσΟεΫϦϓλɺγες
    Ϝίʔϧɺγάφϧ͋ͨΓΛܰ͘ཧղ͢ΔͷʹΑ͍ɻ
    w (PͳΒΘ͔ΔγεςϜϓϩάϥϛϯά ौ઒Α͖͠
    w ೔ຊޠͰಡΈ͚ͨΕ͹͜Ε΋ྑ͍ɻ
    w -JOVYϓϩάϥϛϯάΠϯλϑΣʔε .JDIBFM,FSSJTL
    ઍॅ࣏࿠
    w ϖʔδ௒͑ͷࣙॻɻΦϑΟεʹ͋Δɻ
    w ͳΜͰ΋͍͍ͷͰΞϧΰϦζϜͱσʔλߏ଄ͷຊ
    ࢀߟจݙ ͞Βʹਂ͘ཧղ͍ͨ͠ਓ΁

    View full-size slide

  66. ©2019 Wantedly, Inc.
    w "OOJF4QSBUUIUUQTVOTQMBTIDPNQIPUPTUI*@$;"#.:
    Photo Credit

    View full-size slide