w WJTJUSFDPNNFOEBUJPO w QFPQMFHP w ZBTIJNBVQEBUFT w OPUJpDBUJPOT w NJSBDMFTFBSDIHP Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔ Datadog APM Service Map https://app.datadoghq.com/service/map?env=production
w ϨεϙϯελΠϜϢʔβମݧʹେ͖ͳӨڹ w ͕ݮΔ w ίϯόʔδϣϯ͕૿͑Δ w ʮγΰτͰίίϩΦυϧͻͱΛ;͢ʯ͜ͱ ͕Ͱ͖Δ Why ͳͥ3FEJTʹֶ͍ͭͯͿͷ͔ ৽ଔݚमͰߦ͏*46$0/ͷັྗ /BP.JOBNJ https://speakerdeck.com/south37/xin-zu-yan-xiu-dexing-u-isucon-falsemei-li?slide=15
multiplexing layer supported by this system. * The following should be ordered by performances, descending. */ #ifdef HAVE_EVPORT #include "ae_evport.c" #else #ifdef HAVE_EPOLL #include "ae_epoll.c" #else #ifdef HAVE_KQUEUE #include "ae_kqueue.c" #else #include "ae_select.c" #endif #endif #endif ࡞ऀۘͷΠϕϯτϥΠϒϥϦ w FWQPSU4PMBSJTҎ߱ w FQPMM-JOVY w LRVFVF#4% w TFMFDUେମ͋Δ
the keys with an expire set. • allkeys-lru: Evict any key using approximated LRU. • volatile-lfu: Evict using approximated LFU among the keys with an expire set. • allkeys-lfu: Evict any key using approximated LFU. • volatile-random: Remove a random key among the ones with an expire set. • allkeys-random: Remove a random key, any key. • volatile-ttl: Remove the key with the nearest expire time (minor TTL) • noeviction: Don't evict anything, just return an error on write operations. Eviction Policy ϝϞϦ͔ΒᷓΕͦ͏ͳ࣌ͷڍಈΛઃఆͰ͖Δ
-101LFZ w ઌ಄Λআɻ0 w --&/LFZ w Ϧετͷ͞ɻ0 List w -*/4&35LFZ#&'03&c"'5&3QJWPUWBMVF w ్தʹૠೖɻ0 / w -53*.LFZTUBSUTUPQ w TUBSU͔ΒTUPQ൪ͷཁૉ͚ͩʹ͢Δɻ0 / w -4&5LFZJOEFYWBMVF w JOEFY൪ͷΛมߋɻ0 / w -*/%&9LFZJOEFY w JOEFY൪ͷཁૉΛऔಘɻ0 / w - 3ͷόϦΤʔγϣϯ w ϒϩοΩϯάͷόϦΤʔγϣϯ #Ͱ͡·Δ
)(&5LFZpFME w pMFEͷΛಡΈࠐΈɻ0 w )(&5"--LFZ w શͯͷpFMEWBMVFϖΞΛಡΈࠐΈɻ0 / w ).4&5LFZpFMEWBMVF<pFMEWBMVF> w ෳpFMEʹΛॻ͖ࠐΈɻ0 ࢦఆpFME w ).(&5LFZpFME<pFME> w ෳpFMEͷΛಡΈࠐΈɻ0 ࢦఆpFME Hash w )&9*454LFZpFME w pFME͕ଘࡏ͢Δ͔Ͳ͏͔ɻ0 w )%&-LFZpFME<pFMEʜ> w pFMEΛআɻ0 pFME w )-&/LFZ w pFMEɻ0 w )*/$3#:LFZpFMEJODSFNFOU w ࢦఆpFMEͷΛ૿ݮɻ0
ҎԼͷ߹ɺϝϞϦޮͷΑ ͍z[JQMJTUzͰΤϯίʔσΟϯά w ͦ͏Ͱͳ͍߹lEJDUz IBTIUBCMF w ϝϞϦͱ$16ͷτϨʔυΦϑ Hash // t_hash.c /* Check the length of a number of objects to see if we need to convert a * ziplist to a real hash. Note that we only check string encoded objects * as their string length can be queried in constant time. */ void hashTypeTryConversion(robj *o, robj **argv, int start, int end) { int i; if (o->encoding != OBJ_ENCODING_ZIPLIST) return; for (i = start; i <= end; i++) { if (sdsEncodedObject(argv[i]) && sdslen(argv[i]->ptr) > server.hash_max_ziplist_value) { hashTypeConvert(o, OBJ_ENCODING_HT); break; } } }
encoded dually linked list that is designed * to be very memory efficient. It stores both strings and integer values, * where integers are encoded as actual integers instead of a series of * characters. It allows push and pop operations on either side of the list * in O(1) time. However, because every operation requires a reallocation of * the memory used by the ziplist, the actual complexity is related to the * amount of memory used by the ziplist.
43&.LFZNFNCFS<NFNCFSʜ> w ཁૉͷআɻ0 w 4.&.#&34LFZ w શཁૉͷऔಘɻ0 / w 4*4.&.#&3LFZNFNCFS w ཁૉ͕ू߹ʹଘࡏ͢Δ͔Ͳ͏͔ɻ0 w 4101LFZ<DPVOU> w ϥϯμ ϜʹཁૉΛऔΓग़͠ɻ0 Set w 46/*0/LFZ<LFZ> w ू߹ɻ0 ཁૉͷ߹ܭ w 4*/5&3LFZ<LFZ> w ੵू߹ɻ0 Ұ൪খ͍͞ཁૉ TFU w 4%*''LFZ<LFZ> w ࠩू߹ɻ0 ཁૉͷ߹ܭ w 4$"3%LFZ w ू߹ͷཁૉɻ0
߹ɻ Set // t_set.c /* Add the specified value into a set. * * If the value was already member of the set, nothing is done and 0 is * returned, otherwise the new element is added and 1 is returned. */ int setTypeAdd(robj *subject, sds value) { long long llval; if (subject->encoding == OBJ_ENCODING_HT) { dict *ht = subject->ptr; dictEntry *de = dictAddRaw(ht,value,NULL); if (de) { dictSetKey(ht,de,sdsdup(value)); dictSetVal(ht,de,NULL); return 1; } } else if (subject->encoding == OBJ_ENCODING_INTSET) { if (isSdsRepresentableAsLongLong(value,&llval) == C_OK) { uint8_t success = 0; subject->ptr = intsetAdd(subject->ptr,llval,&success); if (success) { /* Convert to regular set when the intset contains * too many entries. */ if (intsetLen(subject->ptr) > server.set_max_intset_entries) setTypeConvert(subject,OBJ_ENCODING_HT); return 1; } } else { /* Failed to get integer from object, convert to regular set. */ setTypeConvert(subject,OBJ_ENCODING_HT); /* The set *was* an intset and this value is not integer * encodable, so dictAdd should always work. */ serverAssert(dictAdd(subject->ptr,sdsdup(value),NULL) == DICT_OK); return 1; } } else { serverPanic("Unknown set encoding"); } return 0; }
kind of objects like Strings and Hashes can be * internally represented in multiple ways. The 'encoding' field of the object * is set to one of this fields for this object. */ #define OBJ_ENCODING_RAW 0 /* Raw representation */ #define OBJ_ENCODING_INT 1 /* Encoded as integer */ #define OBJ_ENCODING_HT 2 /* Encoded as hash table */ #define OBJ_ENCODING_ZIPMAP 3 /* Encoded as zipmap */ #define OBJ_ENCODING_LINKEDLIST 4 /* No longer used: old list encoding. */ #define OBJ_ENCODING_ZIPLIST 5 /* Encoded as ziplist */ #define OBJ_ENCODING_INTSET 6 /* Encoded as intset */ #define OBJ_ENCODING_SKIPLIST 7 /* Encoded as skiplist */ #define OBJ_ENCODING_EMBSTR 8 /* Embedded sds string encoding */ #define OBJ_ENCODING_QUICKLIST 9 /* Encoded as linked list of ziplists */ #define OBJ_ENCODING_STREAM 10 /* Encoded as a radix tree of listpacks */ typedef struct redisObject { unsigned type:4; unsigned encoding:4; unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or * LFU data (least significant 8 bits frequency * and most significant 16 bits access time). */ int refcount; void *ptr; } robj;
w 99طଘͷཁૉͷείΞߋ৽ͷΈɻ w /9৽͍͠ཁૉͷՃͷΈɻ w $)είΞߋ৽͞ΕͨཁૉΛΓʹؚΉ ௨ৗ৽͘͠Ճ͞Εͨ ɻ w */$3είΞͷมԽࠩΛࢦఆɻ w ;4$03&LFZNFNCFS w ࢦఆͨ͠ཁૉͷείΞɻ0 w ;$"3%LFZ w ཁૉɻ0 SortedSet
w ;3"/,LFZNFNCFS w ࢦఆͨ͠ϝϯόʔ͕Կ൪͔ɻ0 MPH / w ;3"/(&LFZTUBSUTUPQ<8*5)4$03&4> w TUBSU͔ΒTUPQ൪ͷཁૉΛऔಘɻ0 MPH શཁૉ औಘཁૉ w ;3"/(&#:4$03&LFZNJONBY<8*5)4$03&4><-*.*5PGGTFUDPVOU> w ࢦఆͨ͠ൣғͷείΞͷཁૉΛऔಘɻ0 MPH શཁૉ औಘཁૉ SortedSet
class VisitorCounter def initialize(app) @app = app end def call(env) $redis.pfadd 'visitors', Rack::Request.new(env).ip @app.call(env) end end use VisitorCounter get '/' do visitors = $redis.pfcount 'visitors' "This website has been visited by #{visitors} unique visitors." end HyperLogLog ग़య: https://thoughtbot.com/blog/hyperloglogs-in-redis
JE AΛ͍ͬͯΔ w (MPCBM*%4FSJBMJ[BUJPOIUUQTHJUIVCDPNSBJMTHMPCBMJE w "DUJPO.BJMFSͱ૬ੑ͕Α͍ EFMJWFS@MBUFS w ։ൃڥͰ3FEJTͳ͠Ͱ͑Δ w ผSVCZUISFBEͰͷ࣮ߦͱͳΔ w +PC࡞࣌ͷA*OMPDBMFAΛอଘͯ͠ɺͦͷϩέʔϧͰ+PCΛ࣮ߦͯ͘͠ΕΔ w ੜ4JEFLJRΛ͏߹ w 4JEFLJRͷػೳΛϑϧʹ͑Δ w EF TFSJBMJ[FͷΦʔόʔϔου͕ݮΔ
ॲཧʹ͕͔͔࣌ؒΓλΠϜΞτ͢Δ͜ͱ͕͋Δ w ϝʔϧϓογϡ௨Λૹ৴͢ΔͱϢʔβମݧ͕ѱ͍ w ૢ࡞͕ႈͰϦτϥΠ͍ͨͭ͠໌ࣔతʹࢦఆ͢Δ w 4JEFLJREFGBVMU@XPSLFS@PQUJPOT<SFUSZ> GBMTF w "DUJWF3FDPSEͷίʔϧόοΫͰΤϯΩϡʔ͢Δ߹ɺBGUFS@DPNNJUͰΔ w BGUFS@DSFBUFBGUFS@VQEBUFͰΤϯΩϡʔ͢Δͱɺ3%#ʹDPNNJU͞ΕΔ·͑ʹɺ KPC͕࣮ߦ͞Εͯ͠·͏͜ͱ͕͋Δ w TJEFLJRϓϩηεͷ4*(5&3.ͱ4*(,*--ͷؒ LTͷEFGBVMUT ʹUJNFPVUͤ͞Δ https://github.com/wantedly/dev-docs/blob/master/sidekiq.md
{ compress: true, expires_in: 90.minutes } w .BSTIBMEVNQ .BSTIBMMPBE Ͱ EF TFSJBMJ[F w 1SJNJUJWFͬΆ͘ͳ͍σʔλܕΛ͍Εͳ͍ w ൵͍͠ࣄނͷྫXBOUFEMZQPTUNPSUFNT wantedly/post-mortems#40
ϒϥοΫϘοΫεԽͤͣʹڵຯΛ࣋ͬͯಡΜͰΈͯ΄͍͠ w ͦΕ͕ɺιϑτΣΞΤϯδχΞͱͯ͠ͷثΛ૿͢͜ͱʹͳΔ w ·ͨɺ্ͷϨΠϠʔʹڵຯΛͻΖ͛ͯ΄͍͠ w Ϣʔβମݧɺ6*ઃܭɺϏδωεϞσϧɺϚʔέςΟϯάɺϓϩδΣΫτϚωʔδϝϯτɺ ϓϩμΫτϚωʔδϝϯτɺͳͲͳͲ ͍ͭͰʹ͔͑ͨͬͨ͜ͱ
kawasy
okashoi
kazumain
.png){kind=avatar}
elith
fsubal
ogurotakayuki
shinoa
ivargrimstad
mraible
pospome
honnibal
gunnarmorling
pinkumohikan
quramy
phodgson
andyhume
dougneiner
tanoku
tammyeverts
jonrohan
sferik
zakiyama
rasmusluckow
myddelton
robhawkes
![©2019 Wantedly, Inc. 3BJMTDBDIFͷόοΫΤϯυͱͯ͠ # config/environments/production.rb config.cache_store = :redis_store, ENV["REDIS_RAILS_CACHE_URL"],](https://files.speakerdeck.com/presentations/96002f59f1e74d52b016775de06621f3/slide_53.jpg){kind=link}
![©2019 Wantedly, Inc. ίϚϯυςʔϒϧ struct redisCommand redisCommandTable[] = { {"module",moduleCommand,-2,"as",0,NULL,0,0,0,0,0},](https://files.speakerdeck.com/presentations/96002f59f1e74d52b016775de06621f3/slide_60.jpg){kind=link}
