Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Redisの調査についてとrmlp
Konboi
March 22, 2016
Technology
5
3.3k
Redisの調査についてとrmlp
shibuya.go#2 での発表資料にします
Konboi
March 22, 2016
Tweet
Share
More Decks by Konboi
See All by Konboi
konboi
5
2.8k
konboi
1
170
konboi
8
8.3k
konboi
1
5.6k
konboi
1
7.9k
konboi
0
3k
konboi
0
500
Other Decks in Technology
See All in Technology
nakashin1
1
1.1k
udzura
2
240
grapecity_dev
0
130
yaegashi
0
210
viva_tweet_x
1
380
grapecity_dev
0
170
yshr1200
0
170
mito201
1
1.1k
subroh0508
4
220
willnet
12
4k
gamella
3
1.4k
thockin
3
840
Featured
See All Featured
chriscoyier
499
130k
3n
163
22k
michaelherold
224
8.5k
cherdarchuk
71
260k
andyhume
64
3.7k
malarkey
393
61k
zenorocha
296
40k
carmenhchung
30
1.5k
vanstee
117
4.8k
kastner
54
1.9k
addyosmani
1347
190k
hursman
107
9.2k
Transcript
Redisͷௐࠪʹ͍ͭͯͱ rmlp shibuya.go#2 3/22
ࣗݾհ • @Konboi • ໘ന๏ਓΧϠοΫ • ϞόΠϧήʔϜͷ։ൃͱӡ༻
ࠓ͢ࣄ • Redisͷௐࠪ • CPU͕ • ϝϞϦ͕ • 1Ͱ •
rmlpʹ͍ͭͯ
ࠓ͞ͳ͍ࣄ • Redisʹ͍ͭͯͷ • Redisͷνϡʔχϯά • GoΛͬͯͷRedisͷૢ࡞
RedisΛۀͰͬͯΔ ͱ͍͏ํ ✋
Redisͷௐࠪʹ͍ͭͯ
ͷલʹ
ӡ༻͍ͯ͠ΔҊ݅Ͱ RedisͷCPU͕͜ͷ༷ʹ ͳͬͨܦݧ͋Γ·͔͢
ಥൃతʹ͕͋ΔCPUෛՙ
None
!
ʹͳΔʹͭΕ͕ͯ͋Δ CPUෛՙ
None
!
ʹʹߴ͘ͳΔCPUෛՙ
None
!!!
ා͍Ͱ͢Ͷ
ͱ͍͏͜ͱͰຊ
CPUͷෛՙ͕ߴ͍࣌
CPUͷෛՙ͕ߴ͍࣌ • ͳʹ͔ॏ͍ॲཧ͕ߦΘΕ͍ͯΔ͕֬ߴ͍Ͱ͢ • RedisγϯάϧεϨουͳͷͰॏ͍ॲཧ͕͋ΔͱͦΕ͕ऴΘ Δ·Ͱଞͷॲཧͨ͞ΕͯͭΒ͍ • ॏ͍ͱ͍ͬͯϛϦඵͰॲཧऴΘΔͷͰؾ͖ͮʹ͍͘ • devͩͱΞΫηε͕গͳ͍ͷͰ࿐ఄ͠ʹ͍͘
Ͳ͏͢Δ...
slow logΛݟ·͠ΐ͏
# ࠓճݕূͷͨΊ slowlogͷ݅Λ10microඵʹ͍ͯ͠·͢ $ redis-cli -h <host> slowlog get <log
num> 1) 1) (integer) 34990 2) (integer) 1457071463 3) (integer) 11 4) 1) "slowlog" 2) "get" 3) "1" ... 3) 1) (integer) 34988 2) (integer) 1457071448 3) (integer) 51 4) 1) "slowlog" 2) "get" 3) "5"
ݟํ 1) 1) (integer) 34990 2) (integer) 1457071463 3) (integer)
11 4) 1) "slowlog" 2) "get" 3) "1" 1) 1) slowlogͷID 2) ࣮ߦऴྃ࣌ͷtimestamp(UNIXඵ) 3) ࣮ߦ࣌ؒ(microඵ) 4) ࣮ߦίϚϯυͱΦϓγϣϯ
CPUͷෛՙ͕ߴ͍࣌ • ͍͍ͩͨ͜ΕͰ͍(ෛՙͷߴ͍)ίϚϯυ͕͔ΔͣͰ͢ • ޙ֘ͷιʔείʔυΛ͠·͠ΐ͏ • ܦݧ্ keys * େྔͷσʔλಉ࢜ͰͷUNION
DIFF Λ͢Δ ͱCPUͷෛՙ͕άϯάϯ্͕Γ·͢
ϝϞϦ
None
ʹʹ૿͑Δσʔλྔ
͜ΕͰ͋ͱXͰ༰ྔ ͷݶքΛܴ͑ͯ͠·͏
!!!
dumpΛऔΓ rdb tool Λͬͯௐࠪ͠·͠ΐ͏
rdb tool ͱ • sripathikrishnan/redis-rdb-tools • Rdbtools is a parser
for Redis' dump.rdb files. • python • rdbϑΝΠϧ͔Βσʔλͷ༷ʑͳใΛग़ྗͰ͖Δπʔϧ
αϯϓϧ dumpslaveͳͲ͔ΒऔΓ·͠ΐ͏ # αʔόʔ͔Βdump $ redis-cli -h <host> --rdb dump.rdb
# memoryͱkeyΛग़ྗ $ rdb -c memory dump.rdb > dump_memory.csv # memoryͷେ͖͞ॱʹsort $ cat dump_memory.csv | awk -F "," '{print $4 "\t" $3'} | sort -n
ϝϞϦ • ༰ྔॱʹstore͞Ε͍ͯΔσʔλ͕Θ͔ΔͷͰ֘ͷσʔλΛ ফ͢ • ඞཁແ͘ͳͬͨσʔλΛআ͢Δ • ܦݧ্ temporary తͳσʔλ͕͍ͬͯΔࣄ͕ଟ͍
• SET <key> <VALUE> "EX" <EXPIRE_TIME> • ͳͲexpire͢Δ࣌ؒΛ͜·Ίʹࢦఆ • ফ͜͠Έॲཧ͕࣮͞Ε͍ͯΔ͔֬ೝ
͖ͪΜͱফ͜͠ΈΛߦ͏ Α͏ʹͨ݁͠Ռ
None
! ! !
ͦΕͰ1Ͱݶք͕...
ׂ͝ͱʹෳʹ͠Α͏ʂ
ׂ͝ͱʹෳʹ͠Α͏ • session༻ • cache༻ • ϥϯΩϯά࡞༻ • ͳͲͳͲ
Ͱͪΐͬͱͬͯ ✋
ͦͷׂຊʹޮ͘...?
ͦͷׂຊʹޮ͘...? • $ redis-cli slowlog get Ͱ͍ίϚϯυ͕Θ͔Δ • rdb tool
Ͱ৭ʑղੳͰ͖Δ • ͜Ε͚ͩͰŊ֤keyͷར༻ස·Ͱ͔Βͳ͍ • ಛఆͷΠϕϯτͷ͚࣌ͩଟ͍͔͠Εͳ͍ • ͋Δkeyͷ࣮ߦ͕΄͔ͷkeyͷ10ഒҎ্͔͠Εͳ͍ • ͳͲ
redis-cli monitor
redis-cli monitor $ redis-cli monitor 1457077479.354522 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__"
1457077479.354621 [0 127.0.0.1:59247] "INCR" 1457077479.352548 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx""counter:__rand_int__" 1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.352812 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx" 1457077479.354522 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__" 1457077479.354621 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__" 1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.353138 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx" ݱࡏ࣮ߦ͞Ε͍ͯΔίϚϯυ͕͔Δ एׯෛՙ͕͕͋ΔͷͰؾΛ͚࣮ͭͯߦ͍ͯͩ͘͠͞
1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.354956 [0 127.0.0.1:59248] "LPUSH"
"mylist" "xxx" <command start timestamp> [<DB> <HOST>] <COMMAND> <KEY> ͳͷͰ2ߦ͔Β1ߦͷtimestampΛҾ͚1ߦͷ࣮ߦ࣌ؒ ͕Θ͔Δ (ݫີͰͳ͍)
ίϚϯυϥΠϯͰΔͱ $ awk 'function fetchcommand() { c =""; for (i
= 4; i < NF; i++) { c = c " " $i }; c = c " " $NF; return c } NR==1 { time=$1; command=fetchcommand() } NR!=1 { print command, $1-time; time=$1; command=fetch_command(); }' /tmp/redis.log surported by @tkuchiki
ίϚϯυʹ͠·ͨ͠
ͬͱGoͷΛ ͪΐΖͬͱ͠·͢
rmlp
rmlp • Konboi/rmlp • Redis Monitor Log Profiler • Go
• ઌ΄ͲͷlogΛݟ͍͢ܗʹܗ • ίϚϯυͷ ฏۉॱ ࣮ߦॱ ࣮ߦճॱ ͰιʔτՄೳ
$ redis-cli monitor > redis-monitor.log $ rmlp -f redis-monitor.log Overall
Stats ================================== LineCount 15000 Commands Rate ================================== LRANGE 4000 PING 2000 ... Heavy Commands ================================== Command Sum(msec) LRANGE 0.803558 PING 0.106445 ... Slowest Calls ================================== KEY Count Max(msec) Avg(msec) PING 2000 0.001511 0.000053 LRANGE mylist 4000 0.001358 0.000201
rmlp • खݩ(Mac)Ͱσόοάͯ͠Ϗϧυ͢Εຊ൪(EC2)Ͱಈ͘࠷ߴ • Ͳͷkey͕ͲͷఔͷසͰൃߦ͞Ε͍ͯΔͷ͔ѲͰ͖Δ • ࿈ଓతʹredis͕ར༻͞Ε͍ͯͳ͍ͱਖ਼֬ͳܭଌ͕Ͱ͖ͳ͍ • ࣍ͷίϚϯυͷ։࢝࣌ؒΛݩʹ࣮ߦ࣌ؒΛग़͍ͯ͠ΔͨΊ •
͚Δඞཁੑ͕͋Δͷ͔ͷஅࡐྉʹͰ͖Δ͔ͳͱ
rmlp • facebookarchive/redis-faina • python • hirose31/redis-traffic-stats • tcpdump࣮ͯ͠ߦ࣌ؒͱසΛܭଌ͢Δπʔϧ͋Δ
ҙࣄ߲ • ࠓճग़ͨ͠ௐࠪͷྫ͋͘·ͰҰྫͰ͢ͷͰଞʹௐࠪํ๏ ͋Δͱࢥ͍·͢ • dumpͱ͔ෛՙͰ࣮αʔϏε͕Өڹͳ͍ঢ়گͰͦ͏ͧ !
շదͳRedisੜ׆Λʂ
! ͝੩ௌ ! ͋Γ͕ͱ͏͍͟͝·ͨ͠