Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Redisの調査についてとrmlp
Konboi
March 22, 2016
Technology
5
3.8k
Redisの調査についてとrmlp
shibuya.go#2 での発表資料にします
Konboi
March 22, 2016
Tweet
Share
More Decks by Konboi
See All by Konboi
煩雑な運用を Goを使って楽にする / Go Conference 2019 Spring
konboi
4
3.4k
あなたってどんな方(型)?/ kamakura.go#4
konboi
1
360
中規模チームを支える自動化とノウハウ共有の仕組み/CEDEC2017
konboi
8
8.9k
golang tokyo #6 / ゲーム開発には欠かせない?! あれをシュッと見る
konboi
1
6.5k
git-schemlexとddl-makerを使ったDB migrationの紹介 / git-schemalex and ddl-maker migration #golangtokyo
konboi
1
8.9k
カヤックの新卒研修 #師弟登壇2015
konboi
0
3.4k
開発環境をどっかんどっかん立てる
konboi
0
540
Other Decks in Technology
See All in Technology
データベースの発表には RDBMS 以外もありますよ
maroon1st
0
230
データエンジニアを助けてくれるFivetranとSnowflakeの仕様&機能のご紹介
sagara
0
450
230125 古いタブレットの活用 かーでぃさん
comucal
PRO
0
9.9k
日本ディープラーニング協会主催 NeurIPS 2022 技術報告会講演資料
tdailab
0
1k
re:Inventで発表があったIoT事例の紹介と考察
kizawa2020
0
160
経営統合をきっかけに会社をエンジニアリングした話 / btconjp-2023
carta_engineering
0
110
NGINXENG JP#2 - 3-NGINX Plus・プロダクトのアップデート
hiropo20
0
180
IoT から見る AWS re:invent 2022 ― AWSのIoTの歴史を添えて/Point of view the AWS re:invent 2022 with IoT - with a history of IoT in AWS
ma2shita
0
220
OpenShiftでスポットVMを使おう.pdf
jpishikawa
1
210
ECテックカンファレンス2023 EC事業部のモバイル開発2023
tatsumi0000
0
180
データ分析基盤の要件分析の話(202201_JEDAI)
yabooun
0
180
プログラミング支援AI GitHub Copilot すごいの話
moyashi
0
280
Featured
See All Featured
Designing Dashboards & Data Visualisations in Web Apps
destraynor
224
50k
How STYLIGHT went responsive
nonsquared
89
4.2k
Bash Introduction
62gerente
601
210k
What’s in a name? Adding method to the madness
productmarketing
12
1.9k
Making Projects Easy
brettharned
102
4.8k
A Philosophy of Restraint
colly
193
15k
5 minutes of I Can Smell Your CMS
philhawksworth
198
18k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
101
6.2k
How To Stay Up To Date on Web Technology
chriscoyier
779
250k
Thoughts on Productivity
jonyablonski
49
2.7k
Rebuilding a faster, lazier Slack
samanthasiow
69
7.5k
Intergalactic Javascript Robots from Outer Space
tanoku
261
26k
Transcript
Redisͷௐࠪʹ͍ͭͯͱ rmlp shibuya.go#2 3/22
ࣗݾհ • @Konboi • ໘ന๏ਓΧϠοΫ • ϞόΠϧήʔϜͷ։ൃͱӡ༻
ࠓ͢ࣄ • Redisͷௐࠪ • CPU͕ • ϝϞϦ͕ • 1Ͱ •
rmlpʹ͍ͭͯ
ࠓ͞ͳ͍ࣄ • Redisʹ͍ͭͯͷ • Redisͷνϡʔχϯά • GoΛͬͯͷRedisͷૢ࡞
RedisΛۀͰͬͯΔ ͱ͍͏ํ ✋
Redisͷௐࠪʹ͍ͭͯ
ͷલʹ
ӡ༻͍ͯ͠ΔҊ݅Ͱ RedisͷCPU͕͜ͷ༷ʹ ͳͬͨܦݧ͋Γ·͔͢
ಥൃతʹ͕͋ΔCPUෛՙ
None
!
ʹͳΔʹͭΕ͕ͯ͋Δ CPUෛՙ
None
!
ʹʹߴ͘ͳΔCPUෛՙ
None
!!!
ා͍Ͱ͢Ͷ
ͱ͍͏͜ͱͰຊ
CPUͷෛՙ͕ߴ͍࣌
CPUͷෛՙ͕ߴ͍࣌ • ͳʹ͔ॏ͍ॲཧ͕ߦΘΕ͍ͯΔ͕֬ߴ͍Ͱ͢ • RedisγϯάϧεϨουͳͷͰॏ͍ॲཧ͕͋ΔͱͦΕ͕ऴΘ Δ·Ͱଞͷॲཧͨ͞ΕͯͭΒ͍ • ॏ͍ͱ͍ͬͯϛϦඵͰॲཧऴΘΔͷͰؾ͖ͮʹ͍͘ • devͩͱΞΫηε͕গͳ͍ͷͰ࿐ఄ͠ʹ͍͘
Ͳ͏͢Δ...
slow logΛݟ·͠ΐ͏
# ࠓճݕূͷͨΊ slowlogͷ݅Λ10microඵʹ͍ͯ͠·͢ $ redis-cli -h <host> slowlog get <log
num> 1) 1) (integer) 34990 2) (integer) 1457071463 3) (integer) 11 4) 1) "slowlog" 2) "get" 3) "1" ... 3) 1) (integer) 34988 2) (integer) 1457071448 3) (integer) 51 4) 1) "slowlog" 2) "get" 3) "5"
ݟํ 1) 1) (integer) 34990 2) (integer) 1457071463 3) (integer)
11 4) 1) "slowlog" 2) "get" 3) "1" 1) 1) slowlogͷID 2) ࣮ߦऴྃ࣌ͷtimestamp(UNIXඵ) 3) ࣮ߦ࣌ؒ(microඵ) 4) ࣮ߦίϚϯυͱΦϓγϣϯ
CPUͷෛՙ͕ߴ͍࣌ • ͍͍ͩͨ͜ΕͰ͍(ෛՙͷߴ͍)ίϚϯυ͕͔ΔͣͰ͢ • ޙ֘ͷιʔείʔυΛ͠·͠ΐ͏ • ܦݧ্ keys * େྔͷσʔλಉ࢜ͰͷUNION
DIFF Λ͢Δ ͱCPUͷෛՙ͕άϯάϯ্͕Γ·͢
ϝϞϦ
None
ʹʹ૿͑Δσʔλྔ
͜ΕͰ͋ͱXͰ༰ྔ ͷݶքΛܴ͑ͯ͠·͏
!!!
dumpΛऔΓ rdb tool Λͬͯௐࠪ͠·͠ΐ͏
rdb tool ͱ • sripathikrishnan/redis-rdb-tools • Rdbtools is a parser
for Redis' dump.rdb files. • python • rdbϑΝΠϧ͔Βσʔλͷ༷ʑͳใΛग़ྗͰ͖Δπʔϧ
αϯϓϧ dumpslaveͳͲ͔ΒऔΓ·͠ΐ͏ # αʔόʔ͔Βdump $ redis-cli -h <host> --rdb dump.rdb
# memoryͱkeyΛग़ྗ $ rdb -c memory dump.rdb > dump_memory.csv # memoryͷେ͖͞ॱʹsort $ cat dump_memory.csv | awk -F "," '{print $4 "\t" $3'} | sort -n
ϝϞϦ • ༰ྔॱʹstore͞Ε͍ͯΔσʔλ͕Θ͔ΔͷͰ֘ͷσʔλΛ ফ͢ • ඞཁແ͘ͳͬͨσʔλΛআ͢Δ • ܦݧ্ temporary తͳσʔλ͕͍ͬͯΔࣄ͕ଟ͍
• SET <key> <VALUE> "EX" <EXPIRE_TIME> • ͳͲexpire͢Δ࣌ؒΛ͜·Ίʹࢦఆ • ফ͜͠Έॲཧ͕࣮͞Ε͍ͯΔ͔֬ೝ
͖ͪΜͱফ͜͠ΈΛߦ͏ Α͏ʹͨ݁͠Ռ
None
! ! !
ͦΕͰ1Ͱݶք͕...
ׂ͝ͱʹෳʹ͠Α͏ʂ
ׂ͝ͱʹෳʹ͠Α͏ • session༻ • cache༻ • ϥϯΩϯά࡞༻ • ͳͲͳͲ
Ͱͪΐͬͱͬͯ ✋
ͦͷׂຊʹޮ͘...?
ͦͷׂຊʹޮ͘...? • $ redis-cli slowlog get Ͱ͍ίϚϯυ͕Θ͔Δ • rdb tool
Ͱ৭ʑղੳͰ͖Δ • ͜Ε͚ͩͰŊ֤keyͷར༻ස·Ͱ͔Βͳ͍ • ಛఆͷΠϕϯτͷ͚࣌ͩଟ͍͔͠Εͳ͍ • ͋Δkeyͷ࣮ߦ͕΄͔ͷkeyͷ10ഒҎ্͔͠Εͳ͍ • ͳͲ
redis-cli monitor
redis-cli monitor $ redis-cli monitor 1457077479.354522 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__"
1457077479.354621 [0 127.0.0.1:59247] "INCR" 1457077479.352548 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx""counter:__rand_int__" 1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.352812 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx" 1457077479.354522 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__" 1457077479.354621 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__" 1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.353138 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx" ݱࡏ࣮ߦ͞Ε͍ͯΔίϚϯυ͕͔Δ एׯෛՙ͕͕͋ΔͷͰؾΛ͚࣮ͭͯߦ͍ͯͩ͘͠͞
1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.354956 [0 127.0.0.1:59248] "LPUSH"
"mylist" "xxx" <command start timestamp> [<DB> <HOST>] <COMMAND> <KEY> ͳͷͰ2ߦ͔Β1ߦͷtimestampΛҾ͚1ߦͷ࣮ߦ࣌ؒ ͕Θ͔Δ (ݫີͰͳ͍)
ίϚϯυϥΠϯͰΔͱ $ awk 'function fetchcommand() { c =""; for (i
= 4; i < NF; i++) { c = c " " $i }; c = c " " $NF; return c } NR==1 { time=$1; command=fetchcommand() } NR!=1 { print command, $1-time; time=$1; command=fetch_command(); }' /tmp/redis.log surported by @tkuchiki
ίϚϯυʹ͠·ͨ͠
ͬͱGoͷΛ ͪΐΖͬͱ͠·͢
rmlp
rmlp • Konboi/rmlp • Redis Monitor Log Profiler • Go
• ઌ΄ͲͷlogΛݟ͍͢ܗʹܗ • ίϚϯυͷ ฏۉॱ ࣮ߦॱ ࣮ߦճॱ ͰιʔτՄೳ
$ redis-cli monitor > redis-monitor.log $ rmlp -f redis-monitor.log Overall
Stats ================================== LineCount 15000 Commands Rate ================================== LRANGE 4000 PING 2000 ... Heavy Commands ================================== Command Sum(msec) LRANGE 0.803558 PING 0.106445 ... Slowest Calls ================================== KEY Count Max(msec) Avg(msec) PING 2000 0.001511 0.000053 LRANGE mylist 4000 0.001358 0.000201
rmlp • खݩ(Mac)Ͱσόοάͯ͠Ϗϧυ͢Εຊ൪(EC2)Ͱಈ͘࠷ߴ • Ͳͷkey͕ͲͷఔͷසͰൃߦ͞Ε͍ͯΔͷ͔ѲͰ͖Δ • ࿈ଓతʹredis͕ར༻͞Ε͍ͯͳ͍ͱਖ਼֬ͳܭଌ͕Ͱ͖ͳ͍ • ࣍ͷίϚϯυͷ։࢝࣌ؒΛݩʹ࣮ߦ࣌ؒΛग़͍ͯ͠ΔͨΊ •
͚Δඞཁੑ͕͋Δͷ͔ͷஅࡐྉʹͰ͖Δ͔ͳͱ
rmlp • facebookarchive/redis-faina • python • hirose31/redis-traffic-stats • tcpdump࣮ͯ͠ߦ࣌ؒͱසΛܭଌ͢Δπʔϧ͋Δ
ҙࣄ߲ • ࠓճग़ͨ͠ௐࠪͷྫ͋͘·ͰҰྫͰ͢ͷͰଞʹௐࠪํ๏ ͋Δͱࢥ͍·͢ • dumpͱ͔ෛՙͰ࣮αʔϏε͕Өڹͳ͍ঢ়گͰͦ͏ͧ !
շదͳRedisੜ׆Λʂ
! ͝੩ௌ ! ͋Γ͕ͱ͏͍͟͝·ͨ͠