Redisの調査についてとrmlp

935fa3c3cfe0ae00d891c958864e3de1?s=47 Konboi
March 22, 2016

 Redisの調査についてとrmlp

shibuya.go#2 での発表資料にします

935fa3c3cfe0ae00d891c958864e3de1?s=128

Konboi

March 22, 2016
Tweet

Transcript

  1. 7.
  2. 10.
  3. 11.

    !

  4. 13.
  5. 14.

    !

  6. 16.
  7. 17.

    !!!

  8. 24.

    # ࠓճ͸ݕূͷͨΊ slowlogͷ৚݅Λ10microඵʹ͍ͯ͠·͢ $ redis-cli -h <host> slowlog get <log

    num> 1) 1) (integer) 34990 2) (integer) 1457071463 3) (integer) 11 4) 1) "slowlog" 2) "get" 3) "1" ... 3) 1) (integer) 34988 2) (integer) 1457071448 3) (integer) 51 4) 1) "slowlog" 2) "get" 3) "5"
  9. 25.

    ݟํ 1) 1) (integer) 34990 2) (integer) 1457071463 3) (integer)

    11 4) 1) "slowlog" 2) "get" 3) "1" 1) 1) slowlogͷID 2) ࣮ߦऴྃ࣌ͷtimestamp(UNIXඵ) 3) ࣮ߦ࣌ؒ(microඵ) 4) ࣮ߦίϚϯυͱΦϓγϣϯ
  10. 27.
  11. 28.
  12. 31.

    !!!

  13. 33.

    rdb tool ͱ͸ • sripathikrishnan/redis-rdb-tools • Rdbtools is a parser

    for Redis' dump.rdb files. • python੡ • rdbϑΝΠϧ͔Βσʔλͷ༷ʑͳ৘ใΛग़ྗͰ͖Δπʔϧ
  14. 34.

    αϯϓϧ dump͸slaveͳͲ͔ΒऔΓ·͠ΐ͏ # αʔόʔ͔Βdump $ redis-cli -h <host> --rdb dump.rdb

    # memoryͱkeyΛग़ྗ $ rdb -c memory dump.rdb > dump_memory.csv # memoryͷେ͖͞ॱʹsort $ cat dump_memory.csv | awk -F "," '{print $4 "\t" $3'} | sort -n
  15. 35.

    ϝϞϦ • ༰ྔॱʹstore͞Ε͍ͯΔσʔλ͕Θ͔ΔͷͰ֘౰ͷσʔλΛ ফ͢ • ඞཁແ͘ͳͬͨσʔλΛ࡟আ͢Δ • ܦݧ্ temporary తͳσʔλ͕࢒͍ͬͯΔࣄ͕ଟ͍

    • SET <key> <VALUE> "EX" <EXPIRE_TIME> • ͳͲexpire͢Δ࣌ؒΛ͜·Ίʹࢦఆ • ফ͜͠Έॲཧ͕࣮૷͞Ε͍ͯΔ͔֬ೝ
  16. 37.
  17. 38.
  18. 44.

    ͦͷ෼ׂຊ౰ʹޮ͘...? • $ redis-cli slowlog get Ͱ͸஗͍ίϚϯυ͕Θ͔Δ • rdb tool

    Ͱ৭ʑղੳͰ͖Δ • ͜Ε͚ͩͰ͸Ŋ֤keyͷར༻ස౓·Ͱ͸෼͔Βͳ͍ • ಛఆͷΠϕϯτͷ͚࣌ͩଟ͍͔΋͠Εͳ͍ • ͋Δkeyͷ࣮ߦ͕΄͔ͷkeyͷ10ഒҎ্͔΋͠Εͳ͍ • ͳͲ
  19. 46.

    redis-cli monitor $ redis-cli monitor 1457077479.354522 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__"

    1457077479.354621 [0 127.0.0.1:59247] "INCR" 1457077479.352548 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx""counter:__rand_int__" 1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.352812 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx" 1457077479.354522 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__" 1457077479.354621 [0 127.0.0.1:59247] "INCR" "counter:__rand_int__" 1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.353138 [0 127.0.0.1:59245] "SET" "key:__rand_int__" "xxx" ݱࡏ࣮ߦ͞Ε͍ͯΔίϚϯυ਺͕෼͔Δ एׯෛՙ͕͕͋ΔͷͰؾΛ͚࣮ͭͯߦ͍ͯͩ͘͠͞
  20. 47.

    1457077479.354852 [0 127.0.0.1:59248] "LPUSH" "mylist" "xxx" 1457077479.354956 [0 127.0.0.1:59248] "LPUSH"

    "mylist" "xxx" <command start timestamp> [<DB> <HOST>] <COMMAND> <KEY> ͳͷͰ2ߦ໨͔Β1ߦͷtimestampΛҾ͚͹1ߦ໨ͷ࣮ߦ࣌ؒ ͕Θ͔Δ (ݫີͰ͸ͳ͍)
  21. 48.

    ίϚϯυϥΠϯͰ΍Δͱ $ awk 'function fetchcommand() { c =""; for (i

    = 4; i < NF; i++) { c = c " " $i }; c = c " " $NF; return c } NR==1 { time=$1; command=fetchcommand() } NR!=1 { print command, $1-time; time=$1; command=fetch_command(); }' /tmp/redis.log surported by @tkuchiki
  22. 51.
  23. 52.

    rmlp • Konboi/rmlp • Redis Monitor Log Profiler • Go੡

    • ઌ΄ͲͷlogΛݟ΍͍͢ܗʹ੔ܗ • ίϚϯυͷ ฏۉॱ ࣮ߦॱ ࣮ߦճ਺ॱ ͰιʔτՄೳ
  24. 53.

    $ redis-cli monitor > redis-monitor.log $ rmlp -f redis-monitor.log Overall

    Stats ================================== LineCount 15000 Commands Rate ================================== LRANGE 4000 PING 2000 ... Heavy Commands ================================== Command Sum(msec) LRANGE 0.803558 PING 0.106445 ... Slowest Calls ================================== KEY Count Max(msec) Avg(msec) PING 2000 0.001511 0.000053 LRANGE mylist 4000 0.001358 0.000201