Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
(Advanced) Redis
Search
Jan De Poorter
February 12, 2013
Programming
8
400
(Advanced) Redis
Everything we learned at a client project using Redis - Talk given at #ArrrrUG
Jan De Poorter
February 12, 2013
Tweet
Share
Other Decks in Programming
See All in Programming
レガシープロジェクトで最大限AIの恩恵を受けられるようClaude Codeを利用する
tk1351
4
1.6k
CSC305 Summer Lecture 12
javiergs
PRO
0
130
ソフトウェアテスト徹底指南書の紹介
goyoki
1
130
さようなら Date。 ようこそTemporal! 3年間先行利用して得られた知見の共有
8beeeaaat
1
320
ECS初心者の仲間 – TUIツール「e1s」の紹介
keidarcy
0
150
testingを眺める
matumoto
1
130
AIレビュアーをスケールさせるには / Scaling AI Reviewers
technuma
2
240
ProxyによるWindow間RPC機構の構築
syumai
3
780
1から理解するWeb Push
dora1998
2
410
MCPで実現するAIエージェント駆動のNext.jsアプリデバッグ手法
nyatinte
7
1k
時間軸から考えるTerraformを使う理由と留意点
fufuhu
9
3.9k
「手軽で便利」に潜む罠。 Popover API を WCAG 2.2の視点で安全に使うには
taitotnk
0
390
Featured
See All Featured
Raft: Consensus for Rubyists
vanstee
140
7.1k
Building Better People: How to give real-time feedback that sticks.
wjessup
368
19k
Code Review Best Practice
trishagee
70
19k
KATA
mclloyd
32
14k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
111
20k
We Have a Design System, Now What?
morganepeng
53
7.8k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
510
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
A better future with KSS
kneath
239
17k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
31
2.2k
Building Adaptive Systems
keathley
43
2.7k
Transcript
(advanced) Redis Advanced is between brackets because we don’t claim
to be advanced users, we just have gained a lot of experience
Jan De Poorter @defv
Redis?
NoSQL
:-D
No, but srsly
Redis is an in-memory key-value data-structure server
in memory with “best effort” persistence
Data structure server Strings SET, SETNX, GET, INCR, DECR, APPEND,
... Hashes HSET, HGET, HGETALL, HDEL, HLEN, ... Lists LPUSH, LPOP, LINDEX, LLEN, LRANGE, ... Sets & Sorted sets SADD, SPOP, SMEMBERS, SDIFF, ZADD, ...
some libraries Ruby redis-rb (gem) hiredis-rb (gem) Node redis (npm)
hiredis (npm)
Q-Music Our use case was the API for the new
Q-Music website and the iPhone and Android applications.
High volume
Pub-Sub
BG jobs with Sidekiq
Some do’s
Serialize values
2 class QApi::Redis 3 class << self 4 def redis
5 @redis ||= Redis.new(config) 6 end ... 181 def set(key, data) 182 redis.set key, to_json(data) 183 end 184 185 def get(key) 186 from_json redis.get(key) 187 end 188 216 private 217 def from_json(result) 218 case result 219 when Array 220 result.map { |r| from_json(r) } 221 when Hash 222 result 223 when nil 224 nil 225 else 226 MultiJson.decode(result) 227 end 228 rescue MultiJson::DecodeError 229 result 230 end 231 232 def to_json(data) 233 MultiJson.encode(data) 234 end 262 end 263 end
Save references
# Write LPUSH plays {“id”: 42, ”title”:”99 Bottles of Rum”}
LPUSH plays {“id”: 49, ”title”:”The Drunken Sailor”} LPUSH plays {“id”: 42, ”title”:”99 Bottles of Rum”} # Read LRANGE plays 0 -1 don’t do this
it’s fine with 100 items
but becomes huge with 90 000 items
or 1 000 000 Justin Bieber mentions ;-)
# Write SET tracks:42 {“id”: 42, ”title”:”99 Bottles of Rum”}
SET tracks:49 {“id”: 42, ”title”:” The Drunken Sailor”} LPUSH plays tracks:42 LPUSH plays tracks:49 LPUSH plays tracks:42 # Read LRANGE plays 0 -1 MGET tracks:42 tracks:49 do this instead
or in Ruby 82 def list_from_references(list, options) 83 references =
redis.lrange list, 84 options[:start], options[:stop] 85 if references.any? 86 from_json redis.mget(*references) 87 else 88 [] 89 end 90 end
Some don’t’s
RDB vs AOF
RDB does fork()
doubling memory usage
Our dataset was 4GB
We had ~8GB of memory
No BGSAVE’s for 4 days
:-(
appendonly yes
Slow(er) start-up
Can be optimized
Monitoring
data:~$ redis-cli redis > INFO redis_version:2.4.16 ... connected_clients:152 ... used_memory:2744963744
used_memory_human:2.56G used_memory_peak:6150240632 used_memory_peak_human:5.73G ... changes_since_last_save:9576 last_save_time:1360661315 redis >
PUBLISH & SUBSCRIBE overload
WebSocket clients
1 client == 1 connection == 1 subscription
Works perfect in development
Works perfect in staging
but in production...
1 new event
sent to 3500 connected clients
Takes about 0.3 seconds
Up to 4 messages per second
:-(
Move logic to websocket server
:-(
1 type of msg == 1 subscription
:-)
MULTI / EXEC
> MULTI OK > RENAME bl:1 bld:1 QUEUED > RENAME
bl:2 bld:2 QUEUED > EXEC 1) OK 2) OK
Works great on 1000’s of commands
Not so much on 1 million commands...
Don’t exceed ~10k commands in 1 transaction
master/slave
no library support
DIY
We didn’t
Limited connections
Redis < 2.6
/* Max number of fd supported */ #define AE_SETSIZE (1024*10)
No more then ~10k connections allowed
Redis >= 2.6
The sky is the limit
(actually, the linux max FD is the limit)
Questions?