Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LXJS 2013: backpack — scalable photo storage

LXJS 2013: backpack — scalable photo storage

Ivan Babrou

October 02, 2013

Transcript

78 slides
  1. 16 photos on main page up to 200 in feed

    many small “previews”
  2. redis 3.5gb data + index 3.5gb data + index 3.5gb

    data + index memory disk all together:
  3. NO!

  4. SHARDS (aka buckets) backpack #1 backpack #2 backpack #3 backpack

    #4 backpack #5 backpack #6 shard #1 (50%) shard #2 (50%) 1:lol.jpg 2:wtf.jpg
  5. ADDING SHARD backpack #1 backpack #3 backpack #3 backpack #4

    backpack #5 backpack #6 1:lol.jpg 1:wtf.jpg shard #1 (50%) shard #2 (50%) backpack #7 backpack #8 backpack #9 50% chance shard #3 (0%)
  6. backpack #1 shard #1 lol.jpg backpack #1 lol.jpg backpack #1

    lol.jpg server #1 server #1 server #1 coordinator replicator
  7. backpack #1 backpack #4 backpack #2 backpack #5 backpack #3

    backpack #6 backpack #7 backpack #8 backpack #9 server #1 server #2 server #3 zookeeper #1 zookeeper #2 zookeeper #3 redis-queue #1 redis-queue #2 redis-queue #3 coordinator #1 coordinator #2 coordinator #3 replicator #1 replicator #2 replicator #3

More Decks by Ivan Babrou

Other Decks in Programming