Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Rocks rocks

Reed Allman
December 04, 2014

Rocks rocks

A high level overview with some details of different ways you can torment yourself trying to build a message queue on disk.

Talk: https://www.youtube.com/watch?v=HTjt6oj-RL4

Reed Allman

December 04, 2014

More Decks by Reed Allman

Other Decks in Technology


  1. That slide telling you who I am and why you

    should listen to me: I work at Previously, I was an undergrad at Auburn There’s really no good reason to listen to me
  2. First, a legend: Enqueue: put a message on a queue

    Dequeue: pull a message off a queue, but don’t delete Delete/Ack: delete the message from the queue Timeout: After a timeout, a message goes back onto its queue Count: Increment or decrement queue size (can’t do tail-head)
  3. • Shard on queues • Gossip a little bit •

    Commit Log • Transactions • RPC • Proxies
  4. We’re bottlenecked on http front end / GC now for

    per node throughput. Haven’t even tested rocks’ limits.
  5. I started writing a fancy storage interface queue benchmarker thing

    but ran out of time. If for some reason these things interest you, it’s here: http://github.com/rdallman/rocksrocks