Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Message Queue is Dead, Long Live the Message Queue

4d9dd9bd8d3d4d0ba8af2acc41d14006?s=47 Mathias Meyer
April 22, 2013
Technology
4
590

The Message Queue is Dead, Long Live the Message Queue

Ramblings and thought experiments on message queues

4d9dd9bd8d3d4d0ba8af2acc41d14006?s=128

Mathias Meyer

April 22, 2013
Tweet

More Decks by Mathias Meyer

See All by Mathias Meyer
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
0
890
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
4
210
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
1
250
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
11
1.6k
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
82
14k
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
2
270
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
5
960
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
10
1k
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
6
300

Other Decks in Technology

See All in Technology
F831177123a4ddda9aa1802721f0a0db?s=48 ozontech
0
1.5k
79dcb04101764d7bf66f898dd446838e?s=48 tsukubachallenge
0
140
665aa20c57717b4f34d0635308d272f7?s=48 kwada
0
130
Cdf97a1b1b132c56582773c3ba09a3a6?s=48 rakus_dev
0
280
93dcbac1a838bbec898747bb92aa47cc?s=48 sadonosake
0
1.7k
50fc0fdc2327ecbe7350645812de52e3?s=48 zakiyama
4
1.3k
2219597a939c895e5db6f8b7c94e2859?s=48 pojiro
2
270
012418ca05dff2e4ac6bdf3e739be4e8?s=48 minorun365
1
340
B3d3a2cce932eca144b8c13a63966404?s=48 alblue
2
430
9cc3d9dc67c048c3820f26f30168076e?s=48 jsaseminar
0
110
79dcb04101764d7bf66f898dd446838e?s=48 tsukubachallenge
0
140
4c5769c58c5b504cd196dbc51d1d3858?s=48 dmiyamoto
0
270

Featured

See All Featured
Fde6c3a50ca518a909ef35c22c0ee0e4?s=48 destraynor
148
20k
168ccec72eee0530b818d44f3fedaacf?s=48 shlominoach
178
8.3k
245cee81a9c424266e5e401d844ea881?s=48 lara
596
62k
812e1705ff0f8bc1bcf18587bde687d5?s=48 62gerente
595
210k
465724d73fe3a92c0879fdfb43a3a6f3?s=48 philhawksworth
195
9.1k
6804f1775cb4babfcc3851298566fbce?s=48 tanoku
91
9k
C9b18d9dff88a9dd2393364c2b3b21bd?s=48 colly
191
14k
7edec06b5435e0dac07a353b2cc26253?s=48 geeforr
337
30k
7c8469ee8c9e594c65c59b919626c08d?s=48 scottboms
253
11k
9952dfcd5d338f8a8e7175c8a8f65fb5?s=48 wjessup
342
16k
828ace851b606e1206900f26459f55ad?s=48 speakerdeck
PRO
2
570
433acaea1012b25d97ae66da9b998dad?s=48 malarkey
194
9.1k

Transcript

  1. the message queue is dead railsberry 2013, mathias meyer, @roidrage

  2. travis-ci.org

  3. None
  4. None
  5. None

  6. the golden rules of message queues

  7. #1 everything is a queue

  8. www email upload live updates

  9. rabbitmq activemq mqseries nsq beanstalkd sqs gearman qless kafka kestrel

    qpid sidekiq resque zeromq celery mqseries stormmq amqp jms hornetq swiftmq delayed_job

  10. tasks

  11. publish/subscribe

  12. publish/subscribe

  13. broker

  14. None

  15. log streaming databass

  16. None

  17. unbounded queues

  18. #2 a queue is either empty or infinitely full

  19. the art of capacity planning

  20. 50 msg * 60 seconds * 30 minutes = 90000

    msg

  21. peak capacity: 120 msg/s

  22. 90000 msg / 120 msg/s = 12,5 min

  23. None

  24. #3 whenever something breaks, it's a queue backing up or

    dropping messages
  25. None

  26. databass x x P x

  27. None

  28. retry databass x x P x

  29. exponential backoff databass x x P x

  30. handling queues involves business decisions

  31. a thought experiment

  32. message queues introduce tight coupling

  33. hide the queue api

  34. api

  35. api

  36. the queue is dead, long live the queue

  37. no berries were harmed © railsberry