Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Message Queue is Dead, Long Live the Message Queue

4d9dd9bd8d3d4d0ba8af2acc41d14006?s=47 Mathias Meyer
April 22, 2013
Technology
4
580

The Message Queue is Dead, Long Live the Message Queue

Ramblings and thought experiments on message queues

4d9dd9bd8d3d4d0ba8af2acc41d14006?s=128

Mathias Meyer

April 22, 2013
Tweet

More Decks by Mathias Meyer

See All by Mathias Meyer
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
0
860
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
4
210
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
1
250
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
11
1.5k
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
83
14k
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
2
270
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
5
960
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
10
1k
4d9dd9bd8d3d4d0ba8af2acc41d14006?s=48 roidrage
6
300

Other Decks in Technology

See All in Technology
1e210dc7083cc282628ff071926b9351?s=48 nnstt1
2
120
65d8522e6ea1d11c8f6343027b2e1688?s=48 yashi8484
0
140
3115a782126be714b5f94d24073c957d?s=48 oracle4engineer
9
6.2k
8aba6de431668fc8d09977c0e33c63c1?s=48 jaguar_imo
0
120
Fe475df4e28cef1175a787fb032e0489?s=48 bolonio
4
570
05e8adce66be4e80390a29ace0075161?s=48 ymatsuwitter
0
240
847a328633b1df6b11cc2f72430025e6?s=48 ks91
PRO
0
350
0e8fbeb3dc2da3e502483edea80be7d6?s=48 htomine
0
170
2cf373725ded741824c50fd571eda6e1?s=48 udzura
0
130
7e6a435700dda6cdb22105d601f20892?s=48 picardparis
4
2.4k
53817d765d91855d2fabcaef60f8bba0?s=48 chipstar_light
0
490
C265a1a7dadb2713ff2262025e91d133?s=48 akabekobeko
0
230

Featured

See All Featured
3d4519fd34b76fb265fc0237f3792bd4?s=48 danielanewman
2
560
245cee81a9c424266e5e401d844ea881?s=48 lara
172
9.8k
1b75d89772b7eea1f6c89fbf362607d9?s=48 moore
125
21k
A9704266587836f7e784235e5073b93e?s=48 jmmastey
10
700
282d599b414022eba5096510f675b6e2?s=48 chrislema
231
16k
06f8b41980eb4c577fa40c41d5030c19?s=48 keathley
20
760
0c6fd6a08d0f898159cda7cafcacf07f?s=48 afnizarnur
176
14k
32de0bd2ba869609d26fd052a4622778?s=48 cromwellryan
104
6.3k
C157b16234f1e75e8eac3698c1d4414a?s=48 ddemaree
273
31k
C4de88ea47538e4594750e3861a341c8?s=48 brettharned
93
3.1k
Ba530e786553390f3fb271848485681c?s=48 3n
163
22k
D8fc2580cfaca035f666d9e4ee79a7f7?s=48 mongodb
23
3.9k

Transcript

  1. the message queue is dead railsberry 2013, mathias meyer, @roidrage

  2. travis-ci.org

  3. None
  4. None
  5. None

  6. the golden rules of message queues

  7. #1 everything is a queue

  8. www email upload live updates

  9. rabbitmq activemq mqseries nsq beanstalkd sqs gearman qless kafka kestrel

    qpid sidekiq resque zeromq celery mqseries stormmq amqp jms hornetq swiftmq delayed_job

  10. tasks

  11. publish/subscribe

  12. publish/subscribe

  13. broker

  14. None

  15. log streaming databass

  16. None

  17. unbounded queues

  18. #2 a queue is either empty or infinitely full

  19. the art of capacity planning

  20. 50 msg * 60 seconds * 30 minutes = 90000

    msg

  21. peak capacity: 120 msg/s

  22. 90000 msg / 120 msg/s = 12,5 min

  23. None

  24. #3 whenever something breaks, it's a queue backing up or

    dropping messages
  25. None

  26. databass x x P x

  27. None

  28. retry databass x x P x

  29. exponential backoff databass x x P x

  30. handling queues involves business decisions

  31. a thought experiment

  32. message queues introduce tight coupling

  33. hide the queue api

  34. api

  35. api

  36. the queue is dead, long live the queue

  37. no berries were harmed © railsberry