Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Message Queue is Dead, Long Live the Message Queue

The Message Queue is Dead, Long Live the Message Queue

Ramblings and thought experiments on message queues

4d9dd9bd8d3d4d0ba8af2acc41d14006?s=128

Mathias Meyer

April 22, 2013
Tweet

Transcript

  1. the message queue is dead railsberry 2013, mathias meyer, @roidrage

  2. travis-ci.org

  3. None
  4. None
  5. None
  6. the golden rules of message queues

  7. #1 everything is a queue

  8. www email upload live updates

  9. rabbitmq activemq mqseries nsq beanstalkd sqs gearman qless kafka kestrel

    qpid sidekiq resque zeromq celery mqseries stormmq amqp jms hornetq swiftmq delayed_job
  10. tasks

  11. publish/subscribe

  12. publish/subscribe

  13. broker

  14. None
  15. log streaming databass

  16. None
  17. unbounded queues

  18. #2 a queue is either empty or infinitely full

  19. the art of capacity planning

  20. 50 msg * 60 seconds * 30 minutes = 90000

    msg
  21. peak capacity: 120 msg/s

  22. 90000 msg / 120 msg/s = 12,5 min

  23. None
  24. #3 whenever something breaks, it's a queue backing up or

    dropping messages
  25. None
  26. databass x x P x

  27. None
  28. retry databass x x P x

  29. exponential backoff databass x x P x

  30. handling queues involves business decisions

  31. a thought experiment

  32. message queues introduce tight coupling

  33. hide the queue api

  34. api

  35. api

  36. the queue is dead, long live the queue

  37. no berries were harmed © railsberry