Upgrade to Pro — share decks privately, control downloads, hide ads and more …

RabbitMQ and Node.js

Zlatko
June 25, 2015

RabbitMQ and Node.js

RabbitMQ and Node.js

Zlatko

June 25, 2015
Tweet

More Decks by Zlatko

Other Decks in Technology

Transcript

  1. Disclaimer (AKA about me) • Experience – Node.js? Lots –

    NoSQL? Lots. – Messaging? Little • Backend Developer at ThinxNet.com (http://thinxnet.com/#jobs)
  2. Ready to serve. How? • Event collection and processing •

    Webapp, mobile app • Backoffice and support apps, notification services etc. • Reporting, third party software integration etc • REST approach: – Lots of requests – Parsing of data is blocking – Avoid choking devices – Communication overhead – Error management
  3. About RabbitMQ • Messaging broker based on AMQP protocol •

    Platform/language independent • Extensible (Erlang) • Persistent message queues • Node.js drivers (node-amqp, rabbit.js...)
  4. Exchanges and topics • Publisher and subscriber. Simple as that.

    • Get message, ack or reject (nack) it. • Subscribers listen on a queue • Publishers talk to an exchange • Exchange routes to queues based on topics
  5. Channels and connections • 1 node process = 1 connection

    • Many channels, share same connection
  6. PUSH-PULL/WORKER • Round-robin distribution • Worker socket.ack() is explicit (rabbit.js

    driver) • Prefetch sets the number of msgs received • Problem: concurrent/sequential modification?
  7. Content-based routing • Erlang extensions? Not standard. • App-side: –

    Keep internal mini-queue – Do not ack a msg if it's not first on local queue – Hold internal messages based on internal “topic”, ie “user.<ID>.save” - hold msgs for that user – Item.update() instead of item.save() • Gets nasty fast, should minimize these scenarios
  8. REST API with MQ • REQUEST/REPLY for immediate responses •

    Other patterns for direct access – REST API just forwards the message – replies will go by other route • Pub/sub is easy!
  9. Error handling • Uncaught exceptions are covered • Retry logic

    is simplified – push incomplete messages to a 'fixer' queue, requeue failed jobs or when no resources • Dedicated error handling
  10. Error handling • Uncaught exceptions are covered • Retry logic

    is simplified – push incomplete messages to a 'fixer' queue, requeue failed jobs or when no resources • Dedicated error handling
  11. Testing • No good known mocks, please help • RabbitMQ

    always on on Jenkins machine • Test handlers directly
  12. Note to self • Thank the audience for patience •

    Invite audience for a discussion and questions