Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Queues with RabbitMQ

Lorna Mitchell
September 30, 2016

Queues with RabbitMQ

Talk delivered at the PHPNW conference in Manchester

Lorna Mitchell

September 30, 2016

More Decks by Lorna Mitchell

Other Decks in Technology


  1. @lornajane QUEUES WITH RABBITMQ Lorna Mitchell, IBM Cloud Data Services

    PHPNW, September 2016 (slides are available: ) http://lornajane.net/resources 1
  2. @lornajane WHY USE A QUEUE? loose coupling for scalability interaction

    between technology stacks asynchronous processing event-driven patterns 2
  3. @lornajane WHY USE RABBITMQ? Homepage: Open source, uses AMQP standard.

    Scalable, robust and written in Erlang. Alternatives could be: Gearman Beanstalkd A lot of the theory applies regardless https://www.rabbitmq.com/ 3
  4. @lornajane RABBITMQ CHANNELS When we want to talk to RabbitMQ,

    we establish a channel Channels allow multiple connections between servers over a single TCP connection, but each is isolated from the others. 5
  5. @lornajane DECLARING THE QUEUE We declare a queue before we

    write to it. if it doesn't exist, it is created if it does exist, cool to check if a queue exists but not create it, use passive mode Our example uses the default exchange, more on exchanges in a bit 6
  6. @lornajane HANDLE WEBHOOKS <?php require "vendor/autoload.php"; $input = file_get_contents("php://input"); $rabbit

    = new PhpAmqpLib\Connection\AMQPStreamConnection( 'localhost', 5672, 'guest', 'guest'); $channel = $rabbit->channel(); $channel->queue_declare('pushes', false, true, false, false); $message = new PhpAmqpLib\Message\AMQPMessage( $input, ["delivery_mode" => 2]); $channel->basic_publish($message, '', 'pushes'); 9
  7. @lornajane CONSUMING WITH PHP <?php require "vendor/autoload.php"; $rabbit = new

    PhpAmqpLib\Connection\AMQPStreamConnection( 'localhost', 5672, 'guest', 'guest'); $channel = $rabbit->channel(); $channel->queue_declare('pushes', false, true, false, false); echo "waiting for jobs ...\n"; 11
  8. @lornajane CONSUMING WITH PHP $process = function ($message) { $data

    = json_decode($message->getBody(), true); // do the actual processing here ... $message->delivery_info['channel'] ->basic_ack($message->delivery_info['delivery_tag']); }; $channel->basic_consume('pushes', '', false, false, false, false, $process); while(count($channel->callbacks)) { $channel->wait(); } 12
  9. @lornajane WRITING WORKERS Workers are long-running processes, beware scope issues

    and memory leaks. There can be many workers attached to one queue, each one independent. They need to be robust and handle failures. 13
  10. @lornajane COMPLETED MESSAGES acknowledge when messages are processed successfully can

    acknowledge failure reject the message optionally: requeue 14
  11. @lornajane HANDLING FAILURE: RETRIES Implement your own logic to handle

    retries Create a new message with: all the existing message contents plus some metadata such as retry count or backoff time 16
  12. @lornajane RABBITMQ EXCHANGES Exchanges are the routing logic of RabbitMQ

    Messages go to exchanges and the exchanges put them into the correct queues for storage (our first example used the empty-named default exchange) 17
  13. @lornajane TYPES OF EXCHANGE Direct: a given routing key puts

    messages onto the matching queue(s) Topic: queues are bound by key, and messages are routed to as many queues as their routing key matches Fanout: messages go to all queues bound to this exchange 18
  14. @lornajane DECLARE EXCHANGE, BIND QUEUE IN PHP <?php require "vendor/autoload.php";

    $rabbit = new PhpAmqpLib\Connection\AMQPStreamConnection( 'localhost', 5672, 'guest', 'guest'); $channel = $rabbit->channel(); $channel->exchange_declare('images', 'topic', false, true, false); $channel->queue_declare('triagev1', false, true, false, false); $channel->queue_bind('triagev1', 'images', 'image.v1.triage'); 21
  15. @lornajane FEEDBACK MECHANISMS Rabbit is fire-and-forget; work is delegated Common

    pattern: return queue to put updates into for the original producer then to consume. 27
  16. @lornajane QUEUES WITH RABBITMQ Queues are awesome for scalability and

    robustness RabbitMQ is open source, lightweight and fast Queues help us meet the requirements for modern applications 29