Message Queues with PHP

About Me Eric Rodriguez Founder @ Data.be Founder @ Pictawall.com
 CTO @ Auctelia.com ! • Web entrepreneur • Multi-Language: PHP, Java/Groovy/Grails, .Net, … ! be.linkedin.com/in/erodriguez ! github.com/wavyx ! @wavyx

Which Rabbit ?!

RabbitMQ • Message Broker • Open Source • AMPQ implementation • Written in Erlang • Lots of AMQP client libraries • Wide community and commercial support

Message Queues 101 • Producer = Sender • Consumer = Receiver • Queue = Mailbox

Use Cases • Send notifications to users • Process heavy work/tasks with multiple workers
 Upload picture, make thumbnails, clear CDN caches… • Distribute logs/messages to multiple endpoints • Interoperability between different platforms • Remote Procedure Call

AMQP
 Advanced Message Queuing Protocol • Open Standard Protocol coming from finance • Interoperable Messaging Middleware • Queuing • Routing (point-to-point and publish-and-subscribe) • Reliability • Security

AMQP Model • Exchange • Queue • Routes

Exchanges • 4 Types - Direct, Fanout, Topic, Headers • Durability - Exchanges survive a restart • Auto-delete - Exchange deleted when queues empty

Direct Exchange A Direct Exchange sends a message to a queue if the message's routing key is identical to the binding key for the queue

Fanout Exchange A Fanout Exchange sends messages 
 to every queue bound to the exchange

Topic Exchange A Topic Exchange sends a message to a queue if the message's 
 routing key matches the binding key for the queue, using wildcard matching

Alternatives Check http://queues.io/ • ZeroMQ - http://zeromq.org/ (ZMTP) • Beanstalkd - http://kr.github.io/beanstalkd/ (Job Server) • Gearman - http://gearman.org/ (Job Server) • HornetQ - http://www.jboss.org/hornetq/ (JMS/STOMP) • Apache Kafka -http://kafka.apache.org/ (PubSub) • Apache Qpid - http://qpid.apache.org/ (AMQP) • Apache ActiveMQ - http://activemq.apache.org/ (AMQP) • Apache Apollo - https://activemq.apache.org/apollo/ (AMQP)

Cloud Alternatives • IronMQ
 http://www.iron.io/mq • Amazon SQS
 http://aws.amazon.com/sqs/ • StormMQ
 http://stormmq.com/ • Rackspace Queues
 http://www.rackspace.com/cloud/queues/ • Windows Azure Service Bus
 http://www.windowsazure.com/en-us/services/messaging/

PHP Implementation • php-amqplib - pure php client
 https://github.com/videlalvaro/php-amqplib • PECL AMQP library - built on RabbitMQ C client
 http://pecl.php.net/package/amqp • amqphp - pure php client
 https://github.com/BraveSirRobin/amqphp • Thumper - library of messaging patterns
 https://github.com/videlalvaro/Thumper

Hello MQ World - send

Hello MQ World - receive

Hello MQ World
 queue_declare

Hello MQ World
 basic_publish & basic_consume

Work Queues / Task Queues • Avoid doing a resource-intensive task immediately • Schedule the task to be done later • Encapsulate a task as a message and send it to a queue • A worker process running in the background will execute the job • Multiple workers will share tasks

Work Queues - new_task

Work Queues - worker

Publish/Subscribe • Task queues: task is delivered to exactly one worker • Pub/Sub: deliver a message to multiple consumers • Published messages are going to be broadcast to all the receivers • Fanout exchanges

Pub/Sub - emit_log

Pub/Sub - receive_logs

Routing • Fanout => Direct exchange • Subscribe only to a subset of the messages • Filter messages based on their routing key

Routing - emit_log_direct

Routing - receive_logs_direct

Topics • Direct exchanges cannot do routing based on multiple criteria • Example: how to combine “severity” and “resource” • Topic exchange: messages are sent with a routing key composed by a list of words, delimited by dots • * (star) can substitute for exactly one word • # (hash) can substitute for zero or more words

Topics - emit_log_topic

Topics - receive_logs_topic

RPC • Work Queues = distribute time-consuming tasks among multiple workers • Remote Procedure Call or RPC = run a function on a remote computer and wait for the result

RPC - rpc_client 1/2

RPC - rpc_client 2/2

RPC - rpc_server

• Work Queues • Publish/Subscribe • Routing • Topics • Remote Procedure Call (RPC)

RabbitMqBundle • Fast and Easy to use RabbitMQ with Symfony2 • Use php-amqplib library • Provide messaging pattern (similar to Thumper)

RabbitMqBundle - Setup

RabbitMqBundle - Config

RabbitMqBundle • Producer / Consumer • Callback • RPC • Parallel RPC • Anonymous Consumer • STDIN Producer

Resources • http://www.rabbitmq.com/getstarted.html • http://www.rabbitmq.com/how.html • RabbitMQ - AMQP concepts • RedHat - Messaging tutorial • http://www.slideshare.net/old_sound/theres-a-rabbit-on-my-symfony • http://www.slideshare.net/jasonlotito/php-rabbitmq-and-you • https://github.com/videlalvaro/rabbitmqbundle • http://reddikh.com/rabbitmq-bundle-in-symfony2/ • http://techportal.inviqa.com/2013/11/18/let-rabbitmq-do-the-work-in-your- symfony2-application/

Books • RabbitMQ in Action
 http://manning.com/videla/ • RabbitMQ Cookbook
 http://www.packtpub.com/ rabbitmq-cookbook/book

Thank you! [email protected] - @wavyx be.linkedin.com/in/erodriguez - github.com/wavyx http://www.meetup.com/Symfony-User-Group-Belgium/