Housekeeping… • Talk – Slides will be posted after the talk. • Me – Sr. Web Architect Manager at NOOK Developer – Prior MNPHP Organizer – Open Source Contributor – Where you can find me: • Twitter: mwillbanks G+: Mike Willbanks • IRC (freenode): mwillbanks Blog: http://blog.digitalstruct.com • GitHub: https://github.com/mwillbanks
A Definition “Message queues and mailboxes are software-engineering components used for interprocess communication, or for inter- thread communication within the same process. They use a queue for messaging – the passing of control or of content.”
What is messaging? “Messaging describes the sending and receiving of data (in the form of messages) between systems. Messages are exchanged between programs or applications, similar to the way people communicate by email but with guarantees on delivery, speed, security and the absence of spam.”
General Anatomy Task Producer Consumer Messages Messages Messages Producer creates a message and pushes it to the queue; the consumer reads from the queue and processes the message.
l Pub/Sub l FIFO buffer l Push / Pull l A way to communicate between applications / systems. l A way to decouple components. l A way to offload work. Describing Message Queues
l Offload Heavy Work l Integration with Legacy Systems l Asynchronous Processing l Parallel Processing l Process consistency l Scalability Why to use a Message Queue
l AMQP Working Group (Community and Vendor) l Platform agnostic protocol. l Completely open, interoperable and broadly applicable. l Many severs available and many client libraries. Overview of AMQP
l AMQP utilizes exchanges, queues and bindings. l An exchange are routers with routing tables. l A binding defines the routing rules. l A queue is where the messages wait for a consumer. How it Works
l Fanout Exchange l No routing keys involved. Any message that is sent to the exchange is sent to all queues bound to that exchange. l Direct Exchange l Routing keys involved. A queue binds to the exchange to request messages that match a routing key exactly. l Topic Exchange l Routing keys involved. A queue binds to the exchange to request messages that match a routing key pattern. Understanding Exchanges
Implementations www.rabbitmq.com Very popular and common message queue owned by VMware. qpid.apache.org Long standing project; apache foundation. www.openamq.org Long standing project; ZeroMQ partner, no news since 2009.
l An exchange, queue and bindings must be defined first. Publishing can then commence after. l Create the queue l Create the exchange l Bind to the queue. Building a Queue
l Default behavior is no persistence. l How important are the messages? l Just about all items have a level of persistence if you would like them to survive on reboot. l Mark exchanges, queues and messages as DURABLE. Persistence?
How It Works l When you send in a message, you tell it which queue to go to. l When you subscribe you request a queue. Connect Send Disconnect /queue/ msg P H P S T O M P S E R V E R Connect Subscribe Disconnect /queue/ msg Read Ack
Sever Implementations activemq.apache.org One of the oldest message queues existing; a apache foundation project activemq.apache.org/apollo Next generation ActiveMQ www.rabbitmq.com Very popular and common message queue owned by VMware. www.jboss.org/hornetq Supported by Red Hat Middle Ware division, picking up steam.
Overview l Best for real-time data. l Leveraging pub/sub can turn it into more of a generic message system. l Multiple libraries l JAXL - https://github.com/abhinavsingh/JAXL l Xmpp - https://github.com/alexmace/Xmpp
XEP-0060: Publish-Subscribe l Specification for implementing Publish Subscribe models. l Extension to the original XMPP specification. Publish Subscribe Sub1 Sub2 Sub3 to, id, message from, to, id, message
l Pass a job to the job server l Worker receives the job and processes l Ability to persist to binlog; default is in-memory Beanstalkd Application Code Beanstalkd Client API (Pheanstalk) Beanstalkd Server Beanstalkd Worker API (Pheanstalk) Worker Application Code
Standards • A recognized standard? – AMQP? STOMP? XMPP? • How many developers? – Will an unfortunate event kill off the product? • Undocumented protocol? – Forget about it!
Messages • Formatting – JSON or XML are great options. • Please no serialized PHP objects. • Message Size – Only as large as necessary. • Don’t send a binary object through the queue.
PHP Daemons • Prevent Memory Leaks – Detection – Cycle Workers • Handle Signals – Properly shutdown! – Watch out for OS service kills • Sleeping is good J
High Availability Web Server Database Worker Server Message Queue Message Queue Worker Server Send to 1 Insert multiple message queue servers + multiple worker nodes. Each worker node can connect to as many message queue servers as necessary.
mwillbanks
syumai
stewemetal
kokuyouwind
mozu1206
ryu_hu03
soutaro
jmatsu
squer
tmokmss
treastrain
77web
doggy
andyhume
deanohume
jlugia
tanoku
bluesmoon
trallard
michaelherold
jrom
carmenintech
bkeepers
smashingmag
sstephenson