Message Queues a basic overview

Talk at PHP Developers meet up #12 held on 23-Aug-2014.

Transcript

1. Message Queues A basic overview
2. Geshan Manandhar Quality and Maintenance Lead, Namshi.com @geshan geshan.com.np
3. Index What is Message Queue(MQ) MQ basics - AMQP Benefits Rabbit MQ Our setup/stack When to use it /Practical Usecase Logging and monitoring Conclusion
4. What is Message Queue "message queues and mailboxes are software-engineering components used for inter-process communication (IPC), or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content." - Wikipedia Message queue is a software components that allow producers to produce message to a queue managed by a message broker, these message are asynchronously consumed by consumers.
5. Message Queue Basics Protocols : Advanced Message Queueing Protocol(AMQP) and Streaming Text Oriented Messaging Protocol(STOMP) Types of Queue: Direct, Topic and Fanout Producer/Publisher Message Broker - Queue Exchange Queue Consumer
6. Benefits of MQ Decoupling Scalability Reliability Fanout - Same message to multiple queues (Pub/Sub)
7. Rabbit MQ
8. Rabbit MQ - features Robust messaging for applications Easy to use Runs on all major operating systems Supports a huge number of developer platforms Open source and commercially supported
9. Our Software stack/services relating to with Rabbit MQ Symfony 2 with old sound rabbit mq bundle and Monolog Rabbit Mq on http://www.cloudamqp.com/ Service Oriented Architecture (SOA) with 3rd party API integrations
10. When to use message queues Some practical use cases Backend -> Frontend sync (db to NOSQL/Memcache) Communication between multiple internal systems (without API) Asynchronous 3rd party API calls Notifications like email and sms Handling large request asynchronously Processing intensive tasks- like image resize
11. Logging and monitoring is vital Only access and error logs from web server and database are surely not enough Logs are first class citizen, adopt active logging from application level.(Monolog) Use dedicated log management system like GrayLog 2 Monitor application activity in real-time with services like NewRelic Log optimally and verbose enough to give enough information about the context.
12. Conclusion/Recap Communication between two systems can be done even without APIs Language agnostic implementation is enable by using message queues Message queues could be that answer you were looking for, to communicate between different systems For scalability, decoupling and reliability utilize message queues You could potentially scale to 1 million message per second : Pivotal Logging and monitoring are inevitable when doing asynchronous processing
14. Questions???
15. Credits http://keyholesoftware.com/2013/05/13/messaging-with-rabbitmq/ https://www.flickr.com/photos/i8ipod/9028050858 https://www.flickr.com/photos/oberazzi/318947873/
16. Some programming mantras to remember.