Introduction to PHP-Queue

Presented at Singapore PHP User Group Meetup on 28 Nov 2012.

Transcript

1. PHP-Queue
   A unified front-end for different queuing backends.
   1
   Wednesday, 28 November, 12
   

   View Slide

2. Michael Cheng
   Senior Software Engineer, mig33
   http://twitter.com/coderkungfu
   http://github.com/miccheng
   2
   Wednesday, 28 November, 12
   

   View Slide

3. What is a Queue?
   • Software engineering components used for
   inter-process communication.
   • They use a queue for messaging - the
   passing of control or of content between
   different parts of your software.
   • Asynchronous communication protocol.
   3
   Wednesday, 28 November, 12
   

   View Slide

4. Examples
   • Email/SMS/Push Notification delivery.
   • Processing uploaded photos/videos.
   • Pushing content to all the home feeds of a
   user’s friends.
   • Anything that takes a long time to process.
   4
   Wednesday, 28 November, 12
   

   View Slide

5. Example
   Web App Messaging
   Queue
   Worker
   Worker
   Worker
   Amazon SQS, IronMQ,
   Beanstalkd, Gearman,
   RabbitMQ, MSMQ
   5
   Wednesday, 28 November, 12
   

   View Slide

6. Benefits
   • Improve responsiveness of your web apps.
   • Reduce load of your web server/database.
   • Distribute workload to multiple threads &
   servers.
   • Helps you scale.
   6
   Wednesday, 28 November, 12
   

   View Slide

7. How about...?
   • Why not use a cron job?
   • Batch processed < time allocated (unused
   capacity)
   • Time taken > time allocated (overlap)
   • Why not use a normal MySQL table?
   • Not crash resistant (workers). Race
   conditions.
   7
   Wednesday, 28 November, 12
   

   View Slide

8. Why PHP-Queue?
   • Its not easy implementing a messaging
   queue.
   • Which backend? Hosted/Self-host?
   • Learning curve - protocols, libraries.
   • Not responsive to changing use cases.
   • Vendor locked-in.
   8
   Wednesday, 28 November, 12
   

   View Slide

9. Benefits of PHP-Queue
   • Backend agnostic.
   • Flexible job queue implementation.
   • 1 queue 1 type of jobs.
   • 1 queue many types of jobs.
   • Independent workers. Chainable.
   • Powerful.
   • REST server, CLI, daemons.
   9
   Wednesday, 28 November, 12
   

   View Slide

10. Demo
    • Installation with Composer.
    • Creating a Queue.
    • Creating a Worker.
    • Using the REST server.
    • Using the CLI.
    10
    Wednesday, 28 November, 12
    

    View Slide

11. PHP-Queue
    https://github.com/miccheng/php-queue
    https://packagist.org/packages/coderkungfu/php-queue
    11
    Wednesday, 28 November, 12
    

    View Slide

12. Usage Model
    Job Queue
    Job
    Job
    Job
    Job
    SMS Worker
    Email Worker
    Runner
    Job
    12
    Wednesday, 28 November, 12
    

    View Slide

13. SMS Worker
    Single Worker
    Job Queue
    Job
    Job
    Job
    Job
    Email Worker
    Runner
    SMS Worker
    Email Worker
    SMS Worker
    Email Worker
    Runner
    Runner
    Job
    13
    Wednesday, 28 November, 12
    

    View Slide

14. SMS Worker
    Multiple Workers
    Job Queue
    Job
    Job
    Job
    Job
    Email Worker
    Runner
    SMS Worker
    Email Worker
    SMS Worker
    Email Worker
    Runner
    Runner
    Job
    14
    Wednesday, 28 November, 12
    

    View Slide

15. Caveat
    • Please know what you are doing.
    • You have to maintain the queue server.
    • Monitor performance and adjust
    accordingly.
    • Secure it!
    15
    Wednesday, 28 November, 12
    

    View Slide

16. Thank You.
    16
    Wednesday, 28 November, 12
    

    View Slide

17. PHP-Queue
    https://github.com/miccheng/php-queue
    https://packagist.org/packages/coderkungfu/php-queue
    17
    Wednesday, 28 November, 12
    

    View Slide