Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to PHP-Queue

Introduction to PHP-Queue

Presented at Singapore PHP User Group Meetup on 28 Nov 2012.

Michael Cheng

November 28, 2012
Tweet

More Decks by Michael Cheng

Other Decks in Programming

Transcript

  1. PHP-Queue
    A unified front-end for different queuing backends.
    1
    Wednesday, 28 November, 12

    View Slide

  2. Michael Cheng
    Senior Software Engineer, mig33
    http://twitter.com/coderkungfu
    http://github.com/miccheng
    2
    Wednesday, 28 November, 12

    View Slide

  3. What is a Queue?
    • Software engineering components used for
    inter-process communication.
    • They use a queue for messaging - the
    passing of control or of content between
    different parts of your software.
    • Asynchronous communication protocol.
    3
    Wednesday, 28 November, 12

    View Slide

  4. Examples
    • Email/SMS/Push Notification delivery.
    • Processing uploaded photos/videos.
    • Pushing content to all the home feeds of a
    user’s friends.
    • Anything that takes a long time to process.
    4
    Wednesday, 28 November, 12

    View Slide

  5. Example
    Web App Messaging
    Queue
    Worker
    Worker
    Worker
    Amazon SQS, IronMQ,
    Beanstalkd, Gearman,
    RabbitMQ, MSMQ
    5
    Wednesday, 28 November, 12

    View Slide

  6. Benefits
    • Improve responsiveness of your web apps.
    • Reduce load of your web server/database.
    • Distribute workload to multiple threads &
    servers.
    • Helps you scale.
    6
    Wednesday, 28 November, 12

    View Slide

  7. How about...?
    • Why not use a cron job?
    • Batch processed < time allocated (unused
    capacity)
    • Time taken > time allocated (overlap)
    • Why not use a normal MySQL table?
    • Not crash resistant (workers). Race
    conditions.
    7
    Wednesday, 28 November, 12

    View Slide

  8. Why PHP-Queue?
    • Its not easy implementing a messaging
    queue.
    • Which backend? Hosted/Self-host?
    • Learning curve - protocols, libraries.
    • Not responsive to changing use cases.
    • Vendor locked-in.
    8
    Wednesday, 28 November, 12

    View Slide

  9. Benefits of PHP-Queue
    • Backend agnostic.
    • Flexible job queue implementation.
    • 1 queue 1 type of jobs.
    • 1 queue many types of jobs.
    • Independent workers. Chainable.
    • Powerful.
    • REST server, CLI, daemons.
    9
    Wednesday, 28 November, 12

    View Slide

  10. Demo
    • Installation with Composer.
    • Creating a Queue.
    • Creating a Worker.
    • Using the REST server.
    • Using the CLI.
    10
    Wednesday, 28 November, 12

    View Slide

  11. PHP-Queue
    https://github.com/miccheng/php-queue
    https://packagist.org/packages/coderkungfu/php-queue
    11
    Wednesday, 28 November, 12

    View Slide

  12. Usage Model
    Job Queue
    Job
    Job
    Job
    Job
    SMS Worker
    Email Worker
    Runner
    Job
    12
    Wednesday, 28 November, 12

    View Slide

  13. SMS Worker
    Single Worker
    Job Queue
    Job
    Job
    Job
    Job
    Email Worker
    Runner
    SMS Worker
    Email Worker
    SMS Worker
    Email Worker
    Runner
    Runner
    Job
    13
    Wednesday, 28 November, 12

    View Slide

  14. SMS Worker
    Multiple Workers
    Job Queue
    Job
    Job
    Job
    Job
    Email Worker
    Runner
    SMS Worker
    Email Worker
    SMS Worker
    Email Worker
    Runner
    Runner
    Job
    14
    Wednesday, 28 November, 12

    View Slide

  15. Caveat
    • Please know what you are doing.
    • You have to maintain the queue server.
    • Monitor performance and adjust
    accordingly.
    • Secure it!
    15
    Wednesday, 28 November, 12

    View Slide

  16. Thank You.
    16
    Wednesday, 28 November, 12

    View Slide

  17. PHP-Queue
    https://github.com/miccheng/php-queue
    https://packagist.org/packages/coderkungfu/php-queue
    17
    Wednesday, 28 November, 12

    View Slide