PHP and Queues: Beanstalkd

A presentation at the Lagos PHP Meetup about Queues in PHP and how to use Beanstalkd

Transcript

1. PHP and Queue: Beanstalkd BY VICTOR ALUKO 1
2. About Me Victor Aluko Software Engineer @Konga.com Ag Software Dev Manager , Konga’s SellerHQ team ex-CTO, Jobberman Zend Certified Engineer, PHP 5.3 • Backend, server-side and Devops Engineer • Coding in PHP for … longer than I expected! • Gamer • https://www.linkedin.com/in/valuko • https://github.com/valuko 2
3. What are Queues?!  Data structure for keep tasks/jobs to be executed later  Usually FIFO  Can implement priorities  3 elements are involved  Producer or Sender  Consumer or Receiver  Queue 3
4. Job Queues?!  Something like these guys  Not these guys! 4
5. Queue Patterns  Job Queues  Delayed Jobs  Fanout  Message Groups  Remote Procedure Call (RPC) 5
6. Use Cases  Sending Notifications to users  Log entries  Decoupling heavy-weight processing from a live user request e.g. Upload picture, make thumbnails, push to CDN…  Batch Processing  Breaking up a long complex process between workers Calculating stats, Indexing…  Delayed Processing 6
7. And the options are…  Active MQ: http://activemq.apache.org/  RabbitMQ: https://www.rabbitmq.com/tutorials/tutorial-one-php.html  Celery: http://www.celeryproject.org/  Gearman: http://gearman.org/  Beanstalkd: http://kr.github.io/beanstalkd/  PHP-Resque: https://github.com/chrisboulton/php-resque/  Amazon SQS: https://aws.amazon.com/sqs/  IronMQ: https://www.iron.io/platform/ironmq/ 7
8. Crons vs Queues: The face-off Crons Message Queues Best used for scheduled jobs Best for asynchronous tasks Easier to setup and manage More power, more control Basic solution. Good for recruits. More refined. Good for Captains. Only one server Scalable. Multiple servers* Jobs wait till next cron run Jobs can be processed much faster Gets bulky after awhile More flexible 8
9. Beanstalkd  Open-source C daemon  *nix only. Sorry Windows!  Fast  Persistence – bin log file  Distributed  Simple protocol – As simple as memcached  Supports priority queues, delayed tasks and time-to-run  Transactional – 1 reserved job to 1 worker 9
10. Beanstalkd PHP Libraries  Pheanstalk: https://github.com/pda/pheanstalk  Beanstalk: https://github.com/davidpersson/beanstalk  Other libraries: https://github.com/kr/beanstalkd/wiki/client-libraries 10
11. Management Interfaces  Beanstalk Console: https://github.com/ptrofimov/beanstalk_console 11
12. Management Interfaces  phpBeanstalkAdmin: http://mnapoli.github.com/phpBeanstalkdAdmin/ 12
13. Code Snippet  Sender Code 13
14. Code Snippet  Receiver code 14
15. Its been fun talking Now you Ask! 15