out work to other machines, dispatching function calls to machines that are better suited to do work, to do work in parallel, to load balance lots of function calls, or to call functions between languages.” ✤ History: ✤ Originally written for LiveJournal in Perl ✤ The name is an anagram for “Manager,” since it dispatches jobs to be done, but does not do anything useful itself. ✤ More modern version is written in C Friday, 20 April 12
front end latency by offloading longer running tasks - not everything needs to be done now ✤ Run jobs synchronously or asynchronously, in parallel ✤ Run jobs on machines better suited for the work ✤ Run jobs cross language (queue a job using PHP, which is run by Ruby / C / ...) Friday, 20 April 12
the machine with Gearmand. ✤ Running more workers means you can run more jobs at once. ✤ Workers can return statuses, fail, warning, complete, etc Friday, 20 April 12
GearmanClient(); $client->addServer(); print $client->do("reverse", "Hello World!"); # Reverse Worker Code $worker= new GearmanWorker(); $worker->addServer(); $worker->addFunction("reverse", "my_reverse_function"); while ($worker->work()); function my_reverse_function($job) { return strrev($job->workload()); } Client Worker You can add as many functions as you wish, though each worker will (obviously) only run one job at a time The worker will run until it crashes or is killed. In this example, we connect to gearman (default is localhost) and request “reverse” be run. “do” means the job is enqueued and will run synchronously. Friday, 20 April 12
common task, however it can be unpredictable and slow ✤ Sending using a Gearman worker makes it asynchronous, resulting in a faster front-end interface ✤ When sending bulk email multiple workers can increase throughput Friday, 20 April 12
to run on a different ip / port; ✤ gearmand -d -L xxx.xxx.xxx.xxx -p 4730 ✤ I would also suggest running gearmand under supervise, especially if you are using a checked out version. Friday, 20 April 12
PHP development tools (apt-get install php-dev) ✤ pecl install channel://pecl.php.net/gearman-0.7.0 ✤ Make sure you add the module to the php.ini Friday, 20 April 12
seperate types of workers if you need ✤ Run by: php index.php “gearman/worker” & (or whatever you’ve called your route) ✤ Better version will register the PID somewhere for management and targeted killing Friday, 20 April 12
✤ Error handling must be robust ✤ If you use database connections, beware they can timeout and if kept open each worker will consume resources on the db server too ✤ During deployment you must use a method of restarting workers in a sensible manner across all your machines. The safest way is to stop jobs being created and kill / restart all the workers - having different versions of the same function accepting jobs is not a good idea. Capistrano rocks. ✤ Automatic restarting of workers is important - they do crash. We use deamontools, alternatives include launchd & the PHP based GearmanManager Friday, 20 April 12
(as yet). iptables is your friend. ✤ Persistent queues are available. By default jobs are lost if Gearmand crashes, using a persistent backend stops this. Backends include Drizzle, MySQL ✤ MySQL UDF (User Defined Functions) allow MySQL queries to act as Gearman clients. See: http://gearman.org/index.php?id=mysql_udf_readme Friday, 20 April 12