Asynchronous Awesome

46093583d8895095adb1b0071c505af2?s=47 Eric Mann
October 23, 2019

Asynchronous Awesome

Sometimes, our use of PHP grows beyond the typical request/response cycle of dynamic page generation. Unfortunately, the threaded nature of PHP - and the stateless nature of the server - betrays any efforts to expand our utilization of the server. Image processing, video rendering, APNS (Apple Push Notification Service) integration - any of these can easily take longer than is reasonable for a simple page request.

Enter tools like message and job queues that empower daemonized PHP workers to handle data processing in the background. Yet further tools enable long-running event loops and asynchronous Promise-driven operations. PHP isn’t multi-threaded, but that doesn’t mean you’re limited to a single-thread paradigm. I will demonstrate various use cases necessitating asynchronous operations, then delve into the code and the tools that make these systems work. Every attendee will leave armed with new ways to think about the management of large data jobs in PHP and an understanding of the tools they can use to make it happen.

46093583d8895095adb1b0071c505af2?s=128

Eric Mann

October 23, 2019
Tweet

Transcript

  1. Asynchronous Awesome Eric Mann

  2. None
  3. None
  4. Problems this Won’t Solve Report generation PubSub events Parallel I/O

    Parallel data access
  5. Naive Approaches to Async Pseudo-cron (WordPress native and tools like

    wp-async-task) Mechanical Turk-like job systems Using another language entirely
  6. There’s a Better Way

  7. Parallel vs Concurrent Concurrent Parallel (not covered today) Concurrent Parallel

  8. Concurrent and Parallel Programming - Joe Armstrong https://joearms.github.io/published/2013-04-05-concurrent-and-parallel-programming.html

  9. Tools Available Gearman RabbitMQ Amp (promises) pthreads

  10. Gearman

  11. Initialize a Standalone Worker <?php // worker.php $importer = new

    PDFImporter(); $worker = new GearmanWorker(); $worker->addServer('localhost', 4730); $worker->addFunction('import', array($importer, 'import'); while($worker->work());
  12. Dispatch Jobs to the Worker <?php // app.php $client =

    new GearmanClient(); $client->addServer('localhost', 4730); $job = $client->doBackground('import', $file . '|:|' . $email); $db->record($job);
  13. RabbitMQ

  14. None
  15. Publish to the Queue <?php $connection = new AMQPStreamConnection('localhost', 5672,

    'u', 'p'); $channel = $connection->channel(); $channel->queue_declare('demo', false, false, false, false); $msg = new AMQPMessage('Hello World!'); $channel->basic_publish($msg, '', 'demo'); $channel->close(); $connection->close();
  16. Consume from the Queue <?php $connection = new AMQPStreamConnection('localhost', 5672,

    'u', 'p'); $channel = $connection->channel(); $channel->queue_declare('demo', false, false, false, false); $cb = function($msg) { // Send Tweet with $msg contents } $channel->basic_consume('demo', '', false, true, false, false, $cb); while(count($channel->callbacks)) { $channel->wait(); }
  17. None
  18. Interact with Multiple Queues <?php $channel->queue_declare('demo', false, false, false, false);

    $channel->queue_declare('demo-rc', false, false, false, false); $cb = function($msg) { // Send Tweet with $msg contents $result = new AMQPMessage( /* Tweet ID */ ); $channel->basic_publish($result, '', 'demo-rc'); } $channel->basic_consume('demo', '', false, true, false, false, $cb); while(count($channel->callbacks)) { $channel->wait(); }
  19. Amp (promises)

  20. Deferred Empowers Async <?php use Amp\Loop; function asyncMultiply($x, $y) {

    $deferred = new Amp\Deferred; // Resolve the async result one second from now Amp\Loop::delay(1000, function () use ($deferred, $x, $y) { $deferred->resolve($x * $y); }); return $deferred->promise(); } asyncMultiply(6, 7)->onResolve(function ($err, $result) { var_dump($result); // int(42) });
  21. Chain Multiple Promises Together <?php Loop::run(function () { $httpClient =

    new Amp\Artax\DefaultClient(); $uris = [ "google" => "http://www.google.com", "bing" => "http://www.bing.com", ]; $responses = yield array_map(function ($uri) use ($httpClient) { return $httpClient->request($uri); }, $uris); foreach ($responses as $key => $response) { printf("%s | %d\n", $key, $response->getStatus()); } Loop::stop(); });
  22. pthreads

  23. Create a Worker <?php class WorkerThread extends Thread { private

    $workerId; public function __construct($id) { $this->workerId = $id; } public function run() { // Do some heavy processing } }
  24. Start the Workers <?php $workers = []; foreach (range(0, 5)

    as $id) { $worker = new WorkerThread($id); $workers[] = $worker; $worker->start(); // Now they'll each run independently } foreach (range(0, 5) as $id) { $workers[$id]->join(); // Synchronously rejoin the main process }
  25. What Use is Any of This? Gearman – powers things

    like PDF conversion, video transpiling, APNS RabbitMQ – powers cross-app messaging in load balanced environments Promises – power concurrent data migrations for large data sources pthreads – powers task management in a performance, scalable, managed fashion
  26. Further Study The amphp/parallel library wraps pthreads entirely – use

    both at the same time! Look at what the Node community is doing – we have the same exact tools available, but a much larger community and lower barriers Challenge yourself to look at traditional programming problems from an asynchronous viewpoint – can things run faster in parallel?
  27. Questions?

  28. Thank you eric.mann@vacasa.com | 503.925.6266