Asynchronous Awesome

46093583d8895095adb1b0071c505af2?s=47 Eric Mann
February 07, 2020

Asynchronous Awesome

Sometimes, our use of PHP grows beyond the typical request/response cycle of dynamic page generation. Unfortunately, the threaded nature of PHP - and the stateless nature of the server - betrays any efforts to expand our utilization of the server. Image processing, video rendering, APNS (Apple Push Notification Service) integration - any of these can easily take longer than is reasonable for a simple page request. Enter tools like message and job queues that empower daemonized PHP workers to handle data processing in the background. Yet further tools enable long-running event loops and asynchronous Promise-driven operations. PHP isn't multi-threaded, but that doesn't mean you're limited to a single-thread paradigm. I will demonstrate various use cases necessitating asynchronous operations, then delve into the code and the tools that make these systems work.

Every attendee will leave armed with new ways to think about the management of large data jobs in PHP and an understanding of the tools they can use to make it happen.

46093583d8895095adb1b0071c505af2?s=128

Eric Mann

February 07, 2020
Tweet

Transcript

  1. 1

  2. 2 Hello. Eric Mann Director of Engineering, Vacasa

  3. 3

  4. 4 In 2012, I had to build a way for

    the “Save PDF” button on this screen to work. It required building a PDF in the background and would sometimes take up to 5 minutes to work. Doing things asynchronously was hard!
  5. 5 Report generation PubSub events Parallel I/O Parallel data access

  6. 6 Pseudo-cron (WordPress native and tools like wp-async-task)

  7. None
  8. None
  9. None
  10. 10 Pseudo-cron (WordPress native and tools like wp-async-task) Mechanical Turk-like

    job systems Using another language entirely
  11. 11

  12. 12 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce

    elit ex, consequat et tincidunt non, pharetra non risus. Quisque ut leo pretium, eleifend lectus in, ultrices diam. Quisque ac congue urna, non finibus orci. Concurrent Parallel (not covered today) Concurrent Parallel Image by Joe Armstrong - https://joearms.github.io/published/2013-04-05-concurrent-and-parallel-programming.html
  13. 13 Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce

    elit ex, consequat et tincidunt non, pharetra non risus. Quisque ut leo pretium, eleifend lectus in, ultrices diam. Quisque ac congue urna, non finibus orci. Concurrent Parallel (not covered today) Concurrent Parallel Image by Joe Armstrong - https://joearms.github.io/published/2013-04-05-concurrent-and-parallel-programming.html
  14. 14

  15. 15

  16. 16 <?php // worker.php $importer = new PDFImporter(); $worker =

    new GearmanWorker(); $worker->addServer('localhost', 4730); $worker->addFunction('import', array($importer, 'import'); while($worker->work());
  17. 17 <?php // app.php $client = new GearmanClient(); $client->addServer('localhost', 4730);

    $job = $client->doBackground('import', $file . '|:|' . $email); $db->record($job);
  18. 18

  19. None
  20. 20 <?php $connection = new AMQPStreamConnection('localhost', 5672, 'u', 'p'); $channel

    = $connection->channel(); $channel->queue_declare('demo', false, false, false, false); $msg = new AMQPMessage('Hello World!'); $channel->basic_publish($msg, '', 'demo'); $channel->close(); $connection->close();
  21. 21 <?php $connection = new AMQPStreamConnection('localhost', 5672, 'u', 'p'); $channel

    = $connection->channel(); $channel->queue_declare('demo', false, false, false, false); $cb = function($msg) { // Send Tweet with $msg contents } $channel->basic_consume('demo', '', false, true, false, false, $cb); while(count($channel->callbacks)) { $channel->wait(); }
  22. None
  23. 23 <?php $channel->queue_declare('demo', false, false, false, false); $channel->queue_declare('demo-rc', false, false,

    false, false); $cb = function($msg) { // Send Tweet with $msg contents $result = new AMQPMessage( /* Tweet ID */ ); $channel->basic_publish($result, '', 'demo-rc'); } $channel->basic_consume('demo', '', false, true, false, false, $cb); while(count($channel->callbacks)) { $channel->wait(); }
  24. 24

  25. 25 —Promises/A+

  26. 26 <?php use Amp\Loop; function asyncMultiply($x, $y) { $deferred =

    new Amp\Deferred; Amp\Loop::delay(1000, function () use ($deferred, $x, $y) { $deferred->resolve($x * $y); }); return $deferred->promise(); } asyncMultiply(6, 7)->onResolve(function ($err, $result) { var_dump($result); // int(42) });
  27. 27 <?php Loop::run(function () { $httpClient = new Amp\Artax\DefaultClient(); $uris

    = [ "google" => "http://www.google.com", "bing" => "http://www.bing.com", ]; $responses = yield array_map(function ($uri) use ($httpClient) { return $httpClient->request($uri); }, $uris); foreach ($responses as $key => $response) { printf("%s | %d\n", $key, $response->getStatus()); } Loop::stop(); });
  28. 28

  29. 29 <?php class WorkerThread extends Thread { private $workerId; public

    function __construct($id) { $this->workerId = $id; } public function run() { // Do some heavy processing } }
  30. 30 <?php $workers = []; foreach (range(0, 5) as $id)

    { $worker = new WorkerThread($id); $workers[] = $worker; $worker->start(); // Now they'll each run independently } foreach (range(0, 5) as $id) { $workers[$id]->join(); // Synchronously rejoin the main process }
  31. 31

  32. 32 <?php Thread::run(function (Channel $channel) { printf("Received: %s\n", yield $channel->receive());

    print "Sleeping for 3 seconds...\n"; sleep(3); // Blocking call in thread. yield $channel->send("Data sent from child."); print "Sleeping for 2 seconds...\n"; sleep(2); // Blocking call in thread. return 42; });
  33. 33 <?php Loop::run(function () { $timer = Loop::repeat(1000, function ()

    { static $i; $i = $i ? ++$i : 1; print "Demonstrating a live parent for the {$i}th time.\n"; }); try { // Do stuff } finally { Loop::cancel(); } });
  34. 34 <?php Loop::run(function () { // Timer try { $context

    = yield Thread::run() // Child print "Waiting 2 seconds to send start data...\n"; yield new Delayed(2000); yield $context->send("Start data"); printf("Received: %s\n", yield $context->receive()); printf("Thread ended: %d\n", yield $context->join()); } finally { Loop::cancel(); } });
  35. 35 Powers things like PDF conversion, video transpiling, APNS Powers

    cross-app messaging in load balanced environments Power concurrent data migrations for large data sources Powers task management in a performance, scalable, managed fashion
  36. 36 Look at what the other communities are doing –

    we often have the same exact tools available, but a much larger community and lower barriers Challenge yourself to look at traditional programming problems from an asynchronous viewpoint – can things run faster in parallel?
  37. 37

  38. 38