Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Asynchronous Awesome

Eric Mann
October 23, 2019

Asynchronous Awesome

Sometimes, our use of PHP grows beyond the typical request/response cycle of dynamic page generation. Unfortunately, the threaded nature of PHP - and the stateless nature of the server - betrays any efforts to expand our utilization of the server. Image processing, video rendering, APNS (Apple Push Notification Service) integration - any of these can easily take longer than is reasonable for a simple page request.

Enter tools like message and job queues that empower daemonized PHP workers to handle data processing in the background. Yet further tools enable long-running event loops and asynchronous Promise-driven operations. PHP isn’t multi-threaded, but that doesn’t mean you’re limited to a single-thread paradigm. I will demonstrate various use cases necessitating asynchronous operations, then delve into the code and the tools that make these systems work. Every attendee will leave armed with new ways to think about the management of large data jobs in PHP and an understanding of the tools they can use to make it happen.

Eric Mann

October 23, 2019
Tweet

More Decks by Eric Mann

Other Decks in Technology

Transcript

  1. Asynchronous
    Awesome
    Eric Mann

    View Slide

  2. View Slide

  3. View Slide

  4. Problems this Won’t Solve
    Report generation
    PubSub events
    Parallel I/O
    Parallel data access

    View Slide

  5. Naive Approaches to Async
    Pseudo-cron (WordPress native and tools like wp-async-task)
    Mechanical Turk-like job systems
    Using another language entirely

    View Slide

  6. There’s a Better Way

    View Slide

  7. Parallel vs Concurrent
    Concurrent
    Parallel (not covered today)
    Concurrent Parallel

    View Slide

  8. Concurrent and Parallel Programming - Joe Armstrong
    https://joearms.github.io/published/2013-04-05-concurrent-and-parallel-programming.html

    View Slide

  9. Tools Available
    Gearman
    RabbitMQ
    Amp (promises)
    pthreads

    View Slide

  10. Gearman

    View Slide

  11. Initialize a Standalone Worker
    // worker.php
    $importer = new PDFImporter();
    $worker = new GearmanWorker();
    $worker->addServer('localhost', 4730);
    $worker->addFunction('import', array($importer, 'import');
    while($worker->work());

    View Slide

  12. Dispatch Jobs to the Worker
    // app.php
    $client = new GearmanClient();
    $client->addServer('localhost', 4730);
    $job = $client->doBackground('import', $file . '|:|' . $email);
    $db->record($job);

    View Slide

  13. RabbitMQ

    View Slide

  14. View Slide

  15. Publish to the Queue
    $connection = new AMQPStreamConnection('localhost', 5672, 'u', 'p');
    $channel = $connection->channel();
    $channel->queue_declare('demo', false, false, false, false);
    $msg = new AMQPMessage('Hello World!');
    $channel->basic_publish($msg, '', 'demo');
    $channel->close();
    $connection->close();

    View Slide

  16. Consume from the Queue
    $connection = new AMQPStreamConnection('localhost', 5672, 'u', 'p');
    $channel = $connection->channel();
    $channel->queue_declare('demo', false, false, false, false);
    $cb = function($msg) {
    // Send Tweet with $msg contents
    }
    $channel->basic_consume('demo', '', false, true, false, false, $cb);
    while(count($channel->callbacks)) {
    $channel->wait();
    }

    View Slide

  17. View Slide

  18. Interact with Multiple Queues
    $channel->queue_declare('demo', false, false, false, false);
    $channel->queue_declare('demo-rc', false, false, false, false);
    $cb = function($msg) {
    // Send Tweet with $msg contents
    $result = new AMQPMessage( /* Tweet ID */ );
    $channel->basic_publish($result, '', 'demo-rc');
    }
    $channel->basic_consume('demo', '', false, true, false, false, $cb);
    while(count($channel->callbacks)) {
    $channel->wait();
    }

    View Slide

  19. Amp (promises)

    View Slide

  20. Deferred Empowers Async
    use Amp\Loop;
    function asyncMultiply($x, $y)
    {
    $deferred = new Amp\Deferred;
    // Resolve the async result one second from now
    Amp\Loop::delay(1000, function () use ($deferred, $x, $y) {
    $deferred->resolve($x * $y);
    });
    return $deferred->promise();
    }
    asyncMultiply(6, 7)->onResolve(function ($err, $result) {
    var_dump($result); // int(42)
    });

    View Slide

  21. Chain Multiple Promises Together
    Loop::run(function () {
    $httpClient = new Amp\Artax\DefaultClient();
    $uris = [
    "google" => "http://www.google.com",
    "bing" => "http://www.bing.com",
    ];
    $responses = yield array_map(function ($uri) use ($httpClient) {
    return $httpClient->request($uri);
    }, $uris);
    foreach ($responses as $key => $response) {
    printf("%s | %d\n", $key, $response->getStatus());
    }
    Loop::stop();
    });

    View Slide

  22. pthreads

    View Slide

  23. Create a Worker
    class WorkerThread extends Thread
    {
    private $workerId;
    public function __construct($id)
    {
    $this->workerId = $id;
    }
    public function run()
    {
    // Do some heavy processing
    }
    }

    View Slide

  24. Start the Workers
    $workers = [];
    foreach (range(0, 5) as $id) {
    $worker = new WorkerThread($id);
    $workers[] = $worker;
    $worker->start(); // Now they'll each run independently
    }
    foreach (range(0, 5) as $id) {
    $workers[$id]->join(); // Synchronously rejoin the main process
    }

    View Slide

  25. What Use is Any of This?
    Gearman – powers things like PDF conversion, video transpiling, APNS
    RabbitMQ – powers cross-app messaging in load balanced environments
    Promises – power concurrent data migrations for large data sources
    pthreads – powers task management in a performance, scalable, managed fashion

    View Slide

  26. Further Study
    The amphp/parallel library wraps pthreads entirely – use both at the same time!
    Look at what the Node community is doing – we have the same exact tools
    available, but a much larger community and lower barriers
    Challenge yourself to look at traditional programming problems from an
    asynchronous viewpoint – can things run faster in parallel?

    View Slide

  27. Questions?

    View Slide

  28. Thank you
    [email protected] | 503.925.6266

    View Slide