Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Asynchronous Awesome

Eric Mann
February 07, 2020

Asynchronous Awesome

Sometimes, our use of PHP grows beyond the typical request/response cycle of dynamic page generation. Unfortunately, the threaded nature of PHP - and the stateless nature of the server - betrays any efforts to expand our utilization of the server. Image processing, video rendering, APNS (Apple Push Notification Service) integration - any of these can easily take longer than is reasonable for a simple page request. Enter tools like message and job queues that empower daemonized PHP workers to handle data processing in the background. Yet further tools enable long-running event loops and asynchronous Promise-driven operations. PHP isn't multi-threaded, but that doesn't mean you're limited to a single-thread paradigm. I will demonstrate various use cases necessitating asynchronous operations, then delve into the code and the tools that make these systems work.

Every attendee will leave armed with new ways to think about the management of large data jobs in PHP and an understanding of the tools they can use to make it happen.

Eric Mann

February 07, 2020
Tweet

More Decks by Eric Mann

Other Decks in Technology

Transcript

  1. 1

    View Slide

  2. 2
    Hello.
    Eric Mann
    Director of Engineering, Vacasa

    View Slide

  3. 3

    View Slide

  4. 4
    In 2012, I had to build a way for the
    “Save PDF” button on this screen
    to work. It required building a
    PDF in the background and
    would sometimes take up to
    5 minutes to work.
    Doing things asynchronously
    was hard!

    View Slide

  5. 5
    Report generation
    PubSub events
    Parallel I/O
    Parallel data access

    View Slide

  6. 6
    Pseudo-cron (WordPress native and tools like wp-async-task)

    View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. 10
    Pseudo-cron (WordPress native and tools like wp-async-task)
    Mechanical Turk-like job systems
    Using another language entirely

    View Slide

  11. 11

    View Slide

  12. 12
    Lorem ipsum dolor sit amet, consectetur
    adipiscing elit. Fusce elit ex, consequat et
    tincidunt non, pharetra non risus. Quisque ut
    leo pretium, eleifend lectus in, ultrices diam.
    Quisque ac congue urna, non finibus orci.
    Concurrent
    Parallel (not covered today)
    Concurrent Parallel
    Image by Joe Armstrong - https://joearms.github.io/published/2013-04-05-concurrent-and-parallel-programming.html

    View Slide

  13. 13
    Lorem ipsum dolor sit amet, consectetur
    adipiscing elit. Fusce elit ex, consequat et
    tincidunt non, pharetra non risus. Quisque ut
    leo pretium, eleifend lectus in, ultrices diam.
    Quisque ac congue urna, non finibus orci.
    Concurrent
    Parallel (not covered today)
    Concurrent Parallel
    Image by Joe Armstrong - https://joearms.github.io/published/2013-04-05-concurrent-and-parallel-programming.html

    View Slide

  14. 14

    View Slide

  15. 15

    View Slide

  16. 16
    // worker.php
    $importer = new PDFImporter();
    $worker = new GearmanWorker();
    $worker->addServer('localhost', 4730);
    $worker->addFunction('import', array($importer, 'import');
    while($worker->work());

    View Slide

  17. 17
    // app.php
    $client = new GearmanClient();
    $client->addServer('localhost', 4730);
    $job = $client->doBackground('import', $file . '|:|' . $email);
    $db->record($job);

    View Slide

  18. 18

    View Slide

  19. View Slide

  20. 20
    $connection = new AMQPStreamConnection('localhost', 5672, 'u', 'p');
    $channel = $connection->channel();
    $channel->queue_declare('demo', false, false, false, false);
    $msg = new AMQPMessage('Hello World!');
    $channel->basic_publish($msg, '', 'demo');
    $channel->close();
    $connection->close();

    View Slide

  21. 21
    $connection = new AMQPStreamConnection('localhost', 5672, 'u', 'p');
    $channel = $connection->channel();
    $channel->queue_declare('demo', false, false, false, false);
    $cb = function($msg) {
    // Send Tweet with $msg contents
    }
    $channel->basic_consume('demo', '', false, true, false, false, $cb);
    while(count($channel->callbacks)) {
    $channel->wait();
    }

    View Slide

  22. View Slide

  23. 23
    $channel->queue_declare('demo', false, false, false, false);
    $channel->queue_declare('demo-rc', false, false, false, false);
    $cb = function($msg) {
    // Send Tweet with $msg contents
    $result = new AMQPMessage( /* Tweet ID */ );
    $channel->basic_publish($result, '', 'demo-rc');
    }
    $channel->basic_consume('demo', '', false, true, false, false, $cb);
    while(count($channel->callbacks)) {
    $channel->wait();
    }

    View Slide

  24. 24

    View Slide

  25. 25
    —Promises/A+

    View Slide

  26. 26
    use Amp\Loop;
    function asyncMultiply($x, $y)
    {
    $deferred = new Amp\Deferred;
    Amp\Loop::delay(1000, function () use ($deferred, $x, $y) {
    $deferred->resolve($x * $y);
    });
    return $deferred->promise();
    }
    asyncMultiply(6, 7)->onResolve(function ($err, $result) {
    var_dump($result); // int(42)
    });

    View Slide

  27. 27
    Loop::run(function () {
    $httpClient = new Amp\Artax\DefaultClient();
    $uris = [
    "google" => "http://www.google.com",
    "bing" => "http://www.bing.com",
    ];
    $responses = yield array_map(function ($uri) use ($httpClient) {
    return $httpClient->request($uri);
    }, $uris);
    foreach ($responses as $key => $response) {
    printf("%s | %d\n", $key, $response->getStatus());
    }
    Loop::stop();
    });

    View Slide

  28. 28

    View Slide

  29. 29
    class WorkerThread extends Thread
    {
    private $workerId;
    public function __construct($id)
    {
    $this->workerId = $id;
    }
    public function run()
    {
    // Do some heavy processing
    }
    }

    View Slide

  30. 30
    $workers = [];
    foreach (range(0, 5) as $id) {
    $worker = new WorkerThread($id);
    $workers[] = $worker;
    $worker->start(); // Now they'll each run independently
    }
    foreach (range(0, 5) as $id) {
    $workers[$id]->join(); // Synchronously rejoin the main process
    }

    View Slide

  31. 31

    View Slide

  32. 32
    Thread::run(function (Channel $channel) {
    printf("Received: %s\n", yield $channel->receive());
    print "Sleeping for 3 seconds...\n";
    sleep(3); // Blocking call in thread.
    yield $channel->send("Data sent from child.");
    print "Sleeping for 2 seconds...\n";
    sleep(2); // Blocking call in thread.
    return 42;
    });

    View Slide

  33. 33
    Loop::run(function () {
    $timer = Loop::repeat(1000, function () {
    static $i;
    $i = $i ? ++$i : 1;
    print "Demonstrating a live parent for the {$i}th time.\n";
    });
    try {
    // Do stuff
    } finally {
    Loop::cancel();
    }
    });

    View Slide

  34. 34
    Loop::run(function () {
    // Timer
    try {
    $context = yield Thread::run() // Child
    print "Waiting 2 seconds to send start data...\n";
    yield new Delayed(2000);
    yield $context->send("Start data");
    printf("Received: %s\n", yield $context->receive());
    printf("Thread ended: %d\n", yield $context->join());
    } finally {
    Loop::cancel();
    }
    });

    View Slide

  35. 35
    Powers things like PDF conversion, video transpiling, APNS
    Powers cross-app messaging in load balanced environments
    Power concurrent data migrations for large data sources
    Powers task management in a performance, scalable, managed fashion

    View Slide

  36. 36
    Look at what the other communities are doing – we often have the
    same exact tools available, but a much larger community and lower
    barriers
    Challenge yourself to look at traditional programming problems
    from an asynchronous viewpoint – can things run faster in parallel?

    View Slide

  37. 37

    View Slide

  38. 38

    View Slide