Guzzle Promiseを使った
非同期処理によるAPIコールの高速化

Guzzle Promiseを使った
非同期処理によるAPIコールの高速化

JavaScript で非同期処理を実現する Promise という機構はご存知でしょうか?

今回は「Promise の考え方を PHP で実装した Guzzle Promise」を使って、大量の API コールを高速化したときの経験についてお話してみたいと思います。

Talked:
- http://phpcon.fukuoka.jp/

Sample codes:
- https://github.com/suzuki/guzzle-promise-sample

8e528456ff66ec543952daa815353a01?s=128

Norio Suzuki

May 21, 2016
Tweet

Transcript

  1. 6.
  2. 7.
  3. 8.
  4. 11.

    Think of the system • ࣮૷ݴޠ • PHPͰ͸ແཧʁ • Go΍Scala౳͕ద੾ͩͱͯ͠΋࣮૷ܦݧ͸ͳ͍

    • ίʔϧઌͷAPI • ॳΊͯར༻͢ΔAPI • ΞαΠϯ࣌఺Ͱ͸ಈ࡞͕೺ѲͰ͖͍ͯͳ͍ → ॳΊͯͷݴޠͰॳΊͯͷAPIίʔϧ͢Δඞཁ͕ʂʁ
  5. 13.
  6. 18.

    Other magic methods $response = $client->get('http://localhost/'); $response = $client->delete('http://localhost/'); $response

    = $client->head('http://localhost/'); $response = $client->options('http://localhost/'); $response = $client->patch('http://localhost/'); $response = $client->post('http://localhost/'); $response = $client->put('http://localhost/');
  7. 19.

    Using Request object use GuzzleHttp\Client; use GuzzleHttp\Psr7\Request; $client = new

    Client(); $request = new Request( 'GET', 'http://localhost:8008' ); $response = $client->send($request);
  8. 21.
  9. 23.

    server.js var http = require('http'); var sleep = require('sleep-async')(); var

    server = http.createServer(); var sleepMiliSecond = 1000; server.on('request', function(req, res) { console.log('Received.'); sleep.sleep(sleepMiliSecond, function() { res.writeHead(200, {'Content-Type': 'text/json'}); res.write('{"status": "ok"}'); res.end(); }); }); server.listen(8008, 'localhost');
  10. 27.

    sample01.php use GuzzleHttp\Client; $client = new Client(); $response = $client->request(

    'GET', 'http://localhost:8008/' ); echo $response->getBody() . PHP_EOL;
  11. 30.

    sample02.php for ($i = 0; $i < 10; ++$i) {

    $response = $client->request( 'GET', 'http://localhost:8008' ); echo $response->getBody() . PHP_EOL; }
  12. 34.

    ղܾํ๏͸ʁ • σʔλΛ෼ׂ͢Δʁ • ෼ׂ਺ͷ਺͚ͩεΫϦϓτΛىಈʁ • ෳ਺ϓϩηεͰ࣮ߦ͢Δʁ • forkͯ͠ؤுΔʁ •

    ඇಉظϦΫΤετ͕͋ͬͨɺ͜ΕͰͳΜͱ͔ͳΔʂʁ → Guzzle ͷ Async ϦΫΤετΛ࢖ͬͯΈΔ͜ͱʹ
  13. 38.

    sample03.php for ($i = 0; $i < 10; ++$i) {

    $response = $client->requestAsync( 'GET', 'http://localhost:8008/' ); echo $response->getBody() . PHP_EOL; }
  14. 40.
  15. 43.

    sample04.php for ($i = 0; $i < 10; ++$i) {

    $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); // echo $response->getBody() . PHP_EOL; }
  16. 45.

    Summary of sample04.php • PHP Fatal error͸ग़ͳ͘ͳͬͨ • ࣮ߦʹ͸੒ޭ͍ͯ͠Δʁ •

    αʔόଆʹʮReceived.ʯͷද͕ࣔग़͍ͯͳ͍ • ϦΫΤετ͕ಧ͍͍ͯͳ͍ʁʁ • Promise ͬͯͳʹʁ ͋ͷ Promise ͷ͜ͱʁʁ
  17. 46.
  18. 50.

    Promise overview var promise = new Promise(function(resolve, reject) { //

    Async routines if (isSuccess) { resolve(value); } else { reject(error); } });
  19. 51.

    Method Chains sample var promise = new Promise(...); promise .then(function(value)

    { if (isSuccess) { resolve(value); } else { reject(error); }; }) .then(function(value) { if (isSuccess) { resolve(value); } else { reject(error); }; }) .catch(function(error) { // handle error }) ; 1SPNJTF 1SPNJTF 1SPNJTF 1SPNJTF then then catch
  20. 58.

    Guzzle Promise overview use GuzzleHttp\Promise\Promise; $promise = new Promise(function() use

    (&$promise) { // Async routine if ($isSuccess) { $promise->resolve($value); } else { $promise->reject($error); } });
  21. 59.

    Async Request using Promise use GuzzleHttp\Client; use Psr\Http\Message\ResponseInterface; use GuzzleHttp\Exception\RequestException;

    $client = new Client(); $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ->otherwise(function(RequestException $e) { echo $e->getMessage() . PHP_EOL; }) ; $promise->wait();
  22. 60.

    sample05.php for ($i = 0; $i < 10; ++$i) {

    $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ; $promise->wait(); }
  23. 63.

    sample05.php for ($i = 0; $i < 10; ++$i) {

    $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ; $promise->wait(); } Not Async
  24. 64.

    sample06.php use GuzzleHttp\Promise; $promises = []; for ($i = 0;

    $i < 10; ++$i) { $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }); $promises[] = $promise; } Promise\all($promises)->wait();
  25. 66.
  26. 67.

    sample07.php $promises = []; for ($i = 0; $i <

    10000; ++$i) { $promise = $client->requestAsync( 'GET', 'http://localhost:8008/' ); $promise ->then(function(ResponseInterface $response) { echo $response->getBody() . PHP_EOL; }) ; $promises[] = $promise; } Promise\all($promises)->wait();
  27. 69.

    Summary of sample07.php PHP Fatal error: Allowed memory size of

    134217728 bytes exhausted (tried to allocate 72 bytes)
  28. 70.

    Why exhausted memory limit ? • $promises ഑ྻʹ10000ݸͷ Promise Φϒ

    δΣΫτ͕ೖͬͨ··ղ์͞Ε͍ͯͳ͍ • ͦΕͧΕͷ Promise ΦϒδΣΫτ͸ঢ়ଶΛอ ͍࣋ͯ͠ΔͷͰͦΕͳΓʹॏ͍σʔλྔ → Կ͔ղܾํ๏͸ʁ
  29. 71.
  30. 73.

    Generator $requests = function() { for ($i = 0; $i

    < 10; ++$i) { yield new Request( 'GET', 'http://localhost:8008/' ); } };
  31. 74.

    How to use Generator sample08.php use GuzzleHttp\Client; use GuzzleHttp\Psr7\Request; $client

    = new Client(); $requests = function() { for ($i = 0; $i < 10; ++$i) { yield new Request('GET', 'http://localhost:8008/'); } }; foreach ($requests() as $request) { $response = $client->send($request); echo $response->getBody() . PHP_EOL; }
  32. 76.
  33. 77.

    Pool overview use GuzzleHttp\Pool; $pool = new Pool($client, $requests(), [

    'concurrency' => 5, 'fulfilled' => function ($response, $index) { // when success }, 'rejected' => function ($reason, $index) { // when failed }, ]); Generator
  34. 78.

    Pool config LFZ UZQF QVSQPTF DPODVSSFODZ JOUFHFS NBYDPODVSSFODZQBSBNFUFS GVMpMMFE DBMMBCMFGVODUJPO

    $BMMXIFO1SPNJTFJTSFTPMWFE SFKFDUFE DBMMBCMFGVODUJPO $BMMXIFO1SPNJTFJTSFKFDUFE
  35. 79.

    sample09.php $client = new Client(); $requests = function() { for

    ($i = 0; $i < 10; ++$i) { yield new Request('GET', 'http://localhost:8008/'); } }; $pool = new Pool($client, $requests(), [ 'concurrency' => 5, 'fulfilled' => function(ResponseInterface $response, $index) { echo $response->getBody() . PHP_EOL; }, ]); $promise = $pool->promise(); $promise->wait();
  36. 82.

    Concurrency = 10 sample10.php $pool = new Pool($client, $requests(), [

    'concurrency' => 10, 'fulfilled' => function(ResponseInterface $response, $index) { echo $response->getBody() . PHP_EOL; }, ]);
  37. 85.

    sample11.php $client = new Client(); $requests = function() { for

    ($i = 0; $i < 10000; ++$i) { yield new Request('GET', 'http://localhost:8008/'); } }; $pool = new Pool($client, $requests(), [ 'concurrency' => 1000, 'fulfilled' => function(ResponseInterface $response, $index) { echo sprintf("%5d: %s\n", $index, $response->getBody()); }, ]); $promise = $pool->promise(); $promise->wait();
  38. 88.
  39. 89.
  40. 90.
  41. 91.

    ࣮Ҋ݅Ͱͷ Pool ར༻ • concurrency = 100 Λ࠾༻ • APIαʔό΁ͷಉ࣌ΞΫηε਺͸ڐ༰ൣғ

    • ϝϞϦͷ࢖༻ྔ΋ڐ༰ൣғ • ໿50ສϦΫΤετΛ30෼͔͔Βͣʹ׬ྃՄೳ → 1࣌ؒ100ສϦΫΤετΛΫϦΞ
  42. 97.
  43. 98.
  44. 100.

    PHP side links • Guzzle Documentation • http://docs.guzzlephp.org/en/latest/ • Guzzle

    Promises • https://github.com/guzzle/promises • PHP Documentation: Generator • http://php.net/manual/en/language.generators.overview.php • AWS SDK for PHP • https://docs.aws.amazon.com/aws-sdk-php/v3/guide/
  45. 101.

    JavaScript side links • JavaScript Promiseͷຊ • http://azu.github.io/promises-book/ •Promise/A+ •

    https://promisesaplus.com/ •ECMAScript 2015 (ES6) • http://www.ecma-international.org/ecma-262/6.0/ •MDN Generator • https://developer.mozilla.org/en-US/docs/Web/JavaScript/ Reference/Global_Objects/Generator
  46. 102.

    References • Asynchronous API Interaction with Guzzle • https://speakerdeck.com/jeremeamia/asynchronous-api- interaction-with-guzzle

    • JavaScript Promises - Thinking Sync in an Async World • https://speakerdeck.com/kerrick/javascript-promises- thinking-sync-in-an-async-world • guzzleͰඇಉظϦΫΤετΛฒྻॲཧͰ͍ͬͺ͍౤͛Δ • http://hacknote.jp/archives/16095/