Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Async Guzzle - Concurrent HTTP Requests in PHP

Async Guzzle - Concurrent HTTP Requests in PHP

Though PHP is known to be a single-threaded programming language, it's possible to execute HTTP requests concurrently using Guzzle – An HTTP client library for PHP. Guzzle creates a powerful abstraction over multi-cURL and provides a familiar asynchronous interface using Promises. This presentation answers the what, why, and how for using Guzzle's async/concurrent request features. The accompanying demos are located at: https://github.com/azPHP/async-guzzle-demos-2020

Ca57a7cfac69ba3abf517470f3770aae?s=128

Jeremy Lindblom

August 25, 2020
Tweet

Transcript

  1. Async Guzzle Concurrent HTTP Requests in PHP By Jeremy Lindblom

    ( @jeremeamia )
  2. Objectives ▫ Understand why async requests can be helpful. ▫

    Understand what “async” means in Guzzle. ▫ Learn how to perform async requests in Guzzle. ▫ Learn how to construct async request workflows. ▫ Understand and avoid async/Guzzle pitfalls. 2
  3. Important Links ▫ Demos: https:/ /github.com/azPHP/async-guzzle-demos-2020 ▫ Guzzle Code: https:/

    /github.com/guzzle/guzzle ▫ Guzzle Docs: http:/ /docs.guzzlephp.org ▫ HttpBin API: https:/ /httpbin.org/ 3
  4. “ “We become what we behold. We shape our tools,

    and thereafter our tools shape us.” ― Marshall McLuhan 4
  5. 1. What is “Async”? Let’s make sure we’re on the

    same page.
  6. What is Guzzle? 6 Guzzle PHP HTTP client Abstracts cURL

    Implements PSR-7 Broad usage Supports concurrent requests HTTP HTTP-based web services Typically “RESTful” APIs Request and response Status codes (e.g., 200, 404) Request methods (e.g., POST) cURL Powerful CLI tool “Transferring data with URLs” Libcurl C library Lots of features Good HTTP client Answer: It’s a pretty awesome HTTP client library for PHP.
  7. What is “Async”? 7

  8. What is Guzzle’s “Async”? 8

  9. Demo 01–02 9

  10. What is Guzzle’s “Async”? 10

  11. “ 11

  12. What it is not: ▫ Fire and Forget ▫ Parallel

    / Threads ▫ Magic ▫ An Extension ▫ ReactPHP / Amp / Swoole ▫ Async What is Guzzle’s “Async”? 12
  13. What it is not: ▫ Fire and Forget ▫ Parallel

    / Threads ▫ Magic ▫ An Extension ▫ ReactPHP / Amp / Swoole ▫ Async What is Guzzle’s “Async”? What it really is: ▫ Async interface ▫ Optimization for multiple HTTP requests ▫ Concurrent HTTP request execution (non-blocking HTTP I/O) 13
  14. What is Guzzle’s “Async”? An asynchronous HTTP client interface using

    Promises for executing multiple HTTP requests concurrently from PHP, implemented via a cURL-based, non-blocking event loop. 14
  15. PROMISES Represents the eventual result of an asynchronous operation. Defined

    in Promises/A+ spec. 15
  16. Promises ▫ Promises start in a pending state ▫ Promises

    can be resolved in two ways: ▫ fulfilled – With a value (e.g., a Response) ▫ rejected – With a reason (e.g., an Exception) 16 Pending Fulfilled Rejected Value Reason / Exception
  17. Promises ▫ Promises can be operated on in these ways:

    ▫ then($fn) – define what to do with resolved values ▫ otherwise($fn) – define what do with rejected reasons ▫ wait() – blocks until resolved. Either returns the value or throws the reason/exception Note: Any object following the Promises/A+ spec for then() is interoperable with Guzzle’s promises (e.g., React PHP promises). 17
  18. Promises ▫ then() and otherwise() return new promises that get

    resolved once the original promise is. ▫ Chainable: $promise->then()->then()->otherwise() ▫ Within a then/otherwise function: ▫ Throwing an exception, will reject the new promise. ▫ Returning a value, will fulfill the new promise. 18
  19. Demo 03 19

  20. 2. Why Use “Async”? “Why” is the best question to

    ask about anything in tech.
  21. A Practical Use Case ▫ Three APIs: ▫ Orgs API

    – districts, schools, and their relationships ▫ People API – users’ info, affiliations, and roles ▫ Sections API – classes, enrollments, and curriculum ▫ Need to combine data to answer a question: “What teachers in the school district are teaching a class with a 3rd grade math curriculum?” 21
  22. x ea. 22 (Step 4) Combine Results from 2 and

    3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s (Step 3) Sections: Get Teacher Enrollments For each Schools: Filter sections by course/curriculum ID 1.5s Total Time, Synchronous (2 schools) 6.5s (4 schools) 9.5s (6 schools) 12.5s
  23. x ea. 23 (Step 4) Combine Results from 2 and

    3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s (Step 3) Sections: Get Teacher Enrollments For each School: Filter sections by course/curriculum ID 1.5s Total Time, Asynchronous (2 schools) 3.5s (4 schools) 3.5s (6 schools) 6.5s
  24. 24 (Step 4) Combine Results from 2 and 3 (Step

    2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s For each School: Filter sections by course/curriculum ID 1.5s Total Time, Asynchronous (2 schools) 3.5s (4 schools) 3.5s 1.5s 1.5s . . . (6 schools) 3.5s
  25. Demo 04 25

  26. Why Should I Use Async Guzzle? ▫ You can improve

    performance when doing multiple API calls. ▫ You can develop skills that are transferable to JavaScript. ▫ You find it to be fun, and you like to Think In Async™. 26
  27. Why Should I NOT Use Async Guzzle? ▫ You only

    ever need to make 1 API call at a time. ▫ You are managing async/concurrency in some other way. (e.g., React PHP/amphp, Swoole, pthreads, fork, etc.) ▫ You are vehemently opposed to Promises, and refuse to deviate from your Rx/Observable purity). ▫ You want to avoid the pain that can occur with concurrency. 27
  28. 3. How Do I Really Use This? You have to

    learn how to be effective with any new tool.
  29. DO: Share Handlers Between Clients ▫ If working with multiple

    APIs, construct your clients so they share the same underlying request handler. ▫ If you don’t, the “event loops” will be separate and will be blocking to each other. 29
  30. Demo 05-07 30

  31. DO: Optimize Your Handler Stack ▫ The default handler stack

    is designed to support the majority of Guzzle use cases, but is not optimized for any particular case. ▫ Don’t add middleware you don’t need. ▫ Instead of using HandlerStack::create(), use new HandlerStack() and customize. 31
  32. 32

  33. 33

  34. 34 // Create custom, optimized handler $handler = new GuzzleHttp\HandlerStack();

    $handler->push(GuzzleHttp\Middleware::prepareBody(), 'body'); $handler->push(new My\Project\Auth(), 'auth'); $handler->setHandler(new GuzzleHttp\Handler\CurlMultiHandler()); // Instantiate client with custom handler $client = new GuzzleHttp\Client([ 'handler' => $handler, 'base_uri' => 'https://example.org', ]);
  35. DON’T: Interrupt cURL with Other I/O ▫ Guzzle allows you

    to do concurrency, but only within the context of its cURL-based event loop. ▫ I/O triggered by other things (DB, files, other HTTP clients) will block Guzzle’s async loop, and nullify any benefits from the attempted concurrency. ▫ File streams managed by cURL/Guzzle are OK. 35
  36. DO: Wait() ▫ Remember: Guzzle’s Async isn’t _really_ async. ▫

    Async requests don’t start immediately; they get queued up in the cURL event loop. ▫ Don’t expect it to do anything until you wait() on a promise. $result = $promise->wait(); ▫ If the program terminates (exit, exception, etc.), it might not even run before the PHP process is ended. 36
  37. DON’T: Wait Too Early ▫ It’s best to keep things

    within the async context as long as you can. Creates more opportunities for concurrency. ▫ Don’t wait() until you need to leave the async context (e.g., generate HTTP response, access DB, etc.) ▫ Libraries with async operations should never wait(), that is the library consumer’s job. 37
  38. DO: Use Guzzle’s Promise Helpers ▫ promise_for($value) ▫ rejection_for($reason) ▫

    unwrap($promises) ▫ all($promises) ▫ some($count, $promises) ▫ any($promises) ▫ coroutine($fn) ▫ is_fulfilled($promise) ▫ is_rejected($promise) is_settled($promise) 38 ▫ each( $promises, $fulfilledFn, $rejectedFn ) ▫ each_limit( $promises, $concurrency, $fulfilledFn, $rejectedFn )
  39. DO: Limit Concurrency Pool Size ▫ The client’s system resources

    are not unlimited (e.g., CPU, Memory, File Descriptors) ▫ The service’s system resources are not unlimited (e.g., Rate Limiting, TOS Violations, DDOS) ▫ Use tools like each_limit() or GuzzleHttp\Pool to limit how many active requests are being executed. ▫ The “delay” request option is also useful, and is implemented in an async-friendly (i.e., not with sleep). 39
  40. DO: Use Coroutines ▫ Coroutines are generator-based functions that yield

    promises and have results sent back into them. ▫ The entire coroutine itself is also a promise. ▫ Allows for making async code look normal. ▫ About as close to async/await as you can get in PHP. ▫ coroutine ~= async ▫ yield ~= await 40
  41. Demo 08 41

  42. Objectives ▫ ✅ Understand why async requests can be helpful.

    ▫ ✅ Understand what “async” means in Guzzle. ▫ ✅ Learn how to perform async requests in Guzzle. ▫ ✅ Learn how to construct async request workflows. ▫ ✅ Understand and avoid async/Guzzle pitfalls. 42
  43. “ “We become what we behold. We shape our tools,

    and thereafter our tools shape us.” ― Marshall McLuhan 43 I’m excited to see how Async Guzzle shapes you and how you work.
  44. 44 Thanks! Any questions? You can find me at @jeremeamia

    on Twitter
  45. Credits ▫ Presentation template by SlidesCarnival ▫ Guzzle was created

    by Michael Dowling 45