Async Guzzle - Concurrent HTTP Requests in PHP

Though PHP is known to be a single-threaded programming language, it's possible to execute HTTP requests concurrently using Guzzle – An HTTP client library for PHP. Guzzle creates a powerful abstraction over multi-cURL and provides a familiar asynchronous interface using Promises. This presentation answers the what, why, and how for using Guzzle's async/concurrent request features. The accompanying demos are located at: https://github.com/azPHP/async-guzzle-demos-2020

Transcript

1. Async Guzzle Concurrent HTTP Requests in PHP By Jeremy Lindblom ( @jeremeamia )
2. Objectives ▫ Understand why async requests can be helpful. ▫ Understand what “async” means in Guzzle. ▫ Learn how to perform async requests in Guzzle. ▫ Learn how to construct async request workflows. ▫ Understand and avoid async/Guzzle pitfalls. 2
3. Important Links ▫ Demos: https:/ /github.com/azPHP/async-guzzle-demos-2020 ▫ Guzzle Code: https:/ /github.com/guzzle/guzzle ▫ Guzzle Docs: http:/ /docs.guzzlephp.org ▫ HttpBin API: https:/ /httpbin.org/ 3
4. “ “We become what we behold. We shape our tools, and thereafter our tools shape us.” ― Marshall McLuhan 4
5. 1. What is “Async”? Let’s make sure we’re on the same page.
6. What is Guzzle? 6 Guzzle PHP HTTP client Abstracts cURL Implements PSR-7 Broad usage Supports concurrent requests HTTP HTTP-based web services Typically “RESTful” APIs Request and response Status codes (e.g., 200, 404) Request methods (e.g., POST) cURL Powerful CLI tool “Transferring data with URLs” Libcurl C library Lots of features Good HTTP client Answer: It’s a pretty awesome HTTP client library for PHP.
7. What is “Async”? 7
8. What is Guzzle’s “Async”? 8
9. Demo 01–02 9
10. What is Guzzle’s “Async”? 10
11. “ 11
12. What it is not: ▫ Fire and Forget ▫ Parallel / Threads ▫ Magic ▫ An Extension ▫ ReactPHP / Amp / Swoole ▫ Async What is Guzzle’s “Async”? 12
13. What it is not: ▫ Fire and Forget ▫ Parallel / Threads ▫ Magic ▫ An Extension ▫ ReactPHP / Amp / Swoole ▫ Async What is Guzzle’s “Async”? What it really is: ▫ Async interface ▫ Optimization for multiple HTTP requests ▫ Concurrent HTTP request execution (non-blocking HTTP I/O) 13
14. What is Guzzle’s “Async”? An asynchronous HTTP client interface using Promises for executing multiple HTTP requests concurrently from PHP, implemented via a cURL-based, non-blocking event loop. 14
15. PROMISES Represents the eventual result of an asynchronous operation. Defined in Promises/A+ spec. 15
16. Promises ▫ Promises start in a pending state ▫ Promises can be resolved in two ways: ▫ fulfilled – With a value (e.g., a Response) ▫ rejected – With a reason (e.g., an Exception) 16 Pending Fulfilled Rejected Value Reason / Exception
17. Promises ▫ Promises can be operated on in these ways: ▫ then($fn) – define what to do with resolved values ▫ otherwise($fn) – define what do with rejected reasons ▫ wait() – blocks until resolved. Either returns the value or throws the reason/exception Note: Any object following the Promises/A+ spec for then() is interoperable with Guzzle’s promises (e.g., React PHP promises). 17
18. Promises ▫ then() and otherwise() return new promises that get resolved once the original promise is. ▫ Chainable: $promise->then()->then()->otherwise() ▫ Within a then/otherwise function: ▫ Throwing an exception, will reject the new promise. ▫ Returning a value, will fulfill the new promise. 18
19. Demo 03 19
20. 2. Why Use “Async”? “Why” is the best question to ask about anything in tech.
21. A Practical Use Case ▫ Three APIs: ▫ Orgs API – districts, schools, and their relationships ▫ People API – users’ info, affiliations, and roles ▫ Sections API – classes, enrollments, and curriculum ▫ Need to combine data to answer a question: “What teachers in the school district are teaching a class with a 3rd grade math curriculum?” 21
22. x ea. 22 (Step 4) Combine Results from 2 and 3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s (Step 3) Sections: Get Teacher Enrollments For each Schools: Filter sections by course/curriculum ID 1.5s Total Time, Synchronous (2 schools) 6.5s (4 schools) 9.5s (6 schools) 12.5s
23. x ea. 23 (Step 4) Combine Results from 2 and 3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s (Step 3) Sections: Get Teacher Enrollments For each School: Filter sections by course/curriculum ID 1.5s Total Time, Asynchronous (2 schools) 3.5s (4 schools) 3.5s (6 schools) 6.5s
24. 24 (Step 4) Combine Results from 2 and 3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s For each School: Filter sections by course/curriculum ID 1.5s Total Time, Asynchronous (2 schools) 3.5s (4 schools) 3.5s 1.5s 1.5s . . . (6 schools) 3.5s
25. Demo 04 25
26. Why Should I Use Async Guzzle? ▫ You can improve performance when doing multiple API calls. ▫ You can develop skills that are transferable to JavaScript. ▫ You find it to be fun, and you like to Think In Async™. 26
27. Why Should I NOT Use Async Guzzle? ▫ You only ever need to make 1 API call at a time. ▫ You are managing async/concurrency in some other way. (e.g., React PHP/amphp, Swoole, pthreads, fork, etc.) ▫ You are vehemently opposed to Promises, and refuse to deviate from your Rx/Observable purity). ▫ You want to avoid the pain that can occur with concurrency. 27
28. 3. How Do I Really Use This? You have to learn how to be effective with any new tool.
29. DO: Share Handlers Between Clients ▫ If working with multiple APIs, construct your clients so they share the same underlying request handler. ▫ If you don’t, the “event loops” will be separate and will be blocking to each other. 29
30. Demo 05-07 30
31. DO: Optimize Your Handler Stack ▫ The default handler stack is designed to support the majority of Guzzle use cases, but is not optimized for any particular case. ▫ Don’t add middleware you don’t need. ▫ Instead of using HandlerStack::create(), use new HandlerStack() and customize. 31
34. 34 // Create custom, optimized handler $handler = new GuzzleHttp\HandlerStack(); $handler->push(GuzzleHttp\Middleware::prepareBody(), 'body'); $handler->push(new My\Project\Auth(), 'auth'); $handler->setHandler(new GuzzleHttp\Handler\CurlMultiHandler()); // Instantiate client with custom handler $client = new GuzzleHttp\Client([ 'handler' => $handler, 'base_uri' => 'https://example.org', ]);
35. DON’T: Interrupt cURL with Other I/O ▫ Guzzle allows you to do concurrency, but only within the context of its cURL-based event loop. ▫ I/O triggered by other things (DB, files, other HTTP clients) will block Guzzle’s async loop, and nullify any benefits from the attempted concurrency. ▫ File streams managed by cURL/Guzzle are OK. 35
36. DO: Wait() ▫ Remember: Guzzle’s Async isn’t _really_ async. ▫ Async requests don’t start immediately; they get queued up in the cURL event loop. ▫ Don’t expect it to do anything until you wait() on a promise. $result = $promise->wait(); ▫ If the program terminates (exit, exception, etc.), it might not even run before the PHP process is ended. 36
37. DON’T: Wait Too Early ▫ It’s best to keep things within the async context as long as you can. Creates more opportunities for concurrency. ▫ Don’t wait() until you need to leave the async context (e.g., generate HTTP response, access DB, etc.) ▫ Libraries with async operations should never wait(), that is the library consumer’s job. 37
38. DO: Use Guzzle’s Promise Helpers ▫ promise_for($value) ▫ rejection_for($reason) ▫ unwrap($promises) ▫ all($promises) ▫ some($count, $promises) ▫ any($promises) ▫ coroutine($fn) ▫ is_fulfilled($promise) ▫ is_rejected($promise) is_settled($promise) 38 ▫ each( $promises, $fulfilledFn, $rejectedFn ) ▫ each_limit( $promises, $concurrency, $fulfilledFn, $rejectedFn )
39. DO: Limit Concurrency Pool Size ▫ The client’s system resources are not unlimited (e.g., CPU, Memory, File Descriptors) ▫ The service’s system resources are not unlimited (e.g., Rate Limiting, TOS Violations, DDOS) ▫ Use tools like each_limit() or GuzzleHttp\Pool to limit how many active requests are being executed. ▫ The “delay” request option is also useful, and is implemented in an async-friendly (i.e., not with sleep). 39
40. DO: Use Coroutines ▫ Coroutines are generator-based functions that yield promises and have results sent back into them. ▫ The entire coroutine itself is also a promise. ▫ Allows for making async code look normal. ▫ About as close to async/await as you can get in PHP. ▫ coroutine ~= async ▫ yield ~= await 40
41. Demo 08 41
42. Objectives ▫ ✅ Understand why async requests can be helpful. ▫ ✅ Understand what “async” means in Guzzle. ▫ ✅ Learn how to perform async requests in Guzzle. ▫ ✅ Learn how to construct async request workflows. ▫ ✅ Understand and avoid async/Guzzle pitfalls. 42
43. “ “We become what we behold. We shape our tools, and thereafter our tools shape us.” ― Marshall McLuhan 43 I’m excited to see how Async Guzzle shapes you and how you work.
44. 44 Thanks! Any questions? You can find me at @jeremeamia on Twitter
45. Credits ▫ Presentation template by SlidesCarnival ▫ Guzzle was created by Michael Dowling 45