Async Guzzle - Concurrent HTTP Requests in PHP

Slide 1

Slide 1 text

Async Guzzle Concurrent HTTP Requests in PHP By Jeremy Lindblom ( @jeremeamia )

Slide 2

Slide 2 text

Objectives ▫ Understand why async requests can be helpful. ▫ Understand what “async” means in Guzzle. ▫ Learn how to perform async requests in Guzzle. ▫ Learn how to construct async request workflows. ▫ Understand and avoid async/Guzzle pitfalls. 2

Slide 3

Slide 3 text

Important Links ▫ Demos: https:/ /github.com/azPHP/async-guzzle-demos-2020 ▫ Guzzle Code: https:/ /github.com/guzzle/guzzle ▫ Guzzle Docs: http:/ /docs.guzzlephp.org ▫ HttpBin API: https:/ /httpbin.org/ 3

Slide 4

Slide 4 text

“ “We become what we behold. We shape our tools, and thereafter our tools shape us.” ― Marshall McLuhan 4

Slide 5

Slide 5 text

1. What is “Async”? Let’s make sure we’re on the same page.

Slide 6

Slide 6 text

What is Guzzle? 6 Guzzle PHP HTTP client Abstracts cURL Implements PSR-7 Broad usage Supports concurrent requests HTTP HTTP-based web services Typically “RESTful” APIs Request and response Status codes (e.g., 200, 404) Request methods (e.g., POST) cURL Powerful CLI tool “Transferring data with URLs” Libcurl C library Lots of features Good HTTP client Answer: It’s a pretty awesome HTTP client library for PHP.

Slide 7

Slide 7 text

What is “Async”? 7

Slide 8

Slide 8 text

What is Guzzle’s “Async”? 8

Slide 9

Slide 9 text

Demo 01–02 9

Slide 10

Slide 10 text

What is Guzzle’s “Async”? 10

Slide 11

Slide 11 text

“ 11

Slide 12

Slide 12 text

What it is not: ▫ Fire and Forget ▫ Parallel / Threads ▫ Magic ▫ An Extension ▫ ReactPHP / Amp / Swoole ▫ Async What is Guzzle’s “Async”? 12

Slide 13

Slide 13 text

What it is not: ▫ Fire and Forget ▫ Parallel / Threads ▫ Magic ▫ An Extension ▫ ReactPHP / Amp / Swoole ▫ Async What is Guzzle’s “Async”? What it really is: ▫ Async interface ▫ Optimization for multiple HTTP requests ▫ Concurrent HTTP request execution (non-blocking HTTP I/O) 13

Slide 14

Slide 14 text

What is Guzzle’s “Async”? An asynchronous HTTP client interface using Promises for executing multiple HTTP requests concurrently from PHP, implemented via a cURL-based, non-blocking event loop. 14

Slide 15

Slide 15 text

PROMISES Represents the eventual result of an asynchronous operation. Defined in Promises/A+ spec. 15

Slide 16

Slide 16 text

Promises ▫ Promises start in a pending state ▫ Promises can be resolved in two ways: ▫ fulfilled – With a value (e.g., a Response) ▫ rejected – With a reason (e.g., an Exception) 16 Pending Fulfilled Rejected Value Reason / Exception

Slide 17

Slide 17 text

Promises ▫ Promises can be operated on in these ways: ▫ then($fn) – define what to do with resolved values ▫ otherwise($fn) – define what do with rejected reasons ▫ wait() – blocks until resolved. Either returns the value or throws the reason/exception Note: Any object following the Promises/A+ spec for then() is interoperable with Guzzle’s promises (e.g., React PHP promises). 17

Slide 18

Slide 18 text

Promises ▫ then() and otherwise() return new promises that get resolved once the original promise is. ▫ Chainable: $promise->then()->then()->otherwise() ▫ Within a then/otherwise function: ▫ Throwing an exception, will reject the new promise. ▫ Returning a value, will fulfill the new promise. 18

Slide 19

Slide 19 text

Demo 03 19

Slide 20

Slide 20 text

2. Why Use “Async”? “Why” is the best question to ask about anything in tech.

Slide 21

Slide 21 text

A Practical Use Case ▫ Three APIs: ▫ Orgs API – districts, schools, and their relationships ▫ People API – users’ info, affiliations, and roles ▫ Sections API – classes, enrollments, and curriculum ▫ Need to combine data to answer a question: “What teachers in the school district are teaching a class with a 3rd grade math curriculum?” 21

Slide 22

Slide 22 text

x ea. 22 (Step 4) Combine Results from 2 and 3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s (Step 3) Sections: Get Teacher Enrollments For each Schools: Filter sections by course/curriculum ID 1.5s Total Time, Synchronous (2 schools) 6.5s (4 schools) 9.5s (6 schools) 12.5s

Slide 23

Slide 23 text

x ea. 23 (Step 4) Combine Results from 2 and 3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s (Step 3) Sections: Get Teacher Enrollments For each School: Filter sections by course/curriculum ID 1.5s Total Time, Asynchronous (2 schools) 3.5s (4 schools) 3.5s (6 schools) 6.5s

Slide 24

Slide 24 text

24 (Step 4) Combine Results from 2 and 3 (Step 2) People: Search for Teachers Filter people by partial name (e.g., “st” -> “steve”) 3s (Step 1) Orgs: Get Schools in District 0.5s For each School: Filter sections by course/curriculum ID 1.5s Total Time, Asynchronous (2 schools) 3.5s (4 schools) 3.5s 1.5s 1.5s . . . (6 schools) 3.5s

Slide 25

Slide 25 text

Demo 04 25

Slide 26

Slide 26 text

Why Should I Use Async Guzzle? ▫ You can improve performance when doing multiple API calls. ▫ You can develop skills that are transferable to JavaScript. ▫ You find it to be fun, and you like to Think In Async™. 26

Slide 27

Slide 27 text

Why Should I NOT Use Async Guzzle? ▫ You only ever need to make 1 API call at a time. ▫ You are managing async/concurrency in some other way. (e.g., React PHP/amphp, Swoole, pthreads, fork, etc.) ▫ You are vehemently opposed to Promises, and refuse to deviate from your Rx/Observable purity). ▫ You want to avoid the pain that can occur with concurrency. 27

Slide 28

Slide 28 text

3. How Do I Really Use This? You have to learn how to be effective with any new tool.

Slide 29

Slide 29 text

DO: Share Handlers Between Clients ▫ If working with multiple APIs, construct your clients so they share the same underlying request handler. ▫ If you don’t, the “event loops” will be separate and will be blocking to each other. 29

Slide 30

Slide 30 text

Demo 05-07 30

Slide 31

Slide 31 text

DO: Optimize Your Handler Stack ▫ The default handler stack is designed to support the majority of Guzzle use cases, but is not optimized for any particular case. ▫ Don’t add middleware you don’t need. ▫ Instead of using HandlerStack::create(), use new HandlerStack() and customize. 31

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

34 // Create custom, optimized handler $handler = new GuzzleHttp\HandlerStack(); $handler->push(GuzzleHttp\Middleware::prepareBody(), 'body'); $handler->push(new My\Project\Auth(), 'auth'); $handler->setHandler(new GuzzleHttp\Handler\CurlMultiHandler()); // Instantiate client with custom handler $client = new GuzzleHttp\Client([ 'handler' => $handler, 'base_uri' => 'https://example.org', ]);

Slide 35

Slide 35 text

DON’T: Interrupt cURL with Other I/O ▫ Guzzle allows you to do concurrency, but only within the context of its cURL-based event loop. ▫ I/O triggered by other things (DB, files, other HTTP clients) will block Guzzle’s async loop, and nullify any benefits from the attempted concurrency. ▫ File streams managed by cURL/Guzzle are OK. 35

Slide 36

Slide 36 text

DO: Wait() ▫ Remember: Guzzle’s Async isn’t _really_ async. ▫ Async requests don’t start immediately; they get queued up in the cURL event loop. ▫ Don’t expect it to do anything until you wait() on a promise. $result = $promise->wait(); ▫ If the program terminates (exit, exception, etc.), it might not even run before the PHP process is ended. 36

Slide 37

Slide 37 text

DON’T: Wait Too Early ▫ It’s best to keep things within the async context as long as you can. Creates more opportunities for concurrency. ▫ Don’t wait() until you need to leave the async context (e.g., generate HTTP response, access DB, etc.) ▫ Libraries with async operations should never wait(), that is the library consumer’s job. 37

Slide 38

Slide 38 text

DO: Use Guzzle’s Promise Helpers ▫ promise_for($value) ▫ rejection_for($reason) ▫ unwrap($promises) ▫ all($promises) ▫ some($count, $promises) ▫ any($promises) ▫ coroutine($fn) ▫ is_fulfilled($promise) ▫ is_rejected($promise) is_settled($promise) 38 ▫ each( $promises, $fulfilledFn, $rejectedFn ) ▫ each_limit( $promises, $concurrency, $fulfilledFn, $rejectedFn )

Slide 39

Slide 39 text

DO: Limit Concurrency Pool Size ▫ The client’s system resources are not unlimited (e.g., CPU, Memory, File Descriptors) ▫ The service’s system resources are not unlimited (e.g., Rate Limiting, TOS Violations, DDOS) ▫ Use tools like each_limit() or GuzzleHttp\Pool to limit how many active requests are being executed. ▫ The “delay” request option is also useful, and is implemented in an async-friendly (i.e., not with sleep). 39

Slide 40

Slide 40 text

DO: Use Coroutines ▫ Coroutines are generator-based functions that yield promises and have results sent back into them. ▫ The entire coroutine itself is also a promise. ▫ Allows for making async code look normal. ▫ About as close to async/await as you can get in PHP. ▫ coroutine ~= async ▫ yield ~= await 40

Slide 41

Slide 41 text

Demo 08 41

Slide 42

Slide 42 text

Objectives ▫ ✅ Understand why async requests can be helpful. ▫ ✅ Understand what “async” means in Guzzle. ▫ ✅ Learn how to perform async requests in Guzzle. ▫ ✅ Learn how to construct async request workflows. ▫ ✅ Understand and avoid async/Guzzle pitfalls. 42

Slide 43

Slide 43 text

“ “We become what we behold. We shape our tools, and thereafter our tools shape us.” ― Marshall McLuhan 43 I’m excited to see how Async Guzzle shapes you and how you work.

Slide 44

Slide 44 text

44 Thanks! Any questions? You can find me at @jeremeamia on Twitter

Slide 45

Slide 45 text

Credits ▫ Presentation template by SlidesCarnival ▫ Guzzle was created by Michael Dowling 45