Async Guzzle - Concurrent HTTP Requests in PHP

Though PHP is known to be a single-threaded programming language, it's possible to execute HTTP requests concurrently using Guzzle – An HTTP client library for PHP. Guzzle creates a powerful abstraction over multi-cURL and provides a familiar asynchronous interface using Promises. This presentation answers the what, why, and how for using Guzzle's async/concurrent request features. The accompanying demos are located at: https://github.com/azPHP/async-guzzle-demos-2020

Transcript

1. Async Guzzle
   Concurrent HTTP Requests in PHP
   By Jeremy Lindblom ( @jeremeamia )
   

   View Slide

2. Objectives
   ▫ Understand why async requests can be helpful.
   ▫ Understand what “async” means in Guzzle.
   ▫ Learn how to perform async requests in Guzzle.
   ▫ Learn how to construct async request workflows.
   ▫ Understand and avoid async/Guzzle pitfalls.
   2
   

   View Slide

3. Important Links
   ▫ Demos: https:/
   /github.com/azPHP/async-guzzle-demos-2020
   ▫ Guzzle Code: https:/
   /github.com/guzzle/guzzle
   ▫ Guzzle Docs: http:/
   /docs.guzzlephp.org
   ▫ HttpBin API: https:/
   /httpbin.org/
   3
   

   View Slide

4. “
   “We become what we behold. We shape our
   tools, and thereafter our tools shape us.”
   ― Marshall McLuhan
   4
   

   View Slide

5. 1. What is “Async”?
   Let’s make sure we’re on the same page.
   

   View Slide

6. What is Guzzle?
   6
   Guzzle
   PHP HTTP client
   Abstracts cURL
   Implements PSR-7
   Broad usage
   Supports concurrent requests
   HTTP
   HTTP-based web services
   Typically “RESTful” APIs
   Request and response
   Status codes (e.g., 200, 404)
   Request methods (e.g., POST)
   cURL
   Powerful CLI tool
   “Transferring data with URLs”
   Libcurl C library
   Lots of features
   Good HTTP client
   Answer: It’s a pretty awesome HTTP client library for PHP.
   

   View Slide

7. What is “Async”?
   7
   

   View Slide

8. What is Guzzle’s “Async”?
   8
   

   View Slide

9. Demo
   01–02
   9
   

   View Slide

10. What is Guzzle’s “Async”?
    10
    

    View Slide

11. “
    11
    

    View Slide

12. What it is not:
    ▫ Fire and Forget
    ▫ Parallel / Threads
    ▫ Magic
    ▫ An Extension
    ▫ ReactPHP / Amp / Swoole
    ▫ Async
    What is Guzzle’s “Async”?
    12
    

    View Slide

13. What it is not:
    ▫ Fire and Forget
    ▫ Parallel / Threads
    ▫ Magic
    ▫ An Extension
    ▫ ReactPHP / Amp / Swoole
    ▫ Async
    What is Guzzle’s “Async”?
    What it really is:
    ▫ Async interface
    ▫ Optimization for multiple
    HTTP requests
    ▫ Concurrent HTTP request
    execution (non-blocking
    HTTP I/O)
    13
    

    View Slide

14. What is Guzzle’s “Async”?
    An asynchronous HTTP client interface using Promises for
    executing multiple HTTP requests concurrently from PHP,
    implemented via a cURL-based, non-blocking event loop.
    14
    

    View Slide

15. PROMISES
    Represents the eventual result
    of an asynchronous operation.
    Defined in Promises/A+ spec.
    15
    

    View Slide

16. Promises
    ▫ Promises start in a pending state
    ▫ Promises can be resolved in two ways:
    ▫ fulfilled – With a value (e.g., a Response)
    ▫ rejected – With a reason (e.g., an Exception)
    16
    Pending Fulfilled Rejected
    Value
    Reason / Exception
    

    View Slide

17. Promises
    ▫ Promises can be operated on in these ways:
    ▫ then($fn) – define what to do with resolved values
    ▫ otherwise($fn) – define what do with rejected reasons
    ▫ wait() – blocks until resolved. Either returns the value or
    throws the reason/exception
    Note: Any object following the Promises/A+ spec for then() is
    interoperable with Guzzle’s promises (e.g., React PHP promises).
    17
    

    View Slide

18. Promises
    ▫ then() and otherwise() return new promises that get
    resolved once the original promise is.
    ▫ Chainable: $promise->then()->then()->otherwise()
    ▫ Within a then/otherwise function:
    ▫ Throwing an exception, will reject the new promise.
    ▫ Returning a value, will fulfill the new promise.
    18
    

    View Slide

19. Demo
    03
    19
    

    View Slide

20. 2. Why Use “Async”?
    “Why” is the best question to ask about anything in tech.
    

    View Slide

21. A Practical Use Case
    ▫ Three APIs:
    ▫ Orgs API – districts, schools, and their relationships
    ▫ People API – users’ info, affiliations, and roles
    ▫ Sections API – classes, enrollments, and curriculum
    ▫ Need to combine data to answer a question:
    “What teachers in the school district are teaching a
    class with a 3rd grade math curriculum?”
    21
    

    View Slide

22. x ea.
    22
    (Step 4) Combine Results from 2 and 3
    (Step 2) People: Search for Teachers
    Filter people by partial name (e.g., “st” -> “steve”)
    3s
    (Step 1) Orgs: Get Schools in District 0.5s
    (Step 3) Sections: Get Teacher Enrollments
    For each Schools:
    Filter sections by course/curriculum ID
    1.5s
    Total Time, Synchronous (2 schools) 6.5s
    (4 schools) 9.5s
    (6 schools) 12.5s
    

    View Slide

23. x ea.
    23
    (Step 4) Combine Results from 2 and 3
    (Step 2) People: Search for Teachers
    Filter people by partial name (e.g., “st” -> “steve”)
    3s
    (Step 1) Orgs: Get Schools in District 0.5s
    (Step 3) Sections: Get Teacher Enrollments
    For each School:
    Filter sections by course/curriculum ID
    1.5s
    Total Time, Asynchronous (2 schools) 3.5s
    (4 schools) 3.5s
    (6 schools) 6.5s
    

    View Slide

24. 24
    (Step 4) Combine Results from 2 and 3
    (Step 2) People: Search for Teachers
    Filter people by partial name (e.g., “st” -> “steve”)
    3s
    (Step 1) Orgs: Get Schools in District 0.5s
    For each School:
    Filter sections by course/curriculum ID
    1.5s
    Total Time, Asynchronous (2 schools) 3.5s
    (4 schools) 3.5s
    1.5s 1.5s
    . . .
    (6 schools) 3.5s
    

    View Slide

25. Demo
    04
    25
    

    View Slide

26. Why Should I Use Async Guzzle?
    ▫ You can improve performance when doing multiple API calls.
    ▫ You can develop skills that are transferable to JavaScript.
    ▫ You find it to be fun, and you like to Think In Async™.
    26
    

    View Slide

27. Why Should I NOT Use Async Guzzle?
    ▫ You only ever need to make 1 API call at a time.
    ▫ You are managing async/concurrency in some other way.
    (e.g., React PHP/amphp, Swoole, pthreads, fork, etc.)
    ▫ You are vehemently opposed to Promises, and refuse to
    deviate from your Rx/Observable purity).
    ▫ You want to avoid the pain that can occur with concurrency.
    27
    

    View Slide

28. 3. How Do I Really Use This?
    You have to learn how to be effective with any new tool.
    

    View Slide

29. DO: Share Handlers Between Clients
    ▫ If working with multiple APIs, construct your clients
    so they share the same underlying request handler.
    ▫ If you don’t, the “event loops” will be separate and
    will be blocking to each other.
    29
    

    View Slide

30. Demo
    05-07
    30
    

    View Slide

31. DO: Optimize Your Handler Stack
    ▫ The default handler stack is designed to support the
    majority of Guzzle use cases, but is not optimized for
    any particular case.
    ▫ Don’t add middleware you don’t need.
    ▫ Instead of using HandlerStack::create(),
    use new HandlerStack() and customize.
    31
    

    View Slide

32. View Slide

33. View Slide

34. 34
    // Create custom, optimized handler
    $handler = new GuzzleHttp\HandlerStack();
    $handler->push(GuzzleHttp\Middleware::prepareBody(), 'body');
    $handler->push(new My\Project\Auth(), 'auth');
    $handler->setHandler(new GuzzleHttp\Handler\CurlMultiHandler());
    // Instantiate client with custom handler
    $client = new GuzzleHttp\Client([
    'handler' => $handler,
    'base_uri' => 'https://example.org',
    ]);
    

    View Slide

35. DON’T: Interrupt cURL with Other I/O
    ▫ Guzzle allows you to do concurrency, but only within
    the context of its cURL-based event loop.
    ▫ I/O triggered by other things (DB, files, other HTTP
    clients) will block Guzzle’s async loop, and nullify any
    benefits from the attempted concurrency.
    ▫ File streams managed by cURL/Guzzle are OK.
    35
    

    View Slide

36. DO: Wait()
    ▫ Remember: Guzzle’s Async isn’t _really_ async.
    ▫ Async requests don’t start immediately; they get
    queued up in the cURL event loop.
    ▫ Don’t expect it to do anything until you wait() on a
    promise. $result = $promise->wait();
    ▫ If the program terminates (exit, exception, etc.), it
    might not even run before the PHP process is ended.
    36
    

    View Slide

37. DON’T: Wait Too Early
    ▫ It’s best to keep things within the async context as
    long as you can. Creates more opportunities for
    concurrency.
    ▫ Don’t wait() until you need to leave the async context
    (e.g., generate HTTP response, access DB, etc.)
    ▫ Libraries with async operations should never wait(),
    that is the library consumer’s job.
    37
    

    View Slide

38. DO: Use Guzzle’s Promise Helpers
    ▫ promise_for($value)
    ▫ rejection_for($reason)
    ▫ unwrap($promises)
    ▫ all($promises)
    ▫ some($count, $promises)
    ▫ any($promises)
    ▫ coroutine($fn)
    ▫ is_fulfilled($promise)
    ▫ is_rejected($promise)
    is_settled($promise)
    38
    ▫ each(
    $promises,
    $fulfilledFn,
    $rejectedFn
    )
    ▫ each_limit(
    $promises,
    $concurrency,
    $fulfilledFn,
    $rejectedFn
    )
    

    View Slide

39. DO: Limit Concurrency Pool Size
    ▫ The client’s system resources are not unlimited
    (e.g., CPU, Memory, File Descriptors)
    ▫ The service’s system resources are not unlimited
    (e.g., Rate Limiting, TOS Violations, DDOS)
    ▫ Use tools like each_limit() or GuzzleHttp\Pool
    to limit how many active requests are being executed.
    ▫ The “delay” request option is also useful, and is
    implemented in an async-friendly (i.e., not with sleep).
    39
    

    View Slide

40. DO: Use Coroutines
    ▫ Coroutines are generator-based functions that
    yield promises and have results sent back into
    them.
    ▫ The entire coroutine itself is also a promise.
    ▫ Allows for making async code look normal.
    ▫ About as close to async/await as you can get in PHP.
    ▫ coroutine ~= async
    ▫ yield ~= await
    40
    

    View Slide

41. Demo
    08
    41
    

    View Slide

42. Objectives
    ▫ ✅ Understand why async requests can be helpful.
    ▫ ✅ Understand what “async” means in Guzzle.
    ▫ ✅ Learn how to perform async requests in Guzzle.
    ▫ ✅ Learn how to construct async request workflows.
    ▫ ✅ Understand and avoid async/Guzzle pitfalls.
    42
    

    View Slide

43. “
    “We become what we behold. We shape our
    tools, and thereafter our tools shape us.”
    ― Marshall McLuhan
    43
    I’m excited to see how Async Guzzle
    shapes you and how you work.
    

    View Slide

44. 44
    Thanks!
    Any questions?
    You can find me at @jeremeamia on Twitter
    

    View Slide

45. Credits
    ▫ Presentation template by SlidesCarnival
    ▫ Guzzle was created by Michael Dowling
    45
    

    View Slide