Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Async Guzzle - Concurrent HTTP Requests in PHP

Async Guzzle - Concurrent HTTP Requests in PHP

Though PHP is known to be a single-threaded programming language, it's possible to execute HTTP requests concurrently using Guzzle – An HTTP client library for PHP. Guzzle creates a powerful abstraction over multi-cURL and provides a familiar asynchronous interface using Promises. This presentation answers the what, why, and how for using Guzzle's async/concurrent request features. The accompanying demos are located at: https://github.com/azPHP/async-guzzle-demos-2020

Jeremy Lindblom

August 25, 2020
Tweet

More Decks by Jeremy Lindblom

Other Decks in Programming

Transcript

  1. Async Guzzle
    Concurrent HTTP Requests in PHP
    By Jeremy Lindblom ( @jeremeamia )

    View full-size slide

  2. Objectives
    ▫ Understand why async requests can be helpful.
    ▫ Understand what “async” means in Guzzle.
    ▫ Learn how to perform async requests in Guzzle.
    ▫ Learn how to construct async request workflows.
    ▫ Understand and avoid async/Guzzle pitfalls.
    2

    View full-size slide

  3. Important Links
    ▫ Demos: https:/
    /github.com/azPHP/async-guzzle-demos-2020
    ▫ Guzzle Code: https:/
    /github.com/guzzle/guzzle
    ▫ Guzzle Docs: http:/
    /docs.guzzlephp.org
    ▫ HttpBin API: https:/
    /httpbin.org/
    3

    View full-size slide


  4. “We become what we behold. We shape our
    tools, and thereafter our tools shape us.”
    ― Marshall McLuhan
    4

    View full-size slide

  5. 1. What is “Async”?
    Let’s make sure we’re on the same page.

    View full-size slide

  6. What is Guzzle?
    6
    Guzzle
    PHP HTTP client
    Abstracts cURL
    Implements PSR-7
    Broad usage
    Supports concurrent requests
    HTTP
    HTTP-based web services
    Typically “RESTful” APIs
    Request and response
    Status codes (e.g., 200, 404)
    Request methods (e.g., POST)
    cURL
    Powerful CLI tool
    “Transferring data with URLs”
    Libcurl C library
    Lots of features
    Good HTTP client
    Answer: It’s a pretty awesome HTTP client library for PHP.

    View full-size slide

  7. What is “Async”?
    7

    View full-size slide

  8. What is Guzzle’s “Async”?
    8

    View full-size slide

  9. Demo
    01–02
    9

    View full-size slide

  10. What is Guzzle’s “Async”?
    10

    View full-size slide

  11. What it is not:
    ▫ Fire and Forget
    ▫ Parallel / Threads
    ▫ Magic
    ▫ An Extension
    ▫ ReactPHP / Amp / Swoole
    ▫ Async
    What is Guzzle’s “Async”?
    12

    View full-size slide

  12. What it is not:
    ▫ Fire and Forget
    ▫ Parallel / Threads
    ▫ Magic
    ▫ An Extension
    ▫ ReactPHP / Amp / Swoole
    ▫ Async
    What is Guzzle’s “Async”?
    What it really is:
    ▫ Async interface
    ▫ Optimization for multiple
    HTTP requests
    ▫ Concurrent HTTP request
    execution (non-blocking
    HTTP I/O)
    13

    View full-size slide

  13. What is Guzzle’s “Async”?
    An asynchronous HTTP client interface using Promises for
    executing multiple HTTP requests concurrently from PHP,
    implemented via a cURL-based, non-blocking event loop.
    14

    View full-size slide

  14. PROMISES
    Represents the eventual result
    of an asynchronous operation.
    Defined in Promises/A+ spec.
    15

    View full-size slide

  15. Promises
    ▫ Promises start in a pending state
    ▫ Promises can be resolved in two ways:
    ▫ fulfilled – With a value (e.g., a Response)
    ▫ rejected – With a reason (e.g., an Exception)
    16
    Pending Fulfilled Rejected
    Value
    Reason / Exception

    View full-size slide

  16. Promises
    ▫ Promises can be operated on in these ways:
    ▫ then($fn) – define what to do with resolved values
    ▫ otherwise($fn) – define what do with rejected reasons
    ▫ wait() – blocks until resolved. Either returns the value or
    throws the reason/exception
    Note: Any object following the Promises/A+ spec for then() is
    interoperable with Guzzle’s promises (e.g., React PHP promises).
    17

    View full-size slide

  17. Promises
    ▫ then() and otherwise() return new promises that get
    resolved once the original promise is.
    ▫ Chainable: $promise->then()->then()->otherwise()
    ▫ Within a then/otherwise function:
    ▫ Throwing an exception, will reject the new promise.
    ▫ Returning a value, will fulfill the new promise.
    18

    View full-size slide

  18. 2. Why Use “Async”?
    “Why” is the best question to ask about anything in tech.

    View full-size slide

  19. A Practical Use Case
    ▫ Three APIs:
    ▫ Orgs API – districts, schools, and their relationships
    ▫ People API – users’ info, affiliations, and roles
    ▫ Sections API – classes, enrollments, and curriculum
    ▫ Need to combine data to answer a question:
    “What teachers in the school district are teaching a
    class with a 3rd grade math curriculum?”
    21

    View full-size slide

  20. x ea.
    22
    (Step 4) Combine Results from 2 and 3
    (Step 2) People: Search for Teachers
    Filter people by partial name (e.g., “st” -> “steve”)
    3s
    (Step 1) Orgs: Get Schools in District 0.5s
    (Step 3) Sections: Get Teacher Enrollments
    For each Schools:
    Filter sections by course/curriculum ID
    1.5s
    Total Time, Synchronous (2 schools) 6.5s
    (4 schools) 9.5s
    (6 schools) 12.5s

    View full-size slide

  21. x ea.
    23
    (Step 4) Combine Results from 2 and 3
    (Step 2) People: Search for Teachers
    Filter people by partial name (e.g., “st” -> “steve”)
    3s
    (Step 1) Orgs: Get Schools in District 0.5s
    (Step 3) Sections: Get Teacher Enrollments
    For each School:
    Filter sections by course/curriculum ID
    1.5s
    Total Time, Asynchronous (2 schools) 3.5s
    (4 schools) 3.5s
    (6 schools) 6.5s

    View full-size slide

  22. 24
    (Step 4) Combine Results from 2 and 3
    (Step 2) People: Search for Teachers
    Filter people by partial name (e.g., “st” -> “steve”)
    3s
    (Step 1) Orgs: Get Schools in District 0.5s
    For each School:
    Filter sections by course/curriculum ID
    1.5s
    Total Time, Asynchronous (2 schools) 3.5s
    (4 schools) 3.5s
    1.5s 1.5s
    . . .
    (6 schools) 3.5s

    View full-size slide

  23. Why Should I Use Async Guzzle?
    ▫ You can improve performance when doing multiple API calls.
    ▫ You can develop skills that are transferable to JavaScript.
    ▫ You find it to be fun, and you like to Think In Async™.
    26

    View full-size slide

  24. Why Should I NOT Use Async Guzzle?
    ▫ You only ever need to make 1 API call at a time.
    ▫ You are managing async/concurrency in some other way.
    (e.g., React PHP/amphp, Swoole, pthreads, fork, etc.)
    ▫ You are vehemently opposed to Promises, and refuse to
    deviate from your Rx/Observable purity).
    ▫ You want to avoid the pain that can occur with concurrency.
    27

    View full-size slide

  25. 3. How Do I Really Use This?
    You have to learn how to be effective with any new tool.

    View full-size slide

  26. DO: Share Handlers Between Clients
    ▫ If working with multiple APIs, construct your clients
    so they share the same underlying request handler.
    ▫ If you don’t, the “event loops” will be separate and
    will be blocking to each other.
    29

    View full-size slide

  27. DO: Optimize Your Handler Stack
    ▫ The default handler stack is designed to support the
    majority of Guzzle use cases, but is not optimized for
    any particular case.
    ▫ Don’t add middleware you don’t need.
    ▫ Instead of using HandlerStack::create(),
    use new HandlerStack() and customize.
    31

    View full-size slide

  28. 34
    // Create custom, optimized handler
    $handler = new GuzzleHttp\HandlerStack();
    $handler->push(GuzzleHttp\Middleware::prepareBody(), 'body');
    $handler->push(new My\Project\Auth(), 'auth');
    $handler->setHandler(new GuzzleHttp\Handler\CurlMultiHandler());
    // Instantiate client with custom handler
    $client = new GuzzleHttp\Client([
    'handler' => $handler,
    'base_uri' => 'https://example.org',
    ]);

    View full-size slide

  29. DON’T: Interrupt cURL with Other I/O
    ▫ Guzzle allows you to do concurrency, but only within
    the context of its cURL-based event loop.
    ▫ I/O triggered by other things (DB, files, other HTTP
    clients) will block Guzzle’s async loop, and nullify any
    benefits from the attempted concurrency.
    ▫ File streams managed by cURL/Guzzle are OK.
    35

    View full-size slide

  30. DO: Wait()
    ▫ Remember: Guzzle’s Async isn’t _really_ async.
    ▫ Async requests don’t start immediately; they get
    queued up in the cURL event loop.
    ▫ Don’t expect it to do anything until you wait() on a
    promise. $result = $promise->wait();
    ▫ If the program terminates (exit, exception, etc.), it
    might not even run before the PHP process is ended.
    36

    View full-size slide

  31. DON’T: Wait Too Early
    ▫ It’s best to keep things within the async context as
    long as you can. Creates more opportunities for
    concurrency.
    ▫ Don’t wait() until you need to leave the async context
    (e.g., generate HTTP response, access DB, etc.)
    ▫ Libraries with async operations should never wait(),
    that is the library consumer’s job.
    37

    View full-size slide

  32. DO: Use Guzzle’s Promise Helpers
    ▫ promise_for($value)
    ▫ rejection_for($reason)
    ▫ unwrap($promises)
    ▫ all($promises)
    ▫ some($count, $promises)
    ▫ any($promises)
    ▫ coroutine($fn)
    ▫ is_fulfilled($promise)
    ▫ is_rejected($promise)
    is_settled($promise)
    38
    ▫ each(
    $promises,
    $fulfilledFn,
    $rejectedFn
    )
    ▫ each_limit(
    $promises,
    $concurrency,
    $fulfilledFn,
    $rejectedFn
    )

    View full-size slide

  33. DO: Limit Concurrency Pool Size
    ▫ The client’s system resources are not unlimited
    (e.g., CPU, Memory, File Descriptors)
    ▫ The service’s system resources are not unlimited
    (e.g., Rate Limiting, TOS Violations, DDOS)
    ▫ Use tools like each_limit() or GuzzleHttp\Pool
    to limit how many active requests are being executed.
    ▫ The “delay” request option is also useful, and is
    implemented in an async-friendly (i.e., not with sleep).
    39

    View full-size slide

  34. DO: Use Coroutines
    ▫ Coroutines are generator-based functions that
    yield promises and have results sent back into
    them.
    ▫ The entire coroutine itself is also a promise.
    ▫ Allows for making async code look normal.
    ▫ About as close to async/await as you can get in PHP.
    ▫ coroutine ~= async
    ▫ yield ~= await
    40

    View full-size slide

  35. Objectives
    ▫ ✅ Understand why async requests can be helpful.
    ▫ ✅ Understand what “async” means in Guzzle.
    ▫ ✅ Learn how to perform async requests in Guzzle.
    ▫ ✅ Learn how to construct async request workflows.
    ▫ ✅ Understand and avoid async/Guzzle pitfalls.
    42

    View full-size slide


  36. “We become what we behold. We shape our
    tools, and thereafter our tools shape us.”
    ― Marshall McLuhan
    43
    I’m excited to see how Async Guzzle
    shapes you and how you work.

    View full-size slide

  37. 44
    Thanks!
    Any questions?
    You can find me at @jeremeamia on Twitter

    View full-size slide

  38. Credits
    ▫ Presentation template by SlidesCarnival
    ▫ Guzzle was created by Michael Dowling
    45

    View full-size slide