Symfony HttpClient, what else?

Have you heard of Symfony HttpClient? This new component will be released at the end of May with Symfony 4.3. How about experimenting with it by launching "compose require symfony/http-client dev-master"? That is what I propose you do at this conference. We will review its features and essential characteristics. Enough to replace Guzzle et al.? Undoubtedly as far as I am concerned :)

Transcript

1. Symfony HttpClient
   what else?
   #Symfony_Live
   @nicolasgrekas
   

   View Slide

2. @nicolasgrekas
   {
   "name": "symfony/http-client",
   "provide": {
   "php-http/client-implementation": "*",
   "psr/http-client-implementation": "1.0",
   "symfony/http-client-contracts-implementation": "1.1"
   },
   "require": {
   "php": "^7.1.3",
   "psr/log": "^1.0",
   "symfony/http-client-contracts": "^1.1.3",
   "symfony/polyfill-php73": "^1.11"
   },
   ...
   HttpClient is a standalone package
   

   View Slide

3. @nicolasgrekas
   composer require symfony/http-client
   use Symfony\Component\HttpClient\HttpClient;
   $client = HttpClient::create();
   $url = 'https://symfony.com/versions.json';
   $response = $client->request('GET', $url);
   

   View Slide

4. @nicolasgrekas
   Consume JSON APIs in a snap
   $symfonyVersions = $response->toArray();
   dump($symfonyVersions);
   array:23 [
   "lts" => "3.4.23"
   "latest" => "4.2.4"
   "dev" => "4.3.0-DEV"
   "2.0" => "2.0.25"
   "2.1" => "2.1.13"
   "2.2" => "2.2.11"
   …
   

   View Slide

5. @nicolasgrekas
   Abstraction is fully decoupled
   from the component
   Stateless (autowired) service
   @nicolasgrekas
   

   View Slide

6. @nicolasgrekas
   Bye bye boilerplate
   

   View Slide

7. @nicolasgrekas
   Headers? Body? Options to the rescue
   

   View Slide

8. @nicolasgrekas
   Query parameters? Options to the rescue
   

   View Slide

9. @nicolasgrekas
   Bearer? JSON payload? Options to the rescue
   

   View Slide

10. @nicolasgrekas
    Form parameters? Options to the rescue
    

    View Slide

11. @nicolasgrekas
    namespace Symfony\Contracts\HttpClient;
    interface ResponseInterface
    {
    public function getStatusCode(): int;
    public function getHeaders(): array;
    public function getContent(): string;
    public function toArray(): array;
    public function cancel(): void;
    public function getInfo(): array;
    }
    

    View Slide

12. Symfony HttpClient
    80% of use cases now covered
    (in 20% of the symfony/contracts abstraction)
    #Symfony_Live
    @nicolasgrekas
    

    View Slide

13. @nicolasgrekas
    What about PSR-18?
    namespace Symfony\Component\HttpClient;
    class Psr18Client implements \Psr\Http\Client\ClientInterface
    {
    public function __construct(
    \Symfony\Contracts\HttpClient\HttpClientInterface,
    \Psr\Http\Message\ResponseFactoryInterface,
    \Psr\Http\Message\StreamFactoryInterface
    )
    

    View Slide

14. @nicolasgrekas
    composer require nyholm/psr7
    $client = new Psr18Client();
    $url = 'https://symfony.com/versions.json';
    $request = $client->createRequest('GET', $url);
    $response = $client->sendRequest($request);
    $symfonyVersions = json_decode($response
    ->getBody()->getContents(), true);
    Dual PSR-17 + 18
    

    View Slide

15. @nicolasgrekas
    Also for HTTPlug ^1.0|^2.0
    namespace Symfony\Component\HttpClient;
    class HttplugClient implements \Http\Client\HttpClient
    {
    public function __construct(
    \Symfony\Contracts\HttpClient\HttpClientInterface,
    \Http\Message\ResponseFactory,
    \Http\Message\StreamFactory
    )
    

    View Slide

16. @nicolasgrekas
    Most still at ^1.0
    

    View Slide

17. Symfony HttpClient
    What about the remaining 80%?
    (in symfony/contracts)
    #Symfony_Live
    @nicolasgrekas
    

    View Slide

18. @nicolasgrekas
    Options to the rescue
    'auth_basic' => null, // array|string - an array containing the username as first value, and optionally the
    // password as the second one; or string like username:password - enabling HTTP Basic
    // authentication (RFC 7617)
    'auth_bearer' => null, // string - a token enabling HTTP Bearer authorization (RFC 6750)
    'query' => [], // string[] - associative array of query string values to merge with the request's URL
    'headers' => [], // iterable|string[]|string[][] - headers names provided as keys or as part of values
    'body' => '', // array|string|resource|\Traversable|\Closure - the callback SHOULD yield a string
    // smaller than the amount requested as argument; the empty string signals EOF; when
    // an array is passed, it is meant as a form payload of field names and values
    'json' => null, // array|\JsonSerializable - when set, implementations MUST set the "body" option to
    // the JSON-encoded value and set the "content-type" headers to a JSON-compatible
    // value it is they are not defined - typically "application/json"
    'user_data' => null, // mixed - any extra data to attach to the request (scalar, callable, object...) that
    // MUST be available via $response->getInfo('user_data') - not used internally
    'max_redirects' => 20, // int - the maximum number of redirects to follow; a value lower or equal to 0 means
    // redirects should not be followed; "Authorization" and "Cookie" headers MUST
    // NOT follow except for the initial host name
    'http_version' => null, // string - defaults to the best supported version, typically 1.1 or 2.0
    'base_uri' => null, // string - the URI to resolve relative URLs, following rules in RFC 3986, section 2
    'buffer' => true, // bool - whether the content of the response should be buffered or not
    'on_progress' => null, // callable(int $dlNow, int $dlSize, array $info) - throwing any exceptions MUST abort
    // the request; it MUST be called on DNS resolution, on arrival of headers and on
    // completion; it SHOULD be called on upload/download of data and at least 1/s
    'resolve' => [], // string[] - a map of host to IP address that SHOULD replace DNS resolution
    'proxy' => null, // string - by default, the proxy-related env vars handled by curl SHOULD be honored
    'no_proxy' => null, // string - a comma separated list of hosts that do not require a proxy to be reached
    'timeout' => null, // float - the inactivity timeout - defaults to ini_get('default_socket_timeout')
    'bindto' => '0', // string - the interface or the local socket to bind to
    'verify_peer' => true, // see https://php.net/context.ssl for the following options
    'verify_host' => true,
    'cafile' => null,
    'capath' => null,
    'local_cert' => null,
    'local_pk' => null,
    'passphrase' => null,
    'ciphers' => null,
    'peer_fingerprint' => null,
    'capture_peer_cert_chain' => false,
    'extra' => [], // array - additional options that can be ignored if unsupported, unlike regular options
    Options are in the contracts:
    Writing decoupled decorators is easy
    Options are in the contracts: the
    transport is covered by the abstraction
    

    View Slide

19. @nicolasgrekas
    • timeout – control inactivity periods
    • proxy – get through an HTTP proxy
    • on_progress – display a progress bar / abort a request
    • base_uri – resolve relative URLs / build a scoped client
    • resolve – protect webhooks against calls to internal endpoints
    • max_redirects – disable or limit redirects
    Showcase of some options
    

    View Slide

20. @nicolasgrekas
    The resiliency of the Web relies on redirects, you’re part of it
    Robust and failsafe by default
    /**
    * Gets the HTTP headers of the response.
    *
    * @param bool $throw Whether an exception should be thrown on 3/4/5xx status codes
    *
    * @throws TransportExceptionInterface When a network error occurs
    * @throws RedirectionExceptionInterface On a 3xx when $throw is true and the
    * "max_redirects" option has been reached
    * @throws ClientExceptionInterface On a 4xx when $throw is true
    * @throws ServerExceptionInterface On a 5xx when $throw is true
    */
    public function getHeaders(bool $throw = true): array;
    But what happens when the redirection limit is reached? or on 4xx / 5xx? Yours to decide:
    

    View Slide

21. @nicolasgrekas
    • body – array|string|resource|\Traversable|\Closure
    Streameable uploads
    $client->request('POST', 'http://upload.example.com', [
    'body' => $mimeParts->toIterable(), // see e.g. symfony/mime
    ]);
    

    View Slide

22. @nicolasgrekas
    Streameable downloads
    $url = 'http://.../ubuntu-18.04.1-desktop-amd64.iso';
    $response = $client->request('GET', $url, [
    'on_progress' => 'dump', // let's spam the console for fun
    'buffer' => false, // skip buffering the response in memory
    ]);
    // Responses are lazy! This waits only for the headers
    if (200 !== $response->getStatusCode()) {
    throw new \Exception('...');
    }
    $h = fopen('./ubuntu.iso', 'w');
    foreach ($client->stream($response) as $chunk) {
    fwrite($h, $chunk->getContent());
    }
    

    View Slide

23. @nicolasgrekas
    Responses are lazy & requests are concurrent
    for ($i = 0; $i < 379; ++$i) {
    $uri = "https://http2.akamai.com/demo/tile-$i.png";
    $responses[] = $client->request('GET', $uri);
    }
    foreach ($responses as $response) {
    // block until completion of $response
    // but monitor all other responses meanwhile
    $response->getContent();
    }
    

    View Slide

24. @nicolasgrekas
    Asynchronous requests
    for ($i = 0; $i < 379; ++$i) {
    $uri = "https://http2.akamai.com/demo/tile-$i.png";
    $responses[] = $client->request('GET', $uri);
    }
    foreach ($client->stream($responses) as $response => $chunk) {
    if ($chunk->isLast()) {
    // a $response completed
    } else {
    // a $response's got network activity or timeout
    }
    }
    379 requests
    completed in
    0,4 s!
    

    View Slide

25. @nicolasgrekas
    dump($request->getInfo('debug'))
    * Found bundle for host http2.akamai.com: 0x55f8aaa0f8c0 [can multiplex]
    * MAX_CONCURRENT_STREAMS reached, skip (100)
    * Conn: 2 (0x55f8aaa261d0) Receive pipe weight: (-1/0), penalized: FALSE
    * Multiplexed connection found!
    * Found connection 2, with requests in the pipe (86)
    * Re-using existing connection! (#2) with host http2.akamai.com
    * Using Stream ID: c7 (easy handle 0x55f8ac0b7740)
    > GET /demo/tile-248.png HTTP/2
    Host: http2.akamai.com
    User-Agent: Symfony HttpClient/Curl
    Accept-Encoding: deflate, gzip
    < HTTP/2 200
    < content-length: 761
    < content-type: image/png
    < ...
    <
    * Connection #2 to host http2.akamai.com left intact
    

    View Slide

26. @nicolasgrekas
    Non-blocking monitoring / timeout management
    $client->stream($runningResponses, 0.0);
    Max number of
    seconds to wait
    before yielding
    a timeout chunk
    

    View Slide

27. @nicolasgrekas
    ResponseInterface::getInfo()
    /**
    * This method SHOULD NOT throw any ExceptionInterface and SHOULD be non-blocking.
    * The returned info is "live": it can be empty and can change from one call to
    * another, as the request/response progresses.
    *
    * The following info MUST be returned:
    * - response_headers - an array modelled after the special $http_response_header variable
    * - redirect_count - the number of redirects followed while executing the request
    * - redirect_url - the resolved location of redirect responses, null otherwise
    * - start_time - the time when the request was sent or 0.0 when it's pending
    * - http_method - the HTTP verb of the last request
    * - http_code - the last response code or 0 when it is not known yet
    * - error - the error message when the transfer was aborted, null otherwise
    * - user_data - the value of the "user_data" request option, null if not set
    * - url - the last effective URL of the request
    *
    * When the "capture_peer_cert_chain" option is true, the "peer_certificate_chain"
    * attribute SHOULD list the peer certificates as an array of OpenSSL X.509 resources.
    *
    * Other info SHOULD be named after curl_getinfo()'s associative return value.
    *
    * @return array|mixed|null An array of all available info, or one of them when $type is
    * provided, or null when an unsupported type is requested
    */
    

    View Slide

28. Symfony HttpClient
    The component
    (everything before is at the contracts level)
    #Symfony_Live
    @nicolasgrekas
    

    View Slide

29. @nicolasgrekas
    NativeHttpClient & CurlHttpClient
    Both provide:
    • 100% of the contracts
    • secure redirects
    • extended (time) info
    • transparent HTTP compression and (de)chunking
    • automatic HTTP proxy configuration via env vars
    • I18n Domain Names
    

    View Slide

30. @nicolasgrekas
    NativeHttpClient
    • The most portable
    • Based on the HTTP stream wrapper
    • With fixed redirection logic
    • Blocking until response headers arrive
    • Multiplexing response bodies
    • Abstracts all http+ssl+socket stream context options
    

    View Slide

31. @nicolasgrekas
    CurlHttpClient
    • Requires ext-curl
    • With fixed redirection logic
    • Fully multiplexing response headers and bodies
    • Leverages HTTP/2 and PUSH when available
    • Abstracts most/all CURL(M)OPT_*/CURLINFO_* related to HTTP
    

    View Slide

32. @nicolasgrekas
    4 decorators as of now
    • ScopingHttpClient – Auto-configure the options based on the request URL
    • MockHttpClient – A client that doesn't make actual HTTP requests
    • CachingHttpClient – Adds caching on top of an HTTP client
    • Psr18Client – You already know about it
    with TraceableHttpClient and record & replay coming
    

    View Slide

33. Symfony HttpClient
    Yours to combine
    (here are a few ideas that leverage some battle tested packages)
    #Symfony_Live
    @nicolasgrekas
    

    View Slide

34. @nicolasgrekas
    FrameworkBundle / Autowiring
    framework:
    http_client:
    max_host_connections: 4
    default_options:
    # ...
    scoped_clients:
    github_client:
    base_uri: https://api.github.com
    headers:
    Authorization: token abc123
    # creates the HttpClientInterface $githubClient autowiring alias
    # and the HttpClientInterface $scopingHttpClient one
    

    View Slide

35. @nicolasgrekas
    WebProfilerBundle
    

    View Slide

36. @nicolasgrekas
    BrowserKit\HttpBrowser
    use Symfony\Component\BrowserKit\HttpBrowser;
    use Symfony\Component\HttpClient\HttpClient;
    $client = HttpClient::create();
    $browser = new HttpBrowser($client);
    $browser->request('GET', 'https://example.com/');
    $browser->clickLink('Log In');
    $browser->submitForm('Sign In', ['username' => 'me', 'password' => 'pass']);
    $browser->clickLink('Subscriptions')
    ->filter('table tr:nth-child(2) td:nth-child(2)')
    ->each(function ($node) { echo trim($node->text())."\n"; });
    

    View Slide

37. @nicolasgrekas
    R.I.P. Goutte
    HttpClient
    + DomCrawler
    + CssSelector
    + HttpKernel
    + BrowserKit
    

    View Slide

38. Thanks for the ideas
    fopen(), Composer, Guzzle, Buzz, HTTPlug,
    Cake 3, Axios, window.fetch()
    and all reviewers!
    

    View Slide

39. Thank You!
    composer require
    symfony/http-client
    

    View Slide

40. @nicolasgrekas
    for ($i = 0; $i < ...; ++$i) {
    $pendingResponses[$i] = $client->request('GET', "/some{$i}.html", [
    'user_data' => [$i, function ($response) use ($client): array {
    // extract CSS/JS files from $response->getContent()
    // $client->request() them and return the list of responses
    }],
    ]);
    }
    Made for
    HTTP/2 PUSH
    while ($pendingResponses) {
    foreach ($client->stream($pendingResponses) as $response => $chunk) {
    if ($chunk->isLast()) {
    [$responseId, $processResponse] = $response->getInfo('user_data');
    unset($pendingResponses[$responseId]);
    if ($moreResponses = $processResponse($response)) {
    $pendingResponses += $moreResponses;
    break;
    }
    }
    }
    }
    Bonus slide: feed the stream while streaming
    

    View Slide