Symfony HttpClient, what else?

Transcript

1. 1.

   Symfony HttpClient what else? #Symfony_Live @nicolasgrekas

2. 2.

   @nicolasgrekas { "name": "symfony/http-client", "provide": { "php-http/client-implementation": "*", "psr/http-client-implementation": "1.0",

   "symfony/http-client-contracts-implementation": "1.1" }, "require": { "php": "^7.1.3", "psr/log": "^1.0", "symfony/http-client-contracts": "^1.1.3", "symfony/polyfill-php73": "^1.11" }, ... HttpClient is a standalone package
3. 3.

   @nicolasgrekas composer require symfony/http-client use Symfony\Component\HttpClient\HttpClient; $client = HttpClient::create(); $url

   = 'https://symfony.com/versions.json'; $response = $client->request('GET', $url);
4. 4.

   @nicolasgrekas Consume JSON APIs in a snap $symfonyVersions = $response->toArray();

   dump($symfonyVersions); array:23 [ "lts" => "3.4.23" "latest" => "4.2.4" "dev" => "4.3.0-DEV" "2.0" => "2.0.25" "2.1" => "2.1.13" "2.2" => "2.2.11" …
5. 5.

   @nicolasgrekas Abstraction is fully decoupled from the component Stateless (autowired)

   service @nicolasgrekas
6. 6.

   @nicolasgrekas Bye bye boilerplate

7. 7.

   @nicolasgrekas Headers? Body? Options to the rescue

8. 8.

   @nicolasgrekas Query parameters? Options to the rescue

9. 9.

   @nicolasgrekas Bearer? JSON payload? Options to the rescue

10. 10.

    @nicolasgrekas Form parameters? Options to the rescue

11. 11.

    @nicolasgrekas namespace Symfony\Contracts\HttpClient; interface ResponseInterface { public function getStatusCode(): int;

    public function getHeaders(): array; public function getContent(): string; public function toArray(): array; public function cancel(): void; public function getInfo(): array; }
12. 12.

    Symfony HttpClient 80% of use cases now covered (in 20%

    of the symfony/contracts abstraction) #Symfony_Live @nicolasgrekas
13. 13.

    @nicolasgrekas What about PSR-18? namespace Symfony\Component\HttpClient; class Psr18Client implements \Psr\Http\Client\ClientInterface

    { public function __construct( \Symfony\Contracts\HttpClient\HttpClientInterface, \Psr\Http\Message\ResponseFactoryInterface, \Psr\Http\Message\StreamFactoryInterface )
14. 14.

    @nicolasgrekas composer require nyholm/psr7 $client = new Psr18Client(); $url =

    'https://symfony.com/versions.json'; $request = $client->createRequest('GET', $url); $response = $client->sendRequest($request); $symfonyVersions = json_decode($response ->getBody()->getContents(), true); Dual PSR-17 + 18
15. 15.

    @nicolasgrekas Also for HTTPlug ^1.0|^2.0 namespace Symfony\Component\HttpClient; class HttplugClient implements

    \Http\Client\HttpClient { public function __construct( \Symfony\Contracts\HttpClient\HttpClientInterface, \Http\Message\ResponseFactory, \Http\Message\StreamFactory )
16. 16.

    @nicolasgrekas Most still at ^1.0

17. 17.

    Symfony HttpClient What about the remaining 80%? (in symfony/contracts) #Symfony_Live

    @nicolasgrekas
18. 18.

    @nicolasgrekas Options to the rescue 'auth_basic' => null, // array|string

    - an array containing the username as first value, and optionally the // password as the second one; or string like username:password - enabling HTTP Basic // authentication (RFC 7617) 'auth_bearer' => null, // string - a token enabling HTTP Bearer authorization (RFC 6750) 'query' => [], // string[] - associative array of query string values to merge with the request's URL 'headers' => [], // iterable|string[]|string[][] - headers names provided as keys or as part of values 'body' => '', // array|string|resource|\Traversable|\Closure - the callback SHOULD yield a string // smaller than the amount requested as argument; the empty string signals EOF; when // an array is passed, it is meant as a form payload of field names and values 'json' => null, // array|\JsonSerializable - when set, implementations MUST set the "body" option to // the JSON-encoded value and set the "content-type" headers to a JSON-compatible // value it is they are not defined - typically "application/json" 'user_data' => null, // mixed - any extra data to attach to the request (scalar, callable, object...) that // MUST be available via $response->getInfo('user_data') - not used internally 'max_redirects' => 20, // int - the maximum number of redirects to follow; a value lower or equal to 0 means // redirects should not be followed; "Authorization" and "Cookie" headers MUST // NOT follow except for the initial host name 'http_version' => null, // string - defaults to the best supported version, typically 1.1 or 2.0 'base_uri' => null, // string - the URI to resolve relative URLs, following rules in RFC 3986, section 2 'buffer' => true, // bool - whether the content of the response should be buffered or not 'on_progress' => null, // callable(int $dlNow, int $dlSize, array $info) - throwing any exceptions MUST abort // the request; it MUST be called on DNS resolution, on arrival of headers and on // completion; it SHOULD be called on upload/download of data and at least 1/s 'resolve' => [], // string[] - a map of host to IP address that SHOULD replace DNS resolution 'proxy' => null, // string - by default, the proxy-related env vars handled by curl SHOULD be honored 'no_proxy' => null, // string - a comma separated list of hosts that do not require a proxy to be reached 'timeout' => null, // float - the inactivity timeout - defaults to ini_get('default_socket_timeout') 'bindto' => '0', // string - the interface or the local socket to bind to 'verify_peer' => true, // see https://php.net/context.ssl for the following options 'verify_host' => true, 'cafile' => null, 'capath' => null, 'local_cert' => null, 'local_pk' => null, 'passphrase' => null, 'ciphers' => null, 'peer_fingerprint' => null, 'capture_peer_cert_chain' => false, 'extra' => [], // array - additional options that can be ignored if unsupported, unlike regular options Options are in the contracts: Writing decoupled decorators is easy Options are in the contracts: the transport is covered by the abstraction
19. 19.

    @nicolasgrekas • timeout – control inactivity periods • proxy –

    get through an HTTP proxy • on_progress – display a progress bar / abort a request • base_uri – resolve relative URLs / build a scoped client • resolve – protect webhooks against calls to internal endpoints • max_redirects – disable or limit redirects Showcase of some options
20. 20.

    @nicolasgrekas The resiliency of the Web relies on redirects, you’re

    part of it Robust and failsafe by default /** * Gets the HTTP headers of the response. * * @param bool $throw Whether an exception should be thrown on 3/4/5xx status codes * * @throws TransportExceptionInterface When a network error occurs * @throws RedirectionExceptionInterface On a 3xx when $throw is true and the * "max_redirects" option has been reached * @throws ClientExceptionInterface On a 4xx when $throw is true * @throws ServerExceptionInterface On a 5xx when $throw is true */ public function getHeaders(bool $throw = true): array; But what happens when the redirection limit is reached? or on 4xx / 5xx? Yours to decide:
21. 21.

    @nicolasgrekas • body – array|string|resource|\Traversable|\Closure Streameable uploads $client->request('POST', 'http://upload.example.com', [

    'body' => $mimeParts->toIterable(), // see e.g. symfony/mime ]);
22. 22.

    @nicolasgrekas Streameable downloads $url = 'http://.../ubuntu-18.04.1-desktop-amd64.iso'; $response = $client->request('GET', $url,

    [ 'on_progress' => 'dump', // let's spam the console for fun 'buffer' => false, // skip buffering the response in memory ]); // Responses are lazy! This waits only for the headers if (200 !== $response->getStatusCode()) { throw new \Exception('...'); } $h = fopen('./ubuntu.iso', 'w'); foreach ($client->stream($response) as $chunk) { fwrite($h, $chunk->getContent()); }
23. 23.

    @nicolasgrekas Responses are lazy & requests are concurrent for ($i

    = 0; $i < 379; ++$i) { $uri = "https://http2.akamai.com/demo/tile-$i.png"; $responses[] = $client->request('GET', $uri); } foreach ($responses as $response) { // block until completion of $response // but monitor all other responses meanwhile $response->getContent(); }
24. 24.

    @nicolasgrekas Asynchronous requests for ($i = 0; $i < 379;

    ++$i) { $uri = "https://http2.akamai.com/demo/tile-$i.png"; $responses[] = $client->request('GET', $uri); } foreach ($client->stream($responses) as $response => $chunk) { if ($chunk->isLast()) { // a $response completed } else { // a $response's got network activity or timeout } } 379 requests completed in 0,4 s!
25. 25.

    @nicolasgrekas dump($request->getInfo('debug')) * Found bundle for host http2.akamai.com: 0x55f8aaa0f8c0 [can

    multiplex] * MAX_CONCURRENT_STREAMS reached, skip (100) * Conn: 2 (0x55f8aaa261d0) Receive pipe weight: (-1/0), penalized: FALSE * Multiplexed connection found! * Found connection 2, with requests in the pipe (86) * Re-using existing connection! (#2) with host http2.akamai.com * Using Stream ID: c7 (easy handle 0x55f8ac0b7740) > GET /demo/tile-248.png HTTP/2 Host: http2.akamai.com User-Agent: Symfony HttpClient/Curl Accept-Encoding: deflate, gzip < HTTP/2 200 < content-length: 761 < content-type: image/png < ... < * Connection #2 to host http2.akamai.com left intact
26. 26.

    @nicolasgrekas Non-blocking monitoring / timeout management $client->stream($runningResponses, 0.0); Max number

    of seconds to wait before yielding a timeout chunk
27. 27.

    @nicolasgrekas ResponseInterface::getInfo() /** * This method SHOULD NOT throw any

    ExceptionInterface and SHOULD be non-blocking. * The returned info is "live": it can be empty and can change from one call to * another, as the request/response progresses. * * The following info MUST be returned: * - response_headers - an array modelled after the special $http_response_header variable * - redirect_count - the number of redirects followed while executing the request * - redirect_url - the resolved location of redirect responses, null otherwise * - start_time - the time when the request was sent or 0.0 when it's pending * - http_method - the HTTP verb of the last request * - http_code - the last response code or 0 when it is not known yet * - error - the error message when the transfer was aborted, null otherwise * - user_data - the value of the "user_data" request option, null if not set * - url - the last effective URL of the request * * When the "capture_peer_cert_chain" option is true, the "peer_certificate_chain" * attribute SHOULD list the peer certificates as an array of OpenSSL X.509 resources. * * Other info SHOULD be named after curl_getinfo()'s associative return value. * * @return array|mixed|null An array of all available info, or one of them when $type is * provided, or null when an unsupported type is requested */
28. 28.

    Symfony HttpClient The component (everything before is at the contracts

    level) #Symfony_Live @nicolasgrekas
29. 29.

    @nicolasgrekas NativeHttpClient & CurlHttpClient Both provide: • 100% of the

    contracts • secure redirects • extended (time) info • transparent HTTP compression and (de)chunking • automatic HTTP proxy configuration via env vars • I18n Domain Names
30. 30.

    @nicolasgrekas NativeHttpClient • The most portable • Based on the

    HTTP stream wrapper • With fixed redirection logic • Blocking until response headers arrive • Multiplexing response bodies • Abstracts all http+ssl+socket stream context options
31. 31.

    @nicolasgrekas CurlHttpClient • Requires ext-curl • With fixed redirection logic

    • Fully multiplexing response headers and bodies • Leverages HTTP/2 and PUSH when available • Abstracts most/all CURL(M)OPT_*/CURLINFO_* related to HTTP
32. 32.

    @nicolasgrekas 4 decorators as of now • ScopingHttpClient – Auto-configure

    the options based on the request URL • MockHttpClient – A client that doesn't make actual HTTP requests • CachingHttpClient – Adds caching on top of an HTTP client • Psr18Client – You already know about it with TraceableHttpClient and record & replay coming
33. 33.

    Symfony HttpClient Yours to combine (here are a few ideas

    that leverage some battle tested packages) #Symfony_Live @nicolasgrekas
34. 34.

    @nicolasgrekas FrameworkBundle / Autowiring framework: http_client: max_host_connections: 4 default_options: #

    ... scoped_clients: github_client: base_uri: https://api.github.com headers: Authorization: token abc123 # creates the HttpClientInterface $githubClient autowiring alias # and the HttpClientInterface $scopingHttpClient one
35. 35.

    @nicolasgrekas WebProfilerBundle

36. 36.

    @nicolasgrekas BrowserKit\HttpBrowser use Symfony\Component\BrowserKit\HttpBrowser; use Symfony\Component\HttpClient\HttpClient; $client = HttpClient::create(); $browser

    = new HttpBrowser($client); $browser->request('GET', 'https://example.com/'); $browser->clickLink('Log In'); $browser->submitForm('Sign In', ['username' => 'me', 'password' => 'pass']); $browser->clickLink('Subscriptions') ->filter('table tr:nth-child(2) td:nth-child(2)') ->each(function ($node) { echo trim($node->text())."\n"; });
37. 37.

    @nicolasgrekas R.I.P. Goutte HttpClient + DomCrawler + CssSelector + HttpKernel

    + BrowserKit
38. 38.

    Thanks for the ideas fopen(), Composer, Guzzle, Buzz, HTTPlug, Cake

    3, Axios, window.fetch() and all reviewers!
39. 39.

    Thank You! composer require symfony/http-client

40. 40.

    @nicolasgrekas for ($i = 0; $i < ...; ++$i) {

    $pendingResponses[$i] = $client->request('GET', "/some{$i}.html", [ 'user_data' => [$i, function ($response) use ($client): array { // extract CSS/JS files from $response->getContent() // $client->request() them and return the list of responses }], ]); } Made for HTTP/2 PUSH while ($pendingResponses) { foreach ($client->stream($pendingResponses) as $response => $chunk) { if ($chunk->isLast()) { [$responseId, $processResponse] = $response->getInfo('user_data'); unset($pendingResponses[$responseId]); if ($moreResponses = $processResponse($response)) { $pendingResponses += $moreResponses; break; } } } } Bonus slide: feed the stream while streaming