Symfony HttpClient, what else?

Symfony HttpClient
what else?
#Symfony_Live
@nicolasgrekas

View Slide

@nicolasgrekas
{
"name": "symfony/http-client",
"provide": {
"php-http/client-implementation": "*",
"psr/http-client-implementation": "1.0",
"symfony/http-client-contracts-implementation": "1.1"
},
"require": {
"php": "^7.1.3",
"psr/log": "^1.0",
"symfony/http-client-contracts": "^1.1.3",
"symfony/polyfill-php73": "^1.11"
},
...
HttpClient is a standalone package

View Slide

@nicolasgrekas
composer require symfony/http-client
use Symfony\Component\HttpClient\HttpClient;
$client = HttpClient::create();
$url = 'https://symfony.com/versions.json';
$response = $client->request('GET', $url);

View Slide

@nicolasgrekas
Consume JSON APIs in a snap
$symfonyVersions = $response->toArray();
dump($symfonyVersions);
array:23 [
"lts" => "3.4.23"
"latest" => "4.2.4"
"dev" => "4.3.0-DEV"
"2.0" => "2.0.25"
"2.1" => "2.1.13"
"2.2" => "2.2.11"
…

View Slide

@nicolasgrekas
Abstraction is fully decoupled
from the component
Stateless (autowired) service
@nicolasgrekas

View Slide

@nicolasgrekas
Bye bye boilerplate

View Slide

@nicolasgrekas
Headers? Body? Options to the rescue

View Slide

@nicolasgrekas
Query parameters? Options to the rescue

View Slide

@nicolasgrekas
Bearer? JSON payload? Options to the rescue

View Slide

@nicolasgrekas
Form parameters? Options to the rescue

View Slide

@nicolasgrekas
namespace Symfony\Contracts\HttpClient;
interface ResponseInterface
{
public function getStatusCode(): int;
public function getHeaders(): array;
public function getContent(): string;
public function toArray(): array;
public function cancel(): void;
public function getInfo(): array;
}

View Slide

Symfony HttpClient
80% of use cases now covered
(in 20% of the symfony/contracts abstraction)
#Symfony_Live
@nicolasgrekas

View Slide

@nicolasgrekas
What about PSR-18?
namespace Symfony\Component\HttpClient;
class Psr18Client implements \Psr\Http\Client\ClientInterface
{
public function __construct(
\Symfony\Contracts\HttpClient\HttpClientInterface,
\Psr\Http\Message\ResponseFactoryInterface,
\Psr\Http\Message\StreamFactoryInterface
)

View Slide

@nicolasgrekas
composer require nyholm/psr7
$client = new Psr18Client();
$url = 'https://symfony.com/versions.json';
$request = $client->createRequest('GET', $url);
$response = $client->sendRequest($request);
$symfonyVersions = json_decode($response
->getBody()->getContents(), true);
Dual PSR-17 + 18

View Slide

@nicolasgrekas
Also for HTTPlug ^1.0|^2.0
namespace Symfony\Component\HttpClient;
class HttplugClient implements \Http\Client\HttpClient
{
public function __construct(
\Symfony\Contracts\HttpClient\HttpClientInterface,
\Http\Message\ResponseFactory,
\Http\Message\StreamFactory
)

View Slide

@nicolasgrekas
Most still at ^1.0

View Slide

Symfony HttpClient
What about the remaining 80%?
(in symfony/contracts)
#Symfony_Live
@nicolasgrekas

View Slide

@nicolasgrekas
Options to the rescue
'auth_basic' => null, // array|string - an array containing the username as first value, and optionally the
// password as the second one; or string like username:password - enabling HTTP Basic
// authentication (RFC 7617)
'auth_bearer' => null, // string - a token enabling HTTP Bearer authorization (RFC 6750)
'query' => [], // string[] - associative array of query string values to merge with the request's URL
'headers' => [], // iterable|string[]|string[][] - headers names provided as keys or as part of values
'body' => '', // array|string|resource|\Traversable|\Closure - the callback SHOULD yield a string
// smaller than the amount requested as argument; the empty string signals EOF; when
// an array is passed, it is meant as a form payload of field names and values
'json' => null, // array|\JsonSerializable - when set, implementations MUST set the "body" option to
// the JSON-encoded value and set the "content-type" headers to a JSON-compatible
// value it is they are not defined - typically "application/json"
'user_data' => null, // mixed - any extra data to attach to the request (scalar, callable, object...) that
// MUST be available via $response->getInfo('user_data') - not used internally
'max_redirects' => 20, // int - the maximum number of redirects to follow; a value lower or equal to 0 means
// redirects should not be followed; "Authorization" and "Cookie" headers MUST
// NOT follow except for the initial host name
'http_version' => null, // string - defaults to the best supported version, typically 1.1 or 2.0
'base_uri' => null, // string - the URI to resolve relative URLs, following rules in RFC 3986, section 2
'buffer' => true, // bool - whether the content of the response should be buffered or not
'on_progress' => null, // callable(int $dlNow, int $dlSize, array $info) - throwing any exceptions MUST abort
// the request; it MUST be called on DNS resolution, on arrival of headers and on
// completion; it SHOULD be called on upload/download of data and at least 1/s
'resolve' => [], // string[] - a map of host to IP address that SHOULD replace DNS resolution
'proxy' => null, // string - by default, the proxy-related env vars handled by curl SHOULD be honored
'no_proxy' => null, // string - a comma separated list of hosts that do not require a proxy to be reached
'timeout' => null, // float - the inactivity timeout - defaults to ini_get('default_socket_timeout')
'bindto' => '0', // string - the interface or the local socket to bind to
'verify_peer' => true, // see https://php.net/context.ssl for the following options
'verify_host' => true,
'cafile' => null,
'capath' => null,
'local_cert' => null,
'local_pk' => null,
'passphrase' => null,
'ciphers' => null,
'peer_fingerprint' => null,
'capture_peer_cert_chain' => false,
'extra' => [], // array - additional options that can be ignored if unsupported, unlike regular options
Options are in the contracts:
Writing decoupled decorators is easy
Options are in the contracts: the
transport is covered by the abstraction

View Slide

@nicolasgrekas
• timeout – control inactivity periods
• proxy – get through an HTTP proxy
• on_progress – display a progress bar / abort a request
• base_uri – resolve relative URLs / build a scoped client
• resolve – protect webhooks against calls to internal endpoints
• max_redirects – disable or limit redirects
Showcase of some options

View Slide

@nicolasgrekas
The resiliency of the Web relies on redirects, you’re part of it
Robust and failsafe by default
/**
* Gets the HTTP headers of the response.
*
* @param bool $throw Whether an exception should be thrown on 3/4/5xx status codes
*
* @throws TransportExceptionInterface When a network error occurs
* @throws RedirectionExceptionInterface On a 3xx when $throw is true and the
* "max_redirects" option has been reached
* @throws ClientExceptionInterface On a 4xx when $throw is true
* @throws ServerExceptionInterface On a 5xx when $throw is true
*/
public function getHeaders(bool $throw = true): array;
But what happens when the redirection limit is reached? or on 4xx / 5xx? Yours to decide:

View Slide

@nicolasgrekas
• body – array|string|resource|\Traversable|\Closure
Streameable uploads
$client->request('POST', 'http://upload.example.com', [
'body' => $mimeParts->toIterable(), // see e.g. symfony/mime
]);

View Slide

@nicolasgrekas
Streameable downloads
$url = 'http://.../ubuntu-18.04.1-desktop-amd64.iso';
$response = $client->request('GET', $url, [
'on_progress' => 'dump', // let's spam the console for fun
'buffer' => false, // skip buffering the response in memory
]);
// Responses are lazy! This waits only for the headers
if (200 !== $response->getStatusCode()) {
throw new \Exception('...');
}
$h = fopen('./ubuntu.iso', 'w');
foreach ($client->stream($response) as $chunk) {
fwrite($h, $chunk->getContent());
}

View Slide

@nicolasgrekas
Responses are lazy & requests are concurrent
for ($i = 0; $i < 379; ++$i) {
$uri = "https://http2.akamai.com/demo/tile-$i.png";
$responses[] = $client->request('GET', $uri);
}
foreach ($responses as $response) {
// block until completion of $response
// but monitor all other responses meanwhile
$response->getContent();
}

View Slide

@nicolasgrekas
Asynchronous requests
for ($i = 0; $i < 379; ++$i) {
$uri = "https://http2.akamai.com/demo/tile-$i.png";
$responses[] = $client->request('GET', $uri);
}
foreach ($client->stream($responses) as $response => $chunk) {
if ($chunk->isLast()) {
// a $response completed
} else {
// a $response's got network activity or timeout
}
}
379 requests
completed in
0,4 s!

View Slide

@nicolasgrekas
dump($request->getInfo('debug'))
* Found bundle for host http2.akamai.com: 0x55f8aaa0f8c0 [can multiplex]
* MAX_CONCURRENT_STREAMS reached, skip (100)
* Conn: 2 (0x55f8aaa261d0) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 2, with requests in the pipe (86)
* Re-using existing connection! (#2) with host http2.akamai.com
* Using Stream ID: c7 (easy handle 0x55f8ac0b7740)
> GET /demo/tile-248.png HTTP/2
Host: http2.akamai.com
User-Agent: Symfony HttpClient/Curl
Accept-Encoding: deflate, gzip
< HTTP/2 200
< content-length: 761
< content-type: image/png
< ...
<
* Connection #2 to host http2.akamai.com left intact

View Slide

@nicolasgrekas
Non-blocking monitoring / timeout management
$client->stream($runningResponses, 0.0);
Max number of
seconds to wait
before yielding
a timeout chunk

View Slide

@nicolasgrekas
ResponseInterface::getInfo()
/**
* This method SHOULD NOT throw any ExceptionInterface and SHOULD be non-blocking.
* The returned info is "live": it can be empty and can change from one call to
* another, as the request/response progresses.
*
* The following info MUST be returned:
* - response_headers - an array modelled after the special $http_response_header variable
* - redirect_count - the number of redirects followed while executing the request
* - redirect_url - the resolved location of redirect responses, null otherwise
* - start_time - the time when the request was sent or 0.0 when it's pending
* - http_method - the HTTP verb of the last request
* - http_code - the last response code or 0 when it is not known yet
* - error - the error message when the transfer was aborted, null otherwise
* - user_data - the value of the "user_data" request option, null if not set
* - url - the last effective URL of the request
*
* When the "capture_peer_cert_chain" option is true, the "peer_certificate_chain"
* attribute SHOULD list the peer certificates as an array of OpenSSL X.509 resources.
*
* Other info SHOULD be named after curl_getinfo()'s associative return value.
*
* @return array|mixed|null An array of all available info, or one of them when $type is
* provided, or null when an unsupported type is requested
*/

View Slide

Symfony HttpClient
The component
(everything before is at the contracts level)
#Symfony_Live
@nicolasgrekas

View Slide

@nicolasgrekas
NativeHttpClient & CurlHttpClient
Both provide:
• 100% of the contracts
• secure redirects
• extended (time) info
• transparent HTTP compression and (de)chunking
• automatic HTTP proxy configuration via env vars
• I18n Domain Names

View Slide

@nicolasgrekas
NativeHttpClient
• The most portable
• Based on the HTTP stream wrapper
• With fixed redirection logic
• Blocking until response headers arrive
• Multiplexing response bodies
• Abstracts all http+ssl+socket stream context options

View Slide

@nicolasgrekas
CurlHttpClient
• Requires ext-curl
• With fixed redirection logic
• Fully multiplexing response headers and bodies
• Leverages HTTP/2 and PUSH when available
• Abstracts most/all CURL(M)OPT_*/CURLINFO_* related to HTTP

View Slide

@nicolasgrekas
4 decorators as of now
• ScopingHttpClient – Auto-configure the options based on the request URL
• MockHttpClient – A client that doesn't make actual HTTP requests
• CachingHttpClient – Adds caching on top of an HTTP client
• Psr18Client – You already know about it
with TraceableHttpClient and record & replay coming

View Slide

Symfony HttpClient
Yours to combine
(here are a few ideas that leverage some battle tested packages)
#Symfony_Live
@nicolasgrekas

View Slide

@nicolasgrekas
FrameworkBundle / Autowiring
framework:
http_client:
max_host_connections: 4
default_options:
# ...
scoped_clients:
github_client:
base_uri: https://api.github.com
headers:
Authorization: token abc123
# creates the HttpClientInterface $githubClient autowiring alias
# and the HttpClientInterface $scopingHttpClient one

View Slide

@nicolasgrekas
WebProfilerBundle

View Slide

@nicolasgrekas
BrowserKit\HttpBrowser
use Symfony\Component\BrowserKit\HttpBrowser;
use Symfony\Component\HttpClient\HttpClient;
$client = HttpClient::create();
$browser = new HttpBrowser($client);
$browser->request('GET', 'https://example.com/');
$browser->clickLink('Log In');
$browser->submitForm('Sign In', ['username' => 'me', 'password' => 'pass']);
$browser->clickLink('Subscriptions')
->filter('table tr:nth-child(2) td:nth-child(2)')
->each(function ($node) { echo trim($node->text())."\n"; });

View Slide

@nicolasgrekas
R.I.P. Goutte
HttpClient
+ DomCrawler
+ CssSelector
+ HttpKernel
+ BrowserKit

View Slide

Thanks for the ideas
fopen(), Composer, Guzzle, Buzz, HTTPlug,
Cake 3, Axios, window.fetch()
and all reviewers!

View Slide

Thank You!
composer require
symfony/http-client

View Slide

@nicolasgrekas
for ($i = 0; $i < ...; ++$i) {
$pendingResponses[$i] = $client->request('GET', "/some{$i}.html", [
'user_data' => [$i, function ($response) use ($client): array {
// extract CSS/JS files from $response->getContent()
// $client->request() them and return the list of responses
}],
]);
}
Made for
HTTP/2 PUSH
while ($pendingResponses) {
foreach ($client->stream($pendingResponses) as $response => $chunk) {
if ($chunk->isLast()) {
[$responseId, $processResponse] = $response->getInfo('user_data');
unset($pendingResponses[$responseId]);
if ($moreResponses = $processResponse($response)) {
$pendingResponses += $moreResponses;
break;
}
}
}
}
Bonus slide: feed the stream while streaming

View Slide

Symfony HttpClient, what else?

Symfony HttpClient, what else?

Nicolas Grekas

More Decks by Nicolas Grekas

Other Decks in Technology

Featured

Transcript