HTTP Clients: silent heroes of distributed systems

HTTP Clients silent heroes of distributed systems Adam Dubiel @dubieladam

@dubieladam Why am i here?

@dubieladam 12 mln users 350 microservices 500+ engineers

@dubieladam Skylab team lead Andamio: JVM service starter Hermes pubsub

@dubieladam HTTP clients we know

@dubieladam HTTP clients we know RestTemplate restTemplate = new RestTemplate();
Client client = ClientBuilder.newBuilder().build(); AsyncRestTemplate asyncRestTemplate = new AsyncRestTemplate(); Retrofit retrofit = new Retrofit.Builder() .baseUrl("https://api.github.com/") .build();

@dubieladam HTTP clients we argue about restTemplate.getForEntity("https://github.com", Map.class); jerseyClient.target("https://github.com").request().get();

@dubieladam HTTP clients about we argue async.getForEntity("https://allegro.pl",Map.class).addCallback( { ResponseEntity<Map> result
-> /* success! */ }, { Throwable ex -> /* failure :( */ } ) jersey.target("https://allegro.pl").request().async().get();

@dubieladam

@dubieladam We need to go deeper RestTemplate.getForEntity -> RestTemplate.doExecute ->
HttpAccessor.createRequest -> ClientHttpRequestFactory.createRequest -> SimpleClientHttpRequestFactory -> URL.openConnection -> HttpURLConnection

@dubieladam We need to go deeper Client.get -> JerseyInvocation.invoke ->
ClientRuntime.invoke -> Connector.apply -> HttpUrlConnector._apply -> DefaultConnectionFactory.getConnection -> URL.openConnection() -> HttpURLConnection

@dubieladam

@dubieladam Async surely has some extra powers! Umm.. nope

@dubieladam Just throw some async on it AsyncRestTemplate.getForEntity -> AsyncRestTemplate.doExecute
-> AsyncHttpAccessor.createAsyncRequest -> AsyncClientHttpRequestFactory.createAsyncRequest -> ... -> ... -> HttpURLConnection

@dubieladam Just throw some async on it Client.async.get -> JerseyInvocation.submit
-> ClientRuntime.submit -> Connector.apply -> ... -> ... -> ... -> HttpURLConnection

@dubieladam Stop arguing about syntax sugar! We can resume once
we solve problems that do matter

API vs Implementation RestTemplate Jersey AsyncRestTemplate Retrofit HttpURLConnection Apache Client
Apache NIO (Async) Client OkHttp

@dubieladam How to choose the implementation?

@dubieladam Start with the basics The network is reliable. Latency
is zero. Bandwidth is infinite. The network is secure. Topology doesn't change. There is one administrator. Transport cost is zero. The network is homogeneous.

@dubieladam Fallacies of distributed systems The network is reliable. Latency

@dubieladam

@dubieladam Latency is zero Is latency zero? When do i
pay for latency? Can i pay less for connections? What is connection pooling? No With every connection and request Use connection pooling

@dubieladam Connection pooling

@dubieladam Cost of connecting HTTP TCP you are here you
need to care about this as well!

@dubieladam Costs of connecting conn request / response conn request
/ response conn request / response conn request / response

@dubieladam Costs of connecting vs network is reliable conn request
/ response

@dubieladam Costs of connecting conn request / response request /
response request / response request / response

@dubieladam https://shkspr.mobi/blog/wp-content/uploads/2016/05/A-pet-cat-typing-on-a-computer-keyboard.jpg

@dubieladam Connection pool connections available available leased leased

@dubieladam Connection pool connections leased leased leased leased pending

@dubieladam Fallacies of distributed systems The network is reliable. Latency

@dubieladam Network is reliable Are network or my peers reliable?
How can i know if they fail? Timeouts? What timeouts? No Set timeouts

@dubieladam Timeouts

@dubieladam Connection timeout conn request / response this used to
be faster.. should i go get a tea? okay, cancelling any moment.. oh! works again

@dubieladam One might say: impossible

@dubieladam Connection timeout time between initiating and establishing TCP connection
default depends on system (might be infinity) sane defaults: 50ms (accounting for hiccups, milage may vary, please measure!)

@dubieladam Socket/read timeout conn request / response request response response
great, got some data! hello?! waiting for some more..

@dubieladam Socket read/timeout time between receiving first and consecutive chunks
of data default depends on system (measured in seconds) sane default depends on peer response time

@dubieladam http://cdn.viralscape.com/wp-content/uploads/2014/09/Computer-Cat.jpg

@dubieladam Do i know all my dependencies? async.getForEntity("https://allegro.pl",Map.class).addCallback( { ResponseEntity<Map>
result -> /* success! */ }, { Throwable ex -> /* failure :( */ } )

@dubieladam DNS is a hidden dependency!

@dubieladam https://mdelamata.wordpress.com/2013/12/08/you-are-very-welcome/

@dubieladam Timeouts == fail fast waiting is wasting resources waste
enough and your app becomes unresponsive which makes other apps wait for you which makes them unresponsive which...

@dubieladam

@dubieladam Thread model

@dubieladam IO thread model Remote Server My Thread 1 conn
response

@dubieladam NIO thread model Remote Server My Thread 1 OS
IO Thread be careful not to do much here!

IO vs NIO HttpURLConnection Apache HTTP Client OkHttp Apache Async
HTTP Client Netty based Undertow based

@dubieladam For implementation, always ask is there any connection pool
and how big is it? what are the timeouts? what is the thread model?

@dubieladam Getting back to the APIs Now that we know
what really matters

@dubieladam Thread model AsyncRestTemplate.getForEntity -> AsyncRestTemplate.doExecute -> AsyncHttpAccessor.createAsyncRequest -> AsyncClientHttpRequestFactory.createAsyncRequest
-> ... -> ... -> HttpURLConnection

@dubieladam Thread model If my code does not block, who
is waiting for everything to happen?

@dubieladam What are the async APIs hiding? MyThread AsyncRestTemplate Remote
Server MAGIC...

@dubieladam What are the async APIs hiding? MyThread AsyncRestTemplate Remote
Server MAGIC... Thread 1 Thread 2 Thread 3 Thread N

@dubieladam For API, always ask on which thread pool is
my code running? how big is the thread pool and how can i control it?

@dubieladam When choosing HTTP client, always ask is there any
connection pool? how big is the connection pool? what are the timeouts? what is the thread model? on which thread pool is my code running? how big is the thread pool and how can i control it?

@dubieladam github.com/adamdubiel @dubieladam github.com/allegro

@dubieladam When choosing HTTP client, always ask is there any
connection pool? how big is the connection pool? what are the timeouts? what is the thread model? on which thread pool is my code running? how big is the thread pool and how can i control it?

HTTP Clients: silent heroes of distributed systems

HTTP Clients: silent heroes of distributed systems

More Decks by Adam Dubiel

Other Decks in Programming

Featured

Transcript