@me • CTO at Attendify • 5+ years with Clojure in production • Creator of Muse | Aleph & Netty contributor • More: protocols, algebras, Haskell, Idris • @kachayev on Twitter & Github
Dirigiste • aleph.flow/instrumented-pool • max-queue-size is 65,536 by default • remember this ^ when planning backpressure • see more • "HTTP Server: Execution Flow" • "HTTP Client: Connections Pool"
Netty ® Aleph • Netty is super cool, but kinda "low-level" • aleph.netty defines a lot of bridges • helpers to deal with ByteBufs • ChannelFuture → manifold's deferred • Channel represented as manifold's stream • a few macros to define ChannelHandlers • a lot more!
Aleph: In Production • still a lot of corners • or details to take care about • a few config params (at least) to be aware of • will discuss them one by one as it goes
Aleph Server: Step ↺ Step • http/start-server delegates to http.server/start- server • setups executor • mind the defaults ! • delegates to netty/start-server • mind the on-close callback
http.server/start-server • detects epoll when necessary or uses NIO • defines SSL context injection • builds io.netty.bootstrap.ServerBootstrap • sets up chieldHandler to pipeline-initializer • binds to socket and waits for the Channel to be ready
aleph.netty/pipeline-initializer • aleph.netty/pipeline-initializer • creates Netty's ChannelHandler that will • wait until Channel is registered on the Netty's Pipeline instance • call pipeline-builder provided as an argument • passing the instnace of Pipeline as an argument • clean up itself
aleph.http.server/pipeline- builder • pipeline-builder callback is defined here • sets up handlers, notably HttpServerCodec, HttpServerExpectContinueHandler • request handler is either ring-handler or raw-ring- handler • main task: read HTTP request and pass it to handle- request
aleph.http.server/handle-request • converts Netty's request → Ring-compatible request • runs handler (provided by the user) on a given executor • (or inlined!) • catches j.u.c.RejectedExecutionException and passes to rejected-handler • (by default answering with 503) • sends response when ready
Host Header $ curl -v http://localhost:8080 > GET / HTTP/1.1 > Host: localhost:8080 ... • RFC 7230 #5.4 • ... a server MUST respond with a 400 ... • is not enforced neither by Aleph nor by Netty • it's not that practical nowadays...
Host Header $ telnet 127.0.0.1 8080 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GET / HTTP/1.1 HTTP/1.1 200 OK Content-Type: text/plain ...
HTTP: Persistent Connection • persistent connection a.k.a keep-alive • single TCP connection for multiple HTTP requests/ responses • HTTP/1.0 Connection: keep-alive • HTTP/1.1: all connections are persistent by default • HTTP/1.1 Connection: close when necessary
HTTP: Connections in Aleph • to reuse TCP connections Aleph uses pools • aleph.http/connection-pool builds flow/intrumented- pool • generate callback in the intrumented-pool creates a new connection • keep-alive? option is set to true by default
HTTP: Connections • on the client appropriate Connection: * header is set here • meaning you can mess this up a bit by setting header manually • Aleph server detects keep-alive "status" here and here • and uses here to send response • Aleph server adds the header automatically • ... still !
aleph.http.client/http-connection • aleph.http/create-connection delagates to aleph.http.client/http-connection • internal function, not a public API • connection is a function: request → deferred response • manifold streams to represent requests & responses (we can have many) • netty/create-client to build Netty's channel
aleph.http.client/http-connection • when channel is ready • defines a function to get request from public API • locks on channel to put! request and take! response • consumes requests one by one • ...
aleph.http.client/http-connection • builds HttpMessage & writes it to the channel • when response is ready checks errors • converts body to ByteArrayInputStream using buffer of a given size
HTTP Connections: Netty Channel • aleph.netty/create-client • creates io.netty.bootstrap.Bootstrap • set a few options: SO_REUSEADDR, MAX_MESSAGES_PER_READ • detects to use EpollEventLoopGroup or NioEventLoopGroup • sets handler to pipeline-initializer • connects to the remote address
Client: pipeline-builder • pipeline-builder argument for the initializer is defined here ! • updates Pipeline instance with a few new handlers, most notably: • HttpClientCodec with appropriate settings • "main" handler with Aleph's client logic • pipeline-transform option might be useful to rebuild Pipeline when necessary
Client: pipeline-builder • either raw-client-handler or client-handler depending on raw-stream? • raw-client-handler returns body as manifold's stream of Netty's ByteBuf • client-handler converts body to InputStream of bytes (additional copying but less frictions) • both implementations are kinda tricky • most of the complexity: buffering all the way down & chunks
Connections Flow For The Public • aleph.http/request (rarely called directly) • "jumps" to the executor specified (or default-response- executor) • this might throw j.u.c.RejectedExecutionException • aleph.http/request is responsible for cleaning up after response is ready and on timeouts • also responsible for "top-level" middlewares: redirects & cookies
aleph.http/request • acquire connection from the pool specified (or default- connection-pool) • waits for the connection to be realized (either ready/reused or connecting) • "sends" the request applying connection function • chains on response and waits for :aleph/complete • disposes the connection from the pool when not keep-alive and on error
aleph.http/request • each stage tracks its own timeout since • PoolTimeout, ConnectionTimeout, RequestTimeout, ReadTimeout • never perform async operations w/o timeout • flexible error handling, easier to debug (reasoning is different) • you need this when implementing proxies or deciding on retries
HTTP Connections: Idle Timeout • persistent connection is meant to be "persistent" forever • not always the best option ! • idle-timeout option is available both for the client and the server since • when set, just updates the Pipeline builder • heavy lifting is done by Netty's IdleStateHandler • catching IdleStateEvent to close the connection
HTTP Connections: Pipelining • when multiple HTTP requests sent w/o waiting on responses • "allowed" with HTTP/1.1, not used widely (e.g. not used in modern browsers) • might dramatically reduce the number of TCP/IP packets • Aleph • supports pipelining on the server • does not support pipelining on the client
HTTP Connections: Pipelining • previous-response atom handles deferred • of the response that is currently processing • if any • next response is "scheduled" to be sent after
HTTP Connections: Debugging • from time to time you need to trace what's going on with your connections • at least state changes: opened, closed, acquired, released • easiest way: inject a ChannelHandler that listens to all events and logs them • to catch acquire and release you need to wrap flow/ instrumented-pool
HTTP Connections: Takeaways • read the list of connection-pool options • mind the defaults • mind timeouts and exceptions • do not rely on the default pool • especially when sending a lot of requests
Request & Response • ring req/resp are maps • io.netty.handler.codec.http provides • HttpRequest(s): DefaultHttpRequest, DefaultFullHttpRequest • HttpResponse(s): DefaultHttpResponse • also HttpMessage, HttpContent and more
Headers • headers represented with "almost" map • mind that getting a header by the name is not taking element from a map • concatenates with "," in case we have few values • not always the best option • required by Ring Spec
HTTP: Multipart • just use :multipart instead of :body when sending request • generates random boundary, sets appropriate header • would be helpful to "remember" boundary generated in the request (e.g. for testing) !
HTTP: Multipart • follows clj-http :multipart format (at least, visually ) • clj-http uses org.apache.http.entity.mime.MultipartEntityBuilder • Aleph implements "from scratch" on the client • supported Content-Transfer-Encodings • no support for the server • yada's implementation with manifold's stream
100 Continue • client sends Expect: 100-continue and does not transmit body • server replies with status code 100 Continue or 417 Expectation failed • client send body • potentially, less pressure on the networks when sending large requests • rarely used in practise
HTTP: Chunked • server: sending the response • server: reading the request • server: detecting last chunk of the request • client: reading the body • client: detecting last chunk • :max-chunk-size and :response-buffer-size options
aleph.http.core/send-message • good excuse to learn aleph.http.core/send-message • send-contiguous-body • send-file-body (send-chunked-file or send-file- region) • send-streaming-body • mind that try-set-content-length! is not called
HTTP: Chunked & Aleph • Aleph responses with "Transter-Encoding: chunked" when :body is seq, iterator or stream • if Content-Lenght header is not set explicitely • detection client disconnect is still kinda tough • think about buffering and throttling in advance, this talk might help
HTTP: Compression Headers • HTTP/1.1 headers • Accept-Encoding for the client • Content-Encoding for the server (whole connection) • Transfer-Encoding for the server (hop-by-hop)
HTTP: Compression • compression is disabled by default • supports custom compression level since • heavy lifting is done by io.netty.handler.codec.http.HttpContentCompressor
HTTP: Compression In Netty HttpContentCompressor ↳ HttpContentEncoder ... ↳ MessageToMessageCodec ... ... ↳ ChannelDuplexHandler • supports gzip or deflate, Brotli is an open question • respects Accept-Encoding header value
HTTP: Compression • mind the instance of io.netty.handler.stream.ChunkedWriteHandler • this forces send-file-body to use send-chunked-file instead of send-file-region • why? send-file-region uses zero-copy file transfer with io.netty.channel.DefaultFileRegion • does not support user-space modifications, e.g. compression !
HTTP: WebSocket • full-duplex communication • RFC 6455 "The WebSocket Protocol" • handshaking using HTTP Upgrade header (compatibility) • Aleph uses manifold's SplicedStream to represent duplex channel • supports Text and Binary frames, replies to Ping frames • a lot of cases and corners in the protocol (duplex communication is hard)
HTTP: Aleph WebSocket Client • http/websocket-client delegates to aleph.http.client/websocket-connection • mind the difference with aleph.http/websocket- connection ! • http.client/websocket-connection builds a Channel with netty/create-client • websocket-client-handler creates a duplex stream and a handler
HTTP: Aleph WebSocket Client • handler is responsible for • a handshake processing • Close frame sending • reacting appropriatly on incoming frames • non-websocket data payload throws an IllegalStateException
HTTP: Aleph WebSocket Server • not very intuitive ! • the idea here is to process HTTP/1.1 request first and then perform "upgrade" • http.server/websocket-upgrade-request? might be useful to "test" the request
HTTP: Aleph WebSocket Server • http/websocket-connection takes request and delegates to http.server/initialize-websocket-handler • initialize-websocket-handler builds and runs handshaker • .websocket? mark is set to modify response sending behavior • Pipeline is rebuilt appropriately • 2 streams spliced into one, as for the client
HTTP: Aleph & WebSocket • seq of the "connection close" events is "almost RFC" • client sends CloseFrame before closing the connection • on receiving CloseFrame saves status & reason • server sends CloseFrame w/o closing the connection • as it will be done by Netty • Netty behavior is "more RFC-ish"
HTTP: Aleph & WebSocket • client and server support permessage-deflate extension since • fine-grained Ping/Pong support is still an open question • to add ability to send http/websocket-ping manually, and to wait for Pong • helpful for heartbeats, online presence detection etc • pipeline-transform might be used to extend both server and client
DNS (import 'java.net.InetAddress) (InetAddress/getByName "github.com") ;; #object[java.net.Inet4Address 0x471abb1c "github.com/192.30.253.113"] • blocking ! • impossible to employ an alternative cache/retry policy • no way to use alternative name resolution
DNS • Aleph supports pluggable name resolver since • when :dns-options specified, sets up :name-resolver to async DNS resolver (def dns-pool (http/connection-pool {:dns-options {:name-servers ["8.8.4.4"]}})) (d/chain' (http/get "https://github.com/" {:pool dns-pool}) :status)
DNS • heavy lifting is done by io.netty.resolver.dns.DnsAddressResolverGroup • Aleph's part is mostly params juggling • supports epoll detection • and flexible configuration format for name server providers • aleph.http/create-connection uses InetSocketAddress/createUnresolved
HTTP Proxy: Aleph Client • import part of global HTTP infrastructure • used pretty heavily even for internal networks (yeah, servise mesh ! ) • long story, available in Aleph since • implementaion in not compatible with clj-http API, works on the connection-pool level only • heavy lifting is done by io.netty/netty-handler-proxy
ALPN • Application-Layer Protocol Negotiation Extention, RFC 7301 • allows the application layer to negotiate which protocol should be performed • replaced NPN (Next Protocol Negotiation Extension) • emerged from SPDY development
Negotiation with Netty • SPDY server gives a good example • pass ApplicationProtocolConfig to SslContextBuilder • choose NPN or ALPN • tell acceptable protocols
Negotiation with Netty • SslHandler performs negotiation • so, it should be added to the Pipeline earlier • passes to different engines, like OpenSSL, BoringSSL or even JDK • "Don't use the JDK for ALPN! But if you absolutely have to, here's how you do it... :)", grpc-java
HTTP/2 • major revision of the HTTP/1.1, SPDY successor, RFC 7540 • high-level compatibility with HTTP/1.1 • features (notably): • compressed headers (HPACK) • server push • multiplexing over a single TCP connection • more!
HTTP/2 • even tho' Netty 4.1 supports HTTP/2 • Aleph does not • Ring spec does not cover all HTTP/2 features • could be done in a separate library • with smart fallback to Aleph on ALPN (when necessary) • started working, very slow progress
What's Next? • [done] to cover req/resp representations • to talk about cookies & CookieStore • to talk about HTTPS • [done] talk about ALPN • [in progress] more pull requests!
kachayev
ogijun2018
leichteckig
codehex
gennei
mitchan
miso
star_zero
a4lg
hiro_y
speed1313
chatii
yusukebe
brad_frost
gr2m
ufuk
paulrobertlloyd
geeforr
chriscoyier
lara
michaelherold
aarron
bryan
vanstee
scottboms