Enabling Googley microservices with HTTP/2 and gRPC.
Slides from my #JavaDay2016 talk "Enabling Googley microservices with HTTP/2 and gRPC.
A high performance, open source, HTTP/2-based RPC framework."
#gRPC #http2 #microservices
leader experienced in large scale software development. • 10+ years of software engineering experience. • Active gRPC user. Software Engineer @ Google @aiborisov
Publishing 50KB messages at maximum throughput from a single n1-highcpu-16 GPE VM instance, using 9 gRPC channels. More impressive than the almost 3x increase in throughput, is that it took only 1/4 of the CPU resources. 11x difference per CPU 3x increase in throughput https://cloud.google.com/blog/big-data/2016/03/announcing-grpc-alpha-for-google-cloud-pubsub
Remote Procedure Calls. A high performance, open source, general purpose standards-based, feature-rich RPC framework. Open sourced version of Stubby RPC used in Google. Actively developed and production-ready, current version is 1.0.1.
Dial-up connection 22,8 Kbps, 33.6Kbps, 56Kbps Fiber optic 100 - 1,000Mbps # of active web sites ~ 1.1 million ~ 1 billion Internet access Wired PCs PCs, laptops, mobiles, IoT, etc. Internet sites Text and image pages Multimedia applications
HTTP since HTTP/1.1, published in RFC 2068 in 1997. Design goals: • Reduce latency. • Minimize protocol overhead. • Add support for request prioritization • Add support for server push.
previous HTTP standards. The application semantics of HTTP are the same: • HTTP header fields • HTTP Methods • Request-response • Status codes • URIs HTTP/2 modifies how the data is formatted (framed) and transported between the client and server.
previous HTTP standards. The application semantics of HTTP are the same:: • HTTP header fields • HTTP Methods • Request-response • Status codes • URIs HTTP/2 modifies how the data is formatted (framed) and transported between the client and server.
bidirectional flow of bytes within an established connection, which may carry one or more messages. • Message is a complete sequence of frames that map to a logical request or response message. • Frame is the smallest unit of communication in HTTP/2, each containing a frame header, which at a minimum identifies the stream to which the frame belongs: HEADERS for metadata, DATA for payload, RST_STREAM SETTINGS, PUSH_PROMISE, PING, GOAWAY, WINDOW_UPDATE, etc.
into an exchange of binary-encoded frames, which are then mapped to messages that belong to a stream, and all of which are multiplexed within a single TCP connection. Binary Framing Stream 1 HEADERS Stream 2 :method: GET :path: /kyiv :version: HTTP/2 :scheme: https HEADERS :status: 200 :version: HTTP/2 :server: nginx/1.10.1 ... DATA <payload> Stream N Request Response TCP
requests and responses in parallel without blocking on any one. Use a single TCP connection to deliver multiple requests and responses in parallel. Enable flow-control, server push, etc. Stream 1 HEADERS Stream 2 DATA Stream 3 HEADERS Stream 3 DATA Stream 1 DATA Stream Y HEADERS Stream X DATA Requests Responses HTTP/2 connection Client Server
of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value Request #1
of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value Request #1 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame
of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value :method GET :scheme HTTPS :host myhost.com :path /resource custom_header some_value Request #1 Request #2 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame
of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value :method GET :scheme HTTPS :host myhost.com :path /resource custom_header some_value Request #1 Request #2 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame
of previously seen header fields. Indexes are sent for already seen headers. Values are encoded with a static Huffman code. HPACK :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value :method GET :scheme HTTPS :host myhost.com :path /resource custom_header some_value Request #1 Request #2 :method GET :scheme HTTPS :host myhost.com :path /image custom_header some_value HEADERS Frame :path /resource + indexes for already seen values HEADERS Frame
is reliable Latency is zero Bandwidth is infinite The network is secure https://blogs.oracle.com/jag/resource/Fallacies.html Topology doesn't change There is one administrator Transport cost is zero The network is homogeneous
the microservices design philosophy of coarse-grained message exchange between systems while avoiding the pitfalls of distributed objects and the fallacies of ignoring the network. gRPC Principles & Requirements http://www.grpc.io/blog/principles
futureClient.getCurrent(request); Futures.addCallback(future, new FutureCallback<WeatherResponse>() { @Override public void onSuccess(@Nullable WeatherResponse result) { logger.info("Current weather for {}: {}", request, response); } @Override public void onFailure(Throwable t) { logger.info("Cannot get current weather for {}", request); } });
• Multiplexes conn-s on the event loops: EpollEventLoopGroup, NioEventLoopGroup. • Decouples I/O waiting from threads. • gRPC uses Netty event loops for both client and server transports. Request Event Loop Worker thread Worker thread Worker thread Worker thread Worker thread Worker thread Worker thread Event Loop Event Loop Event Loop Callback Request Callback N e t w o r k N e t w o r k
client sends a single request to the server and gets a single response back, just like a normal function call. The client sends a request to the server and gets a stream to read a sequence of messages back. The client reads from the returned stream until there are no more messages. The client send a sequence of messages to the server using a provided stream. Once the client has finished writing the messages, it waits for the server to read them and return its response. Client streaming Both sides send a sequence of messages using a read-write stream. The two streams operate independently. The order of messages in each stream is preserved. BiDi streaming Unary Server streaming
scalable NoSQL database service. A fully-managed real-time messaging service. Speech to text conversion powered by machine learning. Cloud Speech API Cloud Bigtable Cloud PubSub
of gRPC: “Our team has instead building an RPC solution on top of gRPC. We are doing this transition for two main reasons: multi-language support and better extensibility/composability through request interceptors. That’s our current plan moving forward.” https://github.com/Netflix/ribbon Some of the gRPC Adopters
available on every popular development platform and easy for someone to build for their platform of choice. It should be viable on CPU & memory limited devices. gRPC Principles & Requirements http://www.grpc.io/blog/principles
in time. Deadline indicates to the server how long the client is willing to wait for an answer. RPC will fail with DEADLINE_EXCEEDED status code when deadline reached. gRPC Deadlines
gRPC. Lets the client provide information associated with the call to the server and vice versa in the form of a list of key-value pairs. Keys are strings; values can be either strings or binary data.
gRPC. Lets the client provide information associated with the call to the server and vice versa in the form of a list of key-value pairs. Keys are strings; values can be either strings or binary data. Can be sent in headers or trailers. http://www.grpc.io/docs/guides/concepts.html#metadata
a logical connection to a service, which is a collection of servers. NameResolver is a pluggable component that resolves a service name and return addresses to the caller. It has no knowledge of load-balancing. There will be different NameResolvers to support different name-systems, e.g., DNS, ZooKeeper etc. LoadBalancer is a pluggable component that receives resolved addresses from NameResolver and selects a Transport for Channel when asked.
of the stack must be able to evolve independently. A revision to the wire-format should not disrupt application layer bindings. http://www.grpc.io/blog/principles
new WeatherServiceAsync(tempService, humidityService, windService); Server grpcServer = InProcessServerBuilder.forName("weather") .addService(weatherService).build();
new WeatherServiceAsync(tempService, humidityService, windService); Server grpcServer = InProcessServerBuilder.forName("weather") .addService(ServerInterceptors.intercept(weatherService, interceptor)).build();
performance, and useful in testing. In-process client side channel: fully-featured, high performance, and useful in testing. MetadataUtils for testing metadata headers and trailer.
performance, and useful in testing. In-process client side channel: fully-featured, high performance, and useful in testing. MetadataUtils for testing metadata headers and trailer. grpc-testing - additional test utility functions useful for writing unit and integration tests: https://github.com/grpc/grpc-java/tree/master/testing
Systems Tracing Infrastructure: http://research.google.com/pubs/pub36356.html Implementation: Zipkin is a distributed tracing system: zipkin.io (JVM, Go, C#, Python, Ruby). Brave is a pure-Java distributed tracing implementation compatible with Zipkin (no Scala required): https://github.com/openzipkin/brave Brave-gRPC integrates Braves with gRPC.
a universal grpc command line client. grpc-gateway generates a reverse-proxy server which translates a RESTful JSON API into gRPC. OpenTracing is a set of consistent, expressive, vendor-neutral APIs for distributed tracing and context propagation. Prometheus monitoring support for grpc-java and grpc-go.
load-balancing and failover, monitoring, tracing, logging, and so on. Implementations should provide extensions points to allow for plugging in these features and, where useful, default implementations. gRPC Principles & Requirements http://www.grpc.io/blog/principles
multiplexed bidirectional protocol. gRPC (http://grpc.io): • HTTP/2 transport based, open source, general purpose standards-based, feature-rich RPC framework. • Bidirectional streaming over one single TCP connection. • Netty transport provides asynchronous and non-blocking I/O. • Deadline and cancellations propagation. • Client- and server-side flow-control. • Layered, pluggable and extensible. • Supports 10 programming languages. • Build-in testing support. • Production-ready (current version is 1.0.1) and growing ecosystem.
photos: • Andrey Borisenko • Alexandr Gusew Photo from https://www.google.com/about/datacenters page: • https://www.google.com/about/datacenters/gallery/#/tech/12 Photos licenced by Creative Commons 2.0 https://creativecommons.org/licenses/by/2.0/ : • https://www.flickr.com/photos/13800911@N08/3557747851/ by DirectDish • https://www.flickr.com/photos/marcalandavis/45657810/ by Marc Davis • https://www.flickr.com/photos/garryknight/5754661212/ by Garry Knight • https://www.flickr.com/photos/sodaigomi/21128888345/ by sodai gomi • https://www.flickr.com/photos/zengame/15972170944/ by Zengame • https://www.flickr.com/photos/londonmatt/18496249193/ by Matt Brown • https://www.flickr.com/photos/gazeronly/8058105980/ by torbakhopper U.S. Government Works: • https://www.flickr.com/photos/cbpphotos/8653133856/ by Josh Denmark Free of known restrictions: https://commons.wikimedia.org/wiki/File:He_can%27t_fix_guns_in_the_air%5E_Build_%60em_right%5E_Keep_%60em_firing%5E_-_NARA_-_535050.jpg
shimomura
inouehi
takezoux2
n3xem
yuyaabo
knakagawa
hakkadaikon
kuro_kurorrr
m4maruyama
ttnyt8701
daisuketakeda
tomokusaba
malarkey
danielanewman
reverentgeek
jakevdp
orderedlist
brettharned
marktimemedia
ammeep
cromwellryan
keathley
eileencodes
maggiecrowley
![Google Cloud Platform Adding New [Micro]Services Weather Weather Temp Wind](https://files.speakerdeck.com/presentations/84377ae168a74366b9700546fbb8c2c8/slide_67.jpg){kind=link}
![Google Cloud Platform Microservices The D'Artagnan [Our] Service Not D'Artagnan](https://files.speakerdeck.com/presentations/84377ae168a74366b9700546fbb8c2c8/slide_107.jpg){kind=link}
