as wire Protocol? RSocket

• Software Engineer focused on distributed systems development, adopting Reactive Manifesto and Reactive Programming techniques • Open source geek, active contributor of Project Reactor / RSocket • Author of the book "Reactive Programming in Spring 5” • Achieved 4-times better performance by tuning Reactor for RSocket Project About Me @OlehDokuka

Safe Harbor Everything you will hear is not related to any upcoming futures of storage protocols Everything that I’m going to say is nothing more than experiments or my OWN vision

Today’s Outline Wire Protocols Problematics RSocket

What is wire protocol? so

the mechanism for transmitting data from point a to point b

Existing Wire Protocol • MongoDb Wire Protocol • Postgresql Wire Protocol • Kafka Wire Protocol • … • …

WHY? Everyone did that

Reasons • Historically happened (in case of SQL Datas stores) • Kafka: HTTP libraries in many languages to be surprisingly shabby • Because I can (MongoDB)

MongoDB Wire Protocol

Mongo: Handshake Authentication Client Server TCP Error Success

Mongo: Messaging Message Client Server TCP Message

is a simple request-response style protocol MongoDB Wire Protocol

Mongo: Message Format 0 0 1 1 0 1 0 1 1 0 1 0 1 0 0 1 0 0 0 1 1 0 1 1 1 1 1 1 0 1 0 1 1 0 0 0 Logical Message Headers Message struct MsgHeader { int32 messageLength; int32 requestID; int32 responseTo; 
 int32 opCode; } struct MsgHeader { int32 messageLength; int32 requestID; int32 responseTo; 
 int32 opCode; } struct MsgHeader { int32 messageLength; int32 requestID; int32 responseTo; 
 int32 opCode; }

Mongo: Multiplexing Client Server TCP Message Message Message Message Message Message

Mongo: Message Types struct OP_QUERY { MsgHeader header; int32 flags; cstring fullCollectionName ; // "dbname.collectionname" int32 numberToSkip; // number of documents to skip int32 numberToReturn; // number of documents to return // in the first OP_REPLY batch document query; // query object. See below for details. [ document returnFieldsSelector; ] } struct OP_QUERY { MsgHeader header; int32 flags; cstring fullCollectionName ; // "dbname.collectionname" int32 numberToSkip; // number of documents to skip int32 numberToReturn; // number of documents to return // in the first OP_REPLY batch document query; // query object. See below for details. [ document returnFieldsSelector; ] } struct OP_QUERY { MsgHeader header; int32 flags; cstring fullCollectionName ; // "dbname.collectionname" int32 numberToSkip; // number of documents to skip int32 numberToReturn; // number of documents to return // in the first OP_REPLY batch document query; // query object. See below for details. [ document returnFieldsSelector; ] } struct OP_GET_MORE { MsgHeader header; int32 ZERO; cstring fullCollectionName; int32 numberToReturn; int64 cursorID; } struct OP_GET_MORE { MsgHeader header; int32 ZERO; cstring fullCollectionName; int32 numberToReturn; int64 cursorID; }

Mongo: Backpressure by Fetching Client Server TCP Cursor Query More More 2 1 3

Mongo 4.+: Message Types struct OP_MSG { MsgHeader header; uint32 flagBits; Sections[] sections; optional checksum; } struct OP_MSG { MsgHeader header; uint32 flagBits; Sections[] sections; optional checksum; }

Mongo: Streaming Client Server TCP Cursor Query 2 1 3

Mongo: Message Cancelling struct OP_KILL_CURSORS { MsgHeader header; int32 ZERO; int32 numberOfCursorIDs; int64* cursorIDs; }

is a simple request-response style protocol MongoDB Wire Protocol is a simple request-response style protocol is a simple request-response style protocol

Kafka Wire Protocol

Kafka: Message Types Request Header v0 => request_api_key request_api_version correlation_id request_api_key => INT16 request_api_version => INT16 correlation_id => INT32 Fetch Request (Version: 0) => replica_id max_wait_time min_bytes [topics] … … Fetch Response (Version: 0) => [responses] responses => topic [partition_responses] topic => STRING partition_responses => partition_header record_set partition_header => partition error_code high_watermark partition => INT32 error_code => INT16 high_watermark => INT64 record_set => RECORDS

Kafka: Throttling Mechanism Produce Request (Version: 0) => acks timeout [topic_data] acks => INT16 timeout => INT32 topic_data => topic [data] topic => STRING data => partition record_set partition => INT32 record_set => RECORDS Produce Response (Version: 1) => [responses] throttle_time_ms responses => topic [partition_responses] topic => STRING partition_responses => partition error_code base_offset partition => INT32 error_code => INT16 base_offset => INT64 throttle_time_ms => INT32 Produce Response (Version: 1) => [responses] throttle_time_ms responses => topic [partition_responses] topic => STRING partition_responses => partition error_code base_offset partition => INT32 error_code => INT16 base_offset => INT64 throttle_time_ms => INT32

Kafka: Some downsides

is a complex protocol Kafka Wire Protocol is a complex protocol

PostgreSql Wire Protocol

Postgresql: Summary • No multiplexing • Similar mechanisms for messaging as in Kafka and Mongo • It is own message binary format

is a simple request-response style protocol Postgresql Wire Protocol

Problems Summary • Wire Protocols are VERY Similar • Most of them are not ready to proper streaming and other todays requirements

In Vacuum Wire Protocol

Everyone Do: Handshake Storage Selection Authentication Client Server TCP

Everyone Do: Data Exchange Command Client Server Result TCP

Everyone Do: Client <- Server Backpressure Command Client Server Result TCP More Result More Result

Every Wants: Client -> Server Backpressure Client Server TCP Throttling

Some Do: Multiplexing Client Server TCP

Every Wish: Streaming Client Server TCP Query 2 1 3

Every Wish: Transport Variety Client Server TCP / UDP / QUIC / WebSocket

Problems Summary • Wire Protocols are VERY Similar • Most of them are not ready to proper streaming and other todays requirements • Most-likely all of them will reinvent the same again in the future

What if everyone use

What is • Open Source Layer 5/6 communication protocol • Reactive streams semantics • Application-level flow control • Supports both RPC and event-based messaging • Up to 10x faster than HTTP/1.1, using 90% less resources Built by leaders in microservices and cloud computing

42 Binary Messaging

43 Binary Messaging RSocket RSocket Connection

44 Binary Messaging RSocket RSocket Connection

45 Binary Messaging RSocket RSocket Connection

Binary Messaging 0 0 1 1 0 1 0 1 1 0 1 0 1 0 0 1 0 0 0 1 1 0 1 1 1 1 1 1 0 1 0 1 1 0 0 0 Logical Message Metadata Data • Data - allows any message format (including custom binary protocol) with no size limitations • Metadata - stands for headers / meta- info • Composite Metadata Extension - provides a way to provide Tracing / Monitoring / Routing / MimeType / Anything Cloud Applications needs today

47 Performance via Multiplexing

STREAM BLUE STREAM YELLOW STREAM RED 48 Project Reactor / RxJava / Any RS Performance via Multiplexing

49 WebSocket TCP QUIC Aeron Flexibility via Transport Agnostic • Message’s Framing / Logical Streams identity / all infrastructural information are provided on top if necessary • User may use any Reliable Transport

50 10 9 Resilience via Reactive Streams Spec

51 Resilience via Leasing STREAM BLUE STREAM YELLOW STREAM RED STREAM BLUE STREAM YELLOW STREAM RED Requester Requester STREAM BLUE STREAM YELLOW STREAM RED Responder Requester

52 Resilience via Leasing Responder Requester Requests Count : 10 
 Duration : next 10 secs STREAM BLUE STREAM YELLOW STREAM RED Requests Count : 9 
 Duration : next 10 secs Requests Count : 8 
 Duration : next 10 secs Requests Count : 7 
 Duration : next 10 secs Requests Count : 7 
 Duration : next 9 secs Requests Count : 7 
 Duration : next 8 secs Requests Count : 7 
 Duration : next 7 secs Requests Count : 7 
 Duration : next 0 secs Requests Count : 10 
 Duration : next 10 secs

Resilience Advanced • Server sales capacity to its clients • Natural Rate Limiting • Built-in Circuit Breaker

• Request - Response • Request - Stream • Request - Channel 
 (aka Stream - Stream)
 
 • Fire - and - Forget 
 (flush out and that is it) Wide Communication Patterns

Wide Communication Patterns • Clients is required to establish connection • Real Peer-To-Peer • Both sides are equal and can or cannot implements mentioned communication >>> CLIENT REQUEST STREAM >>> <<< SERVER REQUEST STREAM <<< SERVER CLIENT PEER PEER

No content

Summary • Using RSocket you do not have to reinvent wheel • Interoperability between RSocket based protocols

So, if everyone use

Client Server RSocket Message Message Message Message Message 2 1 3 reqiest(5) 2 1 3

Wait I do the same in my web 
 application every day

We all are Building wire protocols

No content

You can use RSocket too And yes!

64 User Experience • RSocket-RPC (via Protobuf Generators) for all supported languages • RSocket-IPC (plain inter-process communication) • RSocket-GraphQL • Spring-Boot-Starter-RSocket (Spring-Messaging integration)

Service A ??? RSocket Setup Frame Metadata Push

Introducing Broker Specification + Broker Spec

Computer A Computer B Application Transport Internet Internet Internet Internet Network Access Network Access Network Access Network Access Router #1 Router #2 Application Transport Components of TCP/IP Communication

API Abstraction Client Library API Abstraction Client Library Service A Service B Broker #1 Broker #2 Components of RSocket Communication Netifi Broker Netifi Broker

But this future we overview in another speech

Summary • RSocket is a powerful binary Protocol • In can Simplify communication between A and B • All the date stores just expose its own API • ALL the database vendors can benefit from using RSocket

What to do next? • http://rsocket.io - official web-page • https://bit.ly/2Fku9VC - video channel • https://community.reactive.foundation/ - community forum @OlehDokuka