Share
Embed
Switching to Asynchronous RPC Stack
gRPC is a really nice RPC system supporting not only classical remote method execution but also many useful features from 21st century like streaming, asynchronous message processing, h2 transport. In this talk we will share pain and experience Wix gained in a journey from handcrafted blocking json-rpc method of RPC to a standard non-blocking gRPC.
Viaceslav Pozdniakov
November 24, 2018
Transcript
48 slides
-
Numbers ▪ ~450 jvm (~Scala) (micro-)services / ~1400 instances ▪
~440 Node.js (micro-)services / ~1040 instances ▪ 228 back-end developers (our users) ▪ 379 front-end developers BEFORE WE EVEN START -
Remote Procedure Call In distributed computing, a remote procedure call
(RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is coded as if it were a normal (local) procedure call, without the programmer explicitly coding the details for the remote interaction. https://en.wikipedia.org/wiki/Remote_procedure_call BEFORE WE EVEN START -
AGENDA Wix (micro-)services Json RPC - problematic legacy gRPC -
challenges and solution How to implement such changes -
Microservices Microservices are a software development technique — a variant
of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. https://en.wikipedia.org/wiki/Microservices (MICRO-)SERVICES -
Good, but we need development velocity ▪ Same metrics reported
by every service ▪ Logs in same format ▪ Support of A/B testing by every service ▪ Context propagation (MICRO-)SERVICES -
Context? ▪ User identity of initial requests ▪ What kind
of user ▪ Experiment conduction data (A/B testing) ▪ User interface language ▪ ... (MICRO-)SERVICES -
Context propagation at Wix If service A calls service B
and service B calls service C, service C must receive all context data sent by service A even though this data is not explicitly used by service B and all calls do not bring any parameters. (MICRO-)SERVICES A B C uid=42 uid=42 uid=42 -
Interface Definition Language An interface description language or interface definition
language (IDL), is a specification language used to describe a software component's application programming interface (API). https://en.wikipedia.org/wiki/Interface_description_language JSON-RPC -
Scala as IDL trait ServiceName { def foo(bar: Bar): Unit
def zoo(value: Int): Option[Zoo] } case class Bar(value: String) case class Zoo(val1: Long, val2: Long) JSON-RPC -
Typical JSON-RPC service //server class ServiceNameImpl extends ServiceName { def
foo(bar: Bar): Unit = () def zoo(value: Int): Option[Zoo] = None } //client val aClient = rpcClientFactory.forTrait[ServiceName](url) aClient.zoo(42) // supports context propagation JSON-RPC -
Jackson is too flexible @JsonDeserialize(as=classOf[BarNew]) case class Bar(@JsonProperty("name") value: String)
case class BarNew(name: String, age: Integer = 0) @JsonTypeInfo(use=Id.CLASS, include=As.PROPERTY, property="class") trait Zoo case class ZooOne(val1: Long, val2: Long) extends Zoo case class ZooTwo(name: String) extends Zoo JSON-RPC -
Binary coupling/dependencies JSON-RPC Somewhy you cannot use off-the-shelf library for
your language to talk easily over a standard protocol -
Protocol Buffers IDL syntax = "proto3"; service ServiceName { rpc
Echo (Message) returns (Message); }; message Message { string field = 1; repeated string fields = 2; } gRPC -
gRPC (gRPC Remote Procedure Call) • Over HTTP/2 ◦ multiplexed,
instead of ordered and blocking ◦ one connection • Netty underneath • Protocol Buffers IDL compiled to almost any programming language gRPC -
Sounds like a plan! 1. Take ScalaPB (https://scalapb.github.io/) 2. Generate
traits and entities 3. Use JSON-RPC right away (+ REST) 4. Eventually switch to gRPC (and expose same service impl. as in json-rpc) 5. PROFIT!!! gRPC -
Protocol Buffers IDL import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {
override def echo(request: Message): Future[Message] = Future(request) } gRPC -
Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {
override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC -
Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {
override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC -
Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName {
override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC -
Stack relies on ThreadLocal 1. java.lang.ThreadLocal[T] is a global variable
per thread 2. ~6 years old code base 3. Own and specialized json-rpc and kafka clients,.. read ThreadLocal 4. Designed for sequential request processing gRPC -
Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName
{ override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC -
Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName
{ override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC -
Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName
{ override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC -
Definition of “thread”, theory A programming structure or process formed
by linking a number of separate elements or subroutines, especially each of the tasks executed concurrently in multithreading. https://en.oxforddictionaries.com/definition/thread gRPC -
What practitioners do? • Finagle: server as a function Req
=> Future[Res] (https://monkey.org/~marius/funsrv.pdf) • Spray: The Magnet Pattern (http://spray.io/blog/2012-12-13-the-magnet-pattern) • Play Framework: implicit request • FP elitists: “Use Reader monad!” • Martin Odersky: Implicit Function Types (dotty) or brand new Witnesses gRPC -
Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends
ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request) _ <- kafka.produce(response) } yield request } gRPC -
Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends
ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request)(cs) _ <- kafka.produce(response)(cs) } yield request } gRPC -
ScalaPB generator plugin def printService(printer: FunctionalPrinter): FunctionalPrinter = { printer
.add("package " + service.getFile.scalaPackageName) .newline .call(internal) .newline .add(s"object ${service.getName}WithCallScope {") .indent .newline //... gRPC -
CallScope • Represents a thread • Contains request context, request
deadline • Immutable/thread safe • A builder for other CallScopes with updated values • Constructed by framework with love. Users cannot create their own one (except TestCallScope) gRPC -
New “CallScoped” infrastructure • Service traits are generated by our
protoc plugin • Service traits, kafka consumers are (implicit) CallScope producers • gRPC and json-rpc clients, kafka producers are (implicit) CallScope consumers • Enables asynchronicity which is predictable gRPC -
Look ma, no globals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends
ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope) = { val f1 = Future(rpcClient.boo()(cs.withUid(42)) val f2 = Future(rpcClient.boo()(cs.withUid(43)) for { _ <- f1 _ <- f2 } yield request } } gRPC -
Architecture we had ~2 years ago Binary coupled distributed monolith
with thread-per-request threading model EXECUTION -
Architecture we are promoting Protocol Buffers IDL based distributed monolith
(must be improved but we are OK with that) which is asynchronous and non-blocking EXECUTION -
Execution of such big changes • Legacy will stay here
forever - deal with it • Create a black market first - no one will give you any resources to change status quo • Start with power-users: they are more agreeable • Be responsive, otherwise even agreeable users will leave you • Claim for legalization after a while - make the black market you created an official one EXECUTION -
Technical stuff • Just do not use global variables (i.e.
ThreadLocal) • Improved coding culture ◦ Future[T] for IO ◦ implicit CallScope - compile time checks • Use FP design patterns (i.e. Reader monad). Just don’t tell your users about that. • Code generation is OK, don’t be squeamish EXECUTION