Switching to Asynchronous RPC Stack

Slide 1

Slide 1 text

Viačeslav Pozdniakov @poznia github.com/vipo Switching to Asynchronous RPC Stack

Slide 2

Slide 2 text

Wix Engineering Locations Ukraine Israel Lithuania Vilnius Kiev Dnipro Tel-Aviv Be’er Sheva

Slide 3

Slide 3 text

Numbers ▪ ~450 jvm (~Scala) (micro-)services / ~1400 instances ▪ ~440 Node.js (micro-)services / ~1040 instances ▪ 228 back-end developers (our users) ▪ 379 front-end developers BEFORE WE EVEN START

Slide 4

Slide 4 text

Remote Procedure Call In distributed computing, a remote procedure call (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is coded as if it were a normal (local) procedure call, without the programmer explicitly coding the details for the remote interaction. https://en.wikipedia.org/wiki/Remote_procedure_call BEFORE WE EVEN START

Slide 5

Slide 5 text

AGENDA Wix (micro-)services Json RPC - problematic legacy gRPC - challenges and solution How to implement such changes

Slide 6

Slide 6 text

Wix (micro-)services 01

Slide 7

Slide 7 text

Microservices Microservices are a software development technique — a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. https://en.wikipedia.org/wiki/Microservices (MICRO-)SERVICES

Slide 8

Slide 8 text

Good, but we need development velocity ▪ Same metrics reported by every service ▪ Logs in same format ▪ Support of A/B testing by every service ▪ Context propagation (MICRO-)SERVICES

Slide 9

Slide 9 text

Context? ▪ User identity of initial requests ▪ What kind of user ▪ Experiment conduction data (A/B testing) ▪ User interface language ▪ ... (MICRO-)SERVICES

Slide 10

Slide 10 text

Context propagation at Wix If service A calls service B and service B calls service C, service C must receive all context data sent by service A even though this data is not explicitly used by service B and all calls do not bring any parameters. (MICRO-)SERVICES A B C uid=42 uid=42 uid=42

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

JSON-RPC 02

Slide 13

Slide 13 text

JSON-RPC https://en.wikipedia.org/wiki/JSON-RPC JSON-RPC

Slide 14

Slide 14 text

Interface Definition Language An interface description language or interface definition language (IDL), is a specification language used to describe a software component's application programming interface (API). https://en.wikipedia.org/wiki/Interface_description_language JSON-RPC

Slide 15

Slide 15 text

Scala as IDL trait ServiceName { def foo(bar: Bar): Unit def zoo(value: Int): Option[Zoo] } case class Bar(value: String) case class Zoo(val1: Long, val2: Long) JSON-RPC

Slide 16

Slide 16 text

Typical JSON-RPC service //server class ServiceNameImpl extends ServiceName { def foo(bar: Bar): Unit = () def zoo(value: Int): Option[Zoo] = None } //client val aClient = rpcClientFactory.forTrait[ServiceName](url) aClient.zoo(42) // supports context propagation JSON-RPC

Slide 17

Slide 17 text

Implementation details ▪ Some Proxy.newProxyInstance magic ▪ Jackson (https://github.com/FasterXML/jackson) JSON-RPC

Slide 18

Slide 18 text

Jackson is too flexible @JsonDeserialize(as=classOf[BarNew]) case class Bar(@JsonProperty("name") value: String) case class BarNew(name: String, age: Integer = 0) @JsonTypeInfo(use=Id.CLASS, include=As.PROPERTY, property="class") trait Zoo case class ZooOne(val1: Long, val2: Long) extends Zoo case class ZooTwo(name: String) extends Zoo JSON-RPC

Slide 19

Slide 19 text

Binary coupling/dependencies JSON-RPC Somewhy you cannot use off-the-shelf library for your language to talk easily over a standard protocol

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

gRPC 03

Slide 22

Slide 22 text

Protocol Buffers IDL syntax = "proto3"; service ServiceName { rpc Echo (Message) returns (Message); }; message Message { string field = 1; repeated string fields = 2; } gRPC

Slide 23

Slide 23 text

gRPC (gRPC Remote Procedure Call) ● Over HTTP/2 ○ multiplexed, instead of ordered and blocking ○ one connection ● Netty underneath ● Protocol Buffers IDL compiled to almost any programming language gRPC

Slide 24

Slide 24 text

Sounds like a plan! 1. Take ScalaPB (https://scalapb.github.io/) 2. Generate traits and entities 3. Use JSON-RPC right away (+ REST) 4. Eventually switch to gRPC (and expose same service impl. as in json-rpc) 5. PROFIT!!! gRPC

Slide 25

Slide 25 text

Protocol Buffers IDL import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = Future(request) } gRPC

Slide 26

Slide 26 text

Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Stack relies on ThreadLocal 1. java.lang.ThreadLocal[T] is a global variable per thread 2. ~6 years old code base 3. Own and specialized json-rpc and kafka clients,.. read ThreadLocal 4. Designed for sequential request processing gRPC

Slide 30

Slide 30 text

Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Definition of “thread”, theory A programming structure or process formed by linking a number of separate elements or subroutines, especially each of the tasks executed concurrently in multithreading. https://en.oxforddictionaries.com/definition/thread gRPC

Slide 34

Slide 34 text

What practitioners do? ● Finagle: server as a function Req => Future[Res] (https://monkey.org/~marius/funsrv.pdf) ● Spray: The Magnet Pattern (http://spray.io/blog/2012-12-13-the-magnet-pattern) ● Play Framework: implicit request ● FP elitists: “Use Reader monad!” ● Martin Odersky: Implicit Function Types (dotty) or brand new Witnesses gRPC

Slide 35

Slide 35 text

Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request) _ <- kafka.produce(response) } yield request } gRPC

Slide 36

Slide 36 text

Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request)(cs) _ <- kafka.produce(response)(cs) } yield request } gRPC

Slide 37

Slide 37 text

ScalaPB generator plugin def printService(printer: FunctionalPrinter): FunctionalPrinter = { printer .add("package " + service.getFile.scalaPackageName) .newline .call(internal) .newline .add(s"object ${service.getName}WithCallScope {") .indent .newline //... gRPC

Slide 38

Slide 38 text

CallScope ● Represents a thread ● Contains request context, request deadline ● Immutable/thread safe ● A builder for other CallScopes with updated values ● Constructed by framework with love. Users cannot create their own one (except TestCallScope) gRPC

Slide 39

Slide 39 text

New “CallScoped” infrastructure ● Service traits are generated by our protoc plugin ● Service traits, kafka consumers are (implicit) CallScope producers ● gRPC and json-rpc clients, kafka producers are (implicit) CallScope consumers ● Enables asynchronicity which is predictable gRPC

Slide 40

Slide 40 text

Look ma, no globals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope) = { val f1 = Future(rpcClient.boo()(cs.withUid(42)) val f2 = Future(rpcClient.boo()(cs.withUid(43)) for { _ <- f1 _ <- f2 } yield request } } gRPC

Slide 41

Slide 41 text

Lessons learned 04

Slide 42

Slide 42 text

Architecture we had ~2 years ago Binary coupled distributed monolith with thread-per-request threading model EXECUTION

Slide 43

Slide 43 text

Architecture we are promoting Protocol Buffers IDL based distributed monolith (must be improved but we are OK with that) which is asynchronous and non-blocking EXECUTION

Slide 44

Slide 44 text

Execution of such big changes ● Legacy will stay here forever - deal with it ● Create a black market first - no one will give you any resources to change status quo ● Start with power-users: they are more agreeable ● Be responsive, otherwise even agreeable users will leave you ● Claim for legalization after a while - make the black market you created an official one EXECUTION

Slide 45

Slide 45 text

Technical stuff ● Just do not use global variables (i.e. ThreadLocal) ● Improved coding culture ○ Future[T] for IO ○ implicit CallScope - compile time checks ● Use FP design patterns (i.e. Reader monad). Just don’t tell your users about that. ● Code generation is OK, don’t be squeamish EXECUTION