Switching to Asynchronous RPC Stack

gRPC is a really nice RPC system supporting not only classical remote method execution but also many useful features from 21st century like streaming, asynchronous message processing, h2 transport. In this talk we will share pain and experience Wix gained in a journey from handcrafted blocking json-rpc method of RPC to a standard non-blocking gRPC.

Transcript

1. Viačeslav Pozdniakov @poznia github.com/vipo Switching to Asynchronous RPC Stack
2. Wix Engineering Locations Ukraine Israel Lithuania Vilnius Kiev Dnipro Tel-Aviv Be’er Sheva
3. Numbers ▪ ~450 jvm (~Scala) (micro-)services / ~1400 instances ▪ ~440 Node.js (micro-)services / ~1040 instances ▪ 228 back-end developers (our users) ▪ 379 front-end developers BEFORE WE EVEN START
4. Remote Procedure Call In distributed computing, a remote procedure call (RPC) is when a computer program causes a procedure (subroutine) to execute in a different address space (commonly on another computer on a shared network), which is coded as if it were a normal (local) procedure call, without the programmer explicitly coding the details for the remote interaction. https://en.wikipedia.org/wiki/Remote_procedure_call BEFORE WE EVEN START
5. AGENDA Wix (micro-)services Json RPC - problematic legacy gRPC - challenges and solution How to implement such changes
6. Wix (micro-)services 01
7. Microservices Microservices are a software development technique — a variant of the service-oriented architecture (SOA) architectural style that structures an application as a collection of loosely coupled services. https://en.wikipedia.org/wiki/Microservices (MICRO-)SERVICES
8. Good, but we need development velocity ▪ Same metrics reported by every service ▪ Logs in same format ▪ Support of A/B testing by every service ▪ Context propagation (MICRO-)SERVICES
9. Context? ▪ User identity of initial requests ▪ What kind of user ▪ Experiment conduction data (A/B testing) ▪ User interface language ▪ ... (MICRO-)SERVICES
10. Context propagation at Wix If service A calls service B and service B calls service C, service C must receive all context data sent by service A even though this data is not explicitly used by service B and all calls do not bring any parameters. (MICRO-)SERVICES A B C uid=42 uid=42 uid=42
12. JSON-RPC 02
13. JSON-RPC https://en.wikipedia.org/wiki/JSON-RPC JSON-RPC
14. Interface Definition Language An interface description language or interface definition language (IDL), is a specification language used to describe a software component's application programming interface (API). https://en.wikipedia.org/wiki/Interface_description_language JSON-RPC
15. Scala as IDL trait ServiceName { def foo(bar: Bar): Unit def zoo(value: Int): Option[Zoo] } case class Bar(value: String) case class Zoo(val1: Long, val2: Long) JSON-RPC
16. Typical JSON-RPC service //server class ServiceNameImpl extends ServiceName { def foo(bar: Bar): Unit = () def zoo(value: Int): Option[Zoo] = None } //client val aClient = rpcClientFactory.forTrait[ServiceName](url) aClient.zoo(42) // supports context propagation JSON-RPC
17. Implementation details ▪ Some Proxy.newProxyInstance magic ▪ Jackson (https://github.com/FasterXML/jackson) JSON-RPC
18. Jackson is too flexible @JsonDeserialize(as=classOf[BarNew]) case class Bar(@JsonProperty("name") value: String) case class BarNew(name: String, age: Integer = 0) @JsonTypeInfo(use=Id.CLASS, include=As.PROPERTY, property="class") trait Zoo case class ZooOne(val1: Long, val2: Long) extends Zoo case class ZooTwo(name: String) extends Zoo JSON-RPC
19. Binary coupling/dependencies JSON-RPC Somewhy you cannot use off-the-shelf library for your language to talk easily over a standard protocol
21. gRPC 03
22. Protocol Buffers IDL syntax = "proto3"; service ServiceName { rpc Echo (Message) returns (Message); }; message Message { string field = 1; repeated string fields = 2; } gRPC
23. gRPC (gRPC Remote Procedure Call) • Over HTTP/2 ◦ multiplexed, instead of ordered and blocking ◦ one connection • Netty underneath • Protocol Buffers IDL compiled to almost any programming language gRPC
24. Sounds like a plan! 1. Take ScalaPB (https://scalapb.github.io/) 2. Generate traits and entities 3. Use JSON-RPC right away (+ REST) 4. Eventually switch to gRPC (and expose same service impl. as in json-rpc) 5. PROFIT!!! gRPC
25. Protocol Buffers IDL import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = Future(request) } gRPC
26. Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC
27. Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC
28. Request Context Strike import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { rpcClient.boo() // works as expected Future { rpcClient.boo() // initial request context is lost request } } gRPC
29. Stack relies on ThreadLocal 1. java.lang.ThreadLocal[T] is a global variable per thread 2. ~6 years old code base 3. Own and specialized json-rpc and kafka clients,.. read ThreadLocal 4. Designed for sequential request processing gRPC
30. Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC
31. Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC
32. Being blocking wastes resources import scala.concurrent.ExecutionContext.Implicits.global class ServiceNameImpl extends ServiceNameGrpc.ServiceName { override def echo(request: Message): Future[Message] = { setUid(42) rpcClient.boo() setUid(43) rpcClient.boo() Future(request) } } gRPC
33. Definition of “thread”, theory A programming structure or process formed by linking a number of separate elements or subroutines, especially each of the tasks executed concurrently in multithreading. https://en.oxforddictionaries.com/definition/thread gRPC
34. What practitioners do? • Finagle: server as a function Req => Future[Res] (https://monkey.org/~marius/funsrv.pdf) • Spray: The Magnet Pattern (http://spray.io/blog/2012-12-13-the-magnet-pattern) • Play Framework: implicit request • FP elitists: “Use Reader monad!” • Martin Odersky: Implicit Function Types (dotty) or brand new Witnesses gRPC
35. Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request) _ <- kafka.produce(response) } yield request } gRPC
36. Reader monad for mortals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope): Future[Message] = for { response <- rpcClient.foo(request)(cs) _ <- kafka.produce(response)(cs) } yield request } gRPC
37. ScalaPB generator plugin def printService(printer: FunctionalPrinter): FunctionalPrinter = { printer .add("package " + service.getFile.scalaPackageName) .newline .call(internal) .newline .add(s"object ${service.getName}WithCallScope {") .indent .newline //... gRPC
38. CallScope • Represents a thread • Contains request context, request deadline • Immutable/thread safe • A builder for other CallScopes with updated values • Constructed by framework with love. Users cannot create their own one (except TestCallScope) gRPC
39. New “CallScoped” infrastructure • Service traits are generated by our protoc plugin • Service traits, kafka consumers are (implicit) CallScope producers • gRPC and json-rpc clients, kafka producers are (implicit) CallScope consumers • Enables asynchronicity which is predictable gRPC
40. Look ma, no globals class ServiceNameImpl(rpcClient: SomeRpcClient, kafka: KafkaClient) extends ServiceNameWithCallScope.ServiceName { override def echo(request: Message)(implicit cs: CallScope) = { val f1 = Future(rpcClient.boo()(cs.withUid(42)) val f2 = Future(rpcClient.boo()(cs.withUid(43)) for { _ <- f1 _ <- f2 } yield request } } gRPC
41. Lessons learned 04
42. Architecture we had ~2 years ago Binary coupled distributed monolith with thread-per-request threading model EXECUTION
43. Architecture we are promoting Protocol Buffers IDL based distributed monolith (must be improved but we are OK with that) which is asynchronous and non-blocking EXECUTION
44. Execution of such big changes • Legacy will stay here forever - deal with it • Create a black market first - no one will give you any resources to change status quo • Start with power-users: they are more agreeable • Be responsive, otherwise even agreeable users will leave you • Claim for legalization after a while - make the black market you created an official one EXECUTION
45. Technical stuff • Just do not use global variables (i.e. ThreadLocal) • Improved coding culture ◦ Future[T] for IO ◦ implicit CallScope - compile time checks • Use FP design patterns (i.e. Reader monad). Just don’t tell your users about that. • Code generation is OK, don’t be squeamish EXECUTION
46. The answer is there is no answer Chuck Palahniuk
47. Thank You @poznia github.com/vipo
48. Q&A @poznia github.com/vipo